Algorithm Design for Optimal Power Flow, Security-ConstrainedUnit Commitment, and Demand Response in Energy SystemsbyShahab BahramiM.S., Sharif University of Technology, 2012B.S., Sharif University of Technology, 2010A THESIS SUBMITTED IN PARTIAL FULFILLMENT OFTHE REQUIREMENTS FOR THE DEGREE OFDOCTOR OF PHILOSOPHYinThe Faculty of Graduate and Postdoctoral Studies(Electrical and Computer Engineering)THE UNIVERSITY OF BRITISH COLUMBIA(Vancouver)August 2017c© Shahab Bahrami, 2017AbstractEnergy management is of prime importance for power system operators to enhance theuse of the existing and new facilities, while maintaining a high level of reliability. Inthis thesis, we develop analytical models and efficient algorithms for energy managementprograms in transmission and distribution networks. First, we study the optimal powerflow (OPF) in ac-dc grids, which is a non-convex optimization problem. We use convexrelaxation techniques and transform the problem into a semidefinite program (SDP). Wederive the sufficient conditions for zero relaxation gap and design an algorithm to obtain theglobal optimal solution. Subsequently, we study the security-constrained unit commitment(SCUC) problem in ac-dc grids with generation and load uncertainty. We introduce theconcept of conditional value-at risk to limit the net power supply shortage. The SCUC is anonlinear mixed-integer optimization problem. We use l1-norm approximation and convexrelaxation techniques to transform the problem into an SDP. We develop an algorithmto determine a near-optimal solution. Next, we target the role of end-users in energymanagement activities. We study demand response programs for residential users anddata centers. For residential users, we capture their coupled decision making in a demandresponse program with real-time pricing as a partially observable stochastic game. To makethe problem tractable, we approximate the optimal scheduling policy of the residentialusers by the Markov perfect equilibrium (MPE) of a fully observable stochastic gamewith incomplete information. We develop an online load scheduling learning algorithm todetermine the users’ MPE policy. Last but not least, we focus on the demand responseiiAbstractprogram for data centers in deregulated electricity markets, where each data center canchoose a utility company from multiple available suppliers. We model the data centers’coupled decisions of utility company choices and workload scheduling as a many-to-onematching game with externalities. We characterize the stable outcome of the game, whereno data center has an incentive to unilaterally change its strategy. We develop a distributedalgorithm that is guaranteed to converge to a stable outcome.iiiLay SummaryNew energy management programs are necessary to reduce cost and increase reliabilityin power grids through improving the energy generation, transmission, and consumption.This thesis focuses on designing algorithms for energy management programs for supplyand demand sides in energy networks. We develop an optimal power flow algorithm todetermine the generation levels of the generators to minimize the grid-wide operationcost and power losses. Subsequently, we propose a security-constrained unit commitmentalgorithm to determine the set of operating generators with the minimum cost. Next,we focus on demand response programs and the role of customers in energy managementactivities. We develop a scheduling algorithm for residential customers to manage theenergy consumption of their electric appliances. Finally, we study data centers’ demandresponse. We develop an algorithm that enables a data center to choose a utility companyamong multiple competing suppliers and schedule workloads to reduce bill payment.ivPrefaceChapters 2–5 encompass work which has been published or currently under review. Thecorresponding papers are under the supervision of Prof. Vincent W.S. Wong. The paperscorresponding to Chapter 2 are co-authored with Prof. Juri Jatskevich and Dr. FrancisTherrien. The papers corresponding to Chapters 4 and 5 are co-authored with Prof. Jian-wei Huang. Prof. Jatskevich and Prof. Huang have provided constructive comments andsuggestions on the corresponding papers. Dr. Therrien has also provided helpful comments.For all chapters, I hereby declare that I am the first author of the corresponding papers.The following publications describe the work completed in this thesis.Journal Papers, Published or Accepted• Shahab Bahrami, Francis Therrien, Vincent W.S. Wong, and Juri Jatskevich, “Semidef-inite relaxation of optimal power flow for ac-dc grids,” IEEE Trans. on Power Sys-tems, vol. 32, no. 1, pp. 289−304, Jan. 2017.• Shahab Bahrami, Vincent W.S. Wong, and Jianwei Huang, “An online learning algo-rithm for demand response in smart grid,” accepted for publication in IEEE Trans.on Smart Grid, 2017.Journal Papers, Submitted• Shahab Bahrami and Vincent W.S. Wong, “Security-constrained unit commitmentfor ac-dc grids with generation and load uncertainty,” submitted, 2017.vPreface• Shahab Bahrami, Vincent W.S. Wong, and Jianwei Huang, “Data centers demandresponse in deregulated markets,” submitted, 2017.Conference Papers, Published• Shahab Bahrami, Vincent W.S. Wong, and Juri Jatskevich, “Optimal power flowfor ac-dc networks,” in Proc. of IEEE International Conference on Smart GridCommunications (SmartGridComm), Venice, Italy, Nov. 2014.• Shahab Bahrami and Vincent W.S. Wong, “A potential game framework for chargingPHEVs in smart grid,” in Proc. of IEEE Pacific Rim Conference on Communica-tions, Computers and Signal Processing (PacRim), Victoria, Canada, Aug. 2015.• Shahab Bahrami and Vincent W.S. Wong, “An autonomous demand response pro-gram in smart grid with foresighted users,” in Proc. of IEEE International Confer-ence on Smart Grid Communications (SmartGridComm), Miami, FL, Nov. 2015.• Shahab Bahrami, Vincent W.S. Wong, and Jianwei Huang, “Demand response fordata centers in deregulated markets: A matching game approach,” accepted for pub-lication in Proc. of IEEE International Conference on Smart Grid Communications(SmartGridComm), Dresden, Germany, Oct. 2017.viTable of ContentsAbstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iiLay Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ivPreface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vTable of Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viiList of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiiList of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiiiList of Abbreviations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xviiAcknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxDedication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxi1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.1 Energy Management in Transmission Systems . . . . . . . . . . . . . . . . 41.1.1 Optimal Power Flow (OPF) . . . . . . . . . . . . . . . . . . . . . . 51.1.2 Security-Constrained Unit Commitment (SCUC) . . . . . . . . . . 81.2 Energy Management in Distribution Systems . . . . . . . . . . . . . . . . 111.2.1 Demand Response Program for Residential Users . . . . . . . . . . 14viiTable of Contents1.2.2 Demand Response Program for Data Centers . . . . . . . . . . . . 171.3 Summary of Results and Contributions . . . . . . . . . . . . . . . . . . . 201.4 Thesis Organization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222 Semidefinite Relaxation of Optimal Power Flow for ac-dc Grids . . . . 242.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242.2 System Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252.2.1 Objective Function and Constraints . . . . . . . . . . . . . . . . . 302.3 SDP Form of the ac-dc OPF Problem . . . . . . . . . . . . . . . . . . . . 342.3.1 Transforming the Objective Function . . . . . . . . . . . . . . . . . 362.3.2 Transforming the Constraints . . . . . . . . . . . . . . . . . . . . . 372.4 Performance Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 452.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 543 Security-Constrained Unit Commitment for ac-dc Grids . . . . . . . . 553.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 553.2 System Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 573.2.1 VSC Station Model . . . . . . . . . . . . . . . . . . . . . . . . . . 573.2.2 Energy Storage System Model . . . . . . . . . . . . . . . . . . . . 593.2.3 Generator and Load Model . . . . . . . . . . . . . . . . . . . . . . 603.2.4 ac-dc Network Model . . . . . . . . . . . . . . . . . . . . . . . . . 633.3 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 643.3.1 SCUC Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 643.3.2 l1-norm Relaxation of the SCUC Problem . . . . . . . . . . . . . . 683.3.3 SDP Relaxation of the Regularized SCUC Problem . . . . . . . . . 693.3.4 SCUC Algorithm Design . . . . . . . . . . . . . . . . . . . . . . . 723.4 Performance Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75viiiTable of Contents3.4.1 Simulation Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . 753.4.2 Evaluating SDP Relaxation Gap . . . . . . . . . . . . . . . . . . . 783.4.3 Performance of Algorithm 3.1 . . . . . . . . . . . . . . . . . . . . . 793.4.4 Addressing the Net Power Supply Uncertainty . . . . . . . . . . . 843.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 874 An Online Learning Algorithm for Demand Response in Smart Grid 884.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 884.2 System Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 904.2.1 Appliances Model . . . . . . . . . . . . . . . . . . . . . . . . . . . 904.2.2 Pricing Scheme and Household’s Cost . . . . . . . . . . . . . . . . 974.3 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 994.3.1 Observation Profile Approximation Algorithm . . . . . . . . . . . . 1014.3.2 Markov Perfect Equilibrium (MPE) Policy . . . . . . . . . . . . . 1034.4 Online Learning Algorithm Design . . . . . . . . . . . . . . . . . . . . . . 1064.5 Performance Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1094.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1195 Demand Response for Data Centers in Deregulated Markets . . . . . 1205.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1205.2 System Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1225.2.1 Bilateral Contract Model . . . . . . . . . . . . . . . . . . . . . . . 1235.2.2 Contract Pricing Model . . . . . . . . . . . . . . . . . . . . . . . . 1245.2.3 Data Center’s Operation Model . . . . . . . . . . . . . . . . . . . . 1265.2.4 Risk-Aware Energy Demand Scheduling . . . . . . . . . . . . . . . 1315.2.5 Preference of the Data Center and Utility Company . . . . . . . . 1345.3 Problem Formulation and Algorithm Design . . . . . . . . . . . . . . . . . 136ixTable of Contents5.3.1 Data Center Many-to-One Matching Game . . . . . . . . . . . . . 1365.3.2 Distributed Algorithm Design . . . . . . . . . . . . . . . . . . . . . 1405.4 Performance Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1435.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1526 Conclusions and Future Work . . . . . . . . . . . . . . . . . . . . . . . . . 1536.1 Summary of Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1536.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159AppendicesA Appendices for Chapter 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175A.1 Proof of Inequality (2.28) . . . . . . . . . . . . . . . . . . . . . . . . . . . 175A.2 Proof of Theorem 2.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176A.3 Proof of Theorem 2.2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177A.4 Proof of Theorem 2.3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180A.5 Proof of Theorem 2.4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182A.6 Proof of Theorem 2.5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182B Appendices for Chapter 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191B.1 Transforming Problem (3.24) into an SDP . . . . . . . . . . . . . . . . . . 191B.2 Proof of Theorem 3.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200B.3 Proof of Theorem 3.2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201B.4 Proof of Theorem 3.3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204B.5 Proof of Theorem 3.4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205xTable of ContentsB.5.1 Proof of Theorem 3.5 . . . . . . . . . . . . . . . . . . . . . . . . . 207C Appendices for Chapter 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209C.1 Proof of Equation (4.1) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209C.2 Proof of Theorem 4.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210C.3 Proof of Theorem 4.2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211C.4 Bellman Error Approximation . . . . . . . . . . . . . . . . . . . . . . . . . 213D Appendices for Chapter 5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215D.1 Multiclass M/M/1 Queuing System Model . . . . . . . . . . . . . . . . . 215D.2 Proof of Theorem 5.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218D.3 Proof of Theorem 5.2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219D.4 Proof of Theorem 5.3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220xiList of Tables2.1 VSC station parameters with converter bus k and filter bus f . . . . . . . . 472.2 The generation cost and system losses obtained from SDP relaxation ap-proach and the approach proposed in [153] using Matpower with differentsolvers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 513.1 Generators’ specifications. . . . . . . . . . . . . . . . . . . . . . . . . . . . 773.2 VSC station parameters with ac bus k, dc bus s, and filter bus f . . . . . . 773.3 The generation cost, system losses, and CVaR obtained for the test systemswith and without renewable generators. . . . . . . . . . . . . . . . . . . . . 813.4 The optimal value and average CPU time for the deterministic multi-stagealgorithm and our proposed algorithm. . . . . . . . . . . . . . . . . . . . . 844.1 Operating specifications of controllable appliances. . . . . . . . . . . . . . . 110xiiList of Figures1.1 The schematic of a power system consisting of the transmission and distri-bution networks. System operators use OPF and SCUC analyses for systemoperation and planning in transmission networks. System operators im-plement demand response programs in distribution networks to motivateconsumers towards managing their energy consumption. . . . . . . . . . . . 22.1 A VSC station schematic in an ac-dc grid. . . . . . . . . . . . . . . . . . . 262.2 Q-P characteristic of the converter for |Vk| = 1 pu, V maxk = 1.05 pu, Imaxk =1 pu, |Vf | = 0.95 pu, RCk = 0.001 pu, XCk = 0.1643 pu, Snomk = 1 pu, andmb = 0.6. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302.3 The converter model in O and its approximated model in Oac. . . . . . . . 442.4 The IEEE 118-bus test system connected to five wind farms and one ac-dcmicrogrid. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 472.5 The value of the upper bound εM in (2.37) for different values of parametersbk, ck, Imaxk , k ∈ N convac , and the scaling coefficient ω. . . . . . . . . . . . . . 492.6 Voltage profile obtained from the SDP relaxation approach and the approachproposed in [153] using Matpower with MIPS solver. . . . . . . . . . . . 512.7 Q-P characteristics of the converters 1, 2, 3, 4, 11, and 12. . . . . . . . . . 532.8 Lagrange multipliers for ac-dc grid O and ac grid Oac. . . . . . . . . . . . . 53xiiiList of Figures3.1 A VSC station schematic with dc converter bus s ∈ N convdc , ac converter busk ∈ N convac , and filter bus f ∈ Nac. . . . . . . . . . . . . . . . . . . . . . . . 583.2 The procedure of solving the original SCUC problem (3.24). . . . . . . . . 733.3 An IEEE 30-bus test system connected to three wind farms in buses 14 and30, two PV panels in buses 3 and 7, and one dc microgrid in bus 28. . . . . 763.4 (a) Aggregate load demand, (b) average output power of the PV panels, (c)average output power of the wind farms over 24 hours. . . . . . . . . . . . 783.5 The output active power of the generators for the grid without renewableenergy generators and the grid with renewable energy generators. . . . . . 813.6 The output active power of the generators for the scenarios with nonlinearac power flow, linearized ac power flow, and dc power flow. . . . . . . . . . 823.7 The probability distribution function of Cres(·) and the value of CVaR forβ = 0.3, 0.6, and 0.9. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 863.8 The objective value for different values of parameter β with Algorithm 3.1and δt, t ∈ T with the multi-stage robust algorithm in [70]. . . . . . . . . . 864.1 The values of ra,i,t, qa,i,t, and δa,i,t for appliance a, which should be operatedfor three time slots with a maximum delay of three time slots. . . . . . . . 924.2 Price parameters over one day: (a) ltht ; (b) λ1,t and λ2,t. . . . . . . . . . . 1124.3 (a) Load demand for household 1 over two days; (b) aggregate load demandof users over one day; (c) aggregate load demands of all users over sevendays with and without load scheduling. . . . . . . . . . . . . . . . . . . . . 1134.4 Daily average cost for myopic and foresighted household 1. . . . . . . . . 114xivList of Figures4.5 (a) The EV’s charging schedule when household 1 is myopic (β = 0.05); (b)the electricity price when users are myopic; (c) the EV’s charging schedulewhen household 1 is foresighted (β = 0.995); (d) the electricity price whenusers are foresighted. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1154.6 Expected PAR with periodic and random price parameters. . . . . . . . . 1164.7 Objective value ||f obji (V pi,ki , piki )|| for households 1, 2, and 3. . . . . . . . . 1174.8 Daily average cost for household 1 with the algorithm based on Q-learningand our proposed LSL algorithm. . . . . . . . . . . . . . . . . . . . . . . . 1184.9 The aggregate load demand with the partially observable load schedulingand fully observable load scheduling. . . . . . . . . . . . . . . . . . . . . . 1195.1 (a) A system composed of five data centers equipped with EMS and threeutility companies; (b) the corresponding bipartite graph representation; (c)a feasible many-to-one matching among the data centers and utility compa-nies. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1245.2 (a) Wholesale market prices; (b) average PV generation over 24 hours. . . . 1445.3 (a) Arrival rates of the workloads of service class 2; (b) arrival rates of theworkloads of service class 5; (c) total number of servers; (d) total energydemand over 24 hours in data center 1 with and without load scheduling. . 1465.4 Total daily cost of data center 1 by considering the opportunities of utilitycompany choice or workload scheduling. . . . . . . . . . . . . . . . . . . . 1485.5 (a) The value of Γd,βd(αd, λd, er,d)in (5.16) for data center 1 with confidencelevel β1 in the range of 0.2 to 0.95; (b) total energy demand of data center1 with β1 = 0.95 and β1 = 0.8. . . . . . . . . . . . . . . . . . . . . . . . . 1485.6 (a) The PAR in the generated power; (b) revenue of the utility companiesin the benchmark scenario, and the scenarios with ωu = 0 and ωu = 106. . 150xvList of Figures5.7 (a) The value of potential function P (si) per iteration for 50 data centers;(b) the required number of iterations for convergence versus the number ofdata centers; (c) the required number of iterations for convergence versusthe number of utility companies. . . . . . . . . . . . . . . . . . . . . . . . 151D.1 The average waiting time Wc,d(k1, τ) versus τ for different values of k1 =Ic,d(pc,d, t). The workloads average arrival and execution rates are λc,d = 4workloads per second and µc,d = 5 workloads per second. . . . . . . . . . . 218xviList of Abbreviationsac Alternating currentADMM Alternating direction method of multipliersAGC Automatic generation controlBNE Bayesian Nash equilibriumCPP Critical peak pricingCPU Central processing unitCRAC Computer room air conditioningCVaR Conditional value-at-riskdc Direct currentECC Energy consumption controllerEIA Energy Information AdministrationEMS Energy management systemEV Electric vehicleGW GigawattHVAC Heating, ventilation, and air conditioningHVDC High-voltage direct currentIBR Inclining block rateIEEE Institute of Electrical and Electronics EngineersIPOPT Interior point optimizerIT Information technologyxviiList of AbbreviationsKKT Karush-Kuhn-TuckerkW KilowattkWh Kilowatt hourLSL Load scheduling algorithmMPE Markov perfect equilibriumMIP Mixed-integer programMIPS Matlab interior point method solverMW MegawattMWh Megawatt hourMVA Megavolt ampereMVAR Megawatt volt ampere reactiveOPF Optimal power flowPAR Peak-to-average ratioPDIP Primal-dual interior pointpu per unitPUE Power usage effectivenessPV PhotovoltaicPWM Pulse-width modulationRTP Real-time pricingSAA Sample average approximationSCUC Security-constrained unit commitmentSDP Semidefinite programSNOPT Sparse nonlinear optimizerTD Temporal differenceTOU Time-of-usexviiiList of AbbreviationsU.K. United KingdomU.S. United StatesVSC Voltage source converterxixAcknowledgmentsI would like to express my sincere gratitude to my supervisor, Prof. Vincent W.S. Wong, forhis patience, motivation, immense knowledge, and continuous support of my Ph.D. study.His guidance helped me in all the time of research and writing of this thesis. My sincerethanks also goes to Prof. Juri Jatskevich, Prof. Jianwei Huang, and Dr. Therrien, whoprovided me insightful comments and continuous encouragement throughout my Ph.D.program.Besides my supervisor, I would like to thank the rest of my thesis committee, Prof.Jose Marti and Prof. Y. Christine Chen, for their valuable guidance, which motivates meto widen my research from various perspectives.Many thanks go to all the colleagues and friends in room 4090 of Kaiser Building. AlsoI thank my friends at the Chinese University at Hong Kong for their support during myshort-term visit in 2016.Last but not least, I would like to thank my parents and my lovely wife for theircontinuous support.This work was supported by the Natural Sciences and Engineering Research Councilof Canada (NSERC) and UBC’s Four Year Doctoral Fellowship.xxDedicationTo my wife ElhamxxiChapter 1IntroductionElectrical power systems provide the necessary infrastructure to generate electricity anddeliver it to the consumers. Conventionally, the electricity is generated in large powerplants that use different energy sources such as fossil fuels (e.g., coal, oil, natural gas),converted fuels (e.g., methane), nuclear fuels, and geothermal steam [1]. The generatedpower is transformed through the transmission network at high voltage levels with eitherdirect current (dc) or alternating current (ac). Then, the distribution network enablestransmission of power at medium and low voltage levels to the consumers.In general, system operators consider the grid-wide generation cost of the operatinggenerators and the system power losses to assess the operation of the power system. Thelower the generation cost and system losses are, the more economic the power systemoperates [2]. Operating the conventional power plants is costly due to the high price oftheir energy sources and maintenance cost [3, pp. 13]. Furthermore, the conventionalpower plants are typically located far from the consumer side, which necessitates usinglong transmission lines to deliver power from the generators to the consumers. Deploymentof renewable energy generators such as wind turbines and photovoltaic (PV) panels is apromising solution to mitigate the cost of conventional generators and the power losses intransmission networks. The cost of generating electricity from renewable energy resourcescontinues to decline, e.g., in the United States (U.S.), the average levelized cost of energyof onshore wind turbines has decreased from 150 $/MWh in 2009 to 50 $/MWh in 2016,which is cost-competetive with conventional power plants [3, pp. 19]. Nevertheless, the1Chapter 1. IntroductionFigure 1.1: The schematic of a power system consisting of the transmission and distri-bution networks. System operators use OPF and SCUC analyses for system operationand planning in transmission networks. System operators implement demand responseprograms in distribution networks to motivate consumers towards managing their energyconsumption.volatile output power of the renewable generators makes it a challenge for system operatorsto provide an adequate balance between generation and demand [4].New energy management programs are necessary to enhance the use of the existingand new facilities in transmission systems. They involve exploring and developing toolsand techniques for operating the transmission systems in a cost-effective manner. As Fig.1.1 shows, in transmission systems, system operators can control the generation level ofthe power plants. System operators use optimal power flow (OPF) [5, 6] and security-constrained unit commitment (SCUC) [7, 8] analyses to operate the transmission systemeconomically efficient by reducing the grid-wide generation cost and power losses, whilesatisfying the operational constraints such as the generation-load balance, bus voltagelimits, and lines congestion limits. The OPF and SCUC programs can be executed on anannual basis for long-term system planning [9], on a monthly/weekly basis for short-termsystem management [10], on a daily basis for day-ahead electricity markets [11], and on anhourly basis for near real-time system management [12].2Chapter 1. IntroductionAlong with deploying the energy management programs such as OPF and SCUC intransmission systems, it is inevitable to adapt the structure of the existing distributionnetworks for the challenges that arise with the integration of the emerging technologiessuch as small-scale renewable generators, electric vehicles (EVs), and energy storage sys-tems. As Fig. 1.1 shows, in distribution networks, system operators focus on controllingthe energy demand of the consumers. One of the goals of the future smart grid is to providenecessary communication infrastructure to facilitate two-way transfer of information be-tween the energy suppliers and consumers. It enables the utility companies to implementmore innovative pricing mechanisms and create more attractive incentive programs to en-courage consumers towards modifying their energy consumption behaviour. Specifically,demand response programs are recognized as an effective way to motivate consumers to-wards reducing their energy demand during peak hours in response to the incentives fromthe utility companies or the changes in the energy price [13]. Hence, demand responseprograms can reduce the need to build expensive peaking plants for peak time periods [14].In this thesis, we propose algorithms for the energy management programs in transmis-sion and distribution systems. In particular, we focus on designing algorithms to determinethe near-optimal solution to the OPF and SCUC problems in ac-dc transmission systems.We also focus on the demand response programs for residential electricity consumers anda special type of industrial electricity consumers − data centers in distribution networks.We propose demand response algorithms to schedule the electric appliances in residentialhouseholds and the workloads in data centers in response to dynamic pricing information.The rest of this chapter is organized as follows. In Section 1.1, we provide an overviewof the OPF and SCUC analyses. In Section 1.2, we describe the role of demand responseprograms in distribution systems. The contributions made in this thesis are summarizedin Section 1.3, and the thesis organization is provided in Section 1.4.3Chapter 1. Introduction1.1 Energy Management in Transmission SystemsA power grid is a time-varying interconnected system, in which the generation is dispatchedconstantly in response to the changes in the load demand [5]. For the daily operationand generation dispatch, system operators schedule the power plants for each hour of thenext day based on the forecasted load demand. The operating power plants for the day-ahead dispatch are selected according to the operation cost of the power plants includingthe generation, start-up, and shut down costs, as well as the operation limits includingthe ramp rate (i.e., how quickly the generator’s output can be changed), maximum andminimum limits of generation level, minimum run time (i.e., the minimum amount of timethat the generator should be operated once it is turned on), and minimum down time(i.e., the minimum amount of time that the generator must stay off once it is turned off)[8, 15, 16]. For real-time dispatch, the actual load demand variation is monitored and theautomatic generation control (AGC) is used to change the generation dispatch to ensure thedemand-supply balance. Meanwhile, the flows on the transmission system are monitoredto keep the line flows and bus voltage levels within the reliable range [12].In both day-ahead and real-time generation dispatches in transmission systems, onetypical objective is to determine a secured and economic schedule for the generators. A se-cured generation dispatch takes into account different parameters and constraints includingthe power plants availability and operation constraints, the physical characteristics of theunderlying power grid, and the operation constraints imposed by other components of thesystem (e.g., ac-dc converters, transmission lines, phase shifters, transformers) [2, 11]. Onthe other hand, the economic operation of the grid is largely a matter of reducing the grid-wide operation cost and the system losses. In general, the generators are dispatched in theorder of their operation costs from the lowest to the highest to meet the load demand [16].Moreover, the active power losses can result in a significant cost. For example, according4Chapter 1. Introductionto data from the Energy Information Administration (EIA) in the U.S. [17], the powerlosses amounted to 341 million megawatt hour (MWh), or 8.38% of the net generation.Multiplying the amount of power losses by the national average retail price of electricityfor 2013, we can estimate those losses came at a cost of $34.4 billion.The OPF and SCUC are among the most popular analytic programs for system oper-ators to determine a secured and economic generation dispatch in transmission systems.The OPF program is mainly executed for the hourly-basis system analysis. The SCUC pro-gram is executed for the day-ahead or week-ahead planning. In the following subsections,we describe the OPF and SCUC analyses, discuss the existing challenges, and summarizethe related works.1.1.1 Optimal Power Flow (OPF)The OPF analysis plays an important role in power system operation. In the OPF problem,the output power of the generators is determined by optimizing an objective function suchas the generation cost and the system losses. The OPF problem in ac grids is subject tothe operating constraints such as the power balance constraints, voltage limit of the buses,and power flow limit of the transmission lines [6]. The OPF in ac grids takes the form of anonconvex optimization problem, and is generally difficult to be solved. The nonconvexityof the problem arises from the nonlinear power flow equations, apparent power flow limitof the transmission lines, and quadratic dependency on the set of bus voltages.There is a rich body of work on various techniques to solve the OPF problem suchas linear approximations [18], mixed-integer linear programming [19], gradient method[20], Newton-Raphson method [21], branch-and-cut method [22], quadratic programming[23], alternating direction method of multipliers (ADMM) [24, 25, 26], Lagrange relaxationmethod [27, 28, 29], and metaheuristics [30]. These methods offer no optimality guarantee5Chapter 1. Introductionand may return an infeasible solution to the OPF problem. Jabr [31] was one of the firstto introduce a second-order cone relaxation of the power flow equations. Bai et al. [32]presented a semidefinite program (SDP) relaxation for the OPF problem, which was usedby Lavaei and Low [33] to show that the SDP relaxation for ac grids has zero relaxationgap for practical power grids including the IEEE test power systems. Thereafter, convexrelaxation of the OPF problem in ac grids have attracted significant research attention [34,35, 36, 37, 38, 39, 40, 41]. These techniques are guaranteed to determine the global optimalsolution when the relaxation gap is zero. The work in [34] presented the bus injection andbranch flow models of the OPF problem and proves their equivalence. In [35], the convexrelaxations of the OPF problem based on SDP, chordal extension, and second-order coneprogramming are studied. The sufficient conditions that guarantee the exactness of theserelaxations are provided. In [36], a nested optimization approach is proposed to decomposethe multi-period joint OPF and EV charging problem into separable subproblems. Thedecomposed problem is solved using a nonsmooth separable programming technique. Thework in [37] presented the geometric properties of the set of all vectors of bus powerinjections satisfying the operating constraints of a radial power network. The convexrelaxation of the OPF problem is obtained and the sufficient condition to determine aglobal optimal solution of the OPF problem is derived. The work in [38] presented abranch flow model for the analysis of meshed and radial networks. The proposed convexrelaxation method is exact for radial networks provided there are no upper bounds on loadsor voltage magnitudes. The work in [39] proposed the second-order cone relaxation of theOPF problem for resistive networks. The uniqueness of the optimal solution is characterizedfor radial and meshed networks. The work in [40] presented a model for the power linescapacity and studied the zero relaxation gap for weakly cyclic networks. The upper boundon the rank of the minimum-rank solution of the SDP relaxation is provided. In [41],6Chapter 1. Introductionthe ac OPF problem is formulated as a nonconvex quadratically-constrained quadraticprogram with complex variables and is solved using the SDP relaxation for cases where therelaxation gap is zero. For the case of nonzero relaxation gap, the global optimum solutionis determined using a spatial branch-and-bound method.The transition from ac grids to ac-dc grids is accelerating due to the advance in powerelectronics and the increase of the applications using dc power. Many renewable energyresources, such as PV panels, generate dc power. High voltage direct current (HVDC)transmission lines facilitate transporting power to or from remote areas. Many electricalloads, such as motor loads, pumps, and lighting consume dc power. The dc microgrid canbe a viable alternative as an energy source for data centers. Batteries, supercapacitors, andfuel cells store energy as dc [42, 43, 44, 45, 46]. Another motivation for moving towardsac-dc grids is the increase of energy efficiency by eliminating losses associated with dc-ac-dc conversions. Furthermore, ac-dc grids can be more reliable than ac grids since theadditional conversion steps in ac grids may introduce potential faults [47, 48].In ac-dc grids, the OPF problem includes the constraints imposed by the limits on thevoltage and current ratings of the converters in addition to the constraints imposed by theac and dc grids. The converter losses can add up to a significant fraction of the systemlosses. Thus, the converter losses are usually included in the ac-dc OPF problem [49]. Theconverter losses can be approximated by a quadratic function of its current magnitude[50]. The ac-dc OPF is a nonconvex problem due to the quadratic form of the converterlosses and nonlinear power flow equations. Several methods have been proposed to addressthe ac-dc OPF such as heuristic and interior point methods [51, 52], the Newton-Raphsonmethod [53], second-order cone programming [54], and sequential approaches [55].It is a challenge to determine the global optimal solution among the multiple localoptimal solutions of the ac-dc OPF problem. Most of the works on the ac-dc OPF problem7Chapter 1. Introduction(e.g., [51, 52, 53, 54, 55]) do not guarantee the global optimality of the solution. Theefficiency of SDP in solving the ac OPF problem (e.g., in [33, 34, 35, 36, 37, 38, 39, 40, 41])has motivated us to use this approach to solve the ac-dc OPF problem. Nevertheless, theproposed convex relaxation techniques for the ac OPF problem cannot be applied to theac-dc OPF directly due to the inclusion of the ac-dc converter losses and the operationconstraints imposed by the converters. It is a challenge to obtain the SDP relaxation ofthe OPF problem in ac-dc grid, and addressing this challenge is one of the objectives ofthis thesis.1.1.2 Security-Constrained Unit Commitment (SCUC)The advancement in power electronic technologies in ac-dc converters has led to the revivalof dc power for high voltage transmission, particularly for connecting off-shore wind farmsto the power grid [56]. In an ac-dc grid with renewable energy sources, the SCUC plays animportant role to determine the set of operating generators in a day-ahead basis with theminimum cost and system losses. The SCUC is multi-time version of the OPF problemthat includes the commitment constraints of the generators. Specifically, the objective ofthe SCUC problem includes the generators’ operation cost and the system losses [57]. TheSCUC problem in an ac-dc grid is typically subject to power balance equations, power flowlimits of the transmission lines, bus voltage limits, voltage and current limits of the ac-dcconverters, as well as the generators’ constraints such as the capacity limits, on/off states,minimum on/off time requirements, and ramping up/down rates limits [8].There are several challenges in solving the ac-dc SCUC problem. First, the systemoperator uses the forecasted load demand and the renewable generation to execute theSCUC program. However, the actual system condition may deviate from the presumedcondition due to the uncertainties in the load demand and renewable generator. Thus, the8Chapter 1. Introductionsystem operator requires to take into account some corrective actions such as committingreserve generators. Second, the SCUC is a nonlinear mixed-integer optimization problemwith nonconvex constraints imposed by the power flow equations and the ac-dc converters,as well as a large number of binary variables associated with the on/off states of thegenerators. Therefore, the SCUC is in the class of NP-hard problems [58] and is difficultto be solved optimally.There have been some efforts to address the above-mentioned challenges in the SCUCliterature. We divide the related works into two main threads. The first thread is concernedwith the solution approaches for the deterministic SCUC problem (i.e., without uncertaintyabout the load demand and generation). One approach is to divide the SCUC problem intoa master problem that solves the unit commitment and some subproblems that evaluatethe feasibility of the network constraints. The nonlinear mixed-integer program (MIP) andLagrange relaxation approach are generally used to formulate the master problem [8, 59].Different techniques such as the Benders cut method [60], Newton-Raphson method [61],and branch-and-bound method [62] are used to solve the subproblems. The multi-stagealgorithms are not guarantee to converge to a good local optimal solution, since the unitcommitment decisions from the first stage are fixed in the subsequent stages. Furthermore,the MIP and the Lagrange relaxation approach with large number of multipliers for theconstraints generally suffer from the curse of dimensionality in large networks. Convexrelaxation techniques such as quadratically-constrained quadratic program [63] and SDPrelaxation [64] are the alternative approaches to solve the SCUC problem. However, thezero relaxation gap condition and the optimality of the solution are among the remainingchallenges in these techniques.The second thread is concerned with the uncertainties in the load demand and re-newable generators. The scenario-based stochastic optimization is used to address the9Chapter 1. Introductionuncertainty issues using the approximate probability distribution [65, 66] and the availablehistorical data record [67] for the uncertain variables. The drawback of stochastic optimiza-tion approach is the requirement to represent uncertainty in a detailed fashion, by usinga large number of appropriately weighted scenarios. Generating these scenarios requiresdetailed information about the probability distribution function of the uncertain variables.Moreover, the resulting SCUC problem is generally large with many decision variables anddifficult to be solved in timely fashion. Robust optimization methods [68, 69, 70] are usedto address the unavailibility of probability distribution functions of the uncertain variables.The robust SCUC is a bilevel optimization problem, which can be solved using the Bendersdecomposition approach. In these studies, the objective is to minimize the system oper-ation cost against the worst-case realization of the load and generation uncertainty. Theworst-case nature of these approaches reflects the tendency to operate the system in a con-servative fashion. Robust SCUC models have the advantage that they require only limitedinformation about the stochastic process of the uncertain variables. However, consideringthe worst-case scenario leads to ignoring the severity of other possible scenarios.In this thesis, we focus on the SCUC problem in ac-dc grids with load and generationuncertainty. To address the conservative solution in the robust SCUC models (e.g., [68,69, 70]), we use the risk measure, conditional value-at-risk (CVaR), to limit the likelihoodof excessive net power supply within a confidence level. To address the computationalcomplexity and the optimality of the solution in the existing works (e.g., [8, 59, 60, 61, 62,63, 64]), we apply l1-norm approximation and convex relaxation techniques to develop anefficient SCUC algorithm to determine a near-optimal solution.10Chapter 1. Introduction1.2 Energy Management in Distribution SystemsWith the fast growing electricity demand [71], it is necessary to optimize the operation ofpower grids to deliver more power to end-users, while reducing the need for new powersources and infrastructure. Decreasing the power losses and generation cost in transmissionnetworks are key aspects of energy management in power grids. However, the energymanagement can go beyond the transmission systems by taking into account the efficientelectricity consumption at the user side in distribution networks. This entails exploringopportunities to change the consumption behaviour of the electricity users [72, 73].The two-way communication network in the emerging smart grid can support real-timemetering and pricing, intelligent load scheduling, cost savings from peak load reduction,as well as the integration of EVs and distributed generation sources (e.g., wind turbine,PV panel) [74, 75, 76, 77]. Specifically, it allows utility companies to provide users withincentives for demand reduction efforts in times when the grid is under high load stress. Oneviable incentive is a price-based service through a demand response program [78, 79, 80, 81],which can be implemented by utility companies to motivate users towards distributingtheir energy consumption across different times of the day and interrupting energy use fortheir electric appliances temporarily. With a properly designed demand response program,the utility companies can decrease their generation cost due to the reduction of peak-to-average ratio (PAR) in the aggregate load. Meanwhile, users can reduce their bill paymentby taking advantage of low prices at off-peak hours [82].Different types of demand response programs have been implemented in Canada. Theprovince of Ontario has held an annual demand response auction from 2015 for summerand winter commitment periods [83]. For example, Hydro One in Ontario has introduceddemand response program with dynamic pricing scheme with tariffs vary from 8.3 cents to17.5 cents for off-peak hours to peak hours, respectively [84]. The BC Hydro Cooperation11Chapter 1. Introductionin the province of British Columbia has introduced winter load curtailment program forits electricity users [85]. Hydro Que´bec has introduced interruptible load programs forlarge and medium power customers [86]. SaskPower in the province of Saskatchewan hasintroduced demand response program for large industrial customers (5 megavolt ampere(MVA) or larger) [87]. In the province of Alberta, users can voluntarily reduce energy usein response to spot electricity prices [88].The level of success of different methods in a demand response program depends onvarious factors including the amount of information that is provided to the users as wellas the ability of the users to respond to the price signals. In price-based demand responseprograms, the information provided to the users are based on different pricing schemes suchas real-time pricing (RTP), time-of-use (TOU), critical peak pricing (CPP), and incliningblock rate (IBR) [89]. In RTP, the price values vary across different times of the day toreflect the hourly changes in the wholesale market price. In TOU scheme, different butpredetermined prices are used over a prespecified period (e.g., summer weekdays between12 pm and 6 pm). In CPP scheme, a single predetermined price or sometimes variableprice schedules are applied during specific system operating or market conditions (e.g.,operating reserve shortages, the time periods with high wholesale prices). For the IBRscheme, the marginal price increases with the total consumed power. The RTP and IBRare among the most effective schemes for demand response, since the price value dependson the real-time total load demand of the system, which can prevent load synchronization(i.e., shift a large amount of load to the off-peak period) and motivate users to distributetheir load across different times of the day [89, 90].The ability of the participating users to respond to the price signals mainly dependson the users’ type. In residential sectors, there are five major electric load categories:space heating, space cooling, water heating, lighting, and appliances. A residential user12Chapter 1. Introductionin a demand response program responds to the price information in different ways. Theuser may reduce usage at times of high prices without making it up later, for example, byturning off lights or air conditioner. The user may reschedule usage away from times of highprices, for example, by deferring running a dishwasher until later in the day. According to[71, 91], in the U.S., the electricity consumption of a typical household was about 11,700kilowatt hour (kWh) in 2015, in France it is 6,400 kWh, in the United Kingdom (U.K.)it was 4,600 kWh and in China it was around 1,300 kWh. The global average electricityconsumption for households was roughly 3,500 kWh in 2015. The global average energyconsumption of residential sectors was roughly 30% of the total electricity consumption.It means that residential sectors have a high potential in peak power reduction through aproperly designed demand response program.Commercial and industrial sectors also offer unique demand response opportunities asthey are able to contribute a large amount of load reduction. Commercial sectors canparticipate in demand response programs through controlling the heating, ventilation andair conditioning (HVAC) system and lighting. Industrial loads can turn off productionlines and processes completely during peak demand periods [91]. Among different indus-trial/commercial users, data centers represent a particularly promising electricity consumerfor the adoption of the demand response programs. Data centers’ aggregate energy con-sumption is large and is increasing rapidly. For example, in 2014, data centers in the U.S.consumed an estimated 70 billion kWh, representing about 1.8% of total U.S. electricityconsumption. Based on current trend estimates, U.S. data centers are projected to consumeapproximately 73 billion kWh in 2020 [92]. Data centers are highly automated and moni-tored, and thus there is a potential for a high-degree of responsiveness. The power load andstate of information technology (IT) equipment (e.g., server, storage, networking devices)and cooling facility (e.g., computer room air conditioning (CRAC) units) can be moni-13Chapter 1. Introductiontored and adjusted continuously. Many typical workloads (e.g., high complexity scientificcomputations, data analytics) in data centers are delay-tolerant, hence may be rescheduledto off-peak hours. Moreover, data centers can benefit from renewable energy adoption toreduce the need from the grid by supplying the demand at critical times [93, 94].There are challenges in implementing demand response programs. One challenge is toattract the interest of users to change their usage behavior through providing incentivesand reducing their discomfort experienced from changing their time and the amount ofenergy consumption. In particular, the utility companies need to identify their users’ goalsfrom participating in the demand response program to incentivize them properly. Thesecond challenge is to protect the users’ privacy. This is illustrated by the significantprivacy concerns raised in response to the introduction of smart meters [95, 96]. The thirdchallenge is the inherent uncertainties in the users’ load demand (e.g., time of operation,power consumption) and energy price changes. The uncertainty issue is becoming morechallenging by integrating intermittent renewable generators into the distribution networks.In the following subsections, we describe the demand response programs for residentialsectors and data centers, discuss the viable solutions to address the existing challenges,and summarize the related works.1.2.1 Demand Response Program for Residential UsersA demand response program with time-varying pricing scheme is an effective approach toencourage residential users toward rescheduling their electric appliances from the times thatthe price is high (e.g., during the peak load period) [13, 97, 98]. Nevertheless, it is difficultfor users to manually follow the updated information and respond to the real-time pricefluctuations. Instead, users can deploy technologies that enable a fully automated loadscheduling. Specifically, an automated energy consumption controller (ECC) can make14Chapter 1. Introductionprice-responsive scheduling decisions on behalf of the users [90, 99], which necessitates theincrease of decentralized data processing and real-time information that can better reflectthe generation and demand condition of the grid.The implementation of demand response techniques and algorithms for a large numberof users is nontrivial. There are challenges for the ECC to optimally determine the energyschedule of its corresponding household in a demand response program [90, 100, 101].First, if the utility company uses RTP or IBR, the users’ scheduling decisions are coupled,since the appliances’ energy schedule of a user affects the price that is charged to all users,hence affects other users’ cost. Second, each user is uncertain about the total demand ofother users, as well as the time of use and operation constraints of his/her own appliances.In particular, the operation of an appliance depends on its task specifications (e.g., taskduration, start time/deadline of the task), which are not known a priori until the userdecides to turn on that appliance. Third, the users may not know the price informationahead of time. Fourth, the users may experience discomfort for changing their lifestyleby scheduling the time of use of the appliances. It is a difficult task to price the users’discomfort caused by interruptions and shifting the operation of the appliances.The demand response program for residential households has been studied extensivelyin the literature. We divide the related works into two main threads. The first thread isconcerned with techniques for scheduling the energy usage of the appliances in a householdwith a myopic user, who aims to minimize the cost in a short period of time (e.g., one day).Samadi et al. in [90] proposed pricing algorithms based on stochastic approximation to min-imize the PAR of the aggregate load in one day for a single household. Chen et al. in [102]proposed a robust optimization approach to minimize the worst-case daily bill payment ofa myopic user in a market with the RTP scheme. Eksin et al. in [103] captured the inter-actions among myopic users with heterogeneous but correlated consumption preferences as15Chapter 1. Introductiona Bayesian game. Forouzandehmehr et al. in [104] proposed a differential stochastic gameframework to capture the interactions among myopic users with controllable appliances.In these works, however, it was not mentioned how the proposed algorithms can be usedfor foresighted users, who aim to minimize their long-term costs.The second thread is concerned with techniques for scheduling the appliances in ahousehold with a foresighted user. Wen et al. in [105] proposed a reinforcement learningalgorithm to address the appliances scheduling problem in a household. Kim et al. in [106]proposed a load scheduling algorithm based on Q-learning for a microgrid with TOU pricingscheme. Liang et al. in [107] proposed a Q-learning approach to minimize the bill paymentand discomfort cost of a foresighted user in a household. Ruelens et al. in [108] proposed abatch reinforcement learning algorithm to schedule controllable loads such as water heaterand heat-pump thermostat. These works, however, did not mention how the proposedlearning algorithms can capture the decision making of multiple foresighted users. Xiaoet al. in [109] applied dynamic programming to model the interactions among multipleforesighted suppliers. Yao et al. in [110] studied the electricity sharing problem amongmultiple foresighted users with the RTP scheme. The scheduling problem of each individualuser is formulated as a Markov decision process. A specific structure for the suboptimalpolicy of each user is determined. Jia et al. in [111] proposed a learning algorithm basedon stochastic approximation for the utility company to determine the day ahead pricevalues in a market with multiple foresighted users. These works, however, did not studythe operation constraints of different electrical appliances in residential sectors.To address the remaining challenges, in this thesis, we study the long-term interactionsamong foresighted residential users and take into account the uncertainty about the priceinformation and load demand. We model the coupled decision making of the users ina market with the RTP scheme as a stochastic game. The Markov perfect equilibrium16Chapter 1. Introduction(MPE) is a standard solution concept for analyzing stochastic games. Several algorithmshave been proposed to determine an MPE in stochastic games [112, 113, 114, 115, 116, 117,118, 119]. Some algorithms are model-based and require knowledge of the dynamics of thesystem, i.e., the state transition probabilities. The model-based learning algorithms includerational learning methods [112, 113, 114], linear programming based algorithms [115, 116],and homotopy method [117]. Some other learning algorithms are model-free and aimto determine an MPE when the system dynamics are unknown. Examples of model-free approaches include Lyapunov optimization method [118] and reinforcement learningalgorithms [119]. In this thesis, we develop a model-free load scheduling algorithm for theECC of the households to schedule the appliances in response to the RTP information.Our algorithm is based on the actor-critic method [120, 121, 122, 123], which is a type ofreinforcement learning algorithm. In the design of the proposed algorithm, we consider theconstraints imposed by the users such as the discomfort level as well as the desirable starttime of use, deadline, and operation duration of the appliances.1.2.2 Demand Response Program for Data CentersWith the proliferation of data centers, it is important to design a proper demand responseprograms to motivate them towards workload scheduling [124]. There exist some large datacenters from Apple and Google that have made significant strides in relying exclusively onlocal renewable energy plants (e.g., PV panel, wind turbine) [125]. However, smaller datacenters mostly rely on the power market due to the space and cost constraints on buildingtheir own generation facilities [92]. This motivates a recent rich body of literature ondemand response algorithm design for data centers (e.g., [94, 126, 127, 128, 129, 130, 131,132]). The works in [94, 126, 127, 128, 129, 130, 133] tackled the workload managementproblem for data centers using approaches such as stochastic optimization [94], convex17Chapter 1. Introductionoptimization [126], mixed-integer linear programming [127], auction [128], ADMM [129],and dual decomposition [130]. In these works, the utility company announces a knownand fixed energy price rates, and thus the cost minimizing problem is addressed from thedata centers’ point of view. The work in [131] applied a two-stage optimization method toaddress the problem of optimizing the energy price rates for a utility company, in additionto the energy demand profile for data centers. For a power grid with multiple utilitycompanies, the interaction among utility companies and data centers is addressed by aStackelberg game approach in [132].Despite the high potential of data centers for demand response activities, there stillexist challenges in implementing demand response programs for data centers in practice.The fundamental challenge is that the today’s demand response programs are not properlydesigned for the load profile and risk tolerance of data centers. Currently, CPP schemeis the most commonly used pricing scheme in demand response programs for data centers[94]. The CPP-based demand response program may not provide sufficent incentives fordata centers, since such a program requires curtailment of usage a few times per month.Hence, it cannot use the high potential of load flexibility in data centers. Furthermore, thestochastic nature of the workloads causes the data center to be uncertain about the arrivaltime, requested service class, deadline, and priority of the workloads during the criticalpeak events [93]. Moreover, data centers often have local resources such as energy storagesystem, renewable generator, and backup generator [134, 135]. These technologies adduncertainty and complexity to the workload scheduling decision of the corresponding datacenter in response to the critical peak events. Consequently, designing a proper demandresponse program for data centers is a challenge and a fully automated energy managementsystem (EMS) is required to perform the workload scheduling for a data center.At least two factors should be taken into account to address the above-mentioned chal-18Chapter 1. Introductionlenges. The first factor is the cost of risk. The uncertainty in the workloads and locally-operated renewable generation introduces the risk of excessive energy demand, which re-quires the utility companies to perform some corrective actions such as committing ex-pensive reserve generators or purchasing electricity from the wholesale market with a highprice. Hence, a data center may be charged by the utility companies with a significantlyhigh energy price in case of unpredictable excessive energy demand. The key to minimizingthe cost of risk is to employ an automated and safe operational procedure to manage thestochastic workload migration.The second factor is the regulatory framework for the demand response contracts be-tween the data centers and utility companies. A data center should adopt its schedulingdecision approaches to the underlying regulatory framwork. For example, the schedulingdecision for data centers is more complicated in deregulated markets, since the data cen-ters can choose the power suppliers they want out of several utility companies [136], whilescheduling their workload over the day. The states or provinces which have implementedderegulated markets include the states of Texas and Pennsylvania in the U.S. [137] andthe province of Alberta in Canada [138].In this thesis, we study the emerging deregulated markets, where multiple utility com-panies compete to supply electricity to the same group of geographically dispersed datacenters. Each data center is free to enter a bilateral contract with a utility company andcan schedule its workloads to minimize its bill payment. We use the concept of many-to-one matching game with externalities to capture the interactions among data centers andutility companies in a deregulated electricity market. We develop an algorithm that can beexecuted by the data centers and utility companies in a distributed fashion to determinea stable matching among data centers and utility companies, where no data center has anincentive to change its utility company and workload scheduling unilaterally. Most of the19Chapter 1. Introductionexisting works (e.g., [94, 126, 127, 128, 129, 130, 131, 132, 133]) did not take into accountthe uncertainty in the energy demand of data centers. Whereas, we use the risk measureCVaR to specify a limit for the deviation in the energy demand of the data centers fromthe specified amount in their bilateral contract with the utility companies.1.3 Summary of Results and ContributionsThis thesis proposes algorithms for the OPF and SCUC programs in transmission systems,as well as the demand response program in distribution networks. The results are dividedinto four main chapters. The results and contributions in each chapter are as follows.1. In Chapter 2, we study the OPF problem in an ac grid connected to some dc mi-crogrids via voltage source converters (VSCs) and dc lines. We model the losses andthe limits on the voltage and current ratings of the VSCs in the ac-dc OPF problem.We transform the problem into an SDP using convex relaxation techniques. We de-termine the zero relaxation gap condition and show that the condition can hold inpractical ac-dc grids including the IEEE test systems connected to some dc grids.We develop an algorithm to determine the solution of the original ac-dc OPF prob-lem. We show that the zero relaxation gap condition holds for an IEEE 118-bus testsystem connected to some dc grids. That is, the proposed ac-dc OPF algorithm canprovide the global optimal solution in our case study. Part of the work of Chapter 2has been presented in [139] and the full paper has been published [140].2. In Chapter 3, we use convex relaxation technique incorporated with a risk minimiza-tion approach to solve the ac-dc SCUC problem with load and generation uncertainty.To address the uncertainty in the load demand and renewable generator, we introducea penalty based on CVaR to the objective function. It enables us to limit the risk20Chapter 1. Introductionof shortage in the net power supply. We relax the binary variables and introduce aweighted l1-norm regularization term to the objective function to enforce the relaxedvariables to become either 0 or 1. We use convex relaxation techniques to transformthe problem into an SDP. It enables us to develop an iterative reweighted l1-normapproximation algorithm that solves a sequence of SDPs. Simulations are performedon an IEEE 30-bus test system connected to some dc grids. We show that for 1000different initial conditions, the proposed algorithm returns a near-global optimal so-lution with 2% gap in all scenarios, and returns the near-global optimal solution with1% gap in 98% of the scenarios. The work of Chapter 3 is under review as a fullpaper in IEEE Trans. on Power Systems.3. In Chapter 4, we study the long-term interactions among foresighted residential users.We model the users’ decision making with uncertainty about the price informationand load demand of their appliances as a Markov decision process with differentstates for different possible scenarios. We capture the interactions among users asa stochastic game [141]. The underlying game is partially observable [142, 143,144], since each user only observes his own state and is uncertain about other users’states. We approximate the users’ optimal policy by the MPE of a fully observablestochastic game with incomplete information. We develop an actor-critic method[120, 121, 122, 123]-based distributed learning algorithm that converges to the MPE.Simulation results show that the proposed algorithm can reduce the PAR of theaggregate load and the expected cost of the users by 13% and 28%, respectively,compared with the benchmark of not performing demand response. Furthermore,the proposed algorithm converges to the MPE policy in a reasonable number ofiterations. We have presented part of the work of Chapter 4 in [145]. The full paperhas been accepted for publication in IEEE Trans. on Smart Grid [146].21Chapter 1. Introduction4. In Chapter 5, we model the coupled decisions of utility company choices and workloadscheduling of data centers in deregulated markets as a many-to-one matching game.We apply the concept of CVaR to limit the risk of excessive energy demand of thedata centers. We show that the game admits an exact potential function, its localminima correspond to the stable outcomes of the game. We develop a distributedalgorithm that can guarantee to converge to a stable outcome. Simulations resultsfor a deregulated electricity market with 50 data centers and 10 utility companiesshow that the proposed algorithm can reduce the cost of data centers by 18.7%and the PAR in the generation of the utility companies by 8% compared with thebenchmark of not performing demand response. The work of this chapter is going tobe submitted as a full paper to IEEE Trans. on Smart Grid.1.4 Thesis OrganizationThe rest of the thesis is organized as follows. In Chapter 2, the ac-dc OPF problem isformulated and is transformed to an SDP. The zero relaxation gap condition is studied.An OPF algorithm is developed to determine the global optimal solution to the originalOPF problem. In Chapter 3, we formulate the SCUC problem in ac-dc grids and transformit into an SDP. We propose an iterative algorithm to solve the problem. In Chapter 4, wemodel the interactions among residential users as a partially observable stochastic gameand approximate it by a fully observable stochastic game with incomplete information.We develop a distributed learning algorithm to compute the MPE policy of the users. InChapter 5, we propose a matching game model for the interactions among data centers inderegulated markets. We develop a distributed algorithm to obtain a stable outcome ofthe game. Chapter 6 concludes this thesis. Furthermore, some potential future works areprovided. Chapters 2−5 in this thesis are self-contained and included in separate journal22Chapter 1. Introductionor conference papers. The notations are defined separately for Chapters 2−5.23Chapter 2Semidefinite Relaxation ofOptimal Power Flow for ac-dc Grids2.1 IntroductionIn this chapter, our goal is to determine the global optimal solution of the ac-dc OPFproblem. In ac-dc systems, the ac and dc buses are connected through VSC stations,which are widely used in practice [147]. We use the SDP relaxation technique to solve theac-dc OPF problem. The challenges include obtaining the SDP form of the power lossesand the constraints on the reactive and apparent power flows in the VSCs. Deriving asufficient condition for zero relaxation gap is another challenge that we address in thischapter. The main contributions of this chapter are as follows:• Problem Formulation: We introduce a general ac-dc OPF problem formulation, whichcan be used in different scenarios including (a) an ac grid connected to dc microgridsvia VSCs, and (b) an ac grid embedded with dc cables such as HVDC lines. Wemodel the losses and the limits on the voltage and current ratings of the VSCs in theac-dc OPF problem.• Novel Solution Approach: We transform the problem into SDP, solve the problem,and determine the zero relaxation gap condition. We show that the zero relaxationgap condition can hold in practical ac-dc grids including the IEEE test systems24Chapter 2. Semidefinite Relaxation of Optimal Power Flow for ac-dc Gridsconnected to some dc grids. We describe how the solution of the original ac-dc OPFproblem can be determined from the solution of the SDP form of the ac-dc OPF.• Performance Evaluation: Simulations are performed on an IEEE 118-bus test systemconnected to five off-shore wind farms using HVDC lines and one ac-dc microgrid.We discuss the theoretical findings for the underlying test system. We show that theSDP form of the OPF problem has zero relaxation gap and it can provide the globaloptimal solution.The rest of the chapter is organized as follows. The system model is introduced inSection 2.2. In Section 2.3, we present the problem formulation and the proposed ac-dcOPF algorithm. Numerical results are presented in Section 2.4, and a summary is givenin Section 2.5. The proofs can be found in Appendices A.1−A.6.2.2 System ModelConsider an ac-dc grid consisting of an ac grid connected to multiple dc grids. We representan ac-dc grid by a tuple O(N ,L), where N denotes the set of buses and L denotes theset of transmission lines. The dc grids are connected to the ac grid using VSCs at somebuses. An ac-dc grid with a VSC station is shown in Fig. 2.1. The VSC station consists ofa transformer, ac filter, phase reactor, and converter. Without loss of generality, the VSCstation is assumed to be a two- or three-level converter using the pulse-width modulation(PWM) switching method. Let Nac ⊆ N denote the set of ac buses that are not connectedto the converters. Let Ndc ⊆ N denote the set of dc buses that are not connected to theconverters. Let G ⊆ N denote the set of generator buses. Let N convac ⊆ N denote the setof ac side converter buses. Let N convdc ⊆ N denote the set of dc side converter buses. LetN conv = N convac ∪ N convdc denote the set of all converter buses. Let XCk denote the phase25Chapter 2. Semidefinite Relaxation of Optimal Power Flow for ac-dc GridsFigure 2.1: A VSC station schematic in an ac-dc grid.reactor of the converter connected to ac bus k ∈ N convac . Let RCk denote the resistancemodeling the losses of the non-ideal phase reactor of the converter connected to ac busk ∈ N convac . Let Bf denote the shunt susceptance for the ac filter connected to the filterbus f ∈ Nac. Let XTf and RTf denote the reactance and resistance of the transformerconnecting the ac grid to the filter bus f ∈ Nac, respectively. In Fig. 2.1, buses k ands are in sets N convac and N convdc , respectively. The buses in region 1 are in set Nac, thebuses in region 2 are in set N conv, and the buses in region 3 are in set Ndc. The followingassumptions are made in modeling the VSC station and ac-dc grid.Assumption 2.1 The VSCs can control the active power (or the dc voltage) and the reac-tive power (or the ac voltage) magnitude and phase angle of the ac terminal voltages [148].Assumption 2.2 In a VSC, the resistance RCk is much smaller than the reactance XCkfor k ∈ N convac . Hence, the conductance of the phase reactor is negligible compared to itssusceptance [49].Assumption 2.3 The difference of the phase angles δk−δf for ac bus k ∈ N convac and filterbus f ∈ Nac in a VSC station is small [49].26Chapter 2. Semidefinite Relaxation of Optimal Power Flow for ac-dc GridsAssumption 2.4 A dc grid is modeled as an ac grid with purely resistive transmissionlines and generators operating at unity power factor.The converter between ac bus k ∈ N convac and dc bus s ∈ N convdc converts ac voltage Vk todc voltage Vs. Let ma denote the maximum modulation factor, which can be set accordingto the modulation mode. The voltage magnitude at the converter ac bus is upper boundedby [149]|Vk| ≤ ma|Vs|. (2.1)Let PCk and PCs denote the active power injected into ac bus k ∈ N convac and dc buss ∈ N convdc , respectively. Let P convloss,k denote the losses of the converter connected to acbus k ∈ N convac . The active power balance equation for the converter connected to ac busk ∈ N convac and dc bus s ∈ N convdc isPCk + PCs + Pconvloss,k = 0. (2.2)Let Ik denote the injected current into an arbitrary bus k ∈ N . For a VSC station,the power losses can be determined from the aggregate losses of the components in thestation such as the transformer, ac filter, phase reactor, and converter. The losses of thetransformer, ac filter and phase reactor include the losses through the equivalent seriesresistance, the core losses, and the losses due to the harmonic currents. The losses of theconverter include switching losses and conductance losses [150]. The detailed losses modelof the VSC station can be found in [151] and [152]. However, the losses in a VSC stationwith ac converter bus k ∈ N convac can be approximated by a quadratic function of the accurrent magnitude |Ik| [49, 50, 51, 152]. Hence,P convloss,k = ak + bk|Ik|+ ck|Ik|2, (2.3)27Chapter 2. Semidefinite Relaxation of Optimal Power Flow for ac-dc Gridswhere ak, bk, and ck are positive coefficients representing the constant or no-load losses (e.g.,filter losses, transformer core losses), the linear losses (e.g., switching), and the quadraticlosses (e.g., transformer, phase reactor copper losses, and conduction losses), respectively.The model in (2.3) takes into account the losses of every component of a VSC station. Thevalues of the coefficients ak, bk, and ck depend on the components and the power ratingof the VSC station station, and can be obtained using various approaches such as onlineidentification or by aggregating the loss patterns of each component. The losses model fora sample HVDC link rated at 600 MW and 300 kV is given in [152, pp. 58-60], whereinit is seen that a quadratic losses model as a function of the phase reactor current offerssufficient accuracy for system-level studies.In general, the current magnitude |Ik| of a converter should not exceed an upper limitdenoted by Imaxk . The upper bound of the current amplitude can be replaced by themaximum apparent power flow as an operation constraint [49]. Let SCk and QCk denotethe apparent and reactive power of the converter connected to bus k ∈ N convac , respectively.We have|SCk |2 = (PCk)2 + (QCk)2 ≤ (|Vk|Imaxk )2 . (2.4)From Assumption 2.1, a VSC station can control active and reactive powers to adjustthe voltage of the ac and dc terminal buses. Therefore, the active and reactive powers PCkand QCk are variables and should satisfy constraint (2.4).The operation of the VSC is constrained by the upper and lower limits of the reactiveoutput power of the converter. Let SnomCk denote the nominal value of the apparent powerof the converter connected to bus k ∈ N convac . In practical VSCs, the maximum reactivepower that the converter can absorb is approximately proportional to the nominal value of28Chapter 2. Semidefinite Relaxation of Optimal Power Flow for ac-dc Gridsits apparent power, SnomCk [49]. For the converter connected to ac bus k ∈ N convac , we have−mbSnomCk ≤ QCk , (2.5)where mb is a positive constant and can be determined by the type of the converter.Let V maxk denote the maximum voltage magnitude of bus k ∈ N convac . Let δf and δkdenote the phase angle of the voltages at filter bus f ∈ Nac and converter ac bus k ∈ N convacin the VSC station, respectively. Let BCk =−XCkR2Ck+X2Ckdenote the susceptance of the non-ideal phase reactor connected to ac bus k ∈ N convac . From Assumption 2.2, the conductanceof the phase reactor is negligible compared to its susceptance. For filter bus f ∈ Nac andac bus k ∈ N convac in the VSC station, the injected reactive power is upper bounded byQCk ≤ |BCk | V maxk(V maxk − |Vf | cos (δk − δf )). (2.6)From Assumption 2.3, we obtain cos (δk − δf ) ≈ 1. Hence, the upper limit for theinjected reactive power can be approximated by the minimum value of the right-hand sideof (2.6). We haveQCk ≤ |BCk | V maxk(V maxk − |Vf |). (2.7)The upper bound of the converter apparent power and the upper and lower bounds ofthe converter reactive power are shown in Fig. 2.2 for a sample VSC station. Constraint(2.4) implies that the apparent power is limited by a circle in the Q-P plane. Constraints(2.5) and (2.7) indicate the minimum and maximum reactive power capability. The feasibleoperation region of the VSC is shown by the shaded area.In the ac-dc OPF problem, we aim to minimize a cost function subject to the constraintsimposed by the ac grid, the dc grids, and the VSCs. Let PGk and QGk denote the activeand reactive power generation at bus k ∈ G, respectively. In an ac-dc OPF problem, the29Chapter 2. Semidefinite Relaxation of Optimal Power Flow for ac-dc GridsFigure 2.2: Q-P characteristic of the converter for |Vk| = 1 pu, V maxk = 1.05 pu, Imaxk =1 pu, |Vf | = 0.95 pu, RCk = 0.001 pu, XCk = 0.1643 pu, Snomk = 1 pu, and mb = 0.6.variables include the complex voltage Vk for buses k ∈ N and PGk , QGk for generatorbuses k ∈ G, as well as PCk , QCk and |Ik| for converter buses k ∈ N convac . In the followingsubsection, we present the objective function and the constraints of the ac-dc OPF problem.2.2.1 Objective Function and ConstraintsThe objective function includes the total generation cost Cgen and the total system lossesPloss. The generation cost function for bus k ∈ G is denoted by fk(PGk). It can beapproximated by a quadratic function ck2P2Gk+ ck1PGk + ck0, where ck0, ck1, and ck2 arepositive coefficients [33]. Thus, the total generation cost isCgen =∑k∈Gfk(PGk)=∑k∈G(ck2P2Gk+ ck1PGk + ck0). (2.8)30Chapter 2. Semidefinite Relaxation of Optimal Power Flow for ac-dc GridsThe total system losses are equal to the total generation minus the total load of thesystem. It can be expressed as the summation of the injected active power into all buses.Let PDk denote the active load in bus k ∈ N . We obtainPloss =∑k∈N(PGk − PDk) . (2.9)In (2.9), if bus k is not a generator bus, then PGk = 0. We only consider the active powerlosses of the system since the reactive power does not dissipate energy.Let ω denote a positive scaling coefficient. The objective function fobj of the ac-dc OPFproblem isfobj = Cgen + ωPloss. (2.10)In (2.10), by increasing the value of ω, the total system losses have a larger weight in theobjective function as compared with the total generation cost. The typical value of ω (withunit $/MW) is around the value of coefficients ck1, k ∈ G since the total system losses Plossis a linear function of the generators’ output power PGk , k ∈ G. For an appropriate valueof ω, minimizing fobj will enable timely adjustment of control settings to jointly reducethe generation cost, VSC losses and transmission line losses. Thus, it can improve theeconomic efficiency of the power system operation.The ac-dc OPF problem is subject to a set of equality and inequality constraints im-posed by the ac grid, the dc grids, and the VSCs.Equality ConstraintsLet AT denote the transpose of an arbitrary matrix or vector A. Let Y denote the admit-tance matrix. For the vector of the bus voltages v = (V1, . . . , V|N |) and injected currents31Chapter 2. Semidefinite Relaxation of Optimal Power Flow for ac-dc Gridsi = (I1, . . . , I|N |), we havei = Y vT. (2.11)The equality constraints also consist of the power balance equations. Let z∗ denote theconjugate of an arbitrary complex number z. The active power balance equations for busk ∈ N can be written asPGk − PDk = Re{VkI∗k}, ∀ k ∈ N \ N conv (2.12a)PCk − PDk = Re{VkI∗k}, ∀ k ∈ N conv. (2.12b)From Assumption 2.4, power balance equations (2.12a) and (2.12b) can be used for dcbuses by setting I∗k = Ik and Re{VkI∗k} = VkIk. Let QDk denote the reactive load at busk ∈ N . The reactive power balance equations for ac buses areQGk −QDk = Im{VkI∗k}, ∀ k ∈ Nac (2.13a)QCk −QDk = Im{VkI∗k}, ∀ k ∈ N convac . (2.13b)In (2.12a) and (2.13a), if bus k is not a generator bus, then PGk = QGk = 0. Consider theconverter connected to ac bus k ∈ N convac and dc bus s ∈ N convdc . By substituting (2.12b)into the power balance equation (2.2), we obtainRe{VkI∗k}+ Re{VsI∗s}+ P convloss,k + PDk + PDs = 0. (2.14)32Chapter 2. Semidefinite Relaxation of Optimal Power Flow for ac-dc GridsInequality ConstraintsThe generators output active power PGk , k ∈ G, generators output reactive power QGk ,k ∈ G, the voltage magnitudes |Vk|, k ∈ N , and the apparent power flowing throughthe transmission lines Slm, (l,m) ∈ L are bounded. We use PminGk , PmaxGk , QminGk , QmaxGk ,V mink , and Vmaxk to represent the lower and upper bounds on the generator active power,reactive power, and bus voltage at bus k, respectively. If bus k is not a generator bus, thenPminGk = PmaxGk= QminGk = QmaxGk= 0. Smaxlm is the maximum apparent power flow through theline (l,m) ∈ L. The inequality constraints includePminGk ≤ PGk ≤ PmaxGk , ∀ k ∈ N \ N conv (2.15a)QminGk ≤ QGk ≤ QmaxGk , ∀ k ∈ Nac (2.15b)V mink ≤ |Vk| ≤ V maxk , ∀ k ∈ N (2.15c)|Slm| ≤ Smaxlm , ∀ (l,m) ∈ L. (2.15d)The inequality constraints imposed by the VSCs are (2.1), (2.4), (2.5) and (2.7). Theac-dc OPF problem is formulated as follows:minimize Cgen + ωPloss (2.16a)subject to (2.1), (2.4), (2.5), (2.7), and (2.11)−(2.15d). (2.16b)The minimization is over the complex voltage Vk for all buses k ∈ N , the active outputpower PGk and reactive output power QGk for all generator buses k ∈ G, as well as theactive power flow PCk , injected reactive powerQCk , and current magnitude |Ik| for converterbuses k ∈ N convac .33Chapter 2. Semidefinite Relaxation of Optimal Power Flow for ac-dc Grids2.3 SDP Form of the ac-dc OPF ProblemIn this section, we introduce a semidefinite relaxation of the ac-dc OPF problem (2.16).Our approach and notations are similar to [33]. However, we need to model the powerlosses and the constraints imposed by the active and reactive power flow for the VSCs inSDP form. We first introduce the notations. Then, we transform the objective functionand the constraints to formulate the SDP form of the ac-dc OPF.For k ∈ N and (l,m)∈L, ek is the kth basis vector in R|N |, eTk is its transposed vector,and Yk=ekeTk Y . The row k of matrix Yk is equal to the row k of Y . The other entries of Ykare zero. We use the Π model of the transmission lines (l,m) [2]. Let ylm and y¯lm denotethe value of the series and shunt elements at bus l connected to bus m, respectively. Wedefine Ylm = (y¯lm+ylm)eleTl − (ylm)eleTm, where the entries (l, l) and (l,m) of Ylm are equalto y¯lm + ylm and −ylm, respectively. The other entries of Ylm are zero. We define matricesYk, Y¯k, Ylm, Y¯lm, Mk and Mlm as follows. These matrices will be used to write the SDPform of the ac-dc OPF problem:Yk =12Re{Yk + Y Tk } Im{Y Tk − Yk}Im{Yk − Y Tk } Re{Yk + Y Tk } ,Y¯k = −12Im{Yk + Y Tk } Re{Yk − Y Tk }Re{Y Tk − Yk} Im{Yk + Y Tk } ,Ylm =12Re{Ylm + Y Tlm} Im{Y Tlm − Ylm}Im{Ylm − Y Tlm} Re{Ylm + Y Tlm} ,34Chapter 2. Semidefinite Relaxation of Optimal Power Flow for ac-dc GridsY¯lm = −12Im{Ylm + Y Tlm} Re{Ylm − Y Tlm}Re{Y Tlm − Ylm} Im{Ylm + Y Tlm} ,Mk =ekeTk 00 ekeTk ,Mlm =(el − em)(el − em)T 00 (el − em)(el − em)T .We define the following matrix for converter bus k ∈ N convacSk =ak bk2bk2ck .We define the variable column vector x as the real and imaginary values of the vectorof the complex bus voltages v.x =[Re{v}T Im{v}T]T.We define variable matrix W = xxT . We define the variable column vector ik as followsik =[1 |Ik|]T, ∀ k ∈ N convac .We also define variable matrix Ik = ikiTk for converter bus k ∈ N convac . We use thenotation Tr{A} to represent the trace of an arbitrary square matrix A. In [33], it is shown35Chapter 2. Semidefinite Relaxation of Optimal Power Flow for ac-dc GridsthatRe{VkI∗k} = Tr{YkW}, ∀ k ∈ N (2.17a)Im{VkI∗k} = Tr{Y¯kW}, ∀ k ∈ N (2.17b)|Vk|2 = Tr{MkW}, ∀ k ∈ N (2.17c)|Vl − Vm|2 = Tr{MlmW}, ∀ (l,m) ∈ L (2.17d)|Slm|2 = Tr{YlmW}2 + Tr{Y¯lmW}2, ∀ (l,m) ∈ L (2.17e)P convloss,k = Tr{SkIk}, ∀ k ∈ N convac . (2.17f)We will use (2.17a)−(2.17f) to rewrite the objective function and the constraints in theac-dc OPF problem in terms of variable matrices W and Ik, k ∈ N conv.2.3.1 Transforming the Objective FunctionSubstituting (2.17a) into (2.12a) for k ∈ G, we have PGk = Tr{YkW}+ PDk . The genera-tion cost function (2.8) becomesCgen =∑k∈G(ck2 (Tr{YkW}+ PDk)2 + ck1 (Tr{YkW}+ PDk) + ck0),where the generation cost is expressed as a quadratic function of matrix W. However, inthe SDP form, the objective function must be linear. We can replace Cgen with∑k∈G βk,which is a linear function of auxiliary variables βk, k ∈ G. Then, we can include inequalitiesfk(PGk) ≤ βk into the constraints set for all generator buses. Let τk = ck1PDk + ck0, then36Chapter 2. Semidefinite Relaxation of Optimal Power Flow for ac-dc Gridsthe matrix form of fk(PGk) ≤ βk for bus k ∈ G is βk − ck1Tr{YkW} − τk √ck2(Tr{YkW}+ PDk)√ck2(Tr{YkW}+ PDk) 1 0. (2.18)To represent the total system losses, we can substitute (2.17a) into (2.9). Thus, weobtainPloss =∑k∈NTr{YkW}. (2.19)The SDP form of the objective function in (2.10) can be expressed asfSDPobj =∑k∈Gβk + ω∑k∈NTr{YkW}. (2.20)2.3.2 Transforming the ConstraintsThe active power balance equation in (2.12a) can be combined with constraint (2.15a).Substituting (2.17a) into (2.15a), for k ∈ N \ N conv, we obtainPminGk − PDk ≤ Tr{YkW} ≤ PmaxGk − PDk . (2.21)Similarly, for ac buses k ∈ Nac, we haveQminGk −QDk ≤ Tr{Y¯kW} ≤ QmaxGk −QDk . (2.22)Substituting (2.17c) into (2.15c), for k ∈ N , we obtain(V mink )2 ≤ Tr{MkW} ≤ (V maxk )2. (2.23)37Chapter 2. Semidefinite Relaxation of Optimal Power Flow for ac-dc GridsSubstituting (2.17e) into (2.15d), for (l,m) ∈ L, we haveTr{YlmW}2 + Tr{Y¯lmW}2 ≤ (Smaxlm )2 . (2.24)The matrix form of inequality (2.24) is(Smaxlm )2 Tr{YlmW} Tr{Y¯lmW}Tr{YlmW} 1 0Tr{Y¯lmW} 0 1 0. (2.25)For the converter connected to ac bus k ∈ N convac and dc bus s ∈ N convdc , the SDP formof constraint (2.1) isTr{MkW} ≤ m2a Tr{MsW}. (2.26)The SDP form of constraint (2.5) is−mbSnomCk ≤ Tr{Y¯kW}. (2.27)Let ρk = −|BCk | (V maxk )2 + QDk and ξk = (BCkV maxk )2. Let Ck = (2ρk + 1)Y¯k − ξkMffor ac bus k ∈ N convac connected to filter bus f ∈ Nac. In Appendix A.1, we show thatconstraint (2.7) is equivalent to the following matrix inequalityρ2k + Tr{CkW}1√2Tr{Y¯kW} 1√2Tr{Y¯kW} Tr{Y¯kW}1√2Tr{Y¯kW} Tr{Y¯kW} 0√2 Tr{Y¯kW}1√2Tr{Y¯kW} 0 Tr{Y¯kW} 0Tr{Y¯kW}√2 Tr{Y¯kW} 0 1 0. (2.28)38Chapter 2. Semidefinite Relaxation of Optimal Power Flow for ac-dc GridsThe matrix form of inequality (2.4) is(Imaxk )2 Tr{MkW} Tr{YkW}+ PDk Tr{Y¯kW}+QDkTr{YkW}+ PDk 1 0Tr{Y¯kW}+QDk 0 1 0. (2.29)Substituting (2.17a) and (2.17f) into (2.14), we obtainTr{YkW}+ Tr{YsW}+ Tr{SkIk}+ PDk + PDs = 0. (2.30)Let I22k denote the entry in the second row and the second column of matrix Ik. Wecan obtain I22k = |Ik|2 =|Vk − Vf |2R2Ck +X2Ckfor ac bus k ∈ N convac and filter bus f ∈ Nac of in asame VSC station. From (2.17d), we obtainI22k =Tr{MkfW}R2Ck +X2Ck. (2.31)Let I12k denote the entry in the first row and the second column of matrix Ik. Fork ∈ N convac , we haveI12k ≥ 0. (2.32)We can write the equivalent SDP form of the ac-dc OPF problem (2.16) as follows:minimize∑k∈Gβk + ω∑k∈NTr{YkW} (2.33a)subject to (2.18), and (2.21)−(2.32), (2.33b)rank(Ik) = 1, ∀ k ∈ N convac , (2.33c)rank(W) = 1. (2.33d)39Chapter 2. Semidefinite Relaxation of Optimal Power Flow for ac-dc GridsThe minimization is over variables βk, k ∈ G, Ik, k ∈ N convac , and W. The rank constraints(2.33c) and (2.33d) in problem (2.33) are not convex. We propose a SDP relaxation ofthe ac-dc OPF problem. This optimization problem is obtained from the problem (2.33)by relaxing the rank constraints (2.33c) and (2.33d) and replacing them with constraintsIk 0, k ∈ N convac and W 0, respectively. Hence, the SDP relaxation of the ac-dc OPFproblem is obtained as follows:minimize∑k∈Gβk + ω∑k∈NTr{YkW} (2.34a)subject to (2.18), and (2.21)−(2.32), (2.34b)Ik 0, ∀ k ∈ N convac , (2.34c)W 0. (2.34d)In the following theorem, we state that the solution matrices Ioptk , k ∈ N convac to problem(2.34) return zero values for the current magnitude of the converters.Theorem 2.1 The solution matrices Ioptk , k ∈ N convac to problem (2.34) are all symmetricwith I12,optk = 0.The proof can be found in Appendix A.2. From the definition of the variable matrix Ik, wehave I12k = |Ik|. Theorem 2.1 states that the solutions Ioptk , k ∈ N convac return zero valuesfor the current magnitude |Ik|, k ∈ Nac. Hence, they are rank two. If we enforce problem(2.34) to return symmetric rank one matrices Ioptk , k ∈ N convac with I12,optk ≥ 0, then we candetermine a correct solution for Wopt as well.Motivated by the proof of Theorem 2.1, we introduce a penalty function in problems(2.33) and (2.34) to obtain a modified ac-dc OPF problem and its modified SDP relaxation40Chapter 2. Semidefinite Relaxation of Optimal Power Flow for ac-dc Gridsform. The modified ac-dc OPF problem is obtained as follows:minimize∑k∈Gβk + ω∑k∈NTr{YkW} − ε∑k∈N convacI12k (2.35a)subject to (2.18), and (2.21)−(2.32), (2.35b)rank(Ik) = 1, ∀ k ∈ N convac , (2.35c)rank(W) = 1. (2.35d)By relaxing the rank constraints (2.35c) and (2.35d), the modified SDP relaxation formof the ac-dc OPF is obtained asminimize∑k∈Gβk + ω∑k∈NTr{YkW} − ε∑k∈N convacI12k (2.36a)subject to (2.18), and (2.21)−(2.32), (2.36b)Ik 0, ∀ k ∈ N convac , (2.36c)W 0, (2.36d)where ε is a positive penalty coefficient. In problems (2.35) and (2.36), we have used apenalty function to obtain rank one solution matrices Ioptk , k ∈ N convac with I12,optk ≥ 0. Thefeasible set in problems (2.33) and (2.35) are the same. However, the solution to problem(2.35) is not the same as the solution to problem (2.33) since their objective function arenot the same. Let fSDP,33obj and fSDP,35obj denote the optimal values for the objective functionsof problems (2.33) and (2.35), respectively. In Theorem 2.2, we determine an upper boundfor the difference between the optimal values fSDP,35obj − fSDP,33obj .41Chapter 2. Semidefinite Relaxation of Optimal Power Flow for ac-dc GridsTheorem 2.2 The difference between the optimal values fSDP,33obj and fSDP,35obj is boundedby0 ≤ fSDP,35obj − fSDP,33obj ≤ εM, (2.37)whereM = min{ ∑k∈N convacImaxk ,∑k∈N convac(I12,35k −√I22,34k + Imaxk(√1 +bkckImaxk− 1))}, (2.38)and I22,34k , k ∈ N convac and I12,35k , k ∈ N convac are the solutions to problems (2.34) and (2.35),respectively.The proof can be found in Appendix A.3. Inequality (2.37) implies that the optimalvalue fSDP,35obj obtained from problem (2.35) may not be equal to the optimal value fSDP,33objobtained from problem (2.33). However, if εM approaches zero, then the difference betweenthe optimal values fSDP,35obj − fSDP,33obj will also approach zero. The value of M in (2.38)is small when the number of converters in the system is small. In fact, the value ofI12,35k −√I22,34k , k ∈ N convac is generally small. The value of Imaxk(√1 + bkckImaxk− 1) isalso small in practice. For example, for a VSC with typical values of Imaxk = 1.0526 pu,ck = 0.0036 pu, and bk = 0.0037 pu [153], we have Imaxk(√1 + bkckImaxk− 1) = 0.427 pu. InTheorem 2.3, we show that the value of ε is small in practice. Besides, problems (2.33) and(2.35) have the same feasible sets. Thus, the optimal solution to problems (2.33) and (2.35)are approximately equal. Let bmax denote the maximum value for coefficient bk among allVSC stations in the system. Let cmax1 , cmax2 , and PmaxG denote the maximum value for ck1,ck2, and PGk among all generators k ∈ G, respectively. In the following theorem, we give anapproximation for ε to obtain rank one solution matrices Ioptk , k ∈ N convac with I12,optk ≥ 0.42Chapter 2. Semidefinite Relaxation of Optimal Power Flow for ac-dc GridsTheorem 2.3 To obtain rank one solution matrices Ioptk , k ∈ N convac to problem (2.36) withI12,optk ≥ 0, the penalty coefficient ε can be approximated byε ≈ bmax (2cmax2 PmaxG + cmax1 + ω) . (2.39)The proof can be found in Appendix A.4. The value given in (2.39) is generally not a tightapproximation for the penalty coefficient ε. In Section 2.4, we show that penalty coefficientε in (2.39) is a small number. Therefore, the difference between the optimal values givenin (2.37) is negligible. Problems (2.16) and (2.33) are equivalent. Besides, problems (2.33)and (2.35) are approximately equivalent. However, the relaxation gap between problems(2.35) and (2.36) may not always be zero. In the following theorem, we give a sufficientcondition for zero relaxation gap.Theorem 2.4 Let Wopt and Ioptk , k ∈ N convac be the solution to problem (2.36). If the rankof Wopt is less than or equal to two, and the rank of Ioptk , k ∈ N convac are all one, then theSDP relaxation gap will be zero.The proof can be found in Appendix A.5. The sufficient condition in Theorem 2.4 is thegeneralized form of the sufficient condition proposed in [33] for the ac OPF problem. In[33], it is shown that the sufficient condition holds for practical ac grids including the IEEEtest systems. We approximate the ac-dc grid by an ac grid. Then, we use the results in [33]to show that the sufficient condition in Theorem 2.4 holds in practical ac-dc grids as well.We approximate the ac-dc grid O with an ac grid Oac. As shown in Fig. 2.3, the converterbetween buses k ∈ N convac and s ∈ N convdc is replaced by a small resistor Rks (e.g., 10−5 pu).This resistor is used to maintain the connectivity of the resistive part of the grid Oac. Italso implies that buses k ∈ N convac and s ∈ N convdc have almost the same voltage in Oac.We also connect a generator with only reactive output power to the ac converter buses to43Chapter 2. Semidefinite Relaxation of Optimal Power Flow for ac-dc GridsFigure 2.3: The converter model in O and its approximated model in Oac.model the reactive compensation capability of the converters. Besides, a dc grid in O isconsidered as an equivalent ac grid in Oac with resistive lines and generators operating atunity power factor.To formulate the OPF problem for the ac grid Oac, the converter losses and the oper-ating constraints imposed by the converters will be removed. In Theorem 2.4, we relatethe zero relaxation gap in Oac to the zero relaxation gap in O.Theorem 2.5 If Wopt is at most rank two for the ac OPF in Oac, then it is at most ranktwo for the ac-dc OPF in O.The proof can be found in Appendix A.6. Theorem 2.5 implies that Wopt is at most ranktwo for practical ac-dc grids since an ac-dc grid O can be approximated by an ac grid Oac,and Wopt is rank two for practical ac grids [33]. By solving problem (2.36), we can obtainsolution matrices Wopt and Ioptk , k ∈ N convac , where Wopt is at most rank two and matricesIoptk , k ∈ N convac are all symmetric rank one with I12,optk ≥ 0.In Algorithm 2.1, it is explained how to determine the vector of bus voltages xopt andthe vectors of injected currents ioptk , k ∈ N convac . The steps in Algorithm 2.1 are derivedfrom the proof of Theorem 2.4. In Line 1, problem (2.36) is solved. In Line 2, if thesolution matrices Wopt and Ioptk , k ∈ N convac to problem (2.36) are all rank one, then inLine 3, we calculate the nonzero eigenvalue ϕ with eigenvector ψ of Wopt. Going to Line9, we calculate the solutions as xopt =√ϕψ and Ioptk = ioptk (ioptk )T . In Line 4, if matrices44Chapter 2. Semidefinite Relaxation of Optimal Power Flow for ac-dc GridsAlgorithm 2.1 ac-dc OPF algorithm.1: Solve problem (2.36).2: If Wopt and Ioptk , k ∈ N convac are all rank one3: Calculate the nonzero eigenvalue ϕ with eigenvector ψ of Wopt.4: Else if Wopt is rank two and Ioptk , k ∈ N convac are rank one5: Calculate two nonzero eigenvalues φ1 and φ2 with eigenvectors ν1 and ν2 of Wopt.6: Calculate the rank one matrix Wopt1 =(φ1 + φ2)ν1νT1 .7: Calculate the nonzero eigenvalue ϕ with eigenvector ψ of Wopt1 .8: End if9: Calculate the solution vectors xopt and ioptk , k ∈ N convac from xopt =√ϕψ andIoptk = ioptk (ioptk )T .Ioptk , k ∈ N convac are all rank one, but matrix Wopt is rank two, then in Line 5, we calculatetwo nonzero eigenvalues φ1 and φ2 of Wopt with the corresponding eigenvectors ν1 andν2. It can be shown that the rank one matrix Wopt1 = (φ1 + φ2)ν1νT1 is also the solutionof problem (2.36) [33]. In Line 6, matrix Wopt1 is obtained. In Line 7, we calculate thenonzero eigenvalue ϕ of Wopt1 with its corresponding eigenvector ψ. Then in Line 9, thesolution vector xopt can be obtained from xopt =√ϕψ. If the rank of Wopt is greater thantwo, or at least one of the Ioptk , k ∈ N convac is not rank one, then the relaxation gap may notbe zero and the proposed approach does not return the global solution to the ac-dc OPFproblem. Similar to [40], one may use a heuristic method to enforce the low-rank solutionof problem (2.36) to become rank one or rank two. However, this is beyond the scope ofthis thesis.2.4 Performance EvaluationIn this section, we illustrate the performance of the SDP approach for solving the ac-dcOPF problem. The test system is shown in Fig. 2.4. The IEEE 118-bus test system isconnected to five off-shore wind farms at buses 7 and 9 via HVDC lines. An ac-dc microgridconsisting of three PV systems is connected to bus 5 via an ac-dc converter. Buses 5, 7,45Chapter 2. Semidefinite Relaxation of Optimal Power Flow for ac-dc Gridsand 9 in the IEEE test system are connected to each other via dc cables. The base power ofthe system is 100 MVA. The data for the IEEE 118-bus test system can be found in [154].The generators’ cost function coefficients can be found in Matpower’s library [155]. Therated power of each VSC is 50 MVA. The data for the VSC stations in the grid’s per unitsystem are given in Table 2.1. The VSC losses parameters in Table 2.1 are from [153] andconverted to the grid’s per unit system. The resistance of all HVDC lines is 0.06 pu. Theresistance of all dc cables is 0.001 pu. The maximum apparent power flow through theHVDC lines and other transmission lines is 1.1 pu. Unless stated otherwise, parameter mbis set to 0.5 and the maximum modulation factor ma is set to 1.05 [49].The lower and upper bounds for voltage magnitudes are 0.9 pu and 1.1 pu, respectively.The active output power of the wind farms connected to buses 132 to 135 are 10 MW. Theactive output power of the wind farm connected to bus 125 is 50 MW. Wind farms cancontrol the reactive power at its grid connection point. In this study, the wind farms areoperating at unity power factor. For the ac-dc microgrid, the output active power of eachPV system is 5 MW, and the active load in each dc bus is 20 MW. The active and reactiveloads in ac bus 164 are 20 MW and 20 MVAR, respectively. Total active and reactive loadsof the system are 4302 MW and 1448 MVAR, respectively. The scaling coefficient ω inthe objective function (2.10) is set to 1000 $/pu to jointly minimize the total generationcost and total system losses. There are 12 converters and the set of converter ac buses isN convac = {120, 123, 127, 130, 137, 140, 143, 146, 149, 152, 155, 165}.We discuss the results obtained from Theorems 2.1 to 2.5 in detail. We first solveproblem (2.34) by using CVX with SeDuMi solver in Matlab. As Theorem 2.1 states, thematrices Ioptk , k ∈ N convac are all symmetric and rank two with zero values for I12,optk . Thus,problem (2.34) does not return a correct solution to the ac-dc OPF problem. Then, wesolve problem (2.36).46Chapter 2. Semidefinite Relaxation of Optimal Power Flow for ac-dc GridsFigure 2.4: The IEEE 118-bus test system connected to five wind farms and one ac-dcmicrogrid.Table 2.1: VSC station parameters with converter bus k and filter bus f .VSC parameters (pu)RTk = 0.0005 XTk = 0.0125 Bf = 0.2 SnomCk= 0.5RCk = 0.00025 XCk = 0.04 Vmaxk = 1.05 Imaxk = 1.0526VSC losses data (pu)ak = 0.00265 bk = 0.0037 ck = 0.003647Chapter 2. Semidefinite Relaxation of Optimal Power Flow for ac-dc GridsAccording to Theorem 2.3, the appropriate value for the penalty factor ε can be de-termined from (2.39). In the grid’s per unit system, we have bmax = 0.0037 pu, cmax1 = 40$/MW, cmax2 = 0.02 $/MW2, and PmaxG = 500 MW. Hence, ε = 3.922 $/pu is obtainedfrom (2.39). It returns symmetric rank one matrices Ioptk , k ∈ N convac with positive valuesfor I12,optk . The optimal value of problem (2.36) is $128,171. If the relaxation gap betweenproblems (2.35) and (2.36) is zero, then the optimal value of problems (2.35) and (2.36)are equal. We use inequality (2.37) to determine the upper bound for fSDP,35obj − fSDP,33obj .By solving problem (2.34), we obtain∑k∈N convac(I12,35k −√I22,34k)= 0.11 pu. Accordingto (2.37), for ε = 3.922 $/pu and Imaxk = 1.0526 pu, the difference between the optimal val-ues fSDP,33obj and fSDP,35obj is at most $25.292, which is negligible compared with $128,171, theoptimal value of problem (2.35). Therefore, problems (2.33) and (2.35) are approximatelyequivalent. To assess the approximation in solving the ac-dc OPF problem, we obtain thevalue of the upper bound εM given in (2.37) for different values of parameters bk, ck, Imaxk ,k ∈ N convac , and the scaling coefficient ω. The results are given in Fig. 2.5. The value ofparameters bk and ck, k ∈ N convac are changing from 50% to 200% of their assumed valuesin the simulation setup. The value of Imaxk , k ∈ N convac is changing from 1 pu to 1.1 pu. Thevalue of ω is changing from 0 $/MW to 103 $/MW, which is practical for the case studysince coefficients ck1, k ∈ G in (2.8) are about 40 pu and the base power of the systemis 100 MVA. The results show that for different values of bk, ck, Imaxk , k ∈ N convac , and ω,the value of εM is less than $250. Hence, the solutions to problem (2.35) and the originalac-dc OPF problem (2.33) can be treated as approximately equal.By solving problem (2.36), we obtain a rank two matrix Wopt with nonzero eigenvaluesφ1 = φ2 = 8.831 and the corresponding eigenvectors ν1 and ν2. According to Theorem2.3, when matrix Wopt is rank two, then the relaxation gap between problems (2.35) and(2.36) is zero, and the global optimal solution can be obtained by using Algorithm 2.1.48Chapter 2. Semidefinite Relaxation of Optimal Power Flow for ac-dc Grids 0.0019 0.0028 0.0037 0.0046 0.0055 0.0064 0.0073 0255075100 0.0018 0.0027 0.0036 0.0045 0.0054 0.0063 0.0072 10203040501 1.01 1.02 1.03 1.04 1.05 1.06 1.07 1.08 1.09 1.125.125.225.325.425.50 1000 2000 3000 4000 5000 6000 7000 8000 9000 100000100200300Figure 2.5: The value of the upper bound εM in (2.37) for different values of parametersbk, ck, Imaxk , k ∈ N convac , and the scaling coefficient ω.49Chapter 2. Semidefinite Relaxation of Optimal Power Flow for ac-dc GridsFrom Algorithm 2.1, the rank one matrix Wopt1 = (φ1 + φ2)ν1νT1 is also the solution ofproblem (2.36). Matrix Wopt1 has one nonzero eigenvalue ϕ = 17.662 with correspondingeigenvector ψ. The solution vector xopt is obtained from xopt =√ϕψ. Fig. 2.6 showsthe voltage profile obtained by the proposed approach and Matpower. For the purposeof comparison, we use the primal-dual interior point (PDIP) algorithm using the Matlabinterior point method solver (MIPS) available in Matpower to obtain a solution to theac-dc OPF problem. We use the ac-dc OPF formulation proposed in [153] to includeconstraints of the ac-dc network into the Matpower. The objective function given in[153] for the ac-dc OPF problem is replaced by (2.10). The constraint for the converter dcvoltage lower limit in [153] is replaced by (2.5). The constraints given in [153] for the gridcode of wind farms’ connection are modified for unity power factor in our simulation. TheJacobian and Hessian matrices in Matpower are modified by adding the new variablesfor the VSCs and the set of equality and inequality constraints imposed by the converters.Unity voltage for all buses is assumed as the initial point. Although Matpower can obtaina solution to the ac-dc OPF problem using the PDIP algorithm, it does not guarantee thesolution to be the global optimal. Matpower can also use other OPF solvers such asTSPOPF [156] and a number of modern solvers for convex optimization problems such asinterior point optimizer (IPOPT) [157], sparse nonlinear optimizer (SNOPT) [158, 159].Although these solvers are well implemented, they can only guarantee convergence to alocal optimal solution of the ac-dc OPF problem. The system generation cost and totalsystem losses obtained from the SDP relaxation approach and Matpower are given inTable 2.2. IPOPT returns the best sub-optimal solution. All other solvers did not returna global optimal solution (the solution obtained from the SDP relaxation technique). Thetotal system losses consist of two components: the transmission line and VSC losses. From(2.3), the transmission line losses can be obtained as Ploss−∑k∈N convac (ak + bk|Ik|+ ck|Ik|2),50Chapter 2. Semidefinite Relaxation of Optimal Power Flow for ac-dc Grids1 6 119 124 129 134 139 144 149 154 159 16411.051.1Bus numberVoltagemagnitude(pu)SDP relaxationMATPOWERbus 1to bus 10bus 119to bus 166Figure 2.6: Voltage profile obtained from the SDP relaxation approach and the approachproposed in [153] using Matpower with MIPS solver.Table 2.2: The generation cost and system losses obtained from SDP relaxation approachand the approach proposed in [153] using Matpower with different solvers.Generationcost ($/hr)Totallosses (MW)Converterlosses (MW)Linelosses (MW)SDP relaxation 128,171 73.12 6.28 66.84Matpower with MIPS 129,085 87.26 9.12 78.14Matpower with IPOPT 128,641 78.46 7.08 71.38Matpower with SNOPT 128,905 83.05 7.19 75.86Matpower with TSPOPF 129,516 91.38 9.41 81.97where the second term is the converter losses.The converter and transmission line losses obtained from the SDP relaxation approachare given in Table 2.2. Total system losses are about 1.7% of the total load of the system.Moreover, we can observe that the generation cost and total system losses obtained fromthe SDP relaxation approach are lower since the solution obtained from Matpower isa local optimum. The VSC losses are about 8.6% of the total system losses. It confirmsthat the VSC losses can add up to a significant fraction of total system losses and haveto be included in the OPF problem. The Q-P characteristics of the converters 1, 2, 3,51Chapter 2. Semidefinite Relaxation of Optimal Power Flow for ac-dc Grids4, 11, and 12 in the VSC per unit system are shown in Fig. 2.7. The nominal apparentpower for the converters is SnomCk = 1 pu in the VSC’s per unit system. From (2.5), themaximum reactive power that the converters can absorb is 0.5 pu. Constraint (2.7) forthe upper limit of the injected reactive power is active for converters 1 and 11. That is,converters 1 and 11 are injecting reactive power to the ac grid with maximum rate forvoltage regulation. Constraint (2.4) for the upper limit of the apparent power is active forconverter 3, since the generated active power by the wind farms is flowing through thisconverter. Converters 2 and 4 are also absorbing the generated active power by the windfarms. Converter 12 is injecting reactive power to meet the load at bus 164.To illustrate the results of Theorem 2.5, we have solved the dual OPF problem defined inthe proof of Theorem 2.5 forO andOac. The Lagrange multipliers are shown in Fig. 2.8. Asshown by dashed rectangles, instead of λk, we have Lagrange multiplier θks associated withequality constraint (2.30) for converter ac bus k ∈ N convac and converter dc bus s ∈ N convdcin the same VSC station. We observe that the Lagrange multipliers λk in the ac-dc gridO are all positive and greater than their corresponding Lagrange multiplier λk in the acgrid Oac. Besides, Lagrange multipliers θks are positive though they can be lower (e.g., inbuses 120 to 123) or greater (e.g., in buses 155 and 156) than their corresponding Lagrangemultipliers in Oac. Hence, the conditions given in the proof of Theorem 2.5 are satisfied.The running time of Algorithm 2.1 depends on the code implementation, solver, initialcondition, stopping criteria (e.g. maximum number of iterations, error), and the size ofthe power system (e.g., number of buses and transmission lines). We have used the sparsematrix operations in Matlab to increase the convergence speed. For this case study, therunning time of Algorithm 2.1 is 5 seconds on average with SeDuMi solver and the error of10−4. One may use Mosek as the solver to reduce the algorithm running time at the costof larger error in the solution. Furthermore, the number of variables of problem (2.36) is52Chapter 2. Semidefinite Relaxation of Optimal Power Flow for ac-dc Grids−1.5 −1 −0.5 0 0.5 1 1.5−1.5−1−0.500.511.5Active power (pu)Reactivepower(pu)Converter 1Converter 2Converter 3Converter 4Converter 11Converter 12Reactive power lower limitFigure 2.7: Q-P characteristics of the converters 1, 2, 3, 4, 11, and 12.1 6 119 124 129 134 139 144 149 154 159 164303540455055Bus numberLagrangemultipliers Lagrange multipliers in ac-dc grid OLagrange multipliers in ac grid Oacbus 1 tobus 10bus 119to bus 166Converter buses with Lagrange multiplier θksFigure 2.8: Lagrange multipliers for ac-dc grid O and ac grid Oac.quadratic with respect to the number of buses. Hence, the computational cost of solvingthe SDP relaxation of the ac-dc OPF problem grows rapidly with the size of the system.To reduce the computational burden of SDP relaxation of large OPF problems, differenttechniques have been proposed such as exploiting sparsity in SDP relaxation [160, 161] and53Chapter 2. Semidefinite Relaxation of Optimal Power Flow for ac-dc Gridsmatrix combination algorithm [162]. In our proposed approach, the number of variablesof the dual of ac-dc OPF problem is linear with respect to the number of buses. Thus,solving the dual OPF can save computation time to determine the solution of the OPFproblem in large ac-dc grids.2.5 SummaryIn this chapter, we studied the OPF problem for ac-dc grids. The converter losses and theoperating limits on the voltage and current of the converters were modeled in the OPFproblem. The original problem was nonconvex, which does not guarantee a global optimalsolution. Convex relaxation techniques were used to obtain the SDP form of the ac-dcOPF problem. An algorithm was given to determine the solution to the original ac-dcOPF problem from its SDP form. We provided a sufficient condition for zero relaxationgap that guarantees the OPF algorithm to return the global optimal solution. We alsoshowed that the sufficient condition holds for practical ac-dc grids. Simulation results on amodified IEEE 118-bus test system confirmed that the zero relaxation gap condition holdsfor the case study, and the SDP approach enabled us to obtain the global optimal solutionto the bus voltages and the converters operating points in polynomial time.54Chapter 3Security-Constrained UnitCommitment for ac-dc Grids3.1 IntroductionIn this chapter, we study the SCUC problem in ac-dc grids. The objective of the underlyingSCUC problem includes the generators’ operation cost and the system’s losses. The SCUCproblem in an ac-dc grid is subject to power balance equations, power flow limits of thetransmission lines, bus voltage limits, voltage and current limits of the ac-dc converters,as well as the generators’ constraints such as the capacity limits, minimum on/off timerequirements, and ramping up/down rate limits. We use convex relaxation techniqueincorporated with a risk minimization approach to solve the ac-dc SCUC problem withload and generation uncertainty. We transform the problem into an SDP and use CVaR[163] to minimize the likelihood of high deviations in the load demand and renewablegeneration. The main contributions of this chapter are as follows:• Addressing the Uncertainty Issues : To address the uncertainties in the load demandand renewable generator, we introduce a penalty based on CVaR to the objectivefunction. It enables us to limit the risk of shortage in the net power supply withina confidence level. CVaR is a convex function and can be optimized using samplingtechniques and linear program.55Chapter 3. Security-Constrained Unit Commitment for ac-dc Grids• Novel Solution Approach: Unlike most of the existing approaches (e.g., [8, 59, 60, 61,62, 65, 66, 67, 68, 69, 70]) that apply either dc or linearized ac power flow models, weconsider the full ac power flow model to formulate the SCUC problem as a nonlinearmixed-integer optimization problem. We relax the binary variables and introduce aweighted l1-norm regularization term to the objective function to enforce the relaxedvariables to be either 0 or 1. We use convex relaxation techniques to transformthe problem into an SDP. It enables us to develop an iterative reweighted l1-normapproximation algorithm that solves a sequence of SDPs.• Performance Evaluation: Simulations are performed on an IEEE 30-bus system con-nected to some dc grids. We show that for 1000 different initial conditions, theproposed algorithm returns a near-global optimal solution with 2% gap in all scenar-ios, and returns the near-global optimal solution with 1% gap in 98% of the scenarios.When compared with the multi-stage deterministic SCUC approaches (e.g., in [60]and [61]) in a number of test systems, our algorithm returns a solution with a smallergap from the global optimal solution in a lower central processing unit (CPU) time.When compared with a robust multi-stage SCUC algorithm (e.g., in [68, 69, 70]), theoptimal objective value with our algorithm is smaller, as it takes into account thelikelihood of deviations in the net power supply instead of the worst-case scenario.The rest of this chapter is organized as follows. Section 3.2 introduces the system modeland operational constraints for the ac-dc grid. In Section 3.3, we formulate the SCUCproblem and transform it into an SDP. We propose an iterative algorithm to solve theproblem. In Section 3.4, we evaluate the performance of the proposed algorithm throughextensive simulations. A summary is given in Section 3.5. The proofs can be found inAppendices B.1−B.5.56Chapter 3. Security-Constrained Unit Commitment for ac-dc Grids3.2 System ModelConsider an ac-dc grid comprising a set of buses N and a set of lines L. The componentsoperating on dc power are connected to the ac network through three-phase VSCs. Fig.3.1 shows the schematic of a VSC station consisting of an ac-dc converter, phase reactor,ac filter, and transformer. For simplicity in the notations, we partition the set of buses intofour distinct subsets: The set N convac ⊆ N of ac side converter buses, the set Nac ⊆ N ofac buses that are not connected to the converters, the set N convdc ⊆ N of dc side converterbuses, and the set Ndc ⊆ N of dc buses that are not connected to the converters. LetN conv = N convac ∪ N convdc denote the set of all converter buses. For example, for the VSCstation in Fig. 3.1, we have s ∈ N convdc , k ∈ N convac , and f ∈ Nac.In the following subsections, we introduce the models of the VSC, energy storage,generator, load, and ac-dc network.3.2.1 VSC Station ModelFig. 3.1 shows a VSC station with a two- or three-level converter using the PWM switchingmethod. In this figure, XCk denotes the phase reactor of the converter connected tobus k ∈ N convac and RCk denotes the resistance modeling the losses of this phase reactor.Let BCk =−XCkR2Ck+X2Ckdenote the susceptance of the phase reactor connected to bus k ∈N convac . The ac filter connected to bus f ∈ Nac is modeled by the shunt susceptanceBf . The transformer connected to bus f ∈ Nac is modeled by its series reactance XTf andresistance RTf [147]. We divide the operation cycle into a set T = {1, . . . , T} of T time slotswith equal length. The losses in a VSC station in time slot t ∈ T can be approximated by aquadratic function of its ac current magnitude |Ik(t)| injected into ac bus k ∈ N convac [153].Let P convloss,k(t) denote the losses of the VSC station with ac bus k ∈ N convac in time slot t.57Chapter 3. Security-Constrained Unit Commitment for ac-dc GridsFigure 3.1: A VSC station schematic with dc converter bus s ∈ N convdc , ac converter busk ∈ N convac , and filter bus f ∈ Nac.We haveP convloss,k(t) = ak + bk|Ik(t)|+ ck|Ik(t)|2, (3.1)where ak, bk, and ck are positive coefficients, which depend on the components and thepower rating of the VSC station. Let PCk(t) and QCk(t) denote the injected active andreactive power into ac converter bus k ∈ N convac in time slot t, respectively. Let PCs(t)denote the injected active power into dc bus s ∈ N convdc in time slot t. The active powerbalance in time slot t ∈ T for the converter connected to buses k ∈ N convac and s ∈ N convdc isPCk(t) + PCs(t) + Pconvloss,k(t) = 0. (3.2)Let Vk(t) denote the voltage at bus k ∈ N in time slot t. Let Imaxk and V maxk denotethe maximum current and voltage magnitude at bus k ∈ N , respectively. Let SnomCk denotethe nominal value of the apparent power of the converter connected to bus k ∈ N convac . Intime slot t ∈ T , the active and reactive power flow in a VSC station with converter busk ∈ N convac and filter bus f ∈ Nac are bounded by [49, 149]QCk(t) ≤ |BCk |V maxk (V maxk − |Vf (t)|) , (3.3)−mqk SnomCk ≤ QCk(t), (3.4)58Chapter 3. Security-Constrained Unit Commitment for ac-dc Grids(PCk(t))2 + (QCk(t))2 ≤ (|Vk(t)|Imaxk )2 , (3.5)where mqk is a positive constant and can be determined by the type of the converterconnected to bus k ∈ N convac [49]. Let mvks denote the maximum modulation factor of theconverter connected to buses k ∈ N convac and s ∈ N convdc . In time slot t ∈ T , we have [149]|Vk(t)| ≤ mvks|Vs(t)|. (3.6)3.2.2 Energy Storage System ModelFor the sake of simplicity, we assume that the energy storage systems are lossless andoperate at unity power factor. We can extend our model by considering the active powerlosses and reactive power injection for the energy storage systems. Let PBk(t) denote theactive power injected into (PBk(t) > 0) or absorbed from (PBk(t) < 0) bus k by the batteryin time slot t. The power rating of the battery has limits PminBk < 0 and PmaxBk> 0. That isPminBk ≤ PBk(t) ≤ PmaxBk , k ∈ N , t ∈ T . (3.7)If no battery is connected to bus k ∈ N , then PminBk = PmaxBk = 0. Let Binitk ≥ 0 denote theinitial energy level of the battery in bus k ∈ N at the beginning of the operating cycle.The stored energy in the battery until time T ′ ≤ T is nonnegative and upper bounded bythe limit Bmaxk . We have0 ≤ Binitk −T ′∑t=1PBk(t) ≤ Bmaxk , k ∈ N , T ′ ≤ T. (3.8)59Chapter 3. Security-Constrained Unit Commitment for ac-dc Grids3.2.3 Generator and Load ModelWe use binary variable uk(t) ∈ {0, 1} to indicate whether the generator in bus k is on(uk(t) = 1) or off (uk(t) = 0) in time slot t. The generation cost of a committed gener-ator in bus k with generation level PGk(t) ≥ 0 in time slot t can be approximated by aquadratic function ck2 (PGk(t))2 + ck1PGk(t) + ck0, where ck0, ck1, and ck2 are nonnegativecoefficients [33]. In addition, a generator in bus k has a fixed startup cost csuk when it isturned on and a fixed shutdown cost csdk when it is turned off. We use a binary variablesk(t) ∈ {0, 1} to indicate whether the generator in bus k is started up (sk(t) = 1) or not(sk(t) = 0) in time slot t. We use a binary variable dk(t) ∈ {0, 1} to indicate whetherthe generator in bus k is shut down (dk(t) = 1) or not (dk(t) = 0) in time slot t. Wehave sk(t) = dk(t) = 0 if generator k is neither started up nor shut down in time slott. Let vector PG(t) = (PGk(t), k ∈ N ) denote the generation profile of all generators intime slot t. We define variable vectors u(t) = (uk(t), k ∈ N ), s(t) = (sk(t), k ∈ N ) andd(t) = (dk(t), k ∈ N ) for all generators. The grid-wide operation cost of the generators intime slot t isCgen(PG(t),u(t), s(t), d(t)) =∑k∈Nck2 (PGk(t))2 + ck1PGk(t) + ck0 uk(t)+ csuk sk(t) + csdk dk(t). (3.9)We have ck0 = ck1 = ck2 = csuk = csdk = 0, when either a renewable generator or no generatoris connected to bus k.Let PDk(t) and QDk(t) denote the active and reactive load components in bus k in timeslot t, respectively. We assume that the reactive power uncertainty can be mitigated usingthe reactive power compensator. However, the scheduled output power PGk(t) and theactive load PDk(t) for the next day may not match with the actual output power P̂Gk(t)60Chapter 3. Security-Constrained Unit Commitment for ac-dc Gridsand demand P̂Dk(t) in time slot t, respectively. An alternative is to study the deviationof the actual net power supply P̂Gk(t) − P̂Dk(t) from its presumed value PGk(t) − PDk(t)in bus k in time slot t. The excess net supply can be curtailed or stored in the batteryenergy storage. We also assume that in those buses with a renewable generator or loaddemand, there exists a reserve unit that can be turned on to compensate the shortage inthe net supply. We model the generation cost for the reserve generator in bus k in timeslot t as cres,k[PGk(t) − PDk(t) − (P̂Gk(t) − P̂Dk(t))]+, where [·]+ = max{·, 0} and cres,k isthe nonnegative marginal cost in $/MW of the reserve unit connected to bus k. Let vectorP̂G(t) = (P̂Gk(t), k ∈ N ) denote the actual generation profile in time slot t. Let vectorsPD(t) = (PDk(t), k ∈ N ) and P̂D(t) = (P̂Dk(t), k ∈ N ) denote the presumed and actualload profiles in time slot t, respectively. The total cost of reserve units in time slot t isCres(PG(t), P̂G(t),PD(t), P̂D(t))=∑k∈Ncres,k[PGk(t)−PDk(t)−(P̂Gk(t)− P̂Dk(t))]+. (3.10)The output active power PGk(t) and reactive power QGk(t) in time slot t for an operatinggenerator in bus k are bounded by the limits PminGk , PmaxGk, QminGk , and QmaxGk1. That isuk(t)PminGk≤ PGk(t) ≤ uk(t)PmaxGk , k ∈ N , t ∈ T (3.11a)uk(t)QminGk≤ QGk(t) ≤ uk(t)QmaxGk , k ∈ N , t ∈ T . (3.11b)Let ruk and rdk denote the maximum ramp up and ramp down rates for the generatorin bus k, respectively. Let rsuk and rsdk denote the maximum startup ramp and shutdownramp rates for the generator in bus k, respectively. For t ∈ T , we havePGk(t)− PGk(t− 1) ≤ uk(t− 1) ruk + sk(t) rsuk , (3.12a)1Similar to (3.5), one may also consider the coupling constraint for the output active power and reactivepower of a generator.61Chapter 3. Security-Constrained Unit Commitment for ac-dc GridsPGk(t− 1)− PGk(t) ≤ uk(t− 1) rdk + sk(t) rsdk . (3.12b)Parameters ruk , rdk , rsuk , and rsdk are set to a large number if the generator in bus k ∈ N is arenewable energy source. A generator in bus k has a minimum up time tuk and down timetdk to start up and shut down. We havet∑t′=t−tuk+1sk(t′) ≤ uk(t), k ∈ N , t ∈ T (3.13a)t∑t′=t−tdk+1dk(t′) ≤ 1− uk(t), k ∈ N , t ∈ T . (3.13b)The value of sk(t) and dk(t), k ∈ N at t ≤ 0 are set based on the generators’ state beforestarting the operating horizon. The binary variable sk(t) is equal to 1 only when generatork is off in time slot t − 1 and is on in time slot t. The binary variable dk(t) is equal to 1only when generator k is on in time slot t− 1 and is off in time slot t. Thus, we haveuk(t)− uk(t− 1) = sk(t)− dk(t), k ∈ N , t ∈ T . (3.14)The binary variables uk(t), k ∈ N at t = 0 are set based on the on/off state of thegenerators before the operating horizon. Constraints (3.13a), (3.13b), and (3.14) enforcevariables sk(t), and dk(t) to be either 0 or 1 even if we relax variables sk(t) and dk(t) totake any value in the interval [0, 1]. Thus, we haveuk(t) ∈ {0, 1}, k ∈ N , t ∈ T (3.15a)0 ≤ sk(t), dk(t) ≤ 1, k ∈ N , t ∈ T . (3.15b)62Chapter 3. Security-Constrained Unit Commitment for ac-dc Grids3.2.4 ac-dc Network ModelThe system losses are equal to the total generation minus the total load. We obtainPloss(t) =∑k∈N(PGk(t) + PBk(t)− PDk(t)) , t ∈ T . (3.16)Let AT denote the transpose of an arbitrary matrix or vector A. Let Y denote theadmittance matrix. For the vector of the bus voltages v(t) = (V1(t), . . . , V|N |(t)) andinjected currents i(t) = (I1(t), . . . , I|N |(t)) in time slot t, we havei(t) = Y vT(t). (3.17)Let z∗ denote the conjugate of an arbitrary complex number z. The power balance equa-tions in time slot t ∈ T arePGk(t) + PBk(t)− PDk(t) = Re{Vk(t)I∗k(t)}, k ∈ N \ N conv (3.18a)PCk(t) = Re{Vk(t)I∗k(t)}, k ∈ N conv (3.18b)QGk(t)−QDk(t) = Im{Vk(t)I∗k(t)}, k ∈ Nac (3.18c)QCk(t) = Im{Vk(t)I∗k(t)}, k ∈ N convac . (3.18d)In (3.18a) and (3.18c), if bus k is not a generator bus, then PGk(t) = QGk(t) = 0. LetV mink denote the lower bound on the bus voltage at bus k. Let Smaxlm denote the maximumapparent power flow through the line (l,m) ∈ L. We haveV mink ≤ |Vk(t)| ≤ V maxk , k ∈ N (3.19a)|Slm(t)| ≤ Smaxlm , (l,m) ∈ L. (3.19b)63Chapter 3. Security-Constrained Unit Commitment for ac-dc GridsSimilar to [6, Ch. 7], [164, Ch. 8], [165, Ch. 4], [166, 167, 168, 169, 170, 171, 172,173, 174], one can take into account the contingency scenarios for transmission lines outageand formulate the SCUC with contingencies, which is often referred by the contingency-constrained unit commitment. It is sufficient to construct the admittance matrix Y corre-sponding to the possible contingency scenarios and include constraints (3.17)−(3.19b) forthe bus voltage and injected current profiles in all contingency scenarios.3.3 Problem Formulation3.3.1 SCUC ProblemThe system operator aims to jointly minimize the generation cost Cgen(·) in (3.9), the costCres(·) in (3.10) associated with the net power supply uncertainty, and the total systemlosses Ploss(t) in (3.16). The cost Cres(·) in (3.10) depends on the random variables P̂G(t)and P̂D(t). We consider the risk measure CVaR [175] to limit the risk of the net supplyshortage. For a confidence level β ∈ (0, 1) and vectors PG(t) and PD(t) in time slot t,we defineCVaRβ(PG(t),PD(t))=E{Cres(PG(t), P̂G(t),PD(t), P̂D(t)) ∣∣Cres(PG(t), P̂G(t),PD(t), P̂D(t)) ≥ αβ }, (3.20)where E(·) is the expectation over the random variables P̂G(t) and P̂D(t) and αβ =min{α(t)∣∣∣Pr{Cres(·) ≤ α(t)} ≥ β}. In general, the probability distributions of randomvariables P̂G(t) and P̂D(t) are not available. It is possible to estimate the CVaR by adopt-ing sample average approximation (SAA) technique [175]. We use the set J , {1, . . . , J}of J samples of P jG(t) and PjD(t) of the random variables P̂G(t) and P̂D(t) in time slot64Chapter 3. Security-Constrained Unit Commitment for ac-dc Gridst from the historical record and obtain Pr{P jG(t), PjD(t)}, the probability of the scenariowith sample P jG(t) and PjD(t). Then, (3.20) can be approximated byCVaRβ(PG(t),PD(t)) ≈ minα(t)∈RΓβ(α(t),PG(t),PD(t)), (3.21)whereΓβ(α(t), PG(t), PD(t))=α(t) +∑j∈J(Pr{P jG(t), P jD(t)}1− β[Cres(PG(t),PjG(t),PD(t),PjD(t))− α(t)]+) . (3.22)Equation (3.21) implies that to compute the CVaR, it is sufficient to minimize Γβ(·) in(3.22) over the variable α(t) using the historical samples of the random variables P̂G(t)and P̂D(t). The objective function fobj of the SCUC problem isfobj =∑t∈T(Cgen(PG(t),u(t), s(t),d(t))+ ωloss Ploss(t) + ωcvar Γβ(α(t),PG(t),PD(t))),(3.23)where ωloss and ωcvar are nonnegative weight coefficients. In (3.23), by increasing the valueof ωloss, the total system losses have a larger weight in the objective function as comparedwith the total generation cost and CVaR. For small-scale grids, the typical value of ωloss isaround the value of coefficients ck1, k ∈ N , for the generators, since the total system lossesPloss(t), t ∈ T , is a linear function of the generators’ output power. For large-scale testsystems, the impact of coefficients ck2, k ∈ N , on the grid-wide generation cost increases.Hence, one may set ωloss to be a larger value than ck1, k ∈ N , (e.g., 10 to 100 timeslarger). By increasing the value of ωcvar, the cost associated with the net power supplyshortage has a larger weight in the objective function, and thus the system operator is65Chapter 3. Security-Constrained Unit Commitment for ac-dc Gridsmore risk-averse. The typical value for ωcvar is around the value of ck1/cres,k, k ∈ N , sinceΓβ(α(t), PG(t), PD(t))in (3.22) is a linear function of cres,kPGk(t).Let ψ = (Vk(t), Ik(t), uk(t), sk(t), dk(t), PGk(t), QGk , PDk(t), PBk(t), k ∈ N , Slm(t),(l,m) ∈ L, α(t), PCk(t), QCk(t), PCs(t), k ∈ N convac , s ∈ N convdc , t ∈ T ) denote the vector ofdecision variables. The SCUC problem is formulated asminimizeψfobj (3.24)subject to (3.1)−(3.8) and (3.11a)−(3.19b).Problem (3.24) is a nonlinear mixed-integer optimization problem, which is difficult to besolved. We use the l1-norm and convex relaxation techniques to relax the binary variablesuk(t), k ∈ N , t ∈ T and transform problem (3.24) into an SDP. We develop a reweightedl1-norm algorithm to determine the binary solution variables uk(t) in an iterative fashion.Some of our notations are similar to [33] and [140]. The approaches given in [33] and [140]are not directly applicable to solve problem (3.24), since we have binary decision variables.We define the variable column vector x(t) =[Re{v(t)}T Im{v(t)}T ]T as the real andimaginary values of the vector of the bus voltages v(t) in time slot t. We define variablematrix W(t) = x(t)(x(t))T. The VSC losses are the function of ac converter current. Wedefine the variable column vector ik(t) =[Imaxk + |Ik(t)|2Imaxk − |Ik(t)|]T, k ∈ Nac, t ∈ T .We define variable matrix Ik(t), k ∈ N convac , t ∈ T as Ik(t) = ik(t)ik(t)T. In Appendix B.1,we present the SDP form of problem (3.24). First, we express constraints (3.1)−(3.8) and(3.11a)−(3.19b) in terms of matrix variables W(t) and Ik(t), k ∈ N convac , t ∈ T . Second,we relax the variable uk(t) to take any value in the interval [0, 1] and include the followingl0-norm constraint into the constraint set.‖uk(t)‖0 + ‖1− uk(t)‖0 = 1, k ∈ N , t ∈ T , (3.25)66Chapter 3. Security-Constrained Unit Commitment for ac-dc Gridswhere ‖z‖0 is the l0-norm of an arbitrary vector z. When z is a nonnegative scalar, wehave ‖z‖0 = 0 for z = 0, otherwise ‖z‖0 = 1. Third, we obtain fSDPobj , the SDP form of theobjective function in (3.23). We introduce the auxiliary variables ϑk(t), k ∈ N , t ∈ T andreplace Cgen(·) in (3.9) with∑k∈N ϑk(t) +ck0uk(t)+csuk sk(t)+csdk dk(t). We write the systemlosses in (3.16) in terms of W(t). We introduce the auxiliary variable µj(t) for sample jin time slot t to upper bound each term [Cres(·)− α(t)]+ in Γβ(·) in (3.22). We introducethe auxiliary variable ηjk(t) for bus k and sample j in time slot t to upper bound the term[PGk(t)− PDk(t)− (P̂Gk(t)− P̂Dk(t))]+in function Cres(·) in (3.21). We replace functionΓβ(·) in the objective function (3.23) with α(t)+ 11−β∑j∈J Pr{P jG(t), PjD(t)}µj(t).In Appendix B.1, we obtain the feasible set ΦSDP of decision variables φ = (ϑk(t), uk(t),sk(t), dk(t), PBk(t), µj(t), ηjk(t), j ∈ J , k ∈ N , α(t),W(t), Ik(t), k ∈ N convac , t ∈ T ) ex-cluding the l0-norm constraint (3.25). The SDP form of problem (3.24) is as follows:minimizeφ∈ΦSDPfSDPobj (3.26a)subject to constraint (3.25), (3.26b)rank(Ik(t)) = 1, k ∈ N convac , t ∈ T (3.26c)rank(W(t)) = 1, t ∈ T (3.26d)Ik(t) 0, k ∈ N convac , t ∈ T (3.26e)W(t) 0, t ∈ T . (3.26f)Problem (3.26) is equivalent to the SCUC problem (3.24), and is still a nonconvex opti-mization problem due to the l0-norm constraint (3.26b) and the rank constraints (3.26c)and (3.26d). We tackle the nonconvexity of problem (3.26) in the following subsections.67Chapter 3. Security-Constrained Unit Commitment for ac-dc Grids3.3.2 l1-norm Relaxation of the SCUC ProblemWe remove the nonconvex constraint (3.26b) from the constraint set of problem (3.26) andintroduce the penalty∑k∈N∑t∈T ‖uk(t)‖0 +‖1− uk(t)‖0 with a weight coefficient ς to theobjective function. We have the following l0-regularized SCUC problem:minimizeφ∈ΦSDPfSDPobj + ς∑t∈T∑k∈N(‖uk(t)‖0 + ‖1− uk(t)‖0) (3.27)subject to constraints (3.26c)−(3.26f).The value of ‖uk(t)‖0 + ‖1− uk(t)‖0 is 1, if uk(t) is binary. Otherwise, we have ‖uk(t)‖0 +‖1− uk(t)‖0 = 2. Therefore, in problem (3.27), with a sufficiently large coefficient ς, thesolution to variables uk(t), k ∈ N , t ∈ T are all binary. The term∑t∈T∑k∈N ‖uk(t)‖0 +‖1− uk(t)‖0 is equal to T |N | for all binary variables uk(t), k ∈ N , t ∈ T . Hence, for asufficiently large coefficient ς, problems (3.26) and (3.27) are equivalent.Next, we replace the terms ‖uk(t)‖0 + ‖1− uk(t)‖0 with the l1-regularization termθk1(t) ‖uk(t)‖1 + θk2(t) ‖1− uk(t)‖1 for k ∈ N in the objective function of problem (3.27)[176], where θk1(t) and θk2(t) are weight coefficients and ‖z‖1 is the l1-norm of an arbitraryvector z. When z is a nonnegative scalar, ‖z‖1 = z. As uk(t) and 1−uk(t) are nonnegativevariables, we have ‖uk(t)‖1 = uk(t) and ‖1− uk(t)‖1 = 1 − uk(t). We use the l1-normnotation || · ||1 in the rest of the paper to emphasize that we have applied the weightedl1-norm to approximate the l0-norm. We define vectors θ1(t) = (θk1(t), k ∈ N ) andθ2(t) = (θk2(t), k ∈ N ) in time slot t. We also define vectors θ1 = (θ1(t), t ∈ T ) andθ2 = (θ2(t), t ∈ T ). Under the given vectors θ1 and θ2, the objective function of theSCUC problem with an l1-regularization term becomesfREGobj,θ1,θ2 = fSDPobj +∑t∈T∑k∈N(θk1(t) ‖uk(t)‖1 + θk2(t) ‖1− uk(t)‖1). (3.28)68Chapter 3. Security-Constrained Unit Commitment for ac-dc GridsThe l1-regularized SCUC problem is as follows:minimizeφ∈ΦSDPfREGobj,θ1,θ2 (3.29)subject to constraints (3.26c)−(3.26f).In problem (3.29), if θk1(t) is sufficiently larger than θk2(t), then uk(t) will be equal tozero. If θk2(t) is sufficiently larger than θk1(t), then uk(t) will be equal to one. Therefore,there always exist appropriate vectors θ1 and θ2, for which the optimal solution to problem(3.29) is equal to the optimal solution to problem (3.27).Problem (3.29) is still a nonconvex optimization problem due to the rank constraints(3.26c) and (3.26d). In the next subsection, we propose the SDP relaxation form of prob-lem (3.29) and discuss the zero relaxation gap conditions. Finally, we propose an SCUCalgorithm to determine the appropriate vectors θ1 and θ2 to obtain a near-global optimalsolution to the SCUC problem (3.27) (or the original SCUC problem (3.24)).3.3.3 SDP Relaxation of the Regularized SCUC ProblemWe relax the rank constraints (3.26c) and (3.26d) in problem (3.29) and only keep con-straints (3.26e) and (3.26f). The SDP relaxation of the SCUC problem isminimizeφ∈ΦSDPfREGobj,θ1,θ2 (3.30a)subject to constraints (3.26e) and (3.26f). (3.30b)Under the given vectors θ1 and θ2, problem (3.30) is an SDP and can be solved efficiently.Let Woptθ1,θ2(t) and Ioptk,θ1,θ2(t), k ∈ N convac , t ∈ T denote the solution matrices to problem(3.30) under the given vectors θ1 and θ2. There are some difficulties in determining the69Chapter 3. Security-Constrained Unit Commitment for ac-dc Gridscorrect solution to the original SCUC problem (3.26) by solving problem (3.30). First,we need to determine vectors θ1 and θ2 leading to binary values for uk(t), k ∈ N , t ∈ T .Second, we need to obtain the conditions for zero relaxation gap between problems (3.29)and (3.30). In Theorem 3.1, we show that problem (3.30) always returns rank two solutionmatrices Ioptk,θ1,θ2(t), k ∈ N convac , t ∈ T . Hence, the rank constraint in (3.26c) is not satisfiedand the relaxation gap between problems (3.29) and (3.30) is not zero.Theorem 3.1 Under the given θ1 and θ2, the solution matrices Ioptk,θ1,θ2(t), k ∈ N convac , t ∈T to problem (3.30) are all rank two.The proof can be found in Appendix B.2. We use the trace norm regularization techniqueto obtain rank one solution matrices Ioptk,θ1,θ2(t), k ∈ N convac , t ∈ T . Minimizing the tracenorm of a matrix induces sparsity to its vector of eigenvalues, which can lead to reducingthe rank of the matrix. We use a penalty coefficient ε and introduce the penalty functionsεTr{Ik(t)}, k ∈ N convac , t ∈ T to the objective function of problem (3.30). We obtain thefollowing trace norm-regularized SCUC problem:minimizeφ∈ΦSDPfREGobj,θ1,θ2 + ε∑t∈T∑k∈N convacTr{Ik(t)} (3.31a)subject to constraints (3.26e) and (3.26f). (3.31b)Let bmax denote the maximum value for bk and Imin denote the minimum value for Imaxkamong all VSC stations. Let cmax1 , cmax2 , and PmaxG denote the maximum value for ck1, ck2,and PmaxGk among all generators, respectively. In Theorem 3.2, we provide an approximationfor the penalty coefficient ε to obtain rank one matrices Ioptk,θ1,θ2(t), k ∈ N convac , t ∈ T .Theorem 3.2 To obtain rank one solution matrices Ioptk,θ1,θ2(t), k ∈ N convac , t ∈ T to prob-70Chapter 3. Security-Constrained Unit Commitment for ac-dc Gridslem (3.31), the penalty coefficient ε can be approximated byε ≈ bmax(2cmax2 PmaxG + cmax1 + ωloss)(Imin + 1)3 (Imin)2. (3.32)The proof can be found in Appendix B.3. Note that the approximation for ε in (3.32) isdifferent from the approximation for ε in Theorem 2.3 in Chapter 2. We have obtaineda smaller penalty coefficient ε by a factor of(Imin + 1)3 (Imin)2, which is less than 1 for practicalper unit values of Imin. We can show that if the solution matrices Woptθ1,θ2(t), t ∈ T toproblem (3.31) are at most rank two, then we can construct a rank one solution matrixto problem (3.31) as well. Thus, the relaxation gap between problems (3.29) and (3.31) iszero. In Theorem 2.4 in Chapter 2, it is shown that the relaxation gap is zero for the ac-dcoptimal power flow (OPF) problem in practical ac-dc networks including the IEEE testsystems connected to some dc grids. It is also shown that the zero relaxation gap dependson the topology of the network (i.e., the admittance matrix Y ). We adopt the result inTheorem 2.4 to show that problem (3.31) returns matrices Woptθ1,θ2(t), t ∈ T with rank ofat most two under the given vectors θ1 and θ2 for practical ac-dc networks.Theorem 3.3 The solution matrices Woptθ1,θ2(t), t ∈ T to SCUC problem (3.31) are atmost rank two, if for all set of operating generators, the SDP relaxation gap for the OPFproblem in the underlying ac-dc grid is zero.The proof can be found in Appendix B.4. If the condition in Theorem 3.3 holds, thenunder the given vectors θ1 and θ2, the solution to problem (3.31) becomes a feasiblesolution to problem (3.29). However, their objective functions are not the same due tothe trace norm regularization term in the objective function of problem (3.31). Hence, theoptimal value for problem (3.31) may not be the same as the optimal value for problem(3.29). Let fREG,29obj,θ1,θ2 and fREG,31obj,θ1,θ2denote the optimal values for the objective functions of71Chapter 3. Security-Constrained Unit Commitment for ac-dc Gridsproblems (3.29) and (3.31), respectively. In Theorem 3.4, we provide an upper bound forthe difference between the optimal values fREG,29obj,θ1,θ2 and fREG,31obj,θ1,θ2.Theorem 3.4 The difference between the optimal values of problems (3.29) and (3.31) isbounded by0 ≤ fREG,31obj,θ1,θ2 − fREG,29obj,θ1,θ2 ≤ 0.45T ε∑k∈N convac(Imaxk )2 . (3.33)The proof can be found in Appendix B.5. In Section 3.4, we show that the value for ε in(3.32) is small, and we can approximate the solution to problem (3.29) by the solution toproblem (3.31). In Fig. 3.2, we summarize the steps to solve the original SCUC problem(3.24), which is equivalent to the l0-regularized SCUC problem (3.27). We formulate prob-lem (3.29) to relax the l0-norm regularization term. Next, we obtain the SDP relaxationform of SCUC problem (3.30). To guarantee the zero relaxation gap, we formulate thetrace norm-regularized SCUC problem (3.31). In the final step, we determine a (local)optimal solution of problem (3.27) by solving problem (3.31). In the following subsection,we design an iterative SCUC algorithm, where we solve problem (3.31) in each iteration toupdate vectors θi1 and θi2 until convergence to an optimal solution to problem (3.31) withbinary variables uk(t), k ∈ N , t ∈ T .3.3.4 SCUC Algorithm DesignWe propose Algorithm 3.1 based on the iterative reweighted l1-algorithm in [176] to deter-mine the solution to the SCUC problem. Let i denote the iteration index. In iteration i,we solve problem (3.31) under the given vectors θi1 and θi2 and use the solution variables72Chapter 3. Security-Constrained Unit Commitment for ac-dc GridsFigure 3.2: The procedure of solving the original SCUC problem (3.24).uopt,ik,θi1,θi2(t), k ∈ N , t ∈ T to determine the updated coefficients θi+11 and θi+12 as follows:θi+1k1 (t) =ςuopt,ik,θi1,θi2(t) + σ, (3.34a)θi+1k2 (t) =ς1− uopt,ik,θi1,θi2(t) + σ, (3.34b)where ς and σ are positive constants. If we obtain binary values for uopt,i−1k,θi−11 ,θi−12(t), k ∈N , t ∈ T in Line 6, then the algorithm returns matrices Wopt(t) := Wopt,i−1θi−11 ,θi−12(t) andIoptk (t) := Iopt,i−1k,θi−11 ,θi−12(t), k ∈ N convac , t ∈ T in Line 7. According to Theorem 3.3, for the73Chapter 3. Security-Constrained Unit Commitment for ac-dc GridsAlgorithm 3.1 ac-dc SCUC algorithm.1: Set i := 1. Initialize σ and ς. Randomly initialize θ11 and θ12. Obtain J samples of the randomvariables P̂G(t) and P̂D(t).2: Repeat3: Solve problem (3.31) under given vectors θi1 and θi2.4: Determine vectors θi+11 and θi+12 according to (3.34a) and (3.34b).5: i := i+ 16: Until variables uopt,i−1k,θi−11 ,θi−12(t) ∈ {0, 1} for k ∈ N , t ∈ T .7: Determine solution matrices Wopt(t) := Wopt,i−1θi−11 ,θi−12(t) and Ioptk (t) := Iopt,i−1k,θi−11 ,θi−12(t) for k ∈N convac , t ∈ T .8: If solution matrices Wopt(t), t ∈ T are at most rank two9: Calculate solution vectors xopt(t) and ioptk (t), k ∈ N convac , t ∈ T .coefficient ε in (3.32), matrices Ioptk (t), k ∈ N convac , t ∈ T are all rank one. If the solutionmatrices Wopt(t), t ∈ T are at most rank two, then the SDP relaxation gap is zero. In Line9, we recover the solution vectors xopt(t) and ioptk (t), k ∈ N convac , t ∈ T as follows. If matrixWopt(t) is rank two for some t ∈ T , then we calculate two nonzero eigenvalues λ1(t) andλ2(t) with corresponding eigenvectors ν1(t) and ν2(t) of each rank two matrix Wopt(t). Itcan be shown that the rank one matrix Wopt1 (t) := (λ1(t) + λ2(t))ν1(t)νT2 (t) is also thesolution of problem (3.31). We replace each rank two matrix Wopt(t) with the rank onesolution matrix Wopt1 (t), and calculate the eigenvalue λ(t) with corresponding eigenvectorν(t) of the resulting rank one matrices. We determine the solutions as xopt(t) =√λ(t)ν(t)and Ioptk (t) = ioptk (t)ioptk (t)T. If the rank of Wopt(t) is greater than two for some t ∈ T ,then the SDP relaxation gap may not be zero. One may use a heuristic method to enforcethe low-rank solution of problem (3.31) to become rank one or rank two [40].Theorem 3.5 If the solution matrices Wopt,iθi1,θi2(t), t ∈ T to problem (3.31) are at most ranktwo, then for sufficiently large parameter ς and small parameter σ, Algorithm 3.1 alwaysconverges to a local optimal solution of the original SCUC problem (3.24).The proof can be found in Appendix B.5.1. The obtained solution from Algorithm 3.1depends on the initial point of the algorithm, as well as the values of σ and ς [176], [177, Ch.74Chapter 3. Security-Constrained Unit Commitment for ac-dc Grids5], and [178]. By simulations, we show that an appropriate initialization to determine thenear-optimal solution to problem (3.24) is θ1k1(t) = θ1k2(t), k ∈ N , t ∈ T , which correspondsto the convex relaxations of the binary variables in the original SCUC problem (3.24). Thevalue of σ = 10−3 is sufficiently small to be used in Algorithm 3.1. Regarding the parameterς, the weight of l1-regularization term∑t∈T∑k∈N (θk1(t)‖uk(t)‖1 + θk2(t) ‖1− uk(t)‖1)increases in the objective function (3.28) when ς increases. Hence, Algorithm 3.1 maynot converge to a local optimal with small objective value fSDPobj . A proper value for ςcan be chosen such that the value of ς∑t∈T∑k∈N (θk1(t)‖uk(t)‖1 + θk2(t) ‖1− uk(t)‖1) isaround the value of fSDPobj . The value of ς∑t∈T∑k∈N(θk1(t)‖uk(t)‖1+θk2(t) ‖1− uk(t)‖1)isapproximately equal to ς∑t∈T∑k∈N (‖uk(t)‖0 + ‖1− uk(t)‖0). The value of ‖uk(t)‖0 +‖1− uk(t)‖0 is equal to 1 if uk(t) is binary. Otherwise, we have ‖uk(t)‖0 +‖1− uk(t)‖0 = 2.Hence, the value of ς∑t∈T∑k∈N (‖uk(t)‖0 + ‖1− uk(t)‖0) is at most 2 ς T |N |. We canapproximate the value of fSDPobj from the solution of the first iteration of Algorithm 3.1.Thus, we can choose ς such that 2 ς T |N | is greater than or equal to the value of fSDPobjin the first iteration of Algorithm 3.1. Note that if we consider the binary variable uk(t)only for the buses with conventional generators, then |N | is replaced by the number ofconventional generators in the power network.3.4 Performance EvaluationIn this section, we evaluate the performance of Algorithm 3.1 in solving the SCUC problem.3.4.1 Simulation SetupThe test system is shown in Fig. 3.3, which is an IEEE 30-bus test system connectedto three wind farms in buses 14 and 30, two PV panels in buses 3 and 7, and one dcmicrogrid in bus 28. The data for the IEEE 30-bus test system is from [154]. The base75Chapter 3. Security-Constrained Unit Commitment for ac-dc GridsFigure 3.3: An IEEE 30-bus test system connected to three wind farms in buses 14 and30, two PV panels in buses 3 and 7, and one dc microgrid in bus 28.power of the system is 100 MVA. The generators’ specifications are given in Table 3.1.The coefficients ck0, ck1, and ck2, k ∈ N for the generation cost function in (3.9) of theconventional generators can be found in [154]. The marginal cost of the reserve units isset to cres,k = 200 $/pu. The resistance of the high voltage direct current (HVDC) lines is0.06 pu. The resistance of the dc cables is 0.001 pu. The maximum apparent power flowthrough the HVDC lines and other transmission lines is 1.1 pu. The data for the VSCstations are given in Table 3.2. The base power of the VSC station is 100 MVA. To obtainthe samples for the load demands and the output power of the wind farms and PV panels,76Chapter 3. Security-Constrained Unit Commitment for ac-dc GridsTable 3.1: Generators’ specifications.k csuk ($) csdk ($) ruk (pu) rdk (pu) rsuk (pu) rsdk (pu) tuk (pu) rdk (pu)1 1000 100 0.5 0.5 0.7 0.7 3 32 1500 200 0.7 0.7 0.7 0.7 1 15 1000 100 0.5 0.5 0.7 0.7 3 38 1500 200 0.7 0.7 0.7 0.7 1 111 1000 100 0.5 0.5 0.7 0.7 1 113 1500 200 0.5 0.5 0.7 0.7 1 1Table 3.2: VSC station parameters with ac bus k, dc bus s, and filter bus f .VSC parameters (pu)RTk = 0.0005 XTk = 0.0125 Bf = 0.2 SnomCk= 1RCk = 0.00025 XCk = 0.04 Vmaxk = 1.06 Imaxk = 1.0526mqk = 0.5 mvks = 1.1VSC losses data (pu)ak = 0.0053 bk = 0.0037 ck = 0.0018we use the historical available data from Ontario, Canada power grid database [179] fromJune 1 to August 1, 2016. For each bus, we scale the available historical data such thatthe mean value is equal to the load demand given in [154] for that bus. We consider thefixed power factor for the loads. Fig. 3.4 (a) shows the average overall load demand of allbuses over 24 hours. For each renewable generator, we scale down the available historicaldata such that the maximum output power of each wind farm and PV panel over thehistorical data is equal to 20 MW and 15 MW, respectively. Figs. 3.4 (b) and (c) show theaverage output power of the PV panels and wind farms over 24 hours, respectively. Eachrenewable generator is equipped with a battery energy storage system with capacity of 4MW, maximum charging/discharging rate of 1 MW, and initial energy level of 2 MW. Forthe underlying test system, the base power is 100 MVA and the value of ck1, k ∈ N is 20$/MW [154]. Hence, the the weight coefficients ωloss is set to 2× 103 $/pu. Furthermore,we have cres,k = 200 $/pu, and thus ωcvar is set to 100 × (ck1/cres,k) = 10. Unless statedotherwise, β in (3.22) is set to 0.9. We assume that the conventional generators are turned77Chapter 3. Security-Constrained Unit Commitment for ac-dc Grids 1 4 8 16 20 2415020025030035040012Time (hour) Aggregate/oad'emand (MW)(a) 1 4 8 12 16 20 24101520Time (hour)Average2utput3ower 1 4 8 12 16 20 240510Time (hour)Average2utput3ower(b)(c)RI393DQHO0:RI:LQG)DUP0:Figure 3.4: (a) Aggregate load demand, (b) average output power of the PV panels, (c)average output power of the wind farms over 24 hours.on in all time slots before the planning horizon. For the benchmark scenario, we considerthe test system without renewable generators and batteries and with the average loadprofiles in each bus. We perform simulations using MATLAB/CVX with MOSEK solverin a PC with processor Intel(R) Core(TM) i7-3770K CPU@3.5 GHz.3.4.2 Evaluating SDP Relaxation GapTo evaluate the proposed SDP relaxation technique, we study the results of Theorems3.1 to 3.4. We first solve problem (3.30) for different coefficients θ1 and θ2. All solutionmatrices Ioptk,θ1,θ2(t), k ∈ N convac , t ∈ T becomes rank two, which confirms the result ofTheorem 3.1. Then, we use the result of Theorem 3.2 to compute the appropriate value for78Chapter 3. Security-Constrained Unit Commitment for ac-dc Gridsthe penalty factor ε. For our case study, we have ωloss = 2000, bmax = 0.0037 pu, cmax1 = 40$/MW pu, cmax2 = 0.25 $/MW2, Imin = 1.0526 pu and PmaxG = 180 MW. Hence from (??),we have ε = 4.86. We solve problem (A.29) with ε = 4.86 for several coefficients θ1 and θ2.We obtain rank one solution matrices Ioptk,θ1,θ2(t), k ∈ N convac , t ∈ T and rank two solutionmatrices Woptθ1,θ2(t), t ∈ T . Thus, the relaxation gap between problems (3.29) and (3.31)is zero. This confirms the result of Theorem 3.3, as the SDP relaxation gap is zero forac-dc OPF in the modified IEEE test systems. The optimal value of problem (3.31) is$601,508.1. According to the upper bound in (3.33) in Theorem 3.4, fREG,31obj,θ1,θ2 − fREG,29obj,θ1,θ2is at most $407.085, which is about 0.068% of the optimal value of problem (3.31). Thus,we can approximate the solution to problem (3.29) by the solution to problem (3.31), andthe approximation is tight.3.4.3 Performance of Algorithm 3.1We use Algorithm 3.1 (with nonlinear ac power flow model) to evaluate the solutions to theSCUC problem for the benchmark ac grid without renewable generator and the ac-dc gridwith renewable generators. Algorithm 3.1 guarantees to return a local optimal solutionof the SCUC problem (3.31). Our goal is to demonstrate that Algorithm 3.1 most oftenconverges to a near-global optimal solution. As it is mentioned in Section 3.3.4, σ = 10−3is sufficiently small to be used in Algorithm 3.1. There are five conventional generators, Tis equal to 24, and the value of fSDPobj in the first iteration is $213,182. Hence, parameter ςin the update rules (3.34a) and (3.34b) is set to 103, which is greater than fSDPobj /240.We run Algorithm 3.1 in both cases for 1000 randomly chosen initial weight coefficientsθ1k1 and θ1k2, k ∈ N , t ∈ T from the interval [0, 50]. For both case studies, the smallestobtained objective value corresponds to the global optimal solution. For both cases andfor all initial conditions, Algorithm 3.1 returns a near-global optimal solution within 2%79Chapter 3. Security-Constrained Unit Commitment for ac-dc Gridsgap from the global optimal solution. Moreover, for both case studies, Algorithm 3.1returns the near-global optimal solution within 1% gap in 98% of the initial conditions.We emphasize that such a result for the gap from the global optimal solution is onlyvalid for the underlying test cases, and in general, we cannot guarantee a specific gap.Nevertheless, Theorem 3.5 guarantees the convergence of Algorithm 3.1 to a local optimalsolution. Algorithm 3.1 returns the global optimal solution when θk1 and θk2, k ∈ N , t ∈ Tare (approximately) equal. In fact, the regularization term in (3.28) is a constant whenθk1 = θk2, k ∈ N , t ∈ T . Thus, in the first iteration, the binary variables uk(t), sk(t), anddk(t), k ∈ N , t ∈ T are relaxed to take any value in the interval [0, 1]. Hence, the optimalvalue of problem (3.31) in iteration 1 becomes the lower bound for the global optimalsolution of the original SCUC problem. In our case studies, when Algorithm 3.1 startsfrom the lower bound for the global optimal solution, it converges to the global optimalsolution. The number of iterations is between 5 to 12 and the running time is 35 seconds.We consider the global optimal solutions to the benchmark ac grid without renewablegenerator and the ac-dc grid with renewable generators. The output power of all conven-tional generators are reduced in most of the time slots in Fig. 3.5. The generation costand system losses are given in Table 3.3. The generation cost is lower by 23.4% in a systemwith renewable generators. However, the risk of using renewable generators is $10,430, i.e.,6.2% of the generation cost. The system losses in the grid without renewable generators aredue to the losses on the transmission lines and are equal to 191.994 MW. Using the renew-able generators, the total system losses become 188.577 MW, which include 73.177 MWof losses on the transmission lines and 115.4 MW of losses on the VSC stations. Hence,using the renewable generators can reduce the losses on the transmission lines by about60%. However, the VSC losses can add up to a significant fraction of total losses (65% ofthe total losses) and have to be included in the SCUC problem.80Chapter 3. Security-Constrained Unit Commitment for ac-dc GridsTime (hour) 1 4 8 12 16 20 24ActivePower(MW)050100150200Time (hour) 1 4 8 12 16 20 2402550Time (hour) 1 4 8 12 16 20 240255075100Time (hour) 1 4 8 12 16 20 24ActivePower(MW)02550Time (hour) 1 4 8 12 16 20 2402550Time (hour) 1 4 8 12 16 20 2402550Bus 2Bus 1 Bus 5Bus 8 Bus 13Bus 11:LWKRXW5HQHZDEOH*HQHUDWRU :LWK5HQHZDEOH*HQHUDWRUVFigure 3.5: The output active power of the generators for the grid without renewableenergy generators and the grid with renewable energy generators.Table 3.3: The generation cost, system losses, and CVaR obtained for the test systemswith and without renewable generators.Case study Generation cost ($) Total losses (MW) Converter losses (MW) CVaR ($)Without renewablegenerator217,620.1 191.944 0 0With renewablegenerator166,570.8 188.577 115.4 10,430Next, we apply Algorithm 3.1 to evaluate the solutions to the SCUC problem for thebenchmark ac grid without renewable generator in the following case studies by using (i) thedc power flow equations, (ii) the linearized ac power flow equations, and (iii) the nonlinearfull ac power flow equations (as in problems (3.29) and (3.31)). The dc and linearizedac power flow equations are commonly used in the literature to solve the SCUC problem[8, 59, 60, 61, 62, 65, 66, 67, 68, 69, 70]. For these case studies, we apply the proposediterative reweighted l1-norm approximation algorithm to obtain the binary solution of theSCUC problem. Fig. 3.6 shows the output active power of the generators. By using the dc81Chapter 3. Security-Constrained Unit Commitment for ac-dc GridsTime (hour) 1 4 8 12 16 20 24ActivePower(MW)050100150200250Time (hour) 1 4 8 12 16 20 2402550Time (hour) 1 4 8 12 16 20 240255075100Time (hour) 1 4 8 12 16 20 24ActivePower(MW)0255075100Time (hour) 1 4 8 12 16 20 240255075100Time (hour) 1 4 8 12 16 20 2402550Bus 8Bus 5Bus 2Bus 1Bus 11 Bus 13Nonlinear )XOOac Power Flow Linearized ac Power Flow dc Power FlowFigure 3.6: The output active power of the generators for the scenarios with nonlinear acpower flow, linearized ac power flow, and dc power flow.power flow model, the reactive power flow is ignored and the voltage magnitudes are set tobe 1 pu. Generators in buses 5 and 8 are turned off over the operation cycle due to theirhigh generation cost. The total generation cost is $212,281. However, with the obtainedoutput active power profiles for the generators, no feasible solution for the ac power flowproblem can be obtained. That is, the solution of the SCUC problem with dc power flowis infeasible. By using the linearized ac power flow model, the reactive power flow is takeninto account. Generators in buses 5 and 8 are turned on during peak load periods (timeslots 8 to 21). The generation cost is $216,480.3. In this model, the active power losses areset to be zero. When we solve the ac power flow problem with the obtained output powerprofiles for the generators, the active power losses are 282.210 MW, and hence the objectivevalue is $222,124.5. By using the nonlinear full ac power flow model, the contribution of thegenerator in bus 5 in supplying the load demand increases during peak load period. Thisgenerator helps to compensate the active power losses. The generation cost is $217,620.1and total losses are 191.944 MW. The objective value is $221,458.9, which is smaller than82Chapter 3. Security-Constrained Unit Commitment for ac-dc Gridsthe objective value of the scenario with linearized ac power flow model. The comparisonof these scenarios shows that the generators’ schedule obtained from the dc power flowmodel can be infeasible. The generators’ schedule obtained from the linearized ac powerflow models can deviate from the actual optimal schedule obtained from the full ac powerflow model. This results in a 0.3% gap from the optimal value in our case study. Theseresults justify the usage of nonlinear full ac model in the SCUC problem formulation.For the purpose of comparison, we use the multi-stage optimization technique (e.g.,in [8, 60, 61]) to solve the deterministic SCUC problem in an ac grid without generationand load uncertainty. We perform simulations on six test systems shown in Table 3.4.The data for the test systems can be found in [154] and [155]. We randomly assign thespecifications in Table 3.1 to the generators. We use the average load profiles over 24 hoursin each bus. For IEEE 14-bus and 30-bus test systems, we set ωloss = 2 × 103 $/pu. ForOther four test systems, we set ωloss = 2 × 105 $/pu, to sufficiently increase the weightof power losses in the objective function. The multi-stage optimization technique involvesiterative procedure between the master problem and the sub-problems. The linearized acpower flow model, which is lossless, is used [8]. The objective of the master problem is tominimize the grid-wide generation cost. The constraints include the system power balanceand operation constraints of the generators. The master problem can be formulated as anMIP. We use CPLEX 12.6 as the MIP solver to solve the unit commitment in the masterproblem [60, 61]. In the sub-problem, the network evaluation is performed by consideringthe constraints for the real and reactive nodal power balances, transmission lines flowlimits, and bus voltage limits. We apply the Benders cut method and formulate a linearprogram for checking the network constraints. Benders cuts are introduced to the masterproblem of the next iteration for the violated network constraints in order to adjust thegenerators’ schedule [60]. In Table 3.4, we compare the average CPU time and the lowest83Chapter 3. Security-Constrained Unit Commitment for ac-dc GridsTable 3.4: The optimal value and average CPU time for the deterministic multi-stagealgorithm and our proposed algorithm.Our proposed algorithm Multi-stage algorithmTest system fSDP,optobj ($) CPU time (s) fSDP,optobj ($) CPU time (s)IEEE 14-bus 207,550.1 9 216,826.2 22IEEE 30-bus 221,458.9 35 230,317.3 81IEEE 118-bus 3,372,717.8 225 3,627,233.8 395IEEE 300-bus 21,510,112.1 520 22,277,870.5 905Polish 2383wp 45,933,712.9 4,450 49,552,733.3 6,000Polish 3012wp 63,991,443.8 6,700 72,188,966.1 8,950objective value among 10 runs with different initial conditions. MIP has a low convergencespeed in large networks. Furthermore, in the multi-stage algorithm, the unit commitmentdecisions from the first stage are fixed in the second stage. Thus, it is not guarantee toconverge to a good local optimal. Whereas, Algorithm 3.1 is based on the convex relaxationmethod and the global optimality of the solution in each iteration of Algorithm 3.1 leadsto converging to a near-global optimal solution. It is noteworthy that we use the sparsematrix operations in Matlab to increase the convergence speed and efficient use of memory.3.4.4 Addressing the Net Power Supply UncertaintyWe study the impact of confidence level β on the net power supply variations. We considerthe probability distribution Pr{Cres(P optG (t), P̂G(t),PoptD (t), P̂D(t))= Cˆres} for all Cˆres ≥ 0under the given solution vectors P optG (t) and PoptD (t) to problem (3.31).Such a probability distribution can be approximated by computing the value of Cres(·)from the available historical data for the load demand and output power of the renew-able generators. Fig. 3.7 shows the probability distribution function of Cres(·) for β ={0.3, 0.6, 0.9}. When β increases, the lower values of Cres(·) will have higher probabilities.84Chapter 3. Security-Constrained Unit Commitment for ac-dc GridsThat is, the system operator uses the conventional generators instead of the renewablegenerators, and considers higher values for the load demand to limit the risk of shortagein the net supply. Hence, the value of CVaR decreases when β increases.Finally, we compare the performance of Algorithm 3.1 with the multi-stage robustoptimization technique (e.g., in [68, 69, 70]) in solving the stochastic SCUC problem (3.24)in an IEEE 300-bus test system connected to five PV panels, five off-shore wind farms,and five dc microgrids (with the same characteristics given in our case study) in differentbuses. We apply the proposed two-level algorithm in [70]. The outer level employs aBenders decomposition algorithm to obtain optimal commitment decision using the resultsfrom the inner level, which approximately solves the bilinear optimization problem usingan outer approximation algorithm. We consider the penalty for the shortage in the netpower supply and assume that the uncertainty set in time slot t ∈ T is a polyhedral withparameter ∆t that takes values between 0 and the number of buses with load or generationuncertainty. As ∆t increases, a larger total deviation from the presumed net power supplyis considered. In our case study, 208 buses have uncertainty. Thus, we can set ∆t = 208 δtfor t ∈ T , where δt can take any value in the interval [0, 1]. δt = 0 corresponds to the leastconservative case study, in which the uncertainty set only includes the maximum possiblenet power supply in the historical data record. The value δt = 1 corresponds to the mostconservative case study that takes into account all possible deviations in the load demandand generation. Hence, δt = 0 and δt = 1 correspond to the scenarios with β = 0 andβ = 1 in our proposed algorithm, respectively. We set cres,k = 2000 $/MW, k ∈ N andωcvar = 1 in Algorithm 3.1 and compare the smallest objective value among 10 runs withdifferent initial conditions. Fig. 3.8 shows that the optimal value for different values ofparameter β with Algorithm 3.1 is smaller than the optimal value for different δt, t ∈ Twith the multi-stage robust algorithm. Two reasons can be given. First, our proposed85Chapter 3. Security-Constrained Unit Commitment for ac-dc Grids0 5 10 15 20 25 30 35 40 4500.0050.010.0150.02Cˆres ($)Pr{Cres(·)=Cˆres}β= β= β= ×103$7B3$7B3$7B3Figure 3.7: The probability distribution function of Cres(·) and the value of CVaR forβ = 0.3, 0.6, and 0.9.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 119202122Objective Value (M$)23Multi-stage 5obust $lgorithmProposed $lgorithmFigure 3.8: The objective value for different values of parameter β with Algorithm 3.1and δt, t ∈ T with the multi-stage robust algorithm in [70].algorithm generally returns an optimal solution with a smaller gap from the global optimalsolution. However, the proposed multi-stage robust algorithm in [68] is based on Bendersdecomposition technique and may not return a near-global optimal solution. Second, theCVaR term takes into account the probability distribution of the scenarios. Whereas,in the robust optimization technique, the worst-case scenario in the uncertainty set isconsidered. Due to the long tail of the probability distribution of Cres(·) (as shown in Fig.3.7), the scenarios in the uncertainty set can have a relatively small probability. Thus, themulti-stage robust algorithm returns a conservative solution with a larger objective value.86Chapter 3. Security-Constrained Unit Commitment for ac-dc Grids3.5 SummaryIn this chapter, we studied the SCUC problem for ac-dc grids. The uncertainty in theload demand and renewable generation was addressed by introducing a penalty based onCVaR in the objective function to limit the risk of deviations in the load demand andrenewable generation. The SCUC problem was a nonlinear mixed-integer optimizationproblem. We used l0-norm to model the constraints with binary variables, and then appliedl1-norm relaxation to obtain a problem with continuous variables. Finally, we used convexrelaxation techniques to obtain the SDP form of the problem. An algorithm based on theiterative reweighted l1-norm approximation was proposed to determine the local optimalsolution to the original problem. Simulation results on a modified IEEE 30-bus test systemconfirmed that the proposed algorithm with different initial conditions returns the solutionwith at most 2% gap from the global optimal solution for the underlying test system. Whencompared with the multi-stage algorithm in the literature, our proposed algorithm returneda solution with lower gap from the global optimal solution in a lower CPU time.87Chapter 4An Online Learning Algorithm forDemand Response in Smart Grid4.1 IntroductionIn Chapter 2, we discussed the operation of transmission system by studying the ac-dcOPF problem. In Chapter 3, we focused on the transmission system planning by studyingthe SCUC problem. The OPF and SCUC analyses can be used for energy managementin the transmission system. In this chapter, we study the role of customer side in energymanagement activities. Specifically, we focus on designing a load scheduling learning (LSL)algorithm for multiple residential users, who schedule their appliances in response to RTPinformation. Each user is aware that the total energy consumption (not just his own)will affect the price announced by the utility company. Furthermore, each user is selfishand aims to minimize his own bill payment. We study the long-term interactions amongforesighted users instead of the short-term interactions among myopic users. It enablesus to model the users’ decision making with uncertainty about the price information andload demand of their appliances as a Markov decision process. We capture the interactionsamong users as a stochastic game [141]. In the demand response program, each user onlyobserves his own state and is uncertain about other users’ states. Hence, the underlyinggame is partially observable [142, 143, 144]. The key challenge in our model is to character-ize the MPE under the partial observability of each user and the interdependency among88Chapter 4. An Online Learning Algorithm for Demand Response in Smart Gridthe users’ policies. The contributions of this chapter are as follows:• Novel Solution Approach: The partially observable stochastic game is a realisticframework to model the interactions among users, but it is difficult to solve. To makethe problem tractable, we propose an algorithm executed by each user to approximatethe state of all users using some additional information from the utility company. Itenables us to approximate the users’ optimal policy by the MPE policy in a fullyobservable stochastic game with incomplete information, which is more tractable.• Learning Algorithm Design: We formulate an individual optimization problem foreach household, its global optimal solution corresponds to the MPE policy of theproposed fully observable stochastic game with incomplete information. We developan actor-critic method [120, 121, 122, 123]-based distributed LSL algorithm thatconverges to the MPE policy. The algorithm is online and model-free, which en-ables users to learn from the consequences of their past decisions and schedule theirappliances in an online fashion without knowing the system dynamics.• Performance Evaluation: We evaluate the performance of the LSL algorithm in re-ducing the PAR of the aggregate load and the expected cost of users. Comparedwith the benchmark of not performing demand response, our results show that theLSL algorithm can reduce the PAR of the aggregate load and the expected cost offoresighted users by 13% and 28%, respectively. We compare the policy of the fore-sighted and myopic users, and show that foresighted users can reduce their daily costby 17%. When compared with the Q-learning method (e.g., in [105] and [106]), theLSL algorithm based on the actor-critic method converges faster to the MPE policy.The rest of the chapter is organized as follows. Section 4.2 introduces the systemmodel. In Section 4.3, we model the interactions among users as a partially observable89Chapter 4. An Online Learning Algorithm for Demand Response in Smart Gridstochastic game and approximate it by a fully observable stochastic game with incompleteinformation. In Section 4.4, we develop a distributed learning algorithm to compute theMPE. In Section 4.5, we evaluate the performance of the proposed algorithm throughsimulations. A summary is given in Section 4.6. The proofs can be found in AppendicesC.1−C.4.4.2 System ModelWe consider a system with one utility company and a set N ={1, . . . , N} of N households.Each household is equipped with an ECC responsible for scheduling the appliances in thathousehold. The ECC is connected to the utility company via a two-way communicationnetwork, which enables the exchange of the price information and the household’s loaddemand. Users participate in demand response program for a long period of time (e.g.,several weeks). We divide the time into a set T = {1, . . . , T} of T equal time slots, e.g., 15minutes per time slot. In this chapter, we use ECC, household, and user interchangeably.4.2.1 Appliances ModelLet Ai = {1, . . . , Ai} denote the set of appliances in household i ∈ N , where Ai is the totalnumber of appliances. In each time slot, an appliance is either awake or asleep, indicatingwhether it is ready to operate or not. We define the appliance’s operation state as follows:Definition 4.1 (Appliance Operation State): For household i ∈ N , the operation stateof appliance a ∈ Ai in time slot t ∈ T is a tuple sa,i,t = (ra,i,t, qa,i,t, δa,i,t), where ra,i,t is thenumber of remaining time slots to complete the current task, qa,i,t is the number of timeslots for which the current task can be delayed, and δa,i,t is the number of time slots sincethe most recent time slot that appliance a becomes awake with the most recent new task.90Chapter 4. An Online Learning Algorithm for Demand Response in Smart GridFig. 4.1 shows the values of ra,i,t, qa,i,t, and δa,i,t for appliance a ∈ Ai, which has a taskthat should be operated for three time slots with a maximum delay of three time slots.When appliance a becomes awake in time slot t, ra,i,t and qa,i,t are initialized based onthe current task (e.g., here we have ra,i,t = qa,i,t = 3), and δa,i,t is set to 1. The value ofra,i,t decreases when appliance a executes its task and becomes 0 when the appliance hascompleted its task and is asleep in time slot t. The value of qa,i,t remains unchanged whenthe task is executed, and decreases when the task is delayed. When qa,i,t is 0, the ECCcannot delay the appliance’s task. The value of δa,i,t increases in each time slot and is resetto 1 when appliance a becomes awake with a new task. The appliance may start a newtask right after completing the current task. Thus, without becoming asleep, ra,i,t and qa,i,tare initialized based on the new task, and δa,i,t is set to 1.ECC i does not know when an appliance becomes awake ahead of time. Instead, ithas a belief regarding Pa,i(δa,i,t), the probability that the difference between two sequentialwake-up times for appliance a is δa,i,t, for δa,i,t ≥ 1. Such a probability distribution can beestimated, for example, based on the awake history for appliance a. ECC i can approximatePa,i(δa,i,t) by the ratio of the events that the difference between two consecutive wake-uptimes is δa,i,t in a given historical data record. Appliance a may become awake in the nexttime slot (for a new task) if either appliance a is asleep or it will complete the currenttask in the current time slot. In Appendix C.1, we show that given current time t, theprobability Pa,i,t+1 that appliance a ∈ Ai becomes awake with a new task in the next timeslot t+ 1 ∈ T isPa,i,t+1 =Pa,i(δa,i,t)1−∑δa,i,t−1∆=1 Pa,i(∆) . (4.1)We partition the set of appliances into must-run and controllable. LetAMi denote the setof must-run appliances in household i. Examples of must-run appliances include lighting91Chapter 4. An Online Learning Algorithm for Demand Response in Smart GridFigure 4.1: The values of ra,i,t, qa,i,t, and δa,i,t for appliance a, which should be operatedfor three time slots with a maximum delay of three time slots.and TV. The ECC has no control over the operation of must-run appliances. On the otherhand, the ECC can control the time of use for the controllable appliances. The set ofcontrollable appliances in household i can further be partitioned into two sets: the set ANiof non-interruptible appliances, and the set AIi of interruptible appliances. Examples ofnon-interruptible appliances include washing machine and dish washer, and examples ofinterruptible appliances include air conditioner and EV. The ECC may schedule a non-interruptible appliance during several consecutive time slots, but cannot interrupt its task.The ECC may delay or interrupt the operation of an interruptible appliance.Each time an appliance a ∈ Ai becomes awake, it sends information about its newtask’s specifications to the ECC i.Definition 4.2 (Task’s Specifications): For an appliance a ∈ Ai, the specifications ofits task include the average power consumption pavga,i to execute the task, the schedulingwindow Ta,i = [tsa,i, tda,i] corresponding to a time interval which includes the earliest starttime tsa,i ∈ T and the deadline tda,i ∈ T for the task, the operation duration da,i for amust-run or non-interruptible appliance corresponding to the total number of time slotsrequired to complete the task, and the interval [dmina,i , dmaxa,i ] for an interruptible appliancecorresponding to the range of the operation duration.The value of the average power consumption pavga,i is assumed to be fixed and known apriori for each appliance a. The operation duration da,i for a non-interruptible appliance92Chapter 4. An Online Learning Algorithm for Demand Response in Smart Grida ∈ ANi is fixed. On the other hand, the operation duration da,i for a task of an interruptibleappliance a ∈ AIi can be any value in the range of [dmina,i , dmaxa,i ], and we have dmina,i ≥ 0 anddmaxa,i ≤ tda,i − tsa,i.We use the binary decision variable xa,i,t ∈ {0, 1} to indicate whether an appliancea ∈ Ai is scheduled to operate in time slot t (xa,i,t = 1) or not (xa,i,t = 0). Notice thatxa,i,t is equal to 0 when appliance a is asleep (i.e., ra,i,t = 0). Let xi,t = (xa,i,t, a ∈ Ai)denote the scheduling decision vector for all appliances in household i in time slot t.ECC i can infer the state sa,i,t+1 of appliance a in the next time slot t + 1 from thecurrent state sa,i,t, the probability Pa,i,t+1, appliance’s type, the task’s specifications, andthe scheduling decision xa,i,t as follows:Must-run AppliancesThe feasible action for appliance a ∈ AMi in time slot t ∈ T isxa,i,t = 1, if ra,i,t ≥ 1,0, if ra,i,t = 0. (4.2)When appliance a ∈ AMi becomes awake with a new task, ra,i,t is set to da,i, and ECCi operates the appliance without delay, i.e., qa,i,t is equal to 0. Given current time t, theoperation state in time slot t+ 1 can be obtained as follows:• If either appliance a ∈ AMi is asleep (i.e., ra,i,t = 0) or it will complete its task in thecurrent time slot (i.e., ra,i,t = 1), then appliance a becomes awake in time slot t + 1with probability Pa,i,t+1, with the corresponding next state assa,i,t+1 = (da,i, 0, 1), (4.3)93Chapter 4. An Online Learning Algorithm for Demand Response in Smart Gridand the appliance is asleep in time slot t + 1 with probability 1 − Pa,i,t+1, with thecorresponding next state assa,i,t+1 = (0, 0, δa,i,t + 1). (4.4)• If ra,i,t ≥ 2, then appliance a ∈ AMi has not completed its task yet. With probability1, the corresponding next state assa,i,t+1 = (ra,i,t − 1, 0, δa,i,t + 1). (4.5)Non-interruptible Controllable AppliancesThe feasible action for appliance a ∈ ANi in time slot t ∈ T isxa,i,t =0 or 1, if t ∈ Ta,i, ra,i,t ≥ 1, qa,i,t ≥ 1,1, if t ∈ Ta,i, ra,i,t ≥ 1, qa,i,t = 0,0, if ra,i,t = 0.(4.6)Equation (4.6) implies that ECC i can decide to operate a non-interruptible appliance a ornot when the appliance is awake (ra,i,t ≥ 1) and its current task can be delayed (qa,i,t ≥ 1).ECC i has to operate an awake appliance if the task cannot be delayed (qa,i,t=0). ECC iwill not schedule appliance a if it is asleep (ra,i,t = 0).When appliance a ∈ ANi becomes awake, ra,i,t and qa,i,t are set to da,i and tda,i − tsa,i −da,i + 1, respectively. Given current time t, the operation state in the next time slot is asfollows:• If either appliance a∈ANi is asleep (i.e., ra,i,t = 0) or it will complete the current taskin the current time slot (i.e., ra,i,t = 1 and xa,i,t = 1), then the appliance becomes94Chapter 4. An Online Learning Algorithm for Demand Response in Smart Gridawake in time slot t+1 with probability Pa,i,t+1, with the corresponding next state assa,i,t+1 = (da,i, tda,i − tsa,i − da,i + 1, 1), (4.7)and the appliance is asleep in time slot t + 1 with probability 1−Pa,i,t+1, with thecorresponding next state assa,i,t+1 = (0, 0, δa,i,t + 1). (4.8)• If ra,i,t ≥ 2 and xa,i,t = 1, then appliance a ∈ ANi has not completed its task yet andis scheduled in the current time slot t. The appliance cannot be delayed in the nexttime slot, i.e., qa,i,t+1 = 0. With probability 1, the corresponding next state assa,i,t+1 = (ra,i,t − 1, 0, δa,i,t + 1). (4.9)• If ra,i,t ≥ 1 and xa,i,t = 0, then appliance a ∈ ANi has not completed its task yet andis not scheduled in the current time slot t. With probability 1, we have sa,i,t+1 =(ra,i,t, qa,i,t − 1, δa,i,t + 1). The action set in (4.6) implies that xa,i,t cannot be equalto 0 if qa,i,t is 0 in time slot t.Interruptible Controllable AppliancesEquation (4.6) is the feasible action for appliance a ∈ AIi in time slot t ∈ T . When aninterruptible appliance a ∈ AIi becomes awake with a new task, ra,i,t is set to the maximumoperation duration dmaxa,i . To operate the appliance for at least dmina,i time slots, ECC i candelay the task in at most tda,i− tsa,i− dmina,i + 1 time slots. The maximum operation durationmay not be completed before the deadline within the scheduling horizon Ta,i. In this case,95Chapter 4. An Online Learning Algorithm for Demand Response in Smart Gridif t + 1 6∈ Ta,i, the interruptible appliance will become either asleep or awake with a newtask in the next time slot t+1. The operation state in the next time slot t+1 is as follows:• If the next time slot is not in the scheduling window (i.e., t + 1 6∈ Ta,i), appliancea ∈ AIi is asleep (i.e., ra,i,t = 0), or the appliance will complete its task in the currenttime slot (i.e., ra,i,t = 1 and xa,i,t = 1), then the appliance becomes awake in timeslot t+ 1 with probability Pa,i,t+1, with the next state assa,i,t+1 = (dmaxa,i , tda,i − tsa,i − dmina,i + 1, 1), (4.10)and the appliance is asleep in time slot t + 1 with probability 1 − Pa,i,t+1, with thecorresponding next state assa,i,t+1 = (0, 0, δa,i,t + 1). (4.11)• If the next time slot is in the scheduling window (i.e., t + 1 ∈ Ta,i), ra,i,t ≥ 2, andxa,i,t = 1, then appliance a ∈ AIi is scheduled in the current time slot t. The applianceis awake in the next time slot t+ 1 with probability 1, and the next state issa,i,t+1 = (ra,i,t − 1, qa,i,t, δa,i,t + 1). (4.12)• If t + 1 ∈ Ta,i, ra,i,t ≥ 1, and xa,i,t = 0, then the task of appliance a ∈ AIi is notscheduled in the current time slot t. The appliance is awake in the next time slott+ 1 with probability 1, with the corresponding next state assa,i,t+1 = (ra,i,t, qa,i,t − 1, δa,i,t + 1). (4.13)96Chapter 4. An Online Learning Algorithm for Demand Response in Smart Grid4.2.2 Pricing Scheme and Household’s CostIn a dynamic pricing scheme, the payment by each household depends on the time andtotal amount of energy consumption. Let li,t =∑a∈Ai pavga,i xa,i,t denote the aggregate loadof household i in time slot t. Let lotherst denote the aggregate background load demand ofother users in time slot t that do not participate in the demand response program. Theutility company knows lotherst at the end of time slot t. Let lt = lotherst +∑i∈N li,t denotethe aggregate load demands of all users in time slot t.We assume that the utility company uses a combination of RTP and IBR [90, 180]. Intime slot t ∈ T , the unit price λt isλt(lt)=λ1,t, if 0 ≤ lt ≤ ltht ,λ2,t, if lt > ltht ,(4.14)where λ1,t ≤ λ2,t, t ∈ T . Here, λ1,t and λ2,t are the unit price values in time slot t whenthe aggregate load is lower and higher than the threshold ltht , respectively. We define thevector of price parameters in time slot t as λt = (λ1,t, λ2,t, ltht ). The price parameters areset by the utility company according to different factors such as the time of the day, day ofthe week, wholesale market conditions, and the operation conditions of the power network.We can capture the price changes by making the following assumption:Assumption 4.1 The price parameters are generated according to a hidden Markovmodel.In each hidden state, the price parameters are generated from a probability distributionwhich is unknown to the users [181, 182]. Assumption 4.1 is consistent with many real-istic situations of price determination. For example, the price parameters λt may changeperiodically. In this case, the hidden states correspond to the time of the day, and the97Chapter 4. An Online Learning Algorithm for Demand Response in Smart Gridprice parameters vector for each hidden state is fixed. In a more general model, a hiddenstate corresponds to the time of the day and the price parameters are chosen from a knownprobability distribution (e.g., a truncated normal distribution) in each hidden state. If thisis the case, the probability distribution for each time slot can be estimated by examiningthe historical prices of the same time slot from many days [182]. In Section 4.5, we comparethe users scheduling decisions when the utility company applies the periodic and randomprice parameters, respectively.The payment of household i in time slot t is li,t λt(lt). When the ECC interruptsthe operation of the interruptible appliances, the corresponding user will experience adiscomfort cost. When an interruptible appliance a ∈ AI becomes awake, it sends theuser’s desirable operation schedule xdesa,i,t for all time slots t ∈ Ta,i and the coefficients ωa,i,t,a ∈ AIi, t ∈ Ta,i (measured in terms of $) to the ECC to reflect the user’s discomfort causedby any potential change of the operation schedule of interruptible appliance a. For eachhousehold i, we capture the discomfort cost from scheduling the interruptible appliances bythe weighted Euclidean distance between the operation schedule with demand response andthe desirable operation schedule as∑a∈AIi ωa,i,t∣∣xa,i,t − xdesa,i,t∣∣, which is also used in [183].The total cost for each household i in time slot t involves the payment and discomfortcost. That is,ci,t(lt) = li,t λt(lt) +∑a∈AIiωa,i,t∣∣xa,i,t − xdesa,i,t∣∣ . (4.15)In the long-term scheduling problem, the scheduling horizon T is a large number (e.g.,if the scheduling horizon is six months and each time slot is 15 minutes, then we haveT ≈ 17000). Thus, it is reasonable to approximate the problem with an infinite schedulinghorizon, and consider the expected discounted cost of each household i with the discount98Chapter 4. An Online Learning Algorithm for Demand Response in Smart Gridfactor β [184, pp. 150] as(1− β)∞∑t=1βt−1ci,t(li,t, l−i,t). (4.16)The parameter β in (4.16) can be used to characterize a wide range of users’ behaviour.When β is close to zero, the users are myopic, i.e., they aim to minimize their short-termcost (e.g., daily cost) without considering the consequences of their short-term policy ontheir future cost. When β is close to one, the users are foresighted, i.e., they aim to minimizetheir long-term cost. One may assume different values of β for different participating users.We assume that all users have the same value of β. In a more general future study, onemay consider the case where different users have different values of β.In the cost model (4.16) with an infinite scheduling horizon, we can consider the sta-tionary scheduling decision making that is independent of time. Specifically, the decisionmaking only depends on the price parameters and the appliance operation state in a timeslot, but is independent of time slot index t. Therefore, we can remove time index t fromthe appliances’ states, price parameters, and the household’s cost.4.3 Problem FormulationDue to privacy concerns, each household does not reveal the information about its appli-ances to other households. We haveAssumption 4.2 The ECC can only observe the operation state of the appliances in itsown household.We capture the interactions among households in demand response program as a par-tially observable stochastic game.99Chapter 4. An Online Learning Algorithm for Demand Response in Smart GridGame 4.1 Households’ Partially Observable Stochastic Game:Players : The set of households N .States : The state of household i is si = (sa,i, a ∈ Ai).Observations : The observation of household i is oi = (si,λ)∈Oi, where Oi is the setof possible observations for household i. Let o = (oi, i∈N ) ∈ O denote the observationprofile of all households, where O=∏i∈NOi. We use notations z(oi) and z(o) to denotethe value of an arbitrary parameter z in observation oi of household i and observationprofile of all households o, respectively.Actions : We define the action vector of household i in observation profile o as xi(o)=(xa,i(o), a ∈ Ai). Let x(o) = (xi(o), i ∈ N ) denote the action profile of all households.Let Xi(oi) denote the feasible action space obtained from (4.2), (4.6) for household i withobservation oi.Transition Probabilities : Given the current price parameters, Assumption 4.1 impliesthat the price parameters vector is Markovian. From Section 4.2.1, the next state of anappliance depends only on its current state and action. Thus, the transition betweenthe observations of a household is Markovian. Let Pi(o′i |oi, xi(o)) denote the transitionprobability from observation oi ∈ Oi to o′i ∈ Oi with action xi(o). It depends on theappliances wake-up probability in (4.1). Furthermore, the users have independent preferredplans of using their appliances. Hence, the states of different households are independent.The transition probability from observation o ∈ O to o′ ∈ O with action profile x(o) isP (o′ |o, x(o)) = ∏i∈N Pi(o′i |oi, xi(o)).Stationary Policies : Let pii(o,xi(o)) denote the probability of choosing a feasible actionxi(o) in observation o. Let pii(o) = (pii(o,xi(o)), xi(o) ∈ Xi(oi)) denote the probabilitydistribution over the feasible actions. We define the stationary policy for household i as thevector pii = (pii(o), o ∈ O). Let pi = (pii, i ∈ N ) denote the joint policy of all households,100Chapter 4. An Online Learning Algorithm for Demand Response in Smart Gridand pi−i denote the policy for all households except household i.Value functions : Under a given joint policy pi, the value function V pii :O→R returnsthe expected discounted cost for household i starting with observation profile o. It can beexpressed as the following Bellman equation [141]:V pii (o) = Epii(o){Qpi−ii(o,xi(o))}, ∀ o ∈ O, (4.17)where Epii(o){·} denotes the expectation over the probability distribution pii(o). FunctionQpi−ii(o,xi(o))is the Q-function for household i with action xi(o) in observation profile owhen other households’ policy is pi−i [141]. We haveQpi−ii(o,xi(o))= Epi−i(o){(1− β) ci (o,x(o)) + β∑o′∈OP (o′ |o,x(o)) V pii (o′)}. (4.18)It is computationally difficult to determine the optimal policies for the households insuch a partially observable stochastic game. In a partially observable stochastic gameamong users, each user needs to know what other users are observing in each time slot.Inspired by the works in [142, 143, 144], we propose an algorithm executed by each ECCto estimate the observation profile of all households. It enables us to study the users’optimal policy in a fully observable stochastic game with incomplete information, in whichthe households play a sequence of Bayesian games.4.3.1 Observation Profile Approximation AlgorithmTo make the analysis of Game 4.1 tractable, we propose an algorithm executed by eachECC to approximate the observation of all households using some additional information.Let oˆ denote the approximate observation profile of all households. Algorithm 4.1 describeshow ECC i obtains oˆ. ECC i sends the average load demand lavgi (oi) of all feasible actions101Chapter 4. An Online Learning Algorithm for Demand Response in Smart Gridxi(oi) ∈ Xi(oi) to the utility company. ECC i knows λ and receives the average aggregateload lavg(o) = 1N∑j∈N lavgj (oj). It approximates the observation profile o by vector oˆ =(lavg(o), λ).In Algorithm 4.1, each household receives information on the average aggregate loaddemands. Thus, the privacy of each individual household is protected. All ECCs obtain thesame approximation for an observation profile. Thus, we can consider a fully observablestochastic game with incomplete information. Under a given approximate observationprofile oˆ, the households play a Bayesian game, as each household i may have differentobservations oi, and thus different sets of feasible actions.Game 4.2 Households’ Fully Observable Stochastic Game with Incomplete Information:This game is constructed from Game 4.1 if the households define their actions and policyas follows:Actions: Let Oi(oˆ) ⊆ Oi denote the set of possible observations for household i inthe approximate observation profile oˆ. We define the set of actions for household i in theapproximate observation profile oˆ as Xˆi(oˆ) = {xi(oi) : xi(oi) ∈ Xi(oi), oi ∈ Oi(oˆ)}. Thefeasibility of an action xi(oˆ) ∈ Xˆi(oˆ) depends on the observation oi of household i.Policies: We define the stationary policy pii(oˆ,xi(oi)) as the probability of choosinga feasible action xi(oi) ∈ Xi(oi) in an approximate observation profile oˆ when the obser-vation of household i is oi ∈ Oi(oˆ). Let Pi(oi|oˆ) be the probability that household i hasobservation oi ∈ Oi(oˆ) when the approximate observation profile is oˆ. Hence, the prob-ability of choosing any action xi(oˆ) ∈ Xˆi(oˆ) is pii(oˆ,xi(oˆ)) = Pi(oi|oˆ)pii(oˆ,xi(oi)). Letpii(oˆ) = (pii(oˆ,xi(oˆ)), xi(oˆ) ∈ Xˆi(oˆ)) denote the probability distribution over the actionsfor household i in an approximate observation profile oˆ. We define the policy for householdi in Game 4.2 as the vector pii = (pii(oˆ), oˆ ∈ O).102Chapter 4. An Online Learning Algorithm for Demand Response in Smart GridAlgorithm 4.1 Executed by ECC i ∈ N .1: Communicate the average load demand lavgi (oi) for all feasible actions xi(oi) ∈ Xi(oi) to theutility company.2: Receive the average aggregate load lavg(o) from utility company.3: Approximate the observation profile by oˆ :=(lavg(o),λ).4.3.2 Markov Perfect Equilibrium (MPE) PolicyIn this subsection, we discuss how each household i determines a policy pii(oˆ) in Game4.2 for any approximate observation profiles oˆ to minimize its value function V pii (oˆ). TheMPE is a standard solution concept for the partially observable stochastic games. TheMPE corresponds to the users’ policies with Markov properties and is compatible with theassumption for the appliance model in Section 4.2.1. The MPE in Game 4.2 is defined asfollows:Definition 4.3 A policy piMPE = (piMPEi , i ∈ N ) is an MPE if for every household i ∈ Nwith a policy pii, we haveV(pii,piMPE−i )i (oˆ) ≥ V (piMPEi ,piMPE−i )i (oˆ), ∀ i ∈ N , ∀ oˆ ∈ O. (4.19)The MPE is the fixed point solution of every household’s best response policy. Householdi solves the following Bellman equations when other households’ policies are fixed:V piMPEi (oˆ) = minimizepii(oˆ)Epii(oˆ){QpiMPE−ii (oˆ,xi(oˆ))}, ∀ oˆ ∈ O. (4.20)As the following Theorem states, the existence of the MPE is guaranteed for Game 4.2.Theorem 4.1 Game 4.2 has at least one MPE in stochastic stationary policies.The proof of Theorem 4.1 can be found in Appendix C.2. The MPE is the fixed pointof N recursive problems in (4.20) for all households. Problem (4.20) implies that for103Chapter 4. An Online Learning Algorithm for Demand Response in Smart Gridhousehold i with action xi(oˆ) under observation profile oˆ in the MPE, we have VpiMPEi (oˆ) ≤QpiMPE−ii (oˆ,xi(oˆ)). We introduce an equivalent non-recursive optimization problem for eachhousehold, which is more tractable. For household i ∈ N , we define the Bellman error [141]for an action xi(oˆ) in an approximate observation profile oˆ asBi (Vpii , oˆ,xi(oˆ)) = Qpi−ii (oˆ,xi(oˆ))− V pii (oˆ). (4.21)We define function f obji (Vpii ,pii) as the sum of the expected Bellman errors for allobservations oˆ ∈ O. That isf obji (Vpii ,pii) =∑oˆ∈OEpii(oˆ){Bi (Vpii , oˆ,xi(oˆ))}. (4.22)Each household i aims to determine the policy pii and the value function Vpii to minimizef obji (Vpii ,pii) by solving the following optimization problem.minimizeV pii ,piif obji (Vpii ,pii) (4.23)subject to Bi(Vpii , oˆ,xi(oˆ)) ≥ 0, ∀ oˆ ∈ O, ∀xi(oˆ)∈ Xˆi(oˆ).Problem (4.23) is generally a non-convex problem, and may have several local minima. Weshow that the MPE policy of household i is the global minimum of problem (4.23).Theorem 4.2 The policy piMPE is an MPE of Game 4.2 if and only if for all householdsi∈N with action xi(oˆ)∈ Xˆi(oˆ), we havepiMPEi (oˆ,xi(oˆ)) Bi(V piMPEi , oˆ,xi(oˆ))= 0, ∀ oˆ ∈ O. (4.24)The proof can be found in Appendix C.3. Theorem 4.2 implies that the Bellman error is104Chapter 4. An Online Learning Algorithm for Demand Response in Smart Gridzero for an action with positive probability at the MPE. Thus, f obji (VpiMPEi ,piMPEi ) = 0 andthe MPE is the global optimal solution of problem (4.23) for all households.Solving problem (4.23) is still challenging, as each ECC requires the values of theunavailable transition probabilities between the observations. This motivates us to developa model-free learning algorithm that enables each ECC to schedule the appliances in anonline manner without knowing the system dynamics. Basically, each ECC updates thepolicy and value function based on the consequences of its past decisions.As part of the learning algorithm, we need to record the observation and action spacesfor a household. In order to reduce the complexity, we use the linear function approximationto estimate the value function [185, Ch. 3]. For household i, let φi(oˆ) = (φv,i(oˆ), v ∈ V)denote the row vector of basis functions, where V is the set of basis functions. Let θi =(θv,i, v ∈ V) denote the row vector of weight coefficients. The approximate value functionfor household i isV pii (oˆ,θi) = θiφTi (oˆ), (4.25)where T is the transpose operator. It enables ECC i to compute vector θi with |V| ele-ments instead of the value function V pii (oˆ) for all approximate observation profiles oˆ. Weparameterize the policy pii for household i via softmax approximation [185, Ch. 3]. Letµi(oˆ,xi(oˆ)) =(µp,i(oˆ,xi(oˆ)), p ∈ P)denote the row vector of basis functions, where P isthe set of basis functions. Let ϑi = (ϑp,i, p∈P) denote the row vector of weight coefficients.The approximate probability of choosing action xi(oˆ)∈Xˆi(oˆ) ispii(oˆ,xi(oˆ),ϑi)=e(ϑiµTi (oˆ,xi(oˆ)))∑x′i(oˆ)∈Xˆi(oˆ) e(ϑiµTi (oˆ,x′i(oˆ))). (4.26)To simplify the computation of this approximation, we use the vector of compatible105Chapter 4. An Online Learning Algorithm for Demand Response in Smart Gridbasis functions ψi(oˆ,xi(oˆ)) =(ψp,i(oˆ,xi(oˆ)), p ∈ P), whereψp,i(oˆ,xi(oˆ)) =∂ ln(pii(oˆ,xi(oˆ),ϑi))∂ ϑp,i. (4.27)We can show that for the softmax parameterized policy, the vector of basis functionsµi(oˆ,xi(oˆ)) can be replaced with vector ψi(oˆ,xi(oˆ)) [123].4.4 Online Learning Algorithm DesignIn this section, we propose a load scheduling learning (LSL) algorithm executed by the ECCof each household to determine the MPE policy. We use an actor-critic learning method,which is more robust than the actor-only methods (such as the policy evaluation [119, Ch.2]) and faster than the critic-only methods (such as the Q-learning and temporal difference(TD) learning [119, Ch. 6]). The concept of the actor-critic was originally introduced byWitten in [120] and then elaborated by Barto et al. in [121]. A detailed study of theactor-critic algorithm can be found in [122, 123]. Our LSL algorithm is based on the firstproposed algorithm in [123]. The ECC is responsible for the actor and critic updates. Inthe critic update, the ECC evaluates the policy to update the value function. In the actorupdate, it updates the policy to decrease the objective value of problem (4.23) based onthe updated value function. In the policy update, we use the gradient method with asmaller step size compared with the step size in the value function’s update, thereby usinga two-timescale update process [123].Algorithm 4.2 describes the LSL algorithm executed by ECC i. The index k refers toboth iteration and time slot. Our algorithm involves the initiation and scheduling phases.Line 1 describes the initialization in time slot k = 1. The loop involving Lines 2 to 14describes the scheduling phase, which includes the observation profile approximation, the106Chapter 4. An Online Learning Algorithm for Demand Response in Smart Gridcritic update, the actor update, and the basis function construction.In Line 3, ECC i executes Algorithm 4.1 to obtain the approximate observation oˆ. Intime slot k = 1, ECC i does not have any experience from its past decisions and chooses anaction in Line 11. For k > 1, the critic and actor updates are executed. ECC i determinesthe updated vector θki using the TD approach [119, Ch. 6]. The TD error ek−1TD isek−1TD = (1− β) ci(oˆk−1,xk−1(oˆk−1))+ β V pi,k−1i(oˆk, θk−1i)− V pi,k−1i (oˆk−1, θk−1i ) . (4.28)The critic update for ECC i isθki = θk−1i + γk−1c ek−1TD φi(oˆk−1), (4.29)where γkc is the critic step size in iteration k. In the actor update module, ECC i determinesthe updated vector ϑki using the gradient method with descent direction. In particular,ECC i uses the descent direction pik−1i(oˆk−1,xk−1i (oˆk−1),ϑk−1i)∇ϑk−1i f obji (V pi,k−1i ,pik−1i ) toensure convergence to the MPE. Since the gradient is not available, ECC i uses vectorek−1TD ψi(oˆk−1,xk−1i (oˆk−1)) as an estimate of the gradient [123, Algorithm 1]. Therefore, theconvergence to the MPE is guaranteed, since the TD error ek−1TD is an estimate for theBellman error for action xk−1i in iteration k − 1. Thus, the descent direction is zero ifcondition (4.24) is satisfied. The actor update for ECC i isϑki = ϑk−1i − γka pik−1i(oˆk−1,xk−1i (oˆk−1),ϑk−1i)ek−1TD ψi(oˆk−1,xk−1i (oˆk−1)), (4.30)where γka is the actor step size in iteration k. We use the approach in [186] to autonomouslyconstruct the new basis functions ψ|P|+1,i(oˆ,xi(oˆ)) and φ|V|+1,i(oˆ). The candidate for thebasis function ψ|P|+1,i(oˆ,xi(oˆ)) is the TD error ek−1TD in (4.28) (oˆk is replaced by oˆ), whichestimates the Bellman error. The expectation over the Bellman errors of the feasible actions107Chapter 4. An Online Learning Algorithm for Demand Response in Smart GridAlgorithm 4.2 LSL Algorithm Executed by ECC i ∈ N .1: Set k := 1, := 10−3, and ξ = 10−3. Set φ1,i(·) := 1 and ψ1,i(·) := 1, and randomly initializeθ11,i and ϑ11,i.2: Repeat3: Observe oki := (ski ,λk). Approximate oˆk using Algorithm 4.1.4: If k 6= 1,5: Determine the updated vector θki according to (4.29).6: Determine the updated vector ϑki according to (4.30).7: If |θki − θk−1i | < ,8: Construct new basis functions ψ|P|+1,i(oˆ,xi(oˆ)) and φ|V|+1,i(oˆ) using (4.31) and (4.32).9: End if10: End if11: Choose action xki (oˆk) using policy piki (oˆk,ϑki ).12: Receive the cost ci(oˆk,xk(oˆk))from the utility company.13: k := k + 1.14: Until ||fˆobji (V pi,k−1i ,pik−1i )|| < ξ.xi(oˆk−1) ∈ Xi(ok−1i ) is the candidate for φ|V|+1,i(oˆ). We haveψ|P|+1,i(oˆ,xi(oˆ)) = ek−1TD , (4.31)φ|V|+1,i(oˆ) = E{Bk−1i(V pi,k−1i , oˆk−1,xi(oˆk−1))}. (4.32)The expectation in (4.32) is over the probability of choosing each feasible actions xi(oˆk−1) ∈Xi(ok−1i ) for joint observation oˆ and local observation oki . In Appendix C.4, we explainhow to approximate the Bellman error for each feasible action. In Line 7, ECC i checksthe convergence of θki and decides whether to add the new basis functions or not.In Line 11, ECC i schedules the appliances in the current time slot k. In Line 12, ECCi receives the cost ci(oˆk,xk(oˆk)). Next time slot is started in Line 13. In Line 14, thestopping criterion is given. From Theorem 4.2, LSL algorithm converges to the MPE ifthe objective value f obji (Vpi,k−1i ,pik−1i ) is zero. ECC i computes the approximate objective108Chapter 4. An Online Learning Algorithm for Demand Response in Smart Gridvalue by summing over the expected Bellman errors up to iteration k − 1 asf̂ obji (Vpi,k−1i ,pik−1i ) =k−1∑j=1Epik−1i (oˆj){Bji(V pi,k−1i , oˆj,xi(oˆj))}. (4.33)The sufficient conditions for the actor and critic step sizes to ensure the convergence of theLSL algorithm are given in [122]. In the proposed model-free LSL algorithm, ECC i doesnot know the next states of the appliances until the next time slot begins in Line 13. TheECC updates its value function using the TD error in (4.28), which depends on the nexttime slot observation. Therefore, the ECC only goes through one iteration per time slot.4.5 Performance EvaluationIn this section, we evaluate the performance of the LSL algorithm in a system, whereone utility company serves 200 households that participate in the demand response pro-gram. The scheduling horizon is six months. Each time slot is 15 minutes. We con-sider six controllable appliances for each household, e.g., dish washer, washing machine,and stove are non-interruptible appliances, and EV, air conditioner, and water heaterare interruptible appliances. We model other appliances such as refrigerator and TV asmust-run appliances2. Table 4.1 summarizes the task specifications of the controllableappliances [187]. For the EV in each household i, we have dmina,i = (Bdi − B0i )/pavga,i anddmaxa,i = (Bmaxi − B0i )/pavga,i , where B0i is the initial charging level when the EV awakes, Bdiis the charging demand for the next trip, and Bmaxi is the battery’s maximum capacity.The charging demand of the EV in household i is uniformly chosen at random from theset {18 kWh, 18.75 kWh, . . . , 24 kWh}. The battery capacity is set to 30 kWh. Typically,the user is indifference between the charging patterns for the EV as long as the charging2Although the set of appliances are the same for all households, the users’ preferences in operating theappliances are different.109Chapter 4. An Online Learning Algorithm for Demand Response in Smart GridTable 4.1: Operating specifications of controllable appliances.Appliance(pavga,i , da,i, dmina,i , dmaxa,i)Dish washer (1.5 kW, 2 hr,−,−)Washing machine (2.5 kW, 3 hr,−,−)Stove (3 kW, 3 hr,−,−)EV (3 kW,−, (Bdi −B0i )/pavga,i , (Bmaxi −B0i )/pavga,i )Air conditioner (1.5 kW,−, 2 hr, 8 hr)Water heater (2.5 kW,−, 0 hr, 5 hr)is finished before the deadline. Thus, we set coefficients ωa,i,t, t ∈ T to zero for the EV.Coefficients ωa,i,t, t ∈ T are chosen uniformly at random from the interval [$0, $0.5] forthe air conditioner and water heater. We set the desired load pattern (xdesa,i,t, t ∈ T ) of theair conditioner to a 16-hour period, during which the appliance turns on for an hour andturns off in the next hour in a periodic fashion. We set the desired load pattern of thewater heater to a 5-hour period without interruption. To simulate the non-interruptibleappliances, we consider several scheduling windows selected uniformly between 10 am and10 pm, with a length that is uniformly chosen at random from set {4 hr, 5 hr, 6 hr, 7 hr}.For the washing machine, we model (Pa,i(∆), ∆ ≥ 1) as a truncated normal distributionwhich is lower bounded by zero, and has a mean value of 288 time slots and a standarddeviation of 60 time slots. For other appliances, we use a truncated normal distributionwith a mean value of 96 time slots and a standard deviation of 20 time slots. In practicalimplementations, the probability distribution (Pa,i(∆), ∆ ≥ 1) for each appliance a can beapproximated by using the historical record on the usage behaviour of each user i.Unless stated otherwise, the price parameters vary periodically with a period of oneday. As discussed in Section 4.2.2, the periodic price parameter vector is a special case forthe hidden Markov model in Assumption 4.1. Figs. 4.2 (a) and (b) show ltht , t ∈ T , and110Chapter 4. An Online Learning Algorithm for Demand Response in Smart Gridλ1,t and λ2,t, t ∈ T over one day, respectively3.The actor and critic step sizes in iteration k of the LSL algorithm are set to γka = ma/k23and γkc = mc/k, respectively. Since each ECC may use different values for ma and mc inpractice, we choose ma and mc uniformly from [0.5, 2] for each household. Unless statedotherwise, the discounted factor β is set to 0.995, i.e., the users are foresighted. Forthe benchmark scenario without demand response, the non-interruptible appliances areoperated as soon as they become awake. The air conditioner and water heater are operatedaccording to their desired load patterns. The EV starts to charge when it is plugged in. Wesimulate both the benchmark case and LSL algorithm for several scenarios using Matlabin a PC with processor Intel Core i5 3337U CPU 1.80 GHz.First, we compare the load profiles for household 1 over two days in the benchmarkscenario (without load scheduling) and the LSL algorithm (with load scheduling) in Fig.4.3 (a). The EV charging demands of household 1 in the first and second days are 6and 8 hours, respectively. With the LSL algorithm, the ECC of household 1 schedulesthe operating appliances to reduce the payment. In particular, since the peak load withscheduling in the first day is much lower than that in the second day, the ECC of theforesighted household 1 charges the EV for 8.5 hours in the first day (larger than thedemand of 6 hours in the first day), in order to reduce the charging hour to 5.5 hours inthe second day. Such a charging schedule reduces the peak load in the second day. Fig. 4.3(b) shows the aggregate load demand of all users during one sample day. The peak load isabout 1.9 MW around 8 pm without load scheduling. When the households deploy LSLalgorithm, the ECCs schedule the controllable appliances to off-peak hours at the MPE.The peak load decreases by 27% to 1.4 MW. Fig. 4.3 (c) shows the aggregate load profileof all users over one week. The peak load reduction can be observed in all days.3Parameters λ1,t and λ2,t are set to be relatively large between 12 am and 6 am in order to motivateEV charging management and prevent peak load after midnight due to high charging demand of the EVs.111Chapter 4. An Online Learning Algorithm for Demand Response in Smart Grid12 pm 6 pm 12 am 6 am 0.70.91.11.31.57 am12 pm 6 pm 12 am 6 am 204060801007amFigure 4.2: Price parameters over one day: (a) ltht ; (b) λ1,t and λ2,t.The LSL algorithm benefits the users by reducing their daily average cost. We performsimulations for β=0.995, 0.8, 0.5, 0.2, 0.05, which includes the extreme cases of foresightedusers (β = 0.995) and myopic users (β = 0.05). We present the daily average cost ofhousehold 1 for different values of β in Fig. 4.4. The initial value of $4.8 per day is thedaily average cost without load scheduling. When household 1 is foresighted, its dailyaverage cost decreases by 28% (from $4.8 per day to $3.5 per day). When β decreases, thedaily average cost increases gradually. For a myopic user, the daily average cost decreasesby 11% (from $4.8 per day to $4.3 per day). The reason is that the ECC for the foresightedusers schedules the appliances considering the price in the current and future time slots.Fig. 4.5 (a) shows the charging profile of the EV for household 1 with a myopic user. Fig.4.5 (b) shows the dynamics of electricity price over two days when the users are myopic.The ECC of the myopic user (with β = 0.05) considers the daily price fluctuations andcharges the EV just to fulfill the charging demand (for 6 hours).112Chapter 4. An Online Learning Algorithm for Demand Response in Smart Grid 6 am 12 pm 6 pm 12 am 6 am 12 pm 6 pm 12 am 6 am Time (hour)036912Load demand (kW)Without load schedulingWith load scheduling12 pm 6 pm 12 am 6 am Time (hour)00.40.81.21.62Aggregate load (MW) Without load scheduling With load scheduling Must-run load6 amDay 1 Day 2 Day 3 Day 4 Day 5 Day 6 Day 700.40.81.21.622.4Figure 4.3: (a) Load demand for household 1 over two days; (b) aggregate load demandof users over one day; (c) aggregate load demands of all users over seven days with andwithout load scheduling.113Chapter 4. An Online Learning Algorithm for Demand Response in Smart Grid0 500 1000 1500 2000 2500 30003.544.55Figure 4.4: Daily average cost for myopic and foresighted household 1.Fig. 4.5 (c) shows the charging profile of the EV for household 1 with a foresighted user.Fig. 4.5 (d) shows the dynamics of electricity price when the users are foresighted. TheECC of the foresighted user (with β = 0.995), on the other hand, takes advantage of theprice fluctuations over multiple days and charges the EV more than the current chargingdemand (for 8.5 hours) in order to reduce cost in the following day when the price in thecharging period is high.The LSL algorithm helps the utility company reduce the PAR in the aggregate loaddemand. We compute the expected PAR over a period of 2 months in Fig. 4.6. We considertwo special cases of the hidden Markov model in Assumption 4.1, i.e., the periodic andrandom price parameters, respectively, to evaluate the performance of LSL algorithm.With periodic price parameters, the LSL algorithm performs well and reduces the PARfrom 2.3 to 2.02 (13% reduction) in 3000 time slots (about a month). For random priceparameters, we assume that the utility company chooses ltht , t ∈ T from a truncated normaldistribution with a mean value shown in Fig. 4.2 (a) and a standard deviation of 0.2 MW.The parameters λ1,t and λ2,t, t ∈ T are also chosen from a truncated normal distributionwith a mean value shown in Fig. 4.2 (b) and a standard deviation of 5 $/MW. The randomprice parameters can model abnormal fluctuations (such as spikes in the price values).114Chapter 4. An Online Learning Algorithm for Demand Response in Smart Grid 6 am 12 pm 6 pm 12 am 6 am 12 pm 6 pm 12 am 6 am01234EV charging rate (kW)Myopic user6 am 12 pm 6 pm 12 am 6 am 12 pm 6 pm 12 am 6 amTime (hour)20406080100Price ($/MW)0 6 am 12 pm 6 pm 12 am 6 am 12 pm 6 pm 12 am 6 am1234EV charging rate (kW)Foresighted userc6 am 12 pm 6 pm 12 am 6 am 12 pm 6 pm 12 am 6 amTime (hour)20406080100Price ($/MW)Figure 4.5: (a) The EV’s charging schedule when household 1 is myopic (β = 0.05); (b) theelectricity price when users are myopic; (c) the EV’s charging schedule when household 1is foresighted (β = 0.995); (d) the electricity price when users are foresighted.115Chapter 4. An Online Learning Algorithm for Demand Response in Smart Grid0 1000 2000 3000 4000 5000 6000 700022.12.22.32.4Iteration numberExpectedPAR With random price parameters With periodic price parametersFigure 4.6: Expected PAR with periodic and random price parameters.In practice, the probability distributions for the price parameters can be estimated fromthe historical price data. Our LSL algorithm is model-free, hence the ECCs do not need toknow the probability distributions of the price parameters. Fig. 4.6 shows that the ECCscan still effectively determine their MPE policies through learning, but it takes 6500 timeslots for the PAR to converge to 2.07. Thus, LSL algorithm has a robust performance evenin a market with random fluctuations in the price.We show that Algorithm 4.2 converges to the MPE by using the MPE characterizationin Theorem 4.2. Fig. 4.7 depicts the absolute values of the approximate objective function||f̂ obji (V pi,ki , piki )|| for households 1, 2, and 3. It shows that the objective values converge tozero (we have the same result for other households), which is the global optimal solutionof problem (4.23). Thus from Theorem 4.2, the LSL algorithm converges to the MPE ofGame 4.2. Though the action and state spaces of each household are large, the speed ofconvergence is acceptable as a result of using the value function and policy approximations.The jumps in the curves in Fig. 4.7 correspond to the iterations where the basis functions in(4.31) and (4.32) are added to the basis function sets. In our simulation, the running timeof the LSL algorithm per iteration per household is only a few seconds. As the households116Chapter 4. An Online Learning Algorithm for Demand Response in Smart Grid0 500 1000 1500 2000 2500 300002468Iteration numberfobji(V π,ki,πk )Household 1Household 2Household 3Figure 4.7: Objective value ||f obji (V pi,ki , piki )|| for households 1, 2, and 3.only need to go through one iteration of computation per time slot (e.g., 15 mins), theproposed algorithm is suitable for real-time executions.We compare the LSL algorithm with a scheduling algorithm based on Q-learning todemonstrate the benefit of the actor-critic method. Q-learning has been used in someexisting learning algorithms for demand response (e.g., [105] and [106]). We consider analgorithm based on Q-learning with the same structure as LSL algorithm, with the onlydifference that the ECC updates the Q-functions[119, Ch. 6]. Fig. 4.8 shows the dailyaverage cost of household 1 using the LSL algorithm and the Q-learning benchmark. In eachiteration of the Q-learning benchmark, the policies are obtained from the updated valuesof the Q-functions (which is computed based on the Boltzmann exploration as in [105]). Itsuffers from high fluctuations and slow learning. Our proposed algorithm converges muchsmoother, with a total convergence time around 25% of that of the Q-learning benchmark.To study how the observation profile approximation in Algorithm 4.1 affects the users’policy, we compare the households’ policies in two scenarios. In the first scenario, thestates are partially observable to the ECCs. They will use Algorithm 4.1 to approximatethe observation profile of all households. In the second scenario, the utility company shares117Chapter 4. An Online Learning Algorithm for Demand Response in Smart Grid0 3000 6000 9000 12000 3.544.55Figure 4.8: Daily average cost for household 1 with the algorithm based on Q-learningand our proposed LSL algorithm.the state of all households with each ECC. Thus, the states become fully observable to theECCs. The LSL algorithm can be used in both scenarios to determine the MPE policy ofthe households. Fig. 4.9 shows the aggregate load demand in both scenarios over one day,with and without load scheduling. When the states are partially observable, the ECCs playa sequence of Bayesian games in Game 4.2. As each ECC has incomplete information aboutother households’ states, it determines an optimal policy that minimizes the expected costin all possible states of other households under a given approximate observation profile.When the states are fully observable, the ECCs play a sequence of normal form games. Aseach ECC knows the actual state of other households, its policy becomes the best responsefor the actual state of the system. Fig. 4.9 shows that when the states become fullyobservable, the peak in the aggregate load demand further decreases when the aggregateload is around the threshold ltht . This reduces the expected cost of the households, e.g.,the average cost of household 1 is reduced by 6.3% (from $3.5 per day to $3.28 per day).118Chapter 4. An Online Learning Algorithm for Demand Response in Smart Grid12 pm 6 pm 12 am 6 am Time (hour)00.40.81.21.62Aggregate load (MW) Without load schedulingPartially observable load schedulingFully observable load scheduling6 amFigure 4.9: The aggregate load demand with the partially observable load scheduling andfully observable load scheduling.4.6 SummaryIn this chapter, we formulated the scheduling problem of the controllable appliances inthe residential households as a partially observable stochastic game, where each householdaims at minimizing its discounted average cost in a real-time pricing market. We proposeda distributed and model-free learning algorithm based on the actor-critic method to de-termine the MPE policy. We used the value function and policy approximation techniqueto reduce the action and state spaces of the households and improve the learning speed.Simulation results showed that the expected PAR in the aggregate load can be reduced by13% when users deploy the proposed algorithm. Furthermore, the foresighted users canbenefit from 28% reduction in their expected discounted cost in long-term, which is 17%lower than the expected cost of the myopic users.119Chapter 5Demand Response for Data Centersin Deregulated Markets5.1 IntroductionAlthough residential households can play an active role in demand response programs, theymay experience small change in their daily cost. This can result in a low motivation forresidential households to participate in the demand response programs with their maximumpotential for energy management. In general, consumers with flexible large loads (e.g., datacenters) have higher potential for demand response activities, since they can have a highercontribution in load reduction during peak hours. Consumers with large loads have ahigher motivation for load management, as they can save a large amount of money. In thischapter, we study the emerging deregulated markets, where multiple utility companiescompete to supply electricity to the same group of geographically dispersed data centers.Each data center is free to enter a bilateral contract with a utility company and canschedule its workloads to minimize its bill payment. If the utility companies adopt theRTP scheme, the payments of the data centers will depend on the amount and the timeof their electricity consumption. Hence, the decisions of data centers in a deregulatedmarket are coupled among each other as well as with the pricing decisions of the utilitycompanies. The centralized problem of utility company choices and workload schedulingof data centers can be formulated as a mixed-integer nonlinear program, which is NP hard.120Chapter 5. Demand Response for Data Centers in Deregulated MarketsTo address the complexity of the centralized approach, we model the data centers’ coupleddecisions of utility company choices and workload scheduling as a many-to-one matchinggame. In our problem setting, once the utility companies have announced their pricesfor different times of the day, each data center can schedule its workloads and enter abilateral contract with a utility company. The underlying mechanism is a matching withexternalities, since the payment of the data centers choosing the same utility companydepends on the workload scheduling of each individual data center. We characterize thestable outcome of the underlying game, where no data center has an incentive to changeits utility company and workload schedule unilaterally. The main challenge for analyzing amatching game with externalities is to compute a stable outcome, since there does not exista general algorithm that can guarantee to find a stable outcome in all problem settings.The contributions of this chapter are as follows:• Data Center Workload Model : We model the workloads’ arrivals and executions in adata center by a time-dependent multiclass queuing system. Such a model provides aframework for us to compute the optimal number of active servers and the executiontime of delay-tolerant services, subject to the constraints on the waiting time of theinteractive inflexible services over the contract period.• Risk-Aware Contract : Data centers have uncertainty about their demands due to thestochastic nature of the workloads and renewable generation. We introduce the riskmeasure conditional value-at-risk (CVaR), which enables the data centers to limitthe risk of deviation in the energy demand from the contracted amount. To the bestof our knowledge, this is the first work that incorporates risk management in thedata center demand response problem.• Solution Method and Algorithm Design: We characterize an exact potential functionand show that the stable outcomes of the game correspond to the local minima of121Chapter 5. Demand Response for Data Centers in Deregulated Marketsthe potential function. One can determine a local optimal solution to the potentialfunction by solving a mixed-integer nonlinear optimization problem, which is NPhard. Instead, we develop an algorithm that can be executed by the data centersand utility companies in a distributed fashion and is provably convergent to a stableoutcome of the game.• Performance Evaluation: We perform simulations on a market with 50 data centersand 10 utility companies. When compared with the scenario without the demandresponse, our proposed algorithm reduces the cost of data centers and the peak-to-average ratio (PAR) of the aggregate demand of data centers connected to the sameutility company by 18.7% and 8%, respectively. The proposed algorithm also enablesthe utility companies to attract more data centers by setting lower energy tariffs, andas a result increase their revenue up to 80%. The computational complexity of theproposed algorithm is linear with the number of utility companies and independentof the number of data centers. This is much lower than the NP hard complexity ofthe centralized approach.The rest of the chapter is organized as follows. Section 5.2 introduces the system model. InSection 5.3, we propose a matching game model for the data centers interaction. We alsodevelop a distributed algorithm to obtain a stable outcome. In Section 5.4, we evaluate theperformance of the proposed algorithm. A summary is given in Section 5.5. The proofscan be found in Appendices D.1−D.4.5.2 System ModelConsider a system with D data centers and U utility companies. Let D = {1, . . . , D} andU = {1, . . . , U} denote the set of data centers and the set of utility companies, respectively.122Chapter 5. Demand Response for Data Centers in Deregulated MarketsData center d ∈ D can purchase electricity from a utility company in a predetermined setUd ⊆ U . Utility company u ∈ U is able to serve a predetermined set Du ⊆ D of datacenters. Sets Ud, d ∈ D, and Du, u ∈ U , are determined based on the geographic locationsof the utility companies and data centers as well as the topology of the power network.Fig. 5.1 (a) shows a system with five data centers and three utility companies. Fig. 5.1 (b)shows the corresponding bipartite graph representation. For example, utility company 1can sell electricity to data centers in set D1 = {1, 2, 3}. Data center 3 can choose a utilitycompany from set U3 = {2, 3}.Each data center d possesses an energy management system (EMS), which is connectedto the utility companies in set Ud via a two-way communication network. The EMS enablesexchanging information such as the energy consumption of the corresponding data centerand the energy price for entering a bilateral contract. In the following subsection, we modelthe bilateral contracts among data centers and utility companies.5.2.1 Bilateral Contract ModelIn deregulated markets, a data center can enter a bilteral contract with one utility companyto purchase electricity. Meanwhile, a utility company can supply electricity to multiple datacenters. We can capture the contracts between data centers and utility companies as amany-to-one matching function [188], which is defined as follows.Definition 5.1 A many-to-one matching among the data centers and utility companies isa function m : D ∪ U →P(D ∪ U), where m(u) ⊆ Du represents the set of data centersserved by utility company u ∈ U , and m(d) ⊆ Ud with |m(d)| = 1 represents the utilitycompany choice of data center d ∈ D.Here, |·| denotes the cardinality and P is the power set of a set. Fig. 5.1 (c) shows a feasiblemany-to-one matching, where a data center enters a contract with one utility company,123Chapter 5. Demand Response for Data Centers in Deregulated MarketsFigure 5.1: (a) A system composed of five data centers equipped with EMS and three utilitycompanies; (b) the corresponding bipartite graph representation; (c) a feasible many-to-onematching among the data centers and utility companies.while a utility company can enter contracts with multiple data centers. Although short-term contracts are not common for residential customers in today’s deregulated markets,large-load customers such as data centers can enter a contract with utility companies fora period from a few hours to several days [166, 189, 190]. We assume that a data centercan enter a short-term contract (e.g., one day) with a utility company. Without loss ofgenerality, we assume that the contract period of all data centers starts and finishes at thesame time. We divide the intended contract period into a set T = {1, . . . , T} of T timeslots with an equal length, e.g., 15 minutes per time slot.In matching m, utility company u ∈ U sets its retail price pru(t), t ∈ T , for the contractswith the data centers in set m(u). Meanwhile, data center d ∈ D specifies its energydemand profile ed(t), t ∈ T , to be satisfied by its utility company choice m(d). In thefollowing subsection, we discuss the contract pricing model.5.2.2 Contract Pricing ModelWe assume that a utility company purchases electricity from the wholesale market with aprice p(t), t ∈ T . The utility companies may offer RTP rates to the flexible large loads suchas data centers. A proper RTP scheme can motivate the data centers toward scheduling124Chapter 5. Demand Response for Data Centers in Deregulated Marketstheir workloads from peak periods to off-peak periods to reduce their payments. Hence,the utility companies can attract more data centers as customers by offering lower tariffs,meanwhile improving the performance of the power system during peak hours by reducingthe PAR of the aggregate energy demand.In an RTP scheme, the retail price of utility company u depends on both the volumeand time of the energy consumption of all its customers. In particular, the (unit) retailprice of utility company u ∈ U , in time slot t ∈ T , and matching m is an increasing convexfunction of the total energy demand eu(t) = eotheru (t)+∑d∈m(u) ed(t), where eotheru (t) denotesthe demand in time slot t for the customers served by utility company u excluding the datacenters. The retail price is greater than the wholesale price, in order to guarantee a positiveprofit for the utility company. We use the first order Taylor approximation of the retailprice near the wholesale price as follows:pru(eu(t),m) = p(t)+ κu(t) eu(t), u ∈ U , t ∈ T , (5.1)where κu(t), u ∈ U , t ∈ T , are nonnegative coefficients with the unit of $ / MWh2. Theutility companies can determine κu(t) according to the cost of supplying electricity to thedata centers. The linear retail price model in (5.1) has been used in different problemsettings such as the Cournot and Bertrand competetive markets [191] and retail powermarkets [104]4. The dynamic pricing scheme in (5.1) depends on the energy demandacross all data centers. It motivates data center d towards scheduling its energy demanded(t), t ∈ T to take advantage of the retail price fluctuations and reduce its contract billpayment∑t∈T ed(t) pru(eu(t),m) to utility company u = m(d) in matching m. In the nextsubsection, we describe how a data center can manage its energy demand profile.4The linear price model in (5.1) can be extended to a piecewise linear model, which can be used toapproximate nonlinear dynamic pricing schemes.125Chapter 5. Demand Response for Data Centers in Deregulated Markets5.2.3 Data Center’s Operation ModelIn the underlying deregulated electricity market, a data center specifies its energy demandwhen entering a contract with a utility company. The energy demand of a data centerincludes the demand for the workloads execution[94, 132]. We assume that a data centerpossesses a small-scale renewable generator (e.g., PV panel) to partially supply its demand.We also assume that a data center possesses an energy storage system. The data centercan charge and discharge the energy storage system to smooth out the fluctuations in thedemand and renewable energy generation [134].In the following parts, we propose a workload scheduling approach and discuss themodels for the workloads and energy storage system.Workload SchedulingIn this part, we propose a scheduling approach for execution of the workloads in a datacenter. A data center offers different service classes (e.g., video streaming, data analytics)to its customers. Consider data center d ∈ D. Let Cd = {1, . . . , Cd} denote the set ofservice classes, where Cd= |Cd|. We assume that both the workloads’ inter-arrival time andexecution time follow the exponential distribution [131, 132]. We can model the workloads’arrivals and executions by a time-dependent multiclass M/M/1 queuing system. Let λc,d(t)denote the average arrival rate of workloads requesting service class c ∈ Cd, in time slot t.Let nd(t) denote the number of operating (active) servers in time slot t. Let σc,d denotethe average time it takes for a server to execute a workload of service class c. If all theservers execute the workloads of service class c, then the average execution rate in timeslot t is µc,d(t) =nd(t)σc,d.To manage the energy demand for executing the workloads in a data center, we considerthe possibility of deferring the execution of a workload to future time slots. Deferring the126Chapter 5. Demand Response for Data Centers in Deregulated Marketsworkloads enables a data center to control the value of workloads arrival rates in differenttime slots To meet the quality-of-service requirements, the delay in executing each incomingworkload needs to be controlled within a certain range, which depends on the type of servicerequest. We measure the maximum delay of a service in terms of the number of time slots.Let ∆c,d denote the maximum number of time slots that the execution of a workload ofservice class c ∈ Cd can be delayed. If ∆c,d = 0, then the service cannot be delayed dueto its interactive nature. Examples of such interactive services include web search, onlinegaming, video streaming, and social networking. For the delay-tolerant flexible services,such as scientific applications, data analytics, and file processing, we have ∆c,d ≥ 1 [93].We propose a probabilistic model for workload scheduling in a data center. The EMSin data center d may defer the execution of a workload of service class c from time slott1 ∈ T to t2 ∈ [t1, t1 + ∆c,d] with a “time-shift” probability pc,d(t1, t2). For data center d,let pc,d(t1) = (pc,d(t1, t2), t2 ∈ T ) denote the decision variables of the time-shift probabilitydistribution for the workloads of service class c that arrive in time slot t1. The EMS ofdata center d decides on the probability distributions pc,d = (pc,d(t), t ∈ T ) for all serviceclass c and time slot t. We havepc,d(t1, t2) = 0, if t2 6∈ [t1, t1 + ∆c,d], (5.2a)pc,d(t1, t2) ∈ [0, 1], if t2 ∈ [t1, t1 + ∆c,d], (5.2b)∑t2∈Tpc,d(t1, t2) = 1, t1 ∈ T . (5.2c)By scheduling the workloads’ arrival in data center d, the workloads of service class cin time slot t includes the newly arrived workloads, which will not be deferred to futuretime slots, and the workloads that are initially in the system at the beginning of time slott. These workloads involve the deferred workloads from the previous time slots, and theworkloads with incomplete job from time slot t−1. The newly arrived workloads of service127Chapter 5. Demand Response for Data Centers in Deregulated Marketsclass c have the average arrival rate λc,d = pc,d(t, t)λc,d. The average number of workloadsof service class c and are deferred from the previous time slots to time slot t is obtained asΛ1c,d(pc,d, t) =t′=t−1∑t′=1pc,d(t′, t)λc,d (t′) . (5.3)We now compute the average number of workloads from time slot t−1 with incompletejob. In Appendix D.1, we show that the process of the workloads of service class c canbe modeled by an M/M/1 queuing system with the workloads’ average arrival rate λc,dand execution rate µc,d(t) = (1− (ρd(t)− ρc,d(t)))µc,d(t), where ρd(t) =∑c∈Cdλc,d(t)µc,d(t)is theaverage server utilization in time slot t in data center d and ρc,d(t) =λc,d(t)µc,d(t). We use thesteady state approximation to compute the average number of workloads with incompletejob in time slot t− 1 as [192]Λ2c,d(pc,d, t) =λc,d(t− 1)µc,d(t− 1)− λc,d(t− 1). (5.4)Workloads’ Execution Time ConstraintThe scheduled workloads in each time slot should be executed in a relatively short periodof time. Let δc,d denote the maximum execution time (i.e., the waiting time in the queueplus the service time by the servers) for the workloads of service class c in time slot t. Thevalue of δc,d is usually much smaller than the length of one time slot, e.g., δc,d is a fewseconds. Hence, the data center should operate sufficient number of servers to execute thescheduled workloads in each time slot.In Appendix D.1, we show that a workload of service class c in data center d experiencesthe maximum expected execution time either at the beginning or at the end of each timeslot t. We use the steady state approximation of the expected execution time of an incomingworkload with service class c at the end of time slot t. Thus, we have 1µc,d(t)−λc,d(t)≤ δc,d.128Chapter 5. Demand Response for Data Centers in Deregulated MarketsWe can express µc,d(t) in terms of nd(t). By performing some algebraic manipulations, weobtainσc,dδc,d+∑c′∈Cdσc′,d λc′,d(t) ≤ nd(t), d ∈ D, c ∈ Cd, t ∈ T . (5.5)Inequality 1µc,d(t−1)−λc,d(t−1)≤ δc,d implies that Λ2c,d(pc,d, t) in (5.4) is at most δc,d λc,d(t−1). Hence, we can approximate the number of initial workloads that request service class cat the beginning of time slot t as Ic,d(pc,d, t) = Λ1c,d(pc,d, t) + δc,d λc,d(t− 1). The expectedexecution time for a workload requesting service class c at the beginning of time slot t is1+Ic,d(pc,d,t)µc,d(t). It should be less than or equal to δc,d. We can express µc,d(t) in terms of nd(t).By performing some algebraic manipulations, for d ∈ D, c ∈ Cd, t ∈ T , we obtainσc,dδc,d+∑c′∈Cdσc′,d λc′,d(t) + σc,d(λc,d(t− 1)− λc,d(t))+σc,d Λ1c,d(pc,d, t)δc,d≤ nd(t). (5.6)In data center d, the number of operating servers is also upper bounded by nmaxd . We havend(t) ≤ nmaxd , d ∈ D, t ∈ T . (5.7)Workloads’ Execution Energy DemandThe initial workloads affect the transient behaviour of the underlying queuing system atthe beginning of time slot t, which lasts for a short period of time. After this transientbehaviour, the steady state behaviour dominates until the end of time slot t. We considerthe energy consumption of the operating servers (which have the average utilization ρd(t))over the steady state period in time slot t in order to approximate energy demand ofdata center d for workloads execution. Let Eidled and Epeakd denote the average idle energyconsumption and the peak energy consumption per time slot of a single server in data129Chapter 5. Demand Response for Data Centers in Deregulated Marketscenter d. The average energy demand of data center d ∈ D in time slot t ∈ T can beobtained byewd (t) = ηd(t)nd(t)(Eidled + (Epeakd − Eidled)ρd(t)), (5.8)where ηd(t) > 1 is the power usage effectiveness (PUE) of data center d in time slot t. Thetypical value of ηd(t) for most data centers is between 1.5 and 2 [92].Energy Storage System ModelData center d schedules the energy storage’s charging and discharging profile ebd = (ebd(t), t ∈T ), where ebd(t) denotes the amount of energy being charged (ebd(t) > 0) to or discharged(ebd(t) < 0) from the battery energy storage in time slot t. The charging/discharging rateof the energy storage in data center d has limits eb,mind < 0 and eb,maxd > 0. That iseb,mind ≤ ebd(t) ≤ eb,maxd , d ∈ D, t ∈ T . (5.9)Let Eb,initd denote the initial energy level of the energy storage in data center d. Thestored energy in the storage of data center d until time T ′ ≤ T is nonnegative and upperbounded by the limit Eb,maxd . Thus, we have0 ≤ Eb,initd +T ′∑t=1ebd(t) ≤ Eb,maxd , d ∈ D, T ′ ≤ T. (5.10)Data Center’s Total Energy DemandThe total energy demand of data center d in time slot t includes the workloads’ executionenergy demand ew(t), the energy storage’s charging and discharging demand eb(t), and the130Chapter 5. Demand Response for Data Centers in Deregulated Marketssupplied energy erd(t) of the local renewable generator. We haveed(t) = ewd (t) + ebd(t)− erd(t), d ∈ D, t ∈ T . (5.11)Data centers have uncertainty about their energy demands due to the stochastic natureof the workloads and renewable generation. Utility companies can assign a penalty foran excessive energy demand to encourage data centers toward declaring a close-to-actualenergy demand in the contract. In the next subsection, we model the risk of excessivedemand and explain how a data center can limit this risk.5.2.4 Risk-Aware Energy Demand SchedulingThe accurate prediction of renewable generation is a challenge. The predicted renewablegeneration erd(t) often does not exactly match with the actual generation level êrd(t) intime slot t ∈ T due to the intermittent nature of the renewable energy generation. Thepredicted arrival rate λc,d(t) for the workloads of service class c in time slot t often doesnot exactly match with the actual arrival rate λ̂c,d(t) either.Let vectors ê rd = (êrd(t), t ∈ T ) and erd = (erd(t), t ∈ T ) denote the actual and predictedgeneration profiles of the renewable generator in data center d, respectively. Let vectorsλ̂c,d = (λ̂c,d(t), t ∈ T ) and λc,d = (λc,d(t), t ∈ T ) denote the profiles of actual and predictedarrival rate for the workloads of service class c in data center d, respectively. We definevectors λ̂d = (λ̂c,d, c ∈ Cd) and λd = (λc,d, c ∈ Cd) for data center d. The uncertainty inthe renewable generation and workloads arrival causes the actual demand êd(t) to deviatefrom the predicted demand ed(t). We use (5.8) and (5.11) to express êd(t)− ed(t) asêd(t)− ed(t) =∑c∈Cd(ϕc,d(t)(λ̂c,d(t)− λc,d(t))+t−1∑t′=1ψc,d(t′, t)(λ̂c,d(t′)− λc,d(t′)))− (ê rd(t)− erd(t)), (5.12)131Chapter 5. Demand Response for Data Centers in Deregulated Marketswhereϕc,d(t) = σc,d ηd(t) pc,d(t, t)(Epeakd − Eidled), ∀ t′, t ∈ T ,andψc,d(t′, t) = σc,d ηd(t) pc,d(t′, t)Epeakd , ∀ t′, t ∈ T .The excess energy demand of data center d in time slot t is equal to [êd(t)− ed(t)]+, where[·]+ = max{·, 0}. Utility company u can set penalties p+u (t), t ∈ T with the unit of $/MWhto prevent the data centers from under-estimating their demand. We define the cost of riskassociated with the excess energy demand of data center d in matching m asRd(λ̂d,λd, êrd, erd) =∑t∈Tp+m(d)(t) [êd(t)− ed(t)]+. (5.13)We consider CVaR to determine vectors λd and erd [163]. Optimizing the CVaR enablesa data center to use the historical data record about its workloads and renewable generationto limit the risk of high penalty for the excess energy demand within a confidence level.CVaR is a convex function and can be optimized using sampling techniques. The CVaRfor data center d is defined for a confidence level βd ∈ (0, 1), and vectors λd and erd asCVaRd,βd(λd, erd) = E{Rd(λ̂d,λd, êrd, erd)∣∣Rd(λ̂d,λd, ê rd, erd) ≥ αβd}, (5.14)where E(·) is the expectation over the random variables λ̂d and ê rd, and we have αβd =min{αd∣∣Pr{Rd(·) ≤ αd} ≥ βd}. Minimizing the CVaR in (5.14) will help us determinethe appropriate vectors λd and erd that minimize the expected value of the penalty Rd(·)when it is higher than αβd .132Chapter 5. Demand Response for Data Centers in Deregulated MarketsIn general, the explicit characterization of the probability distributions of the randomvariables λ̂d and êrd are not available. However, it is possible to estimate the CVaR in(5.14) by adopting the sample average approximation (SAA) technique [175]. We use theset J , {1, . . . , J} of J samples of random variables λ̂d and ê rd to estimate Pr{λjd, er,jd },the probability of the scenario with jth sample. The CVaR function in (5.14) can beapproximated by [175]CVaRd,βd(λd, erd) ≈ minαd∈RΓd,βd(αd,λd, erd), (5.15)whereΓd,βd(αd, λd, erd)= αd +∑j∈JPr{λjd, er,jd}1− βd[Rd(λjd,λd, er,jd , erd)− αd]+. (5.16)For the sake of simplicity, we introduce the auxiliary variable θjd for sample j in datacenter d to upper bound each term [Rd(·)− αd]+ in (5.16). We also introduce the auxiliaryvariable ϑjd(t) for sample j in time slot t to upper bound the term [êd(t)− ed(t)]+ in (5.13).We define vectors of auxiliary variables θd = (θjd, j ∈ J ) and ϑd = (ϑjd(t), t ∈ T , j ∈ J )for data center d. We can express function Γd,βd(αd, λd, erd)in (5.16) in terms of variablesαd and θd asΓd,βd(αd,θd)= αd +11− βd∑j∈JPr{λjd, er,jd}θjd. (5.17)Next, we include the following inequalities into the constraints set of data center d ∈ D:∑t∈Tp+u (t)ϑjd(t)− αd ≤ θjd, j ∈ J , d ∈ D. (5.18a)êd(t)− ed(t) ≤ ϑjd(t), j ∈ J , d ∈ D, t ∈ T . (5.18b)133Chapter 5. Demand Response for Data Centers in Deregulated Markets5.2.5 Preference of the Data Center and Utility CompanyTo enter a bilateral contract, the data centers have preferences over the utility companiesbased on the payment and risk of the contract. The utility companies have also preferencesover the data centers based on the contract revenue and the PAR of the energy demand.Data Center’s PreferenceThe total cost of data center d in matching m includes the bill payment and the CVaRfunction Γd,βd(αd,θd). Let ad =((pc,d, c ∈ Cd), ebd, µd, λd, erd, αd, θd, ϑd)denote thescheduling decision vector of data center d. The contract payment of data center d dependson the matching m and the joint decision vector a = (ad, d ∈ D) of all data centers throughthe pricing scheme in (5.1). We havecd(a,m) =∑t∈Ted(t) pru(eu(t),m) + ωcvard Γd,βd(αd,θd), (5.19)where ωcvard is a nonnegative weight coefficient. Data center d ∈ D prefers matching mwith the joint decision vector a to matching m′ with the joint decision vector a′ if cd(a,m)is smaller than cd(a′,m′). It defines preference relation d over the pairs (a,m) and(a′,m′) as(a,m) d (a′,m′) ⇐⇒ cd(a,m) ≤ cd(a′,m′). (5.20)Notice that data center d does not need to know the joint decision vector a to determineits preference. The value of retail price in (5.1) provides sufficient information about vectora to each individual data center.134Chapter 5. Demand Response for Data Centers in Deregulated MarketsUtility Company’s PreferenceUtility company u prefers a contract with a higher revenue. It may also prefer a lower PARof the aggregate energy demand to improve the performance of the energy network duringpeak hours. Specifically, reducing the PAR also let the utility company reduce its retailprice to attract more customers. Under a given matching m and vector a, we consider theweighted subtraction of the revenue and the PAR asfu(a,m) = frevu (a,m)− ωu fPARu (a,m), (5.21)where ωu is a nonnegative weight coefficient. The revenue of utility company u from sellingelectricity to data centers in set m(u) isf revu (a,m) =∑d∈m(u)∑t∈Ted(t) pru(eu(t),m),and the PAR of the total energy demand of the data centers in set m(u) isfPARu (a,m) =maxt∈T{eotheru (t) +∑d∈m(u)ed(t)}1T∑t∈T(eotheru (t) +∑d∈m(u)ed(t)) .Utility company u does not need to know the data centers’ decision vector a. Instead,the aggregate energy demand eu(t), t ∈ T provides sufficient information to utility companyu to determine fu(a,m) in (5.21).The total energy supply from utility company u is upper-bounded by emaxu , due to thelimited energy budget and transmission capacity in the network. Utility company u prefersmatching m with data centers’ decision vector a to matching m′ with decision vector a′ iffu(a,m) is larger than fu(a′,m′), subject to the maximum energy budget. Hence we can135Chapter 5. Demand Response for Data Centers in Deregulated Marketsdefine defines the preference relation u for utility company u ∈ U over the pairs (a,m)and (a′,m′) as(a,m) u (a′,m′)⇐⇒fu(a,m) ≥ fu(a′,m′),eu(t) ≤ emaxu , t ∈ T ,e′u(t) ≤ emaxu , t ∈ T .(5.22)5.3 Problem Formulation and Algorithm DesignIn this section, we model the coupled decisions of utility company choices and workloadscheduling of the data centers as a many-to-one matching game.5.3.1 Data Center Many-to-One Matching GameData center d aims to minimize its cost in (5.19) and achive the most prefered schedulingdecision and utility company choice based on the prefernce relation in (5.20). The decisionmaking of data centers are interdependent, since the cost of each individual data center in(5.19) is a function of the joint decision vector a of all data centers as well as the matchingstructure. We capture the interactions among the data centers as a many-to-one matchinggame, which is defined as follows [188]:Game 5.1 Data Center Many-to-One Matching Game:Players : The set of all data centers D.Strategies : The strategy of data center d is the tuple sd = (ad,m(d)). Let Sd denote thefeasible strategy space for data center d defined by (5.2)−(5.12), (5.17), (5.18a), (5.18b),and constraint m(d) ∈ Ud. Let s = (sd, d ∈ D) denote the strategy profile of all datacenters. Let s−d denote the strategy profile of all data centers except data center d.136Chapter 5. Demand Response for Data Centers in Deregulated MarketsCosts : Data center dd aims to minimize the cost cd(sd, s−d) as in (5.19), which canbe expressed as a function of strategy profiles sd of data center d and the strategy s−d ofother data centers.We emphasize that in the above-mentioned matching game, the utility companies arenot players and their price functions are fixed based on (5.1), i.e., parameters κu(t), t ∈ Tare fixed for each utility company u. However, utility company u enters a contract withthe most preferred data centers based on the preference relations in (5.22).Notice that our game is different from a traditional matching game [193], where thecost of a player only depends on whom it matches with, not depending on the matchingresult of other players. Here, however, the cost of a data center d depends on the demandschedules of other data centers which are matched to the same utility company as d. Hence,our game is a matching game with externalities among the data centers [188]. Matchinggame with externalities has been used in different problem settings [194]. For our problemsetting, the outcome of the game is a matching m and the joint scheduling decision profilea of the data centers. It is said to be a stable outcome, when there exists no data centerthat incurs a lower cost from changing either its matched utility company or its actionprofile unilaterally [188]. It is defined as follows.Definition 5.2 A stable outcome of the matching game is the feasible strategy profiles? = (s?d, d ∈ D) such that for d ∈ Dcd(s?d, s?−d) ≤ cd(s, s?−d), s ∈ Sd. (5.23a)The stable outcome for the underlying matching game is defined from the point of viewof data centers, which is analogous to the Nash equilibrium for normal form games [184, Ch.3]. In general, a stable outcome may not exist in a matching game with externalities [194].Furthermore, there does not exist a general algorithm that can guarantee to determine a137Chapter 5. Demand Response for Data Centers in Deregulated Marketsstable outcome in matching game with externalities. For matching game with externalities,Bando in [195] proposed a distributed deferred acceptance algorithm to compute the stableoutcome. Such an algorithm is not directly applicable to our problem, since it only considersthe interdependency of the players’ strategies through the matching structure. In ourproblem, however, the change of decision vector ad by data center d ∈ D may lead to thestrategy changes of other data centers through the price value in (5.1) as well.To study the stable outcomes in Game 5.1, we consider the concept of best responsestrategy, which is a data center’s best strategy to minimize its own cost assuming that thestrategies of other data centers are fixed. The best response strategy of data center d isdefined assbestd (s−d) ∈ arg minsd∈Sdcd(sd, s−d), d ∈ D. (5.24)A stable outcome is a fixed point of the best responses of all data centers. That is,sbestd (s?−d) = s?d for all d ∈ D.Problem (5.24) for data center d involves choosing a utility company. Hence, it isa non-convex optimization problem with discrete variables. However, under the givenstrategy profile s−d and matching m, problem (5.24) can be transformed into a convexoptimization problem with quadratic objective function and linear constraints over thescheduling decision vector ad. There are two steps involved in solving problem (5.24)for data center d under a given strategy profile s−d: (a) solving a convex optimizationproblem for a fixed matching m, and (b) comparing the objective value for all utilitycompany choices for a data center.We prove the existence of a stable outcome by constructing an exact potential func-tion [196, 197]. Such a function is defined as follows:138Chapter 5. Demand Response for Data Centers in Deregulated MarketsDefinition 5.3 A function P (s) is an exact potential for Game 5.1, if for any feasiblestrategy profiles s = (sd, s−d) and s˜ = (s˜d, s−d), we havecd(sd, s−d)− cd(s˜d, s−d) = P (sd, s−d)− P (s˜d, s−d). (5.25)A potential function P (s) tracks the changes in the data center’s cost when its strategy(i.e., the utility company choice or action profile) changes. In the following theorem, wecharacterize an exact potential function for Game 5.1.Theorem 5.1 Game 5.1 admits the exact potential functionP (s) =∑u∈U∑t∈T( ∑d∈m(u)((p(t) + κu(t) eotheru (t))ed(t) + κu(t) e2d(t))+ κu(t)∑d<d′∈m(u)ed(t) ed′(t))+∑d∈Dωcvard Γd,βd(αd,θd). (5.26)The proof can be found in Appendix D.2. In Theorem 5.1, we obtain the specific form ofthe potential function. There is no generic method of constructing a potential function,and it requires exploring the structure of the problem.Under a given matching m, the potential function (5.26) is a convex function of thedecision vector a. We let am=(amd , d∈D) denote the global minimum of P (s) for a givenmatching m. Let M denote the set of tuples (am,m) for all matchings m. In Theorem5.2, we show that the stable outcomes of the matching game are in set M.Theorem 5.2 Game 5.1 has at least one stable outcome, which is in set M.The proof can be found in Appendix D.3.139Chapter 5. Demand Response for Data Centers in Deregulated Markets5.3.2 Distributed Algorithm DesignOne may consider a trusted third party (e.g., the system operator) that has access to theinformation about the energy demand of the data centers and preference of the utilitycompanies. The trusted third party can determine a stable outcome of Game 1 by com-puting the global minimum of the potential function in (5.26) in a centralized fashion. Thecentralized approach can be formulated as a mixed-integer nonlinear problem, which is NPhard.For Game 1 with an exact potential function, one can also use the existing algorithmsbased on the best response update to determine the stable outcome [198]. These algorithms,however, suffer from a low convergence rate, as only one single data center updates itsstrategy per iteration. To address the challenges in the centralized approach and existingbest response algorithms, we propose Algorithm 5.1 that can be executed by the datacenters and utility companies in a distributed fashion to converge to a stable outcome.The proposed algorithm is based on the gradient decent method and best response updateof multiple data centers. Let i denote the iteration index. The EMS of the data centersare responsible for the computations and message exchange. Our algorithm involves theinitiation phase and matching phase.Initiation phase: Lines 1 to 3 describe the initialization for the data centers and utilitycompanies.Matching phase: The loop involving Lines 4 to 17 describes the matching phase. Itincludes the following parts:a) Information exchange: Lines 5 and 6 describe the information exchange between thedata centers and utility companies about the energy demands and retail prices.b) Utility company choice: Lines 7 to 11 describe how a data center d chooses a utilitycompany and how a utility company u responses to the requests of the data centers.140Chapter 5. Demand Response for Data Centers in Deregulated MarketsTo compute the best response strategy sbest,id (si−d), data center d can solve problem(5.24) for different possible utility company choices m(d) ∈ Ud, as it is a convex opti-mization problem under the given strategy profile si−d and utility company choice m(d).If its current utility company is different from the utility company choice in its best re-sponse strategy, data center d sends a termination request to its current utility company.A utility company u uses the preference relation u in (5.22) to accept the most preferredtermination request.Next, each data center d sends a connection request to the utility company in itsbest response strategy if its termination request has been accepted by its current utilitycompany. A utility company u uses the preference relation u in (5.22) to accept the mostpreferred connection request. If a utility company accepts all the requests from the datacenters, then the algorithm may be stuck in an infinite loop. To prevent this case, we onlyallow each utility company to accept at most one termination request and at most oneconnection request.c) Strategy update: Lines 12 to 15 describe how a data center d updates its strategysid = (aid, mi(d)). If data center d changes its matching, then it updates its action profilewith its best response., i.e., ai+1d := abest,id .By receiving the updated retail price pr,iu (eiu(t),mi+1), each data center d, that does notchange its utility company, computes its updated action profile ai+1d based on the followinggradient-based updating process:ai+1d =[aid − γid ∇aid cd(aid,ai−d,mi+1) ]℘, (5.27)where γid > 0 is a diminishing step size with∑∞i=0 γid =∞ and∑∞i=0(γid)2 <∞, and [·]℘ isthe projection onto the feasible space defined by (5.2), (5.8), (5.11)−(5.10). In Theorem5.3, we show that Algorithm 5.1 converges to a stable outcome.141Chapter 5. Demand Response for Data Centers in Deregulated MarketsAlgorithm 5.1 The Data Center Matching Game Algorithm.Initiation phase1: Set i := 1 and ξ := 10−3.2: Randomly assign each data center d∈D to a utility company m1(d) ∈ Ud, and initializeaction profile a1d.3: Send parameters κu(t), t ∈ T to the data centers in set Du.Matching phase4: Repeata) Information exchange:5: Each data center d sends eid(t), t ∈ T to utility company mi(d).6: Each utility company u updates retail prices pr,iu (eiu(t),mi) for t ∈ T using (5.1) and sendsto the data centers in set Du.b) Utility company choice:7: Each data center d chooses a utility company in set Ud by computing its best responsestrategy in (5.24).8: Each data center d sends termination request to its current utility company if it is differentfrom the chosen one.9: Each utility company u accepts the most preferred termination request based on thepreference relation u in (5.22).10: Each data center d sends connection request to its chosen utility company if its terminationrequest has been accepted.11: Each utility company u accepts the most preferred connection request based on thepreference relation u in (5.22).c) Strategy update:12: Each data center d with an accepted connection request updates mi+1(d) with the chosenutility. Otherwise, mi+1(d) :=mi(d).13: Each data center d, that changes its utility company, updates its action profile with its bestresponse, i.e., ai+1d := abest,id .14: Each utility company u sends retail prices pr,iu (eiu(t),mi+1), t ∈ T for the updated matchingmi+1 to the data centers in Du.15: Each data center d, that does not change its utility company, updates ai+1d accordingto (5.27).16: i := i+ 1.17: Until No data center wants to change its strategy, i.e., mi = mi−1 and ||ai − ai−1|| < ξ.Theorem 5.3 Algorithm 5.1 always converges to a stable outcome of the data centermatching game.The proof can be found in Appendix D.4. The computational complexity of Algorithm 5.1142Chapter 5. Demand Response for Data Centers in Deregulated Marketsis an important factor to evaluate its efficiency. In Line 7 of Algorithm 5.1, each data centerd solves |Ud| optimization problems to determine its best response strategy. Hence, theper-iteration complexity of Algorithm 5.1 for data center d is independent of the numberof the data centers in the system and depends only on the number of utility companies inset Ud, i.e., O(|Ud|).5.4 Performance EvaluationIn this section, we evaluate the performance of the stable outcome of the matching game5.We set the contract period to be one day. We divide a day into T = 96 time slots, whereeach time slot is 15 minutes. We consider the electricity market with 10 utility compa-nies serving 50 data centers, which are free to choose a utility company from a randomsubset of seven utility companies. We use the wholesale market price on Oct. 10, 2016of the Ontario’s wholesale market [179]. The wholesale market price over 24 hours isshown in Fig. 5.2 (a). Parameters κu(t) for utility companies u = 1, 2, . . . , 10 are set to0.224, 0.208, . . . , 0.08 $/(MWh)2 for t ∈ T , respectively. We set p+u (t) = 20 $/MWh, t ∈ Tand ωu = 104 for all utility companies.To simulate the arrival rate of the workloads in a data center, we use the World Cup98 web hits data [199]. Each data center offers five service classes, and the workloadsrequesting service classes c = 1, . . . , 5 can be delayed by at most ∆c,d = 0, 4, 8, 16, 20time slots, respectively. We set δc,d = 3 sec for all service classes in all data centers.We consider nmaxd = 14,000 homogeneous servers with power ratings Eidled = 100 kW andEpeakd = 200 kW per time slot in each data center d. Parameters σc,d, c ∈ Cd for the serversare chosen at random from interval [0.1 sec, 10 sec]. The weight coefficient ωcvard is set to5There can be multiple stable outcomes of the game. Here, we evaluate the performance of the onereached by our distributed algorithm.143Chapter 5. Demand Response for Data Centers in Deregulated Markets 12 am 6 am 12 pm 6 pm 12 am 1 1.522.533.5Time (hour)Wholesalemarketprice(cents/kWh)(a) 12 am 6 am 12 pm 6 pm 12 am 0 0.511.5Time (hour)AveragePVgeneration(MWh)(b)Figure 5.2: (a) Wholesale market prices; (b) average PV generation over 24 hours.20, and parameters PUEd(t), t ∈ T are set to 1.5 for all data centers.Each data center is equipped with a PV plant. We use the historical generation datafor Ontario, Canada power grid database from June 1, 2016 to Oct. 1, 2016 [179] to obtainthe samples for PV generation. For each data center, we scale the available historical datasuch that the average generation level of the PV plants becomes equal to 0.5 MW pertime slot. Fig. 5.2 (b) shows the average output power of a PV unit over 24 hours. Theconfidence level βd is set to 0.8 for all data centers.Each data center is equipped with an energy storage system. The maximum charg-ing/discharging rate and capacity of an energy storage is set to 0.05 MW and 0.5 MWh,respectively. The initial energy level of the storage system in each data center is set to 50%of the maximum capacity. We perform simulations using Matlab in a PC with processorIntel(R) Core(TM) i7-3770K CPU@3.5 GHz.We first demonstrate how Algorithm 5.1 enables a data center to manage its energydemand. Let us consider data center 1, as an example. The step size in iteration i is set to144Chapter 5. Demand Response for Data Centers in Deregulated Marketsγid = 1/(50 + 0.07 × i). For the sake of comparison, we consider the benchmark scenario,where each data center randomly chooses a utility company and does not schedule itsworkloads. Figs. 5.3 (a) and (b) show the predicted arrival rate of the delay-tolerantworkloads in data center 1 requesting service classes 2 and 5, respectively. With workloadscheduling, the number of workloads during peak hours is reduced. The EMS can bettermanage the workloads with service class 5 due to more flexibility in delaying their executiontime. Fig. 5.3 (c) shows that with workload scheduling, the total number of operatingservers in data center 1 decreases over the day, e.g., it is reduced from 14,000 to 12,000around 5 pm. Moreover, Fig. 5.3 (d) shows that the energy demand of data center 1 isreduced by 17.3% (from 11 MWh to 9.1 MWh on average) during the period from 12 pmto 6 pm, as a result of workload scheduling and reducing the number of servers.We then study the changes in the cost of data centers when they implement Algorithm5.1. Fig. 5.4 shows the daily cost of data centers 1 to 10 in three scenarios: (1) thebenchmark scenario without demand response and utility company choice, (2) the scenariowithout demand response and with utility company choice, and (3) the scenario with bothdemand response and utility company choice. When comparing the second scenario withthe first scenario, the total cost of the data centers decreases by 8.8% on average as aresult of choosing their preferred utility company. When comparing the third scenariowith the first scenario, the total cost of the data centers decreases by 18.7% as a resultof both workload scheduling and choosing their preferred utility company. Note that inthe second and third scenarios, there may exist some data centers (e.g., data center 6 and9) with a higher cost comparing with the first scenario. This is due the impacts of theworkload scheduling and utility company choices of other data centers on the price values.Nevertheless, in the stable outcome, no data center has an incentive to change its utilitycompany choice and workload scheduling.145Chapter 5. Demand Response for Data Centers in Deregulated Markets 12 am 6 am 12 pm 6 pm 12 am 2004006008001000Time (hour)Without load scheduling With load scheduling(a)Arrival rate(workload per time slot)(b) 12 am 6 am 12 pm 6 pm 12 am 2004006008001000Time (hour)Arrival rate(workload per time slot)(c) 12 am 6 am 12 pm 6 pm 12 am 7891011121314 103Time (hour)Number of servers 12 am 6 am 12 pm 6 pm 12 am 4681012Time (hour)(d)Peak load reduction Total Energy Demand (MWh)Figure 5.3: (a) Arrival rates of the workloads of service class 2; (b) arrival rates of theworkloads of service class 5; (c) total number of servers; (d) total energy demand over 24hours in data center 1 with and without load scheduling.146Chapter 5. Demand Response for Data Centers in Deregulated MarketsNext, we study the impact of the confidence level βd on the energy demand predictionand risk of high excess energy demand for the data centers. It enables us to study thestrategy of a data center in reducing the risk of penalty for excess energy demand in thecontract. Fig. 5.5 (a) shows the values of of Γ1,β1(·) in (5.17) for data center 1 withβ1 = {0.2, 0.4, 0.6, 0.8, 0.95}. When β1 increases, the data center becomes more riskaverse and will try to decrease the risk of high excess energy demand by assigning a higherpredicted workload energy demand and a lower predicted PV generation. Hence, the valueof Γ1,β1(·) decreases when β1 increases. As an example, Fig. 5.5 (b) shows a higher predictedvalues for the energy demand of data center 1 with β1 = 0.95, comparing with the scenariowith β1 = 0.8.We now compare the PAR and revenue of the utility companies in three scenarios: (1)the benchmark scenario with random response to the connection and termination requestsfrom data centers, (2) the scenario with ωu = 0 or the preference relation in (5.22), and(3) the scenario with ωu = 106. Remind that the value of ωu indicates the importance ofreducing the PAR compared with increasing the the revenue in responding to the connec-tion or termination requests from data centers. Fig. 5.6 (a) shows that, compared withthe first scenario, the PAR is reduced by 4.6% and 8% on average in the second and thirdscenarios, respectively. The PAR is the lowest in the third scenario, since for ωu = 106,reducing the PAR is the most dominant factor for the utility companies in responding tothe connection or termination requests.Fig. 5.6 also shows that the revenue of some utility companies may decrease in themarket compared with the benchmark scenario, though the PAR is reduced for all utilitycompanies. Fig. 5.6 (b) shows that the utility companies with a higher parameter κu(t) havea higher average revenue in the benchmark scenario, since all utility companies randomlyaccept the connection and termination requests from data centers. If all utility companies147Chapter 5. Demand Response for Data Centers in Deregulated Markets1 2 3 4 5 6 7 8 9 10121620242832Cost ($) BenchmarkscenarioWith utility company choice, Without load schedulingWith utility company choice, With load scheduling103Figure 5.4: Total daily cost of data center 1 by considering the opportunities of utilitycompany choice or workload scheduling.0.2 0.4 0.6 0.8 0.95100300500700900111 12 am 6 am 12 pm 6 pm 12 am 4681012= 0.95 = 0.81 1Figure 5.5: (a) The value of Γd,βd(αd, λd, er,d)in (5.16) for data center 1 with confidencelevel β1 in the range of 0.2 to 0.95; (b) total energy demand of data center 1 with β1 = 0.95and β1 = 0.8.148Chapter 5. Demand Response for Data Centers in Deregulated Marketsaccept the data center’s offers according to their preference relation in (5.22), then therevenue of those utility companies with a higher κu(t) decreases (up to 70%) and the revenueof those utility companies with a lower κu(t) increases (up to 80%). When comparing withthe second scenario with the third scenario, the revenue of the utility companies is 9.8%higher, as increasing the revenue is the dominant criterion in the second scenario.Next, we study the matching structure in the stable outcome when data centers im-plement Algorithm 5.1. The matching structure represent how Algorithm 5.1 enables eachutility company to attract data centers as customers. In the stable outcome, the num-ber of data centers connected to utility companies 1 to 10 are 2, 3, 3, 3, 4, 5, 5, 7, 8, and10, respectively. It illustrates that parameter κu(t) affects the market share (hence therevenue for a utility company as mentioned before). For example, ten data centers arematched with utility company 10, which has the lowest parameter κu(t); two data centersare matched with utility company 1, which has the highest parameter κu(t). Althoughutility company 10 has the lowest parameter κu(t), all 50 data centers are not matchedwith this utility company, since the price of utility company 10 will become the highest ifall data centers are matched to it.Finally, we evaluate the average number of iterations of Algorithm 5.1, which can beinterpreted as an indicator of the number of message exchanges among data centers andutility companies. Fig. 5.7 (a) depicts the convergence of the potential function in one ofour simulations for 50 data centers. The potential function decreases in each iteration andconverges to a stable outcome in 40 iterations. Fig. 5.7 (b) shows the required averagenumber of iterations versus the number of data centers for an error tolerance ξ = 10−3.We perform simulations for 20 random initial conditions for the matching structure anddata centers energy demands. The number of utility companies is set to 10 in all scenarios.Each data center is free to choose a utility company from a random predetermined subset of149Chapter 5. Demand Response for Data Centers in Deregulated Markets1 2 3 4 5 6 7 8 9 101.21.31.41.51.6 106a1 2 3 4 5 6 7 8 9 100.20.611.41.82.2Revenue ($)105Figure 5.6: (a) The PAR in the generated power; (b) revenue of the utility companies inthe benchmark scenario, and the scenarios with ωu = 0 and ωu = 106.seven utility companies. For each scenario, we increase the value of step size gradually andreport the smallest number of iterations for convergence. We observe that the requirednumber of iterations increases in the number of data centers. However, our algorithmalways converges in a reasonable number of iterations. Fig. 5.7 (c) shows the requiredaverage number of iterations versus the number of utility companies. The number of datacenters is set to 50 in all scenarios. The set of utility companies that can supply a datacenter includes half of the utility companies. Parameters κu(t), t ∈ T for utility companiesare chosen at random from set {0.08, 0.16, . . . , 0.224} $/(MWh)2. Interestingly, we observe150Chapter 5. Demand Response for Data Centers in Deregulated Markets 1 5 10 15 20 25 30 35 40 0.60.811.21.4 10650 150 250 350 450 550 650 750 850 950Number of data centers050100150200250300Average number of iterations10 20 50 40 50 60 70 80 90 100Number of utility companies01020304050Average number of iterationsFigure 5.7: (a) The value of potential function P (si) per iteration for 50 data centers; (b)the required number of iterations for convergence versus the number of data centers; (c)the required number of iterations for convergence versus the number of utility companies.151Chapter 5. Demand Response for Data Centers in Deregulated Marketsthat the required number of iterations decreases in the number of utility companies. InAlgorithm 5.1, each utility company accepts at most one connection request and one ter-mination request from data centers. When the number of utility companies increases,the number of data centers that can change their utility company choices increases in aparticular iteration. Hence, the required average number of iterations decreases.5.5 SummaryIn this chapter, we addressed the data centers’ problem of choosing utility companiesand scheduling workload in a deregulated electricity market. We showed that if utilitycompanies use RTP scheme, then the decisions of utility company choices and workloadscheduling of the data centers become coupled with each other and the pricing decisions ofthe utility companies. We modeled the interaction among data centers as a many-to-onematching game with externalities. It was a challenge to prove the existence of the stableoutcome of the underlying matching game with externalities. We addressed this challengeby constructing an exact potential function of the game, its local minima correspond tothe stable outcomes of the game. Constructing an exact potential function also enabled usto develop a distributed algorithm to reach a stable outcome. Simulation results showedthat the data centers can decrease their cost by 18.7% if they implement the proposedalgorithm, as they can purchase electricity from their preferred utility company and shifttheir demand to off-peak hours. Meanwhile, the utility companies can take advantageof 8% reduction in the PAR. Furthermore, the utility companies with lower tariffs canincrease their revenue by 80% through attracting more data centers as customers. Finally,we showed that the proposed distributed algorithm has a low complexity per iteration foreach data center.152Chapter 6Conclusions and Future WorkIn this final chapter, we summarize the results and highlight the contributions of this thesisin Section 6.1. We also propose ideas for future research topics in Section 6.2.6.1 Summary of ResultsIn this thesis, we studied energy management programs for energy systems in order toimprove the economic operation of transmission and distribution networks, and facilitatethe proliferation of new technologies such as renewable energy generators, EVs, and energystorage systems. In particular, we proposed algorithms for the OPF, SCUC, and demandresponse programs. On the one hand, determining the set of operating conventional gen-erators with the minimum cost and system losses is of prime importance in transmissionsystems. Thus, we focused on designing algorithms to determine the near-optimal solu-tion of the OPF (for the hourly-basis system analysis) and SCUC (for the day-ahead orweek-ahead planning) problems in ac-dc transmission systems. On the other hand, it isimperative to implement innovative energy management programs to motivate consumerstowards modifying their energy consumption behaviour. Hence, we focused on designingeffective demand response programs for residential electricity consumers and data centersin distribution networks. Specifically, we proposed demand response algorithms to sched-ule the electric appliances in residential households and the workloads in data centers inresponse to dynamic pricing information. In the following, we briefly review the resultsand conclusions.153Chapter 6. Conclusions and Future WorkIn Chapter 2, we studied the ac-dc OPF problem, which was nonconvex and difficultto be solved efficiently. For practical hourly-basis system analysis, an OPF algorithm thatleads to the minimum generation cost and system losses is mandatory. In this regard, weused convex relaxation techniques to transform the original problem into an SDP, whichcan be solved using the available solvers such as CVX in Matlab. Our main contributionwas to determine the zero relaxation gap conditions and to show that the relaxation gap iszero for practical ac-dc grids including the IEEE test systems. We proposed a polynomialtime algorithm to determine the solution of the original ac-dc OPF problem when the SDPrelaxation gap is zero. When compared with the solution obtained by different solversin Matpower, our proposed algorithm returned a solution with lower generation costand power losses. Results showed that the losses on the ac-dc converters can add up toa significant fraction of the total losses, and thus should be included in the ac-dc OPFproblem. Our conclusion was that the proposed algorithm can be a promising analyticaltool to obtain the global optimal solution of the OPF problem in ac-dc grids that compriserenewable generators, HVDC lines, and ac-dc microgrids.In Chapter 3, we focused on the economic operation of power plants for the day-aheaddispatch in ac-dc transmission systems. We studied the SCUC problem for ac-dc grids. Theproliferation of renewable energy generators and price-responsive loads in today’s powergrids motivated us to take into account the uncertainty in the generation and load demand.We found that the concept of CVaR was a promising risk measure for system operatorsto limit the likelihood of shortage in the net power supply within a confidence level. Theoriginal SCUC problem was a nonlinear mixed-integer optimization problem and difficultto be solved efficiently. We proposed an algorithm based on the iterative reweighted l1-norm approximation to determine the near-optimal solution of the original SCUC problem.We evaluated the performance of the proposed SCUC algorithm on several test systems154Chapter 6. Conclusions and Future Workwith renewable energy generators. Simulation results showed that the proposed SCUCalgorithm with different initial conditions returned a near-optimal solution with small gapfrom the global optimal solution, which was an important result, since the existing SCUCalgorithms do not guarantee to converge to a good local optimal solution. Our conclusionwas that the proposed algorithm can be a practical alternative to solve the SCUC problemin ac-dc grids.In Chapter 4, we studied the role of electricity consumers in energy management activ-ities. We focused on designing an autonomous load scheduling algorithm for the interrupt-ible and non-interruptible appliances in residential households. One of our objectives wasto go beyond the mathematical formulation by proposing an algorithm that can be exe-cuted by the ECC of households in practice. In this regard, we focused on learning methodsthat enable residential households to gradually determine their optimal scheduling policyfrom the consequences of their past decisions. We captured the coupled scheduling decisionof multiple residential households in a demand response program as a partially observablestochastic game. The modelling choice of partially observable stochastic game was moti-vated by some practical considerations, such as the uncertainties related to the RTP valuesas well as the lack of information of each individual user about the strategy and load de-mand of other users. To make the problem more tractable, we approximated the optimalpolicy of the households with the MPE of a fully observable stochastic game with incom-plete information. We proposed a distributed and model-free learning algorithm based onthe actor-critic method to determine the MPE policy. To make the algorithm more suit-able for practical applications, we used value function and policy approximation techniquesto reduce the action and state spaces of the households and improve the learning speed.According to simulation results, the utility company could reduce the expected PAR ofthe aggregate load when users deploy the proposed algorithm. The foresighted users could155Chapter 6. Conclusions and Future Workalso benefit from reduction in their expected discounted cost in long-term. The proposedlearning algorithm converged to the MPE policy of the users in a reasonable number ofiterations. Our conclusion was that the proposed learning algorithm can be implementedin the ECC of the participating households in the existing demand response programs.In Chapter 5, we studied the role of electricity consumers with flexible large loadsand higher potential for demand response activities. We focused on the data centersdemand response and studied the emerging deregulated electricity markets in order todesign a practical distributed algorithm for the coupled decisions of utility company choicesand workload scheduling of multiple geographically dispersed data centers. We used theconcept of many-to-one matching game, which provided a realistic framework to modelthe interactions among data centers in a deregulated electricity market. To address theuncertainty in the load demand, we considered the CVaR to limit the risk of excessiveenergy demand of data centers from the set amount in the contract. We characterized thestable outcome of the underlying game by using the best response strategy of the datacenters. We proved the existence of the stable outcome of the game. We also developed adistributed algorithm to determine a stable outcome. According to simulation results, theproposed algorithm reduced the cost of data centers and the PAR in the utility companies’generation. The average CPU time of the proposed algorithm was linear with the numberof utility companies and independent of the number of data centers. Our conclusion wasthat the proposed algorithm can be implemented by the data centers and utility companiesin the existing deregulated markets to enter bilateral contracts.6.2 Future WorkThe communication and computing technologies in the emerging smart grid can openup new opportunities for energy management techniques in transmission and distribution156Chapter 6. Conclusions and Future Worksystems. In the current research on the OPF and SCUC problems, most works have focusedon the mathematical techniques to determine a near-optimal solution, while taking intoaccount the uncertainty in the load demand and renewable generation. In the currentresearch on the demand response programs, most works have focused on the exploration ofpractical mechanisms for scheduling the energy consumption of the users in response to theenergy price fluctuations. In Chapters 2 and 3, we proposed OPF and SCUC algorithmsto determine a near-optimal operation schedule of the generators. In Chapters 4 and 5,we focused on designing practical algorithms for residential users and data centers demandresponse programs. However, there still exist many unsolved problems in the area of energymanagement in power grids. In the following, we present some ideas for further research.1. Online OPF Algorithm: In Chapter 2, we studied the OPF problem for ac-dcgrids. Incorporation of intermittent renewable energy sources introduces uncertaintyto the available generation. Renewable generation is difficult to predict accurately.Therefore, it is imperative to implement an online and adaptive approach of adjustingthe operation of the power system in real-time. An interesting extension of Chapter2 is to develop an online ac-dc OPF algorithm that can adapt controllable devices(e.g., the tap changer, fast-response generator, controllable load) and interact withthe grid and power market. Modelling reactive power compensators (e.g., static VARcompensator, static synchronous compensator) is a challenge in designing a practicalonline OPF algorithm.2. SCUC Algorithm for Three-phase Unbalanced Networks: In Chapter 3, weproposed an SCUC algorithm to determine the near-optimal generation dispatch inac-dc grids. The proposed algorithm is designed for three-phase balanced transmis-sion networks. This assumption is valid for practical transmission networks. However,distribution networks can be unbalanced due to the continuous changes in the phase157Chapter 6. Conclusions and Future Workloads in response to unpredictable factors such as customer behaviour and weathercondition. With the advent of ac-dc microgrids in distribution networks, it is an in-teresting extension of Chapter 3 to design a proper SCUC algorithm for three-phaseunbalanced networks to determine the optimal dispatch of the distributed genera-tors in ac-dc microgrids. Modelling three-phase lines, loads, generators, and ac-dcconverters in the SCUC problem is a challenge that should be addressed.3. Impact of Network Constraints on Demand Response Algorithms: InChapters 4 and 5, we proposed demand response algorithms for real-time load schedul-ing in residential households and day-ahead contracts among data centers and utilitycompanies, respectively. However, we did not take into account the physical andoperational constraints imposed by the distribution network. We can extend thework in Chapters 4 and 5 by considering the constraints imposed by the distributionnetwork in the residential users and data centers demand response algorithms. Thisextension is a combination of the work in Chapter 2 and the work in Chapters 4and 5. The changes in the load demand of the users will change the power flow inthe network. Considering the network constraints will couple the decision making ofthe users in a way, which is unknown to each user as well as the utility companies.Coping with this lack of information is a challenge that should be addressed.158Bibliography[1] T. Elliott, K. Chen, and R. Swanekamp, Standard Handbook of Powerplant Engi-neering. McGraw Hill Professional, 2012.[2] J. D. Glover, M. S. Sarma, and T. Overbye, Power System Analysis and Design,5th ed. CT: Cengage Learning, 2011.[3] (accessed: Jul. 2017) Lazard’s Levelized Cost of EnergyAnalysis−Version 10.0. [Online]. Available: https://www.lazard.com/media/438038/levelized-cost-of-energy-v100.pdf[4] J. Hossain and A. Mahmud, Renewable Energy Integration: Challenges and Solutions.Springer Science & Business Media, 2014.[5] F. Milano, Power System Modelling and Scripting. Springer Science & BusinessMedia, 2010.[6] J. Zhu, Optimization of Power System Operation. NJ: Wiley, 2009.[7] L. Wu, Stochastic Security-Constrained Operational Planning in Power Systems.ProQuest, 2008.[8] Y. Fu, Z. Li, and L. Wu, “Modeling and solution of the large-scale security-constrained unit commitment,” IEEE Trans. on Power Systems, vol. 28, no. 4, pp.3524–3533, Nov. 2013.[9] G. Christensen and S. Soliman, Optimal Long-Term Operation of Electric PowerSystems. Springer Science & Business Media, 2012.[10] J. P. S. Catalao, Electric Power Systems: Advanced Forecasting Techniques and Op-timal Generation Scheduling. CRC Press, 2016.[11] A. Mazer, Electric Power Planning for Regulated and Deregulated Markets. JohnWiley & Sons, 2007.[12] S. Savulescu, Real-Time Stability in Power Systems: Techniques for Early Detectionof the Risk of Blackout, 2nd ed. Springer, 2014.159Bibliography[13] “Interim report on customer acceptance, retention, and response to time-basedrates from the consumer behavior studies,” U.S. Energy Information Administra-tion (EIA), Washington, DC, Tech. Rep., Jun. 2015.[14] J. O’Neill, Demand Response: Electricity Market Benefits and Energy EfficiencyCoordination. Nova Science Pub Inc., 2013.[15] Y. Huang, P. M. Pardalos, and Q. P. Zheng, Electrical Power Unit Commit-ment: Deterministic and Two-Stage Stochastic Programming Models and Algorithms.Springer U.S., 2017.[16] N. P. Padhy, “Unit commitment−A bibliographical survey,” IEEE Trans. on PowerSystems, vol. 19, no. 2, pp. 1196–1205, May 2004.[17] “Electric power annual 2013,” U.S. Energy Information Administration (EIA), Wash-ington, DC, Tech. Rep., 2015.[18] B. Stott, J. Jardim, and O. Alsac, “DC power flow revisited,” IEEE Trans. on PowerSystems, vol. 24, no. 3, pp. 1290–1300, Aug. 2009.[19] C. H. Lin and S. Y. Lin, “Distributed optimal power flow with discrete control vari-ables of large distributed power systems,” IEEE Trans. on Power Systems, vol. 23,no. 3, pp. 1383–1392, Aug. 2008.[20] L. Gan and S. H. Low, “An online gradient algorithm for optimal power flow on radialnetworks,” IEEE Journal on Selected Areas in Communications, vol. 34, no. 3, pp.625–638, Mar. 2016.[21] X. Zhao, Y. Yang, and H. Wei, “An interior point method based on continuousNewton’s method for optimal power flow,” in Proc. of IEEE Innovative Smart GridTechnologies (ISGT), Tianjin, China, May 2012.[22] R. A. Jabr, “Optimization of AC transmission system planning,” IEEE Trans. onPower Systems, vol. 28, no. 3, pp. 2779–2787, Aug. 2013.[23] A. G. Bakirtzis and P. N. Biskas, “A decentralized solution to the DC-OPF of in-terconnected power systems,” IEEE Trans. on Power Systems, vol. 18, no. 3, pp.1007–1013, Aug. 2003.[24] T. Erseghe, “Distributed optimal power flow using ADMM,” IEEE Trans. on PowerSystems, vol. 29, no. 5, pp. 2370–2380, Sept. 2014.[25] Q. Peng and S. H. Low, “Distributed algorithm for optimal power flow on a radialnetwork,” in Proc. of IEEE Conf. on Decision and Control, Dec. 2014, pp. 167–172.160Bibliography[26] S. Magnusson, P. C. Weeraddana, and C. Fischione, “A distributed approach forthe optimal power-flow problem based on ADMM and sequential convex approxima-tions,” IEEE Trans. on Control of Network Systems, vol. 2, no. 3, pp. 238–253, Sept.2015.[27] J. M. Arroyo and F. D. Galiana, “Energy and reserve pricing in security and network-constrained electricity markets,” IEEE Trans. on Power Systems, vol. 20, no. 2, pp.634–643, May 2005.[28] G. Hug-Glanzmann and G. Andersson, “Decentralized optimal power flow controlfor overlapping areas in power systems,” IEEE Trans. on Power Systems, vol. 24,no. 1, pp. 327–336, Feb. 2009.[29] S. Mhanna, A. C. Chapman, and G. Verbicˇ, “A fast distributed algorithm for large-scale demand response aggregation,” IEEE Trans. on Smart Grid, vol. 7, no. 4, pp.2094–2107, 2016.[30] M. R. AlRashidi and M. E. El-Hawary, “Hybrid particle swarm optimization ap-proach for solving the discrete OPF problem considering the valve loading effects,”IEEE Trans. on Power Systems, vol. 22, no. 4, pp. 2030–2038, Nov. 2007.[31] R. A. Jabr, “Optimal power flow using an extended conic quadratic formulation,”IEEE Trans. on Power Systems, vol. 23, no. 3, pp. 1000–1008, Aug. 2008.[32] X. Bai, H. Wei, K. Fujisawa, and Y. Wang, “Semidefinite programming for optimalpower flow problems,” Int.’l Journal of Electrical Power & Energy Systems, vol. 30,no. 67, pp. 383–392, Jul. 2008.[33] J. Lavaei and S. H. Low, “Zero duality gap in optimal power flow problem,” IEEETrans. on Power Systems, vol. 27, pp. 92–107, Feb. 2012.[34] S. H. Low, “Convex relaxation of optimal power flow - Part I: Formulations andequivalence,” IEEE Trans. on Control of Network Systems, vol. 1, no. 1, pp. 15–27,Mar. 2014.[35] ——, “Convex relaxation of optimal power flow - Part II: Exactness,” IEEE Trans.on Control of Network Systems, vol. 1, no. 2, pp. 177–189, Jun. 2014.[36] N. Chen, C. W. Tan, and T. Quek, “Electric vehicle charging in smart grid: Op-timality and valley-filling algorithms,” IEEE Journal of Selected Topics in SignalProcessing, vol. 8, no. 6, pp. 1073–1083, Dec. 2014.[37] J. Lavaei, D. Tse, and B. Zhang, “Geometry of power flows and optimization indistribution networks,” IEEE Trans. on Power Systems, vol. 29, no. 2, pp. 572–583,Mar. 2014.161Bibliography[38] M. Farivar and S. H. Low, “Branch flow model: Relaxations and convexification -Parts I and II,” IEEE Trans. on Power Systems, vol. 28, no. 3, pp. 2554–2572, Aug.2013.[39] C. W. Tan, D.W.H. Cai, and X. Lou, “Resistive network optimal power flow: Unique-ness and algorithms,” IEEE Trans. on Power Systems, vol. 30, no. 1, pp. 263–273,Jan. 2015.[40] S. Sojoudi, R. Madani, and J. Lavaei, “Low-rank solution of convex relaxation for op-timal power flow problem,” in Proc. of IEEE SmartGridComm, Vancouver, Canada,Oct. 2013.[41] C. Chen, A. Atamtrk, and S. S. Oren, “Bound tightening for the alternating currentoptimal power flow problem,” IEEE Trans. on Power Systems, vol. 31, no. 5, pp.3729–3736, Sept. 2016.[42] M. Sechilariu, B. Wang, and F. Locment, “Building integrated photovoltaic systemwith energy storage and smart grid communication,” IEEE Trans. on IndustrialElectronics, vol. 60, no. 4, pp. 1607–1618, Apr. 2013.[43] J. Liang, T. Jing, O. Gomis-Bellmunt, J. Ekanayake, and N. Jenkins, “Operation andcontrol of multiterminal HVDC transmission for offshore wind farms,” IEEE Trans.on Power Delivery, vol. 26, no. 4, pp. 2596–2604, Oct. 2011.[44] B. T. Patterson, “DC come home: DC microgrids and the birth of the Enernet,”IEEE Power and Energy Magazine, vol. 10, no. 6, pp. 60–69, Nov. 2012.[45] D. Salomonsson, L. Soder, and A. Sannino, “An adaptive control system for a DCmicrogrid for data centers,” IEEE Trans. on Industry Applications, vol. 44, no. 6,pp. 1910–1917, Nov. 2008.[46] Amin, R. T. Bambang, A. S. Rohman, C. J. Dronkers, R. Ortega, and A. Sasongko,“Energy management of fuel cell/battery/supercapacitor hybrid power sources usingmodel predictive control,” IEEE Trans. on Industrial Informatics, vol. 10, no. 4, pp.1992–2002, Nov. 2014.[47] D. Chen, L. Xu, and L. Yao, “DC voltage variation based autonomous control of DCmicrogrids,” IEEE Trans. on Power Delivery, vol. 28, no. 2, pp. 637–648, Apr. 2013.[48] A. Bracale, P. Caramiaa, G. Carpinelli, F. Mottola, and D. Proto, “A hybrid AC/DCsmart grid to improve power quality and reliability,” in Proc. of IEEE Int.’l EnergyConference and Exhibition (ENERGYCON), Florence, Italy, Sept. 2012.[49] W. Feng, A. L. Tuan, L. Tjernberg, A. Mannikoff, and A. Bergman, “A new approachfor benefit evaluation of multiterminal VSC-HVDC using a proposed mixed AC/DCoptimal power flow,” IEEE Trans. on Power Delivery, vol. 29, no. 1, pp. 432–443,Feb. 2014.162Bibliography[50] J. Beerten, S. Cole, and R. Belmans, “A sequential AC-DC power flow algorithm fornetworks containing multi-terminal VSC HVDC systems,” in Proc. of IEEE Powerand Energy Society General Meeting (PES), Minneapolis, MN, Jul. 2010.[51] R. Wiget and G. Andersson, “Optimal power flow for combined AC and multi-terminal HVDC grids based on VSC converters,” in Proc. of IEEE Power and EnergySociety General Meeting (PES), San Diego, CA, Jul. 2012.[52] J. Yu, W. Yan, W. Li, C. Chung, and K. Wong, “An unfixed piecewise-optimalreactive power-flow model and its algorithm for AC-DC systems,” IEEE Trans. onPower Systems, vol. 23, no. 1, pp. 170–176, Feb. 2008.[53] A. Pizano-Martinez, C. Fuerte-Esquivel, H. Ambriz-Perez, and E. Acha, “Modelingof VSC-based HVDC systems for a Newton-Raphson OPF algorithm,” IEEE Trans.on Power Systems, vol. 22, no. 4, pp. 1794–1803, Nov. 2007.[54] M. Baradar, M. Hesamzadeh, and M. Ghandhari, “Second-order cone programmingfor optimal power flow in VSC-type AC-DC grids,” IEEE Trans. on Power Systems,vol. 28, no. 4, pp. 4282–4291, Nov. 2013.[55] C. Liu, B. Zhang, Y. Hou, F. F. Wu, and Y. Liu, “An improved approach for AC-DC power flow calculation with multi-infeed DC systems,” IEEE Trans. on PowerSystems, vol. 26, no. 2, pp. 862–869, May 2011.[56] D. Jovcic and K. Ahmed, High Voltage Direct Current Transmission: Converters,Systems and DC Grids. Wiley, 2015.[57] F. Nejabatkhah and Y. W. Li, “Overview of power management strategies of hybridAC/DC microgrid,” IEEE Trans. on Power Electronics, vol. 30, no. 12, pp. 7072–7089, Dec. 2015.[58] M. R. Garey and D. S. Johnson, Computers and Intractability: A Guide to the Theoryof NP-Completeness. New York, NY: W. H. Freeman & Co., 1979.[59] T. Li and M. Shahidehpour, “Price-based unit commitment: A case of Lagrangianrelaxation versus mixed integer programming,” IEEE Trans. on Power Systems,vol. 20, no. 4, pp. 2015–2025, Nov. 2005.[60] S. Seguin, P. Cote, and C. Audet, “Self-scheduling short-term unit commitment andloading problem,” IEEE Trans. on Power Systems, vol. 31, no. 1, pp. 133–142, Jan.2016.[61] A. Lotfjou, M. Shahidehpour, and Y. Fu, “Hourly scheduling of DC transmissionlines in SCUC with voltage source converters,” IEEE Trans. on Power Delivery,vol. 26, no. 2, pp. 650–660, Apr. 2011.163Bibliography[62] L. Wu, “An improved decomposition framework for accelerating LSF and BD basedmethods for network-constrained UC problems,” IEEE Trans. on Power Systems,vol. 28, no. 4, pp. 3977–3986, Nov. 2013.[63] M. Paredes, L. Martins, and S. Soares, “Using semidefinite relaxation to solvethe day-ahead hydro unit commitment problem,” IEEE Trans. on Power Systems,vol. 30, no. 5, pp. 2695–2705, Sept. 2015.[64] X. Bai and H. Wei, “Semidefinite programming-based method for security-constrained unit commitment with operational and optimal power flow constraints,”IET Generation, Transmission & Distribution, vol. 3, no. 2, pp. 182–197, Feb. 2009.[65] A. Kalantari, J. F. Restrepo, and F. D. Galiana, “Security-constrained unit commit-ment with uncertain wind generation: The loadability set approach,” IEEE Trans.on Power Systems, vol. 28, no. 2, pp. 1787–1796, May 2013.[66] H. Wu and M. Shahidehpour, “Stochastic SCUC solution with variable wind energyusing constrained ordinal optimization,” IEEE Trans. on Sustainable Energy, vol. 5,no. 2, pp. 379–388, Apr. 2014.[67] C. Zhao and Y. Guan, “Data-driven stochastic unit commitment for integrating windgeneration,” IEEE Trans. on Power Systems, vol. 31, no. 4, pp. 2587–2596, July 2016.[68] D. Bertsimas, E. Litvinov, X. Sun, J. Zhao, and T. Zheng, “Adaptive robust opti-mization for the security constrained unit commitment problem,” IEEE Trans. onPower Systems, vol. 28, pp. 52–63, Feb. 2013.[69] H. Ye and Z. Li, “Robust security-constrained unit commitment and dispatch withrecourse cost requirement,” IEEE Trans. on Power Systems, vol. 31, no. 5, pp. 3527–3536, Sept. 2016.[70] C. Uckun, A. Botterud, and J. R. Birge, “An improved stochastic unit commitmentformulation to accommodate wind uncertainty,” IEEE Trans. on Power Systems,vol. 31, no. 4, pp. 2507–2517, Jul. 2016.[71] (accessed: Mar. 2017) Global Energy Statistical Yearbook. [Online]. Available: https://yearbook.enerdata.net/electricity-domestic-consumption-data-by-region.html[72] M. Erol-Kantarci and H. T. Mouftah, “Energy-efficient information and communi-cation infrastructures in the smart grid: A survey on interactions and open issues,”IEEE Communication Surveys and Tutorials, vol. 17, no. 1, pp. 179–197, Jan. 2015.[73] B. P. Esther and K. S. Kumar, “A survey on residential demand side managementarchitecture, approaches, optimization models and methods,” Renewable and Sus-tainable Energy Reviews, vol. 59, pp. 342–351, Jun. 2016.164Bibliography[74] A. Ipakchi and F. Albuyeh, “Grid of the future,” IEEE Power Energy Mag., vol. 7,no. 2, pp. 52–62, Mar. 2009.[75] A. D. Dominguez-Garcia, C. N. Hadjicostis, and N. H. Vaidya, “Resilient networkedcontrol of distributed energy resources,” IEEE Journal on Selected Areas in Com-munications, vol. 30, no. 6, pp. 1137–1148, Jul. 2012.[76] M. H. K. Tushar, C. Assi, M. Maier, and M. F. Uddin, “Smart microgrids: Optimaljoint scheduling for electric vehicles and home appliances,” IEEE Trans. on SmartGrid, vol. 5, no. 1, pp. 239–250, Jan. 2014.[77] C. O. Adika and L. Wang, “Autonomous appliance scheduling for household energymanagement,” IEEE Trans. on Smart Grid, vol. 5, no. 2, pp. 673–682, Mar. 2014.[78] A. Conejo, J. Morales, and L. Baringo, “Real-time demand response model,” IEEETrans. on Smart Grid, vol. 1, no. 3, pp. 236–242, Dec. 2010.[79] Federal Energy Regulatory Commission, “Assessment of demand response and ad-vanced metering,” Tech. Rep., Feb. 2011 (accessed: Apr. 2017), Available at:http://www.ferc.gov/legal/staff-reports/2010-dr-report.pdf.[80] P. Siano, “Demand response and smart grids–A survey,” Renewable and SustainableEnergy Reviews, vol. 30, pp. 461–478, 2014.[81] R. Deng, Z. Yang, M. Y. Chow, and J. Chen, “A survey on demand response insmart grids: Mathematical models and approaches,” IEEE Trans. on Industrial In-formatics, vol. 11, no. 3, pp. 570–582, Jun. 2015.[82] G. Strbac, “Demand side management: Benefits and challenges,” Energy Policy,vol. 36, no. 12, pp. 4419–4426, Dec. 2008.[83] (accessed: Mar. 2017) Independent Electricity System Operator (IESO).[Online]. Available: http://www.ieso.ca/en/sector-participants/market-operations/markets-and-related-programs/demand-response-auction[84] (accessed: Feb. 2017) Ontario Hydro One time-of-use pricing. [Online]. Available:http://www.hydroone.com/tou/Pages/Default.aspx[85] “Report on demand-side management activities for fiscal 2015,” British ColumbiaHydro and Power Authority (BC Hydro), Vancouver, BC, Tech. Rep., 2015.[86] (accessed: Mar. 2017) Hydro Que´bec demand response program. [Online]. Avail-able: http://www.hydroquebec.com/sustainable-development/energy-environment/demand-side-management.html165Bibliography[87] (accessed: Mar. 2017) SaskPower demand response program. [On-line]. Available: http://www.saskpower.com/efficiency-programs-and-tips/business-programs-and-offers/demand-response-program[88] “Alberta’s wholesale electricity market transition recommendation,” Alberta ElectricSystem Operation (AESO), Calgary, AB, Tech. Rep., 2016.[89] J. S. Vardakas, N. Zorba, and C. V. Verikoukis, “A survey on demand response pro-grams in smart grids: Pricing methods and optimization algorithms,” IEEE Com-munications Surveys Tutorials, vol. 17, no. 1, pp. 152–178, First quarter 2015.[90] P. Samadi, A. H. Mohsenian-Rad, V.W.S. Wong, and R. Schober, “Real-time pricingfor demand response based on stochastic approximation,” IEEE Trans. on SmartGrid, vol. 5, no. 2, pp. 789–798, Mar. 2014.[91] P. Cappers, J. MacDonald, J. Page, J. Potter, and E. Stewart, “Future opportuni-ties and challenges with using demand response as a resource in distribution systemoperation and planning activities,” Ernest Orlando Lawrence Berkeley National Lab-oratory, CA, Tech. Rep. DE-AC02-05CH11231, 2016.[92] A. Shehabi, S. Smith, D. Sartor, R. Brown, M. Herrlin, J. Koomey, E. Masanet,N. Horner, I. Azevedo, and W. Lintner, “United States data center energy usagereport,” Ernest Orlando Lawrence Berkeley National Laboratory, CA, Tech. Rep.DE-AC02-05CH1131, 2016.[93] A. Wierman, Z. Liu, I. Liu, and A. H. Mohsenian-Rad, “Opportunities and challengesfor data center demand response,” in Proc. of Int’l Green Computing Conference(IGCC), Dallas, TX, Nov. 2014.[94] Z. Liu, A. Wierman, Y. Chen, and B. Razon, “Data center demand response: Avoid-ing the coincident peak via workload shifting and local generation,” in Proc. of IEEEACM Int.’l Conf. on Measurement and Modeling of Computer Systems, New York,NY, Jun. 2013.[95] I. Brown, “Britain’s smart meter programme: A case study in privacy by design,”Int.’l Review of Law, Computers & Technology, vol. 28, no. 2, pp. 172–184, 2014.[96] S. Gutwirth, R. Leenes, P. de Hert, and Y. Poullet, Eds., European Data Protection:Coming of Age. Springer, NY, 2013.[97] (accessed: Apr. 2017) Alberta Energy. [Online]. Available: http://www.energy.alberta.ca/Electricity/679.asp[98] (accessed: Mar. 2017) Galvin Electricity Initiative. [Online]. Available: http://www.galvinpower.org/power-consumers/act/real-time-illinois/enroll-now166Bibliography[99] P. Samadi, A. H. Mohsenian-Rad, R. Schober, and V.W.S. Wong, “Tackling theload uncertainty challenges for energy consumption scheduling in smart grid,” IEEETrans. on Smart Grid, vol. 4, no. 2, pp. 1007–1016, Jun. 2013.[100] L. Jia and L. Tong, “Optimal pricing for residential demand response: A stochasticoptimization approach,” in Proc. of Allerton Conf. on Communication, Control, andComputing, Allerton, IL, Oct. 2012.[101] S.-J. Kim and G. B. Giannakis, “Real-time electricity pricing for demand responseusing online convex optimization,” in Proc. of IEEE Innovative Smart Grid Tech-nologies Conference (ISGT), Washington, DC, Feb. 2014.[102] Z. Chen, L. Wu, and Y. Fu, “Real-time price-based demand response managementfor residential appliances via stochastic optimization and robust optimization,” IEEETrans. on Smart Grid, vol. 3, no. 4, pp. 1822–1831, Dec. 2012.[103] C. Eksin, H. Delic, and A. Ribeiro, “Demand response management in smart gridswith heterogeneous consumer preferences,” IEEE Trans. on Smart Grid, vol. 6, no. 6,pp. 3082–3094, Nov. 2015.[104] N. Forouzandehmehr, M. Esmalifalak, A. H. Mohsenian-Rad, and Z. Han, “Au-tonomous demand response using stochastic differential games,” IEEE Trans. onSmart Grid, vol. 6, no. 1, pp. 291–300, Jan. 2015.[105] Z. Wen, D. O’Neill, and H. Maei, “Optimal demand response using device-basedreinforcement learning,” IEEE Trans. on Smart Grid, vol. 6, no. 5, pp. 2312–2324,Sept. 2015.[106] B. Kim, Y. Zhang, M. van der Schaar, and J. Lee, “Dynamic pricing and energyconsumption scheduling with reinforcement learning,” IEEE Trans. on Smart Grid,vol. 7, no. 5, pp. 2187–2198, Sept. 2016.[107] Y. Liang, L. He, X. Cao, and Z. J. Shen, “Stochastic control for smart grid userswith flexible demand,” IEEE Trans. on Smart Grid, vol. 4, no. 4, pp. 2296–2308,Dec. 2013.[108] F. Ruelens, B. J. Claessens, S. Vandael, B. D. Schutter, R. Babuska, and R. Belmans,“Residential demand response of thermostatically controlled loads using batch rein-forcement learning,” accepted for publication in IEEE Trans. on Smart Grid, 2016.[109] Y. Xiao and M. van der Schaar, “Distributed demand side management among fore-sighted decision makers in power networks,” in Proc. of IEEE Conf. on Signals,Systems and Computers, Pacific Grove, CA, Nov. 2013.[110] J. Yao and P. Venkitasubramaniam, “Optimal end user energy storage sharing indemand response,” in Proc. of IEEE SmartGridComm, Miami, FL, Nov. 2015.167Bibliography[111] L. Jia, Q. Zhao, and L. Tong, “Retail pricing for stochastic demand with unknownparameters: An online machine learning approach,” in Proc. of Allerton Conf. onCommunication, Control, and Computing, Monticello, IL, Oct. 2013.[112] E. Kalai and E. Lehrer, “Rational learning leads to Nash equilibrium,” Econometrica,vol. 39, no. 10, pp. 1019–1045, Jul. 1993.[113] A. Sandroni, “Does rational learning lead to Nash equilibrium in finitely repeatedgames?” Journal of Economic Theory, vol. 78, no. 1, pp. 195 – 218, 1998.[114] M. Bowling and M. Veloso, “Rational and convergent learning in stochastic games,”in Proc. of Int.’l Conf. on Artificial Intelligence, Seattle, WA, Aug. 2001.[115] L. M. Dermed and C. L. Isbell, “Solving stochastic games,” in Proc. of Annual Conf.on Neural Information Processing Systems, Vancouver, Canada, Dec. 2009.[116] L. Li and J. Shamma, “LP formulation of asymmetric zero-sum stochastic games,”in Proc. of IEEE Conf. on Decision and Control, Los Angeles, CA, Dec. 2014.[117] R. N. Borkovsky, U. Doraszelski, and Y. Kryukov, “A user’s guide to solving dynamicstochastic games using the homotopy method,” Operations Research, vol. 58, no. 4-part-2, pp. 1116–1132, Jul. 2010.[118] M. J. Neely, “A Lyapunov optimization approach to repeated stochastic games,” inProc. of Allerton Conf. on Communication, Control, and Computing, Monticello, IL,Oct. 2013.[119] R. Sutton and A. Barto, Reinforcement Learning: An Introduction. Cambridge,MA: MIT Press, 1998.[120] I. H. Witten, “An adaptive optimal controller for discrete-time Markov environ-ments,” Information and Control, vol. 34, no. 4, pp. 286–295, Aug. 1977.[121] A. G. Barto, R. S. Sutton, and C. W. Anderson, “Neuronlike adaptive elements thatcan solve difficult learning control problems,” IEEE Trans. on Systems, Man, andCybernetics, vol. 13, no. 5, pp. 834–846, Sept. 1983.[122] V. Konda and J. Tsitsiklis, “On actor-critic algorithms,” SIAM Journal on Controland Optimization, vol. 42, no. 4, pp. 1143–1166, Aug. 2003.[123] S. Bhatnagar, R. Sutton, M. Ghavamzadeh, and M. Lee, “Natural actor-critic algo-rithms,” Automatica, vol. 45, no. 11, pp. 2471–2482, Nov. 2009.[124] J. Li, Z. Li, K. Ren, and X. Liu, “Towards optimal electric demand management forInternet data centers,” IEEE Trans. on Smart Grid, vol. 3, no. 1, pp. 183–192, Mar.2012.168Bibliography[125] F. Kong and X. Liu, “A survey on green-energy-aware power management for data-centers,” ACM Comput. Surv., vol. 47, no. 2, pp. 1–38, Jan. 2015.[126] M. Ghamkhari and A. H. Mohsenian-Rad, “Energy and performance management ofgreen data centers: A profit maximization approach,” IEEE Trans. on Smart Grid,vol. 4, no. 2, pp. 1017–1025, Jun. 2013.[127] L. Rao, X. Liu, L. Xie, and W. Liu, “Coordinated energy cost management of dis-tributed Internet data centers in smart grid,” IEEE Trans. on Smart Grid, vol. 3,no. 1, pp. 50–58, Mar. 2012.[128] Z. Zhou, F. Liu, Z. Li, and H. Jin, “When smart grid meets geo-distributed cloud:An auction approach to datacenter demand response,” in Proc. of IEEE INFOCOM,Hong Kong, Apr. 2015.[129] L. Yu, T. Jiang, and Y. Zou, “Distributed real-time energy management in datacenter microgrids,” accepted for publication in IEEE Trans. on Smart Grid, 2016.[130] T. Chen, Y. Zhang, X. Wang, and G. B. Giannakis, “Robust workload and energymanagement for sustainable data centers,” IEEE Journal on Selected Areas in Com-munications, vol. 34, no. 3, pp. 651–664, Mar. 2016.[131] H. Wang, J. Huang, X. Lin, and A. H. Mohsenian-Rad, “Proactive demand responsefor data centers: A win-win solution,” IEEE Trans. on Smart Grid, vol. 7, no. 3, pp.1584–1596, Dec. 2015.[132] N. Tran, D. Tran, S. Ren, Z. Han, E. Huh, and C. Hong, “How geo-distributed datacenters do demand response: A game-theoretic approach,” IEEE Trans. on SmartGrid, vol. 7, no. 2, pp. 937–947, Mar. 2016.[133] Y. Guo and M. Pan, “Coordinated energy management for colocation data centersin smart grids,” in Proc. of IEEE SmartGridComm, Miami, FL, Nov. 2015.[134] H. Zhou, J. Yao, H. Guan, and X. Liu, “Comprehensive understanding of operationcost reduction using energy storage for IDCs,” in Proc. of IEEE INFOCOM, HongKong, Apr. 2015.[135] D. Wang, C. Ren, A. Sivasubramaniam, B. Urgaonkar, and H. Fathy, “Energy storagein datacenters: What, where, and how much?” in Proc. of IEEE ACM Int.’l Conf.on Measurement and Modeling of Computer Systems, New York, NY, Jun. 2012.[136] A. Mazer, Electric Power Planning for Regulated and Deregulated Markets. Hoboken,NJ: John Wiley & Sons, 2007.[137] S. Borenstein and J. Bushnell, “The U.S. electricity industry after 20 years of re-structuring,” National Bureau of Economic Research, Working Paper 21113, Apr.2015.169Bibliography[138] (accessed: Mar. 2017) Alberta Electric System Operator (AESO) website. [Online].Available: http://www.aeso.ca[139] S. Bahrami, V.W.S. Wong, and J. Jatskevich, “Optimal power flow for AC-DC net-works,” in Proc. of IEEE SmartGridComm, Venice, Italy, Nov. 2014.[140] S. Bahrami, F. Therrien, V.W.S. Wong, and J. Jatskevich, “Semidefinite relaxationof optimal power flow for AC-DC grids,” IEEE Trans. on Power Systems, vol. 32,no. 1, pp. 289–304, Jan. 2017.[141] J. Filar and K. Vrieze, Competitive Markov Decision Processes. NY: Springer, 1997.[142] R. Emery-Montemerlo, G. Gordon, J. Schneider, and S. Thrun, “Approximate so-lutions for partially observable stochastic games with common payoffs,” in Proc. ofInt.’l Conf. on Autonomous Agents and Multiagent Systems, New York, NY, Jul.2004.[143] F. Oliehoek, S. Whiteson, and M. Spaan, “Approximate solutions for factored Dec-POMDPs with many agents,” in Proc. of Int.’l Conf. on Autonomous Agents andMultiagent Systems, Saint Paul, MN, May 2013.[144] L. MacDermed, C. Isbell, and L. Weiss, “Markov games of incomplete information formulti-agent reinforcement learning,” in Proc. of Int.’l Conf. on Artificial Intelligence,San Fransico, CA, Aug. 2011.[145] S. Bahrami and V.W.S. Wong, “An autonomous demand response program in smartgrid with foresighted users,” in Proc. of IEEE SmartGridComm, Miami, FL, Nov.2015.[146] S. Bahrami, V.W.S. Wong, and J. Huang, “An online learning algorithm for demandresponse in smart grid,” accepted for publication in IEEE Trans. on Smart Grid,2017.[147] D. O. Neacsu, Switching Power Converters: Medium and High Power, 2nd ed. FL:CRC Press, 2013.[148] Z. Zhou, “Co-ordination of converter controls and an analysis of converter operatinglimits in VSC-HVDC grids,” Ph.D. dissertation, The University of Manitoba, 2013.[149] R. W. Erickson and D. Maksimovic, Fundamentals of Power Electronics, 2nd ed.NY: Springer, 2001.[150] M. Bierhoff and F. Fuchs, “Semiconductor losses in voltage source and current sourceIGBT converters based on analytical derivation,” in Proc. of IEEE Power ElectronicsSpecialists Conference (PESC), Aachen, Germany, Jun. 2004.170Bibliography[151] L. Mestha and P. Evans, “Analysis of on-state losses in PWM inverters,” IEE Pro-ceedings B, Electric Power Applications, vol. 136, no. 4, pp. 189–195, 1989.[152] G. Daelemans, “VSC HVDC in meshed networks,” Master’s thesis, Katholieke Uni-versiteit Leuven, Leuven, Belgium, 2008.[153] J. Cao, W. Du, H. Wang, and S. Bu, “Minimization of transmission loss in meshedAC/DC grids with VSC-MTDC networks,” IEEE Trans. on Power Systems, vol. 28,no. 3, pp. 3047–3055, Aug. 2013.[154] (accessed: Apr. 2015) The University of Washington, power systems test casearchive. [Online]. Available: https://www2.ee.washington.edu/research/pstca/[155] R. D. Zimmerman, C. E. Murillo-Sanchez, and R. J. Thomas, “MATPOWER: Steadystate operations, planning and analysis tools for power systems research and educa-tion,” IEEE Trans. on Power Systems, vol. 26, no. 1, pp. 12–19, Feb. 2011.[156] (accessed: Feb. 2015) TSPOPF, high performance AC optimal power flow solversfor MATPOWER. [Online]. Available: http://www.pserc.cornell.edu/tspopf[157] A. Wachter and L. T. Biegler, “On the implementation of a primal-dual interior pointfilter line search algorithm for large-scale nonlinear programming,” MathematicalProgramming, vol. 106, no. 1, pp. 25–57, Mar. 2006.[158] (accessed: Apr. 2015) Stanford Business Software, SNOPT. [Online]. Available:http://ampl.com/products/solvers/solvers-we-sell/snopt[159] P. E. Gill, W. Murray, M. A. Saunders, and E. Wong, “User’s guide for SNOPT7.5: Software for large-scale nonlinear programming,” Department of Mathematics,University of California, San Diego, La Jolla, CA, Center for Computational Math-ematics Report CCoM 15-3, 2015.[160] R. A. Jabr, “Exploiting sparsity in SDP relaxations of the OPF problem,” IEEETrans. on Power Systems, vol. 27, no. 2, pp. 1138–1139, May 2011.[161] D. K. Molzahn and I. A. Hiskens, “Sparsity-exploiting moment-based relaxations ofthe optimal power flow problem,” IEEE Trans. on Power Systems, vol. 30, no. 6, pp.3168–3180, Nov. 2015.[162] D. K. Molzahn, J. T. Holzer, B. C. Lesieutre, and C. L. DeMarco, “Implementationof a large-scale optimal power flow solver based on semidefinite programming,” IEEETrans. on Power Systems, vol. 28, no. 4, pp. 3987–3998, Nov. 2013.[163] G. C. Pflug, “Some remarks on the value-at-risk and the conditional value-at-risk,”in Probabilistic Constrained Optimization, S. P. Uryasev, Ed. Springer, US, 2000,pp. 272–281.171Bibliography[164] M. Shahidehpour, H. Yamin, and Z. Li, Market Operations in Electric Power Sys-tems: Forecasting, Scheduling, and Risk Management. NJ: Wiley-IEEE Press, 2002.[165] A. Wood, B. F. Wollenberg, and B. Sheble, Power Generation, Operation, and Con-trol. NJ: Wiley, 2013.[166] A. Street, F. Oliveira, and J. M. Arroyo, “Contingency-constrained unit commitmentwith n − k security criterion: A robust optimization approach,” IEEE Trans. onPower Systems, vol. 26, no. 3, pp. 1581–1590, Aug. 2011.[167] B. Hu and L. Wu, “Robust SCUC considering continuous/discrete uncertainties andquick-start units: A two-stage robust optimization with mixed-integer recourse,”IEEE Trans. on Power Systems, vol. 31, no. 2, pp. 1407–1419, Mar. 2016.[168] R. L.-Y. Chen, N. Fan, A. Pinar, and J.-P. Watson, “Contingency-constrained unitcommitment with post-contingency corrective recourse,” Annals of Operations Re-search, vol. 249, no. 1, pp. 381–407, 2017.[169] Q. Wang, J. P. Watson, and Y. Guan, “Two-stage robust optimization for n − kcontingency-constrained unit commitment,” IEEE Trans. on Power Systems, vol. 28,no. 3, pp. 2366–2375, Aug. 2013.[170] Y. Yu, P. B. Luh, E. Litvinov, T. Zheng, J. Zhao, F. Zhao, and D. A. Schiro,“Transmission contingency-constrained unit commitment with high penetration ofrenewables via interval optimization,” IEEE Trans. on Power Systems, vol. 32, no. 2,pp. 1410–1421, Mar. 2017.[171] C. Zhao and R. Jiang, “Distributionally robust contingency-constrained unit com-mitment,” accepted for publication in IEEE Trans. on Power Systems, 2017.[172] Z. Guo, R. Chen, N. Fan, and J. P. Watson, “Contingency-constrained unit com-mitment with intervening time for system adjustments,” accepted for publication inIEEE Trans. on Power Systems, vol. 32, no. 4, pp. 3049–3059, Jul. 2017.[173] L. E. Sokoler, P. Vinter, R. Brentsen, K. Edlund, and J. B. Jrgensen, “Contingency-constrained unit commitment in meshed isolated power systems,” IEEE Trans. onPower Systems, vol. 31, no. 5, pp. 3516–3526, Sept. 2016.[174] J. Wang, M. Shahidehpour, and Z. Li, “Contingency-constrained reserve require-ments in joint energy and ancillary services auction,” IEEE Trans. on Power Systems,vol. 24, no. 3, pp. 1457–1468, Aug. 2009.[175] R. T. Rockafellar and S. Uryasev, “Optimization of conditional value-at-risk,” Jour-nal of Risk, vol. 2, pp. 21–42, Apr. 2000.172Bibliography[176] E. Candes, M. Wakin, and S. Boyd, “Enhancing sparsity by reweighted l1 minimiza-tion,” Journal of Fourier Analysis and Applications, vol. 14, no. 5, pp. 877–905, Oct.2008.[177] M. J. Abdi, “Cardinality optimization problems,” Ph.D. dissertation, School of Math-ematics, The University of Birmingham, May 2013.[178] M. W. Jacobson and J. A. Fessler, “Properties of MM algorithms on convex feasiblesets: Extended version,” University of Michigan, Ann Arbor, MI, Tech. Rep. 353,2004.[179] (accessed: Mar. 2017) Independent electricity system operator (IESO). [Online].Available: http://www.ieso.ca[180] A. H. Mohsenian-Rad and A. Leon-Garcia, “Optimal residential load control withprice prediction in real-time electricity pricing environments,” IEEE Trans. on SmartGrid, vol. 1, no. 2, pp. 120–133, Sept. 2010.[181] D. W. Bunn, Modelling Prices in Competitive Electricity Markets. Tornoto, Canada:Wiley Finance, 2004.[182] R. S. Mamon and R. J. Elliott, Hidden Markov Models in Finance. NY: Springer,2014.[183] P. Yang, G. Tang, and A. Nehorai, “A game-theoretic approach for optimal time-of-use electricity pricing,” IEEE Trans. on Power Systems, vol. 28, no. 2, pp. 884–892,Aug. 2013.[184] Y. Shoham and K. Leyton-Brown, Multiagent Systems: Algorithmic, Game-Theoretic, and Logical Foundations. Cambridge University Press, 2008.[185] L. Busoniu, R. Babuska, B. D. Schutter, and D. Ernst, Reinforcement Learning andDynamic Programming Using Function Approximators. FL: CRC Press, 2009.[186] R. Parr, C. Painter-Wakefield, L. Li, and M. L. Littman, “Analyzing feature genera-tion for value-function approximation,” in Proc. of Int.’l Conf. on Machine Learning,New York, NY, Jun. 2007.[187] (accessed: Mar. 2016) Toronto Hydro. [Online].Available: http://www.torontohydro.com/sites\/electricsystem/residential/yourbilloverview/Pages/ApplianceChart.aspx[188] A. E. Roth and M. Sotomayor, Two-Sided Matching: A Study in Game-TheoreticModeling and Analysis. Cambridge University Press, 1992.173Bibliography[189] E. Hausman, R. Hornby, and A. Smith, “Bilateral contracting in deregulated electric-ity markets,” The American Public Power Association, Synapse Energy Economics,Tech. Rep., Apr. 2008.[190] R. Basmadjian, J. F. Botero, G. Giuliani, X. Hesselbach, S. Klingert, and H. D. Meer,“Making data centres fit for demand response: Introducing GreenSDA and GreenSLAcontracts,” accepted for publication in IEEE Trans. on Smart Grid, 2017.[191] J. Haraguchi and T. Matsumura, “Cournot–Bertrand comparison in a mixedoligopoly,” Journal of Economics, vol. 117, no. 2, pp. 117–136, Mar. 2016.[192] L. Kleinrock, Queueing Systems, Vol. 1: Theory. Wiley, 1976.[193] D. Gusfield and R. W. Irving, The Stable Marriage Problem: Structure and Algo-rithms. Boston, MA: MIT Press, 1989.[194] K. Bando, R. Kawasaki, and S. Muto, “Two-sided matching with externalities: Asurvey,” Journal of the Operations Research Society of Japan, vol. 59, no. 1, pp.35–71, Jan. 2016.[195] K. Bando, “A modified deferred acceptance algorithm for many-to-one matching mar-kets with externalities among firms,” Journal of Mathematical Economics, vol. 52,no. 1, pp. 173–181, May 2014.[196] D. Monderer and L. S. Shapley, “Potential games,” Games and Economic Behavior,vol. 14, no. 1, pp. 3124–143, May 1996.[197] E. Bodine-Baron, C. Lee, A. Chong, B. Hassibi, and A. Wierman, “Peer effects andstability in matching markets,” in Proc. of Int.’l Conf. on Algorithmic Game Theory,Amalfi, Italy, Oct. 2011.[198] S. Durand and B. Gaujal, Complexity and Optimality of the Best Response Algorithmin Random Potential Games. Berlin, Heidelberg: Springer Berlin Heidelberg, 2016,pp. 40–51.[199] (accessed: Feb. 2017) World Cup 98 web hits. [Online]. Available: http://ita.ee.lbl.gov/html/contrib/WorldCup.html[200] K. L. Rider, “A simple approximation to the average queue size in the time-dependentM/M/1 queue,” Journal of the ACM, vol. 23, no. 2, pp. 631–367, 1976.174Appendix AAppendices for Chapter 2A.1 Proof of Inequality (2.28)We first combine (2.13b) with constraint (2.7) and obtainIm{VkI∗k}+QDk ≤ |BCk |((V maxk )2 − V maxk |Vf |). (A.1)Substituting (2.17b) into (A.1), we obtainTr{Y¯kW}+QDk ≤ |BCk |((V maxk )2 − V maxk |Vf |). (A.2)We substitute ρk = −|BCk | (V maxk )2 +QDk and ξk = (BCkV maxk )2 into (A.2). Taking thesquare of both sides, we obtain(Tr{Y¯kW}+ ρk)2 ≥ ξk|Vf |2. (A.3)Substituting (2.17c) into (A.3), we obtain(Tr{Y¯kW}+ ρk)2 ≥ ξkTr{MfW}. (A.4)175Appendix A. Appendices for Chapter 2Thus, we haveTr{Y¯kW}2 + 2ρkTr{Y¯kW}+ ρ2k − ξkTr{MfW} ≥ 0. (A.5)Substituting Ck = (2ρk + 1)Y¯k − ξkMf into (A.5), we obtainTr{Y¯kW}2 − Tr{Y¯kW}+ ρ2k + Tr{CkW} ≥ 0. (A.6)We multiply (A.6) by positive number Tr{Y¯kW}2 to obtainTr{Y¯kW}4 − Tr{Y¯kW}3 + Tr{Y¯kW}2(ρ2k + Tr{CkW}) ≥ 0. (A.7)The matrix form of (A.7) is given in (2.28). The proof is completed. A.2 Proof of Theorem 2.1The solution matrices Ioptk , k ∈ N convac to problem (2.34) are symmetric positive semidefinite.Hence, I12,optk = I21,optk , I22,optk ≥ 0 for k ∈ N convac . However, we have I12,optk = 0. In fact, foreach k ∈ N convac , the coefficient bk in (2.3) is positive. Thus, a zero value for I12,optk reducesthe VSC losses. Therefore, it reduces the objective value of problem (2.34). By solvingproblem (2.34), the value of I12,optk , k ∈ N convac becomes as low as possible to minimize theobjective function. Matrix Ioptk , k ∈ N convac is positive semidefinite and constraint (2.32)holds. Thus, for each k ∈ N convac , the lowest value that I12,optk can take is I12,optk = 0. It isclear that the matrix Ioptk , k ∈ N convac is rank two when I12,optk = 0. The proof is completed.176Appendix A. Appendices for Chapter 2A.3 Proof of Theorem 2.2Let β33k , k ∈ G, W33 and I33k , k ∈ N convac denote the optimal solutions to problem (2.33).Also, let β35k , k ∈ G, W35 and I35k , k ∈ N convac denote the optimal solutions to problem(2.35). Since β35k , k ∈ G, W35 and I12,35k , k ∈ N convac minimize the objective function ofproblem (2.35), we have∑k∈Gβ35k + ω∑k∈NTr{YkW35} − ε∑k∈N convacI12,35k ≤∑k∈Gβ33k + ω∑k∈NTr{YkW33} − ε∑k∈N convacI12,33k .(A.8)Moreover, β33k , k ∈ G and W33 minimize the objective function of problem (2.33).Hence, we have∑k∈Gβ33k + ω∑k∈NTr{YkW33} ≤∑k∈Gβ35k + ω∑k∈NTr{YkW35}. (A.9)According to equation (2.20), we have fSDP,33obj =∑k∈G β33k + ω∑k∈N Tr{YkW33} andfSDP,35obj =∑k∈G β35k + ω∑k∈N Tr{YkW35}. Substituting fSDP,33obj and fSDP,35obj into inequal-ities (A.8) and (A.9), we obtainfSDP,35obj − ε∑k∈N convacI12,35k ≤ fSDP,33obj − ε∑k∈N convacI12,33k , (A.10a)fSDP,33obj ≤ fSDP,35obj . (A.10b)After rearranging the terms, inequality (A.10a) becomesfSDP,35obj − fSDP,33obj ≤ ε∑k∈N convac(I12,35k − I12,33k). (A.11)177Appendix A. Appendices for Chapter 2Inequality (A.10b) is equivalent to0 ≤ fSDP,35obj − fSDP,33obj , (A.12)which proves the left-hand side of (2.37). Moreover, converter currents I12,33k and I12,35kfor k ∈ N convac are nonnegative and upper bounded by Imaxk . Therefore,∑k∈N convac I12,35k −∑k∈N convac I12,33k in the right-hand side of (A.11) is bounded by∑k∈N convac(I12,35k − I12,33k) ≤ ∑k∈N convacImaxk . (A.13)By multiplying both sides of (A.13) by ε and substituting into (A.11), we obtainfSDP,35obj − fSDP,33obj ≤ ε∑k∈N convacImaxk . (A.14)The right-hand side of (A.13) is not a tight upper bound for∑k∈N convac(I12,35k − I12,33k).The exact value of I12,33k , k ∈ N convac cannot be obtained since problem (2.33) is difficult tobe solved. However, one can solve problem (2.34) and approximate I12,33k , k ∈ N convac bythe value of√I22,34k , k ∈ N convac , where I22,34k , k ∈ N convac is the solution to problem (2.34).We can show that the relaxation gap between problem (2.33) with bk = 0, k ∈ N convac andproblem (2.34) is zero. Condition bk = 0, k ∈ N convac implies that the linear losses (e.g.,switching losses) in the converters are zero. Thus, the losses in a VSC become lower. Thelosses in a VSC with bk = 0 and current I22,34k is Pconvloss,k = ckI22,34k + ak. It should be lowerthan the losses in a VSC with bk > 0 and current I12,33k obtained from (2.3). Thus, we haveckI22,34k + ak ≤ ck(I12,33k )2 + bkI12,33k + ak, ∀ k ∈ N convac . (A.15)178Appendix A. Appendices for Chapter 2Hence, we obtainI22,34k ≤ (I12,33k )2 +bkckI12,33k , ∀ k ∈ N convac . (A.16)We take the square root of both sides of (A.16). Then, we subtract both sides by I12,33k .For all k ∈ N convac , we obtain√I22,34k − I12,33k ≤√(I12,33k )2 +bkckI12,33k − I12,33k . (A.17)The right-hand side of (A.17),√(I12,33k )2 + bkckI12,33k − I12,33k , is an increasing function ofI12,33k . For all k ∈ N convac , we have√(I12,33k )2 +bkckI12,33k − I12,33k ≤ Imaxk(√1 +bkckImaxk− 1). (A.18)Substituting (A.18) into (A.17), we obtain√I22,34k − I12,33k ≤ Imaxk(√1 +bkckImaxk− 1), ∀ k ∈ N convac . (A.19)We add I12,35k to the both sides of (A.19). After rearranging the terms, we obtainI12,35k − I12,33k ≤ I12,35k −√I22,34k + Imaxk(√1 +bkckImaxk− 1), ∀ k ∈ N convac . (A.20)Substituting (A.20) into (A.11), we havefSDP,35obj − fSDP,33obj ≤ ε∑k∈N convac(I12,35k −√I22,34k + Imaxk(√1 +bkckImaxk− 1)). (A.21)The upper bound in (2.37) is obtained from (A.14) and (A.21). 179Appendix A. Appendices for Chapter 2A.4 Proof of Theorem 2.3We use the penalty function in the objective function of problem (2.36) to obtain rankone solution matrices Ioptk , k ∈ N convac with positive value for I12,optk . By changing the valueof I12,optk , k ∈ N convac from zero to positive, the total losses of the system will increase byapproximately∑k∈N convac bkI12,optk . The increase in the losses results in increasing the totalgeneration level. Let P 0Gk denote the generated power of the generator in bus k ∈ G whenI12,optk is zero for all k ∈ N convac . Let P+Gk denote the generated power of the generator inbus k ∈ G when I12,optk is positive for all k ∈ N convac . Let ∆fk and ∆Cgen denote the changein the generation cost of the generator in bus k ∈ G and the total generation cost of thesystem, respectively. If the generation level for generator k ∈ G increases from P 0Gk to P+Gk ,then the value of ∆fk can be approximated as∆fk ≈(2ck2P0Gk+ ck1) (P+Gk − P 0Gk). (A.22)The term 2ck2P0Gk+ ck1 in (A.22) is the marginal cost for the generator in bus k ∈ G withgeneration level P 0Gk . The value of ∆Cgen is∆Cgen =∑k∈G∆fk≈∑k∈G(2ck2P0Gk+ ck1) (P+Gk − P 0Gk). (A.23)Substituting cmax1 , cmax2 , and PmaxG into (A.23) for all generators k ∈ G, we obtain∆Cgen ≤ (2cmax2 PmaxG + cmax1 )∑k∈G(P+Gk − P 0Gk). (A.24)The increase in the generation level of the generators is due to the increase in the losses.180Appendix A. Appendices for Chapter 2Hence, we have∑k∈G(P+Gk − P 0Gk)=∑k∈N convac bkI12,optk and we obtain∆Cgen ≤ (2cmax2 PmaxG + cmax1 )∑k∈N convacbkI12,optk . (A.25)Substituting bmax into (A.25), we obtain∆Cgen ≤ bmax (2cmax2 PmaxG + cmax1 )∑k∈N convacI12,optk . (A.26)Let ∆fobj denote the change in the objective value of problem (2.34) when the valueof I12,optk , k ∈ N convac changes from zero to a positive number. The objective functionfobj includes the total generation cost and the total system losses. When the value ofI12,optk , k ∈ N convac changes from zero to positive, the system losses will increase by at mostbmax∑k∈N convac I12,optk . Therefore, from (A.26), we have∆fobj ≤ bmax (2cmax2 PmaxG + cmax1 + ω)∑k∈N convacI12,optk . (A.27)To obtain rank one solution matrices Ioptk , k ∈ N convac with positive value for I12,optk , it issufficient that the penalty function ε(∑k∈N convac I12,optk)be greater than the change in theobjective function ∆fobj. Therefore, the penalty coefficient ε can be approximated byε ≈ bmax (2cmax2 PmaxG + cmax1 + ω) . (A.28)The proof is completed. 181Appendix A. Appendices for Chapter 2A.5 Proof of Theorem 2.4If the solution matrices Wopt and Ioptk , k ∈ N convac to problem (2.36) are all rank one, thenWopt = xopt(xopt)T and Ioptk = ioptk (ioptk )T . Thus, the relaxation gap is zero. If Ioptk , k ∈N convac are all rank one, but Wopt is rank two, then Wopt has two nonzero eigenvalues φ1and φ2 with corresponding eigenvectors ν1 and ν2. It can be shown that the rank onematrix Wopt1 = (φ1 + φ2)ν1νT1 is also the solution of the OPF problem [33]. Matrix Wopt1has only one nonzero eigenvalue ϕ with corresponding eigenvector ψ. Then, the solutionvector xopt can be obtained from xopt =√ϕψ. If the rank of Wopt is greater than two, orat least one of the matrices Ioptk , k ∈ N convac is not rank one, then the relaxation gap maynot be zero. The proof is completed. A.6 Proof of Theorem 2.5For the sake of convenience, we rewrite the modified ac-dc OPF problem (2.35) again asfollows:minimize∑k∈Gβk + ω∑k∈NTr{YkW} − ε∑k∈N convacI12k (A.29a)subject to (2.18), and (2.21)−(2.32), (A.29b)rank(Ik) = 1, ∀ k ∈ N convac , (A.29c)rank(W) = 1. (A.29d)We consider the dual problem of problem (A.29). We define the dual variables. Letλk, γk, and µk denote the Lagrange multipliers associated with the lower inequalities in(2.21), (2.22), and (2.23), respectively. Let λk, γk, and µk denote the Lagrange multipliersassociated with the upper inequalities in (2.21), (2.22), and (2.23), respectively. For each182Appendix A. Appendices for Chapter 2transmission line (l,m) ∈ L, the matrixRlm =r1lm r2lm r3lmr2lm r4lm r5lmr3lm r5lm r6lmis the Lagrange multiplier associated with the matrix inequality (2.25). For each generatorbus k ∈ G, the matrixRk = 1 r1kr1k r2kis the Lagrange multiplier for the matrix inequality (2.18). λk, γk, and µk are defined asfollows:λk =λk − λk + ck1+ 2√ck2 r1k, if k ∈ Gλk − λk, otherwise,γk = γk − γk,µk = µk − µk.Let X denote the set of all multipliers λk, λk, γk, γk, µk, and µk. Also, let R denotethe set of all Lagrange multipliers Rlm and Rk.For each converter connected to ac bus k ∈ N convac and dc bus s ∈ N convdc , let ηks andθks denote the Lagrange multipliers associated with inequality (2.26) and equality (2.30),respectively. For each bus k ∈ N convac , let ϑk and υk denote the Lagrange multipliersassociated with inequalities (2.27) and (2.32), respectively. For each ac bus k ∈ N convac andfilter bus f ∈ Nac in a VSC station, let σkf denote the Lagrange multiplier associated with183Appendix A. Appendices for Chapter 2equality (2.31). Also, the matrixTkf =t1kf t2kf t3kf t4kft2kf t5kf t6kf t7kft3kf t6kf t8kf t9kft4kf t7kf t9kf t10kfis the Lagrange multiplier associated with the matrix inequality (2.28). For each converterac bus k ∈ N convac , the matrixTk =t1k t2k t3kt2k t4k t5kt3k t5k t6kis the Lagrange multiplier associated with the matrix inequality (2.29). Let T denote theset of all Lagrange multipliers ηks, θks, ϑk, υk, σkf , Tkf , and Tk.We define an affine function h(X,R, T ) as follows:h(X,R, T ) =∑k∈N\N conv(λkPminGk− λkPmaxGk + λkPDk + γkQminGk − γkQmaxGk + γkQDk+ µk(V mink)2 − µk (V maxk )2 )+∑k∈G(ck0 − r2k)− ∑(l,m)∈L(Smaxlm r1lm + r4lm +r6lm)+∑k∈N convac((2t2k + θks)PDk + 2t3kQDk − ρ2kt1kf − t10kf − t4k− t6k)−∑k∈N convacϑkmb SnomCk+∑s∈N convdc(θksPDs) . (A.30)184Appendix A. Appendices for Chapter 2Furthermore, we define the functionA(X,R, T ) =∑k∈N\N conv(λkYk + γkY¯k + µk Mk)+∑(l,m)∈L(2 r2lmYlm + 2 r3lmY¯lm)+∑k∈N convac((2 t2k + θks)Yk + (√2 t2kf +√2 t3kf + 2 t4kf + t5kf + 2√2 t7kf + t8kf+ 2 t3k − ϑk)Y¯k − t1kfCk+(ηks−(Imaxk )2 t1k)Mk)+∑s∈N convdc(θksYs−m2aηksMs).(A.31)In (A.30) and (A.31), the converter ac bus k ∈ N convac , the converter dc bus s ∈ N convdc andthe filter bus f ∈ Nac are in the same VSC station. Hence, we only use index k in the fifthsummation of (A.30) and the third summation of (A.31).Also, we define the following affine function for converter ac bus k ∈ N convac , converterdc bus s ∈ N convdc , and filter bus f ∈ Nac in the same VSC stationBk(X) =12 0 ε+ υkε+ υk 2σkf+ θksSk. (A.32)Consider the dual of the ac-dc OPF problem as follows:maximizeX,R,Th(X,R, T ) (A.33a)subject to A(X,R, T ) 0, (A.33b)Bk(X) 0, ∀ k ∈ N convac (A.33c)Rlm 0, ∀ (l,m) ∈ L (A.33d)Rk 0, ∀ k ∈ G (A.33e)Tkf 0, ∀ k ∈ N convac , ∀f ∈ Nac (A.33f)185Appendix A. Appendices for Chapter 2Tk 0, ∀ k ∈ N convac (A.33g)λk, λk, γk, γk, µk, µk ≥ 0, ∀ k ∈ N (A.33h)ηks, ϑk ≥ 0, ∀ k ∈ N convac , ∀s ∈ N convdc . (A.33i)Problem (A.33) is the dual of problem (A.29). We can show that problem (A.33) is alsothe dual of problem (2.36) and strong duality holds between these optimization problems.Furthermore, the matrix W in problem (2.36) is the Lagrange multiplier associated withthe inequality constraint (A.33b). Besides, for every k ∈ N convac , the matrix Ik is theLagrange multiplier associated with inequality (A.33c). Let (Xopt, Ropt, T opt) denote thesolution to problem (A.33). From the complementary slackness in Karush-Kuhn-Tucker(KKT) conditions, we obtainTr{A(Xopt, Ropt, T opt)Wopt} = 0, (A.34a)Tr{Bk(Xopt)Ioptk } = 0, ∀ k ∈ N convac . (A.34b)From (A.34a) and (A.34b), the orthogonal eigenvectors of Wopt and Ioptk , k ∈ N convac belongto the null space of A(Xopt, Ropt, T opt) and Bk(Xopt), k ∈ N convac , respectively [33]. Thus,if matrix A(Xopt, Ropt, T opt) has a zero eigenvalue of multiplicity two, then matrix Woptis at most rank 2. If matrix Bk(Xopt), k ∈ N convac has one zero eigenvalue, then matrixIoptk , k ∈ N convac is also rank one. Lemma A.6.1 summarizes the obtained result for an ac-dcgrid O.Lemma A.6.1 The solution matrix Wopt to problem (2.36) is at most rank 2 if the so-lution matrix A(Xopt, Ropt, T opt) to problem (A.33) has a zero eigenvalue of multiplicitytwo.In [33], it is shown that the Lemma A.6.1 remains valid for the ac grid Oac. In the186Appendix A. Appendices for Chapter 2ac grid Oac, the constraints imposed by the converters are removed. Hence, the Lagrangemultipliers in set T will be removed from the affine functions defined in (A.30) and (A.31).Besides, we do not require to define functions Bk(X) in (A.32). Consequently, for the acgrid Oac, function h(X,R, T ) and matrix A(X,R, T ) will be replaced by function h(X,R)and matrix A(X,R), respectively. The dual OPF problem will be simplified to an optimiza-tion problem with objective function h(X,R) and constraints (A.33b), (A.33d), (A.33e),and (A.33h). Details can be found in [33]. Let (Xopt, Ropt) denote the solution to the dualOPF problem in the ac grid Oac. Then, the solution matrix Wopt to the SDP relaxationform of the ac OPF problem is at most rank 2 if the solution matrix A(Xopt, Ropt) to thedual of the ac OPF problem has a zero eigenvalue of multiplicity two [33]. We use this factto prove Theorem 2.5.It is sufficient to show that the solution matrix A(Xopt, Ropt, T opt) to the dual OPFproblem in the ac-dc grid O has a zero eigenvalue of multiplicity two if the solution ma-trix A(Xopt, Ropt) to the dual OPF problem in the ac grid Oac has a zero eigenvalue ofmultiplicity two.Consider the dual OPF problem in the ac grid Oac. The solution matrix A(Xopt, Ropt)has a simple structure as follows:A(Xopt, Ropt) =12 H1(Xopt, Ropt) H2(Xopt, Ropt)−H2(Xopt, Ropt) H1(Xopt, Ropt), (A.35)where H1(Xopt, Ropt) and H2(Xopt, Ropt) are symmetric real matrices. Consider matrixH1(Xopt, Ropt). Let Ylm denote the entry (l,m) of admittance matrix Y . The off-diagonalentry (l,m) ∈ L of H1(Xopt, Ropt) can be obtained asH1(Xopt, Ropt)lm = Re{Ylm}(λl + λm + 2r2lm)− Im{Ylm}(γl + γm + 2r3lm). (A.36)187Appendix A. Appendices for Chapter 2In [33], it is shown that the smallest eigenvalue of A(Xopt, Ropt) is zero if two conditionsare satisfied. First, the graph of the resistive part of the grid is connected. That is, thereexists a connected path between all two buses in the graph of the resistive part of the grid.Second, the off-diagonal entries of H1(Xopt, Ropt) are nonpositive and the entries of matrixH2(Xopt, Ropt) are sufficiently smaller than the entries of H1(Xopt, Ropt). For a practicalac grid with connected resistive part, the off-diagonal entries of H1(Xopt, Ropt) are nonpos-itive because Re{Ylm}, (l,m) ∈ L is nonpositive and Im{Ylm}, (l,m) ∈ L is nonnegative.Furthermore, the Lagrange multipliers λk, k ∈ N are positive and γk, k ∈ N are small ascompared with λk, k ∈ N . Similarly, we can obtain the entries of matrix H2(Xopt, Ropt)and show that they are sufficiently smaller than the entries of matrixH1(Xopt, Ropt). Hence,the smallest eigenvalue of A(Xopt, Ropt) is zero. The structure of A(Xopt, Ropt) guaranteesthat its smallest eigenvalue has multiplicity of two [33].Now, consider the dual OPF problem in the ac-dc grid O. Matrix A(Xopt, Ropt, T opt)has a similar structure to (A.35). There exist symmetric real matrices H1(Xopt, Ropt, T opt)and H2(Xopt, Ropt, T opt), for which we haveA(Xopt, Ropt, T opt) =12 H1(Xopt, Ropt, T opt) H2(Xopt, Ropt, T opt)−H2(Xopt, Ropt, T opt) H1(Xopt, Ropt, T opt) . (A.37)Again, the smallest eigenvalue of A(Xopt, Ropt, T opt) is zero if the graph of the resistive partof the grid is connected and the off-diagonal entries of H1(Xopt, Ropt, T opt) are nonpositive,as well as the entries of matrix H2(Xopt, Ropt, T opt) are sufficiently smaller than the entriesof H1(Xopt, Ropt, T opt). In the ac-dc grid O, the converter buses are not connected toeach other directly. We add a large resistance (e.g., 105 pu) between the converter ac busk ∈ N convac and dc bus s ∈ N convdc in the same VSC station. Since the added resistance issufficiently large, the converter buses k and s will have independent voltage magnitudes188Appendix A. Appendices for Chapter 2as before. The grids O and Oac are different in the constraints imposed by the VSCs withbuses f ∈ Nac, k ∈ N convac and s ∈ N convdc . Moreover, in the ac-dc grid O, the converterlosses are included in the system losses. Hence, at the global optimal solution to the OPFproblem, the generation levels in O will be higher than the generation levels in Oac tocompensate the higher losses in the ac-dc grid. Thus, the optimal value in O is greaterthan the optimal value inOac. The Lagrange multipliers λk, k ∈ N\N conv measure the rateof increase of the objective function at the optimal point as the corresponding constraintis relaxed. Consequently, we have the following proposition.Lemma A.6.2 The Lagrange multipliers λk, k ∈ N \N conv in the ac-dc grid O are greaterthan or equal to their corresponding Lagrange multipliers in Oac.If the off-diagonal entries of H1(Xopt, Ropt) for ac grid Oac are nonpositive, then theoff-diagonal entries (l,m) ∈ L, l,m ∈ N \ N conv of H1(Xopt, Ropt, T opt) for ac-dc grid Oremain nonpositive. The other off-diagonal entries can be obtained as follows:For k ∈ N convac and s ∈ N convdc , we haveH1(Xopt, Ropt, T opt)ks = Re{Yks}(2θks). (A.38)For (f, k) ∈ L, f ∈ Nac, and k ∈ N convac , we haveH1(Xopt, Ropt, T opt)fk = Re{Yfk}(λf + 2r2kf + 2t2k + 2θks)−Im{Ykf}(√2t2kf +√2t3kf + 2t4kf+ t5kf + 2√2t7kf + t8kf + 2t3k − ϑk + λ¯f + 2r3kf − (2ρk + 1)t1kf).(A.39)For (s,m) ∈ L, s ∈ N convdc , and m ∈ N \ N conv, we haveH1(Xopt, Ropt, T opt)sm = Re{Ysm}(2r2sm + 2θks + λm). (A.40)189Appendix A. Appendices for Chapter 2For practical ac-dc grids, θks is nonnegative since over satisfaction of the active loadsPDs and PDk in converter buses leads to higher losses and generation levels in the system.Furthermore, λf , f ∈ Nac and λm, m ∈ N\N conv are positive in the ac-dc gridO since theyare greater than or equal to their corresponding Lagrange multipliers in Oac (Lemma A.6.2)and they are positive in Oac. Entry t1kf is positive since matrix Tk is positive semidefinite.ρk = −|BCk | (V maxk )2 +QDk is a large negative number. Therefore, (2ρk+1)t1kf is a negativenumber. Other Lagrange multipliers associated with the inequality constraints are positive.Other Lagrange multipliers associated with the equality constraints can be either positiveor negative. If they are negative, they have small values compared with the value of(2ρk + 1)t1kf , λf , λm, and θks. Consequently, the off-diagonal entries of H1(Xopt, Ropt, T opt)for the ac-dc grid O are nonpositive. Similarly, we can show that the entries of matrixH2(Xopt, Ropt, T opt) are sufficiently smaller than the entries of H1(Xopt, Ropt, T opt) for theac-dc grid O. Therefore, matrix A(Xopt, Ropt, T opt) has a zero eigenvalue of multiplicitytwo for the ac-dc grid O, and Wopt is at most rank two. The proof is completed. 190Appendix BAppendices for Chapter 3B.1 Transforming Problem (3.24) into an SDPFor k ∈ N , let ek denote the kth basis vector in R|N | and Yk = ekeTk Y . The row k ofmatrix Yk is equal to the row k of the admittance matrix Y . The other entries of Yk arezero. We use the Π model of the transmission lines. Let ylm and y¯lm denote the valueof the series and shunt elements at bus l connected to bus m, respectively. We defineYlm = (y¯lm + ylm)eleTl − (ylm)eleTm, where the entries (l, l) and (l,m) of Ylm are equal toy¯lm + ylm and −ylm, respectively. The other entries of Ylm are zero. We define matricesYk, Y¯k, Ylm, Y¯lm, Mk and Mlm as follows.Yk =12Re{Yk + Y Tk } Im{Y Tk − Yk}Im{Yk − Y Tk } Re{Yk + Y Tk } ,Y¯k = −12Im{Yk + Y Tk } Re{Yk − Y Tk }Re{Y Tk − Yk} Im{Yk + Y Tk } ,Ylm =12Re{Ylm + Y Tlm} Im{Y Tlm − Ylm}Im{Ylm − Y Tlm} Re{Ylm + Y Tlm} ,191Appendix B. Appendices for Chapter 3Y¯lm = −12Im{Ylm + Y Tlm} Re{Ylm − Y Tlm}Re{Y Tlm − Ylm} Im{Ylm + Y Tlm} ,Mk =ekeTk 00 ekeTk ,Mlm =(el − em)(el − em)T 00 (el − em)(el − em)T .We use the notation Tr{A} to represent the trace of an arbitrary square matrix A. Itcan be shown thatRe{Vk(t)I∗k(t)} = Tr{YkW(t)}, k ∈ N (B.1a)Im{Vk(t)I∗k(t)} = Tr{Y¯kW(t)}, k ∈ N (B.1b)|Vk(t)|2 = Tr{MkW(t)}, k ∈ N (B.1c)|Vl(t)− Vm(t)|2 = Tr{MlmW(t)}, (l,m) ∈ L (B.1d)|Slm(t)|2 =Tr{YlmW(t)}2+Tr{Y¯lmW(t)}2, (l,m) ∈ L. (B.1e)We use (B.1a)−(B.1e) to represent the constraints and objective function of the SCUCproblem (3.24) in terms of matrix W(t), t ∈ T . Since Ik(t) = ik(t)ik(t)T, for a VSCstation with ac converter bus k ∈ N convac , we haveIk(t) =(Imaxk + |Ik(t)|)24(Imaxk )2 − |Ik(t)|22(Imaxk )2 − |Ik(t)|22(Imaxk − |Ik(t)|)2. (B.2)192Appendix B. Appendices for Chapter 3We define symmetric matrix Sk for k ∈ N convac asSk =S11k S12kS21k S22k , (B.3)where S12k = S21k =2ak + bk(Imaxk − 1) + ck(2− 4 (Imaxk )2)4 (Imaxk )2 , S11k =ak + bkImaxk + ck(Imaxk )2, andS22k =ak − bk + ck4 (Imaxk )2 . The VSC losses in (3.1) can be written asP convloss,k(t) = Tr{SkIk(t)}, k ∈ N convac , t ∈ T . (B.4)In the following parts, we transform the constraints and objective function of the SCUCproblem (3.24) into an SDP.Transforming the ConstraintsSubstituting (B.1a) into (3.18b), we have PCk(t) = Tr{YkW(t)}, k ∈ N convac and PCs(t) =Tr{YsW(t)}, s ∈ N convdc . We also substitute P convloss,k(t) = Tr{SkIk(t)} into (3.2). For k ∈N convac , s ∈ N convdc , and t ∈ T , we obtainTr{YkW(t)}+ Tr{YsW(t)}+ Tr{SkIk(t)} = 0. (B.5)We define ρk(t) = −|BCk | (V maxk )2, ξk = (BCkV maxk )2, and Ck(t) = (2ρ(t)k+1)Y¯k−ξkMffor ac bus k ∈ N convac connected to filter bus f ∈ Nac. In Chapter 2, we have used constraint(3.3) in the constraint set of the ac-dc OPF problem. In Appendix A.1, it is shown that193Appendix B. Appendices for Chapter 3constraint (3.3) is equivalent to the following matrix inequality:ρ2k(t) + Tr{Ck(t)W(t)}Tr{Y¯kW(t)}√2Tr{Y¯kW(t)}√2Tr{Y¯kW(t)}Tr{Y¯kW(t)}√2Tr{Y¯kW(t)} 0√2Tr{Y¯kW(t)}Tr{Y¯kW(t)}√20 Tr{Y¯kW(t)} 0Tr{Y¯kW(t)}√2Tr{Y¯kW(t)} 0 1 0.(B.6)The SDP form of constraint (3.4) is−mqkSnomCk ≤ Tr{Y¯kW(t)}. (B.7)The matrix form of inequality (3.5) is(Imaxk )2 Tr{MkW(t)} Tr{YkW(t)}+ PDk(t) Tr{Y¯kW(t)}Tr{YkW(t)} 1 0Tr{Y¯kW(t)} 0 1 0. (B.8)The SDP form of constraint (3.6) isTr{MkW(t)} ≤ (mvks)2 Tr{MsW(t)}. (B.9)The active power balance equation in (3.18a) can be combined with constraint (3.11a).Substituting (B.1a) into (3.11a), for k ∈ N \ N conv and t ∈ T , we obtainuk(t)PminGk+ PBk(t)− PDk(t)≤Tr{YkW(t)} ≤uk(t)PmaxGk + PBk(t)− PDk(t) (B.10)194Appendix B. Appendices for Chapter 3Similarly, for ac buses k ∈ Nac and t ∈ T , constraint (3.11b) becomesuk(t)QminGk−QDk(t) ≤ Tr{Y¯kW(t)} ≤ uk(t)QmaxGk −QDk(t). (B.11)The active power balance equation in (3.18a) for time slots t−1 and t can be combinedwith constraints (3.12a) and (3.12b). By using (B.1a), for k ∈ N and t ∈ T , we obtainTr{YkW(t)} − Tr{YkW(t− 1)} − PBk(t) + PBk(t− 1)≤ uk(t− 1)ruk + sk(t)rsuk + PDk(t− 1)− PDk(t), (B.12a)Tr{YkW(t− 1)} − Tr{YkW(t)} − PBk(t− 1) + PBk(t)≤ uk(t− 1)rdk+ sk(t) rsdk + PDk(t)− PDk(t− 1). (B.12b)By including (3.25) into the constraint set, the variable uk(t) can take any value in theinterval [0, 1]. Thus, constraint (3.15a) can be relaxed as0 ≤ uk(t) ≤ 1, k ∈ N , t ∈ T . (B.13)Substituting (B.1c) into (3.19a), we obtain(V mink )2 ≤ Tr{MkW(t)} ≤ (V maxk )2, k ∈ N , t ∈ T . (B.14)Substituting (B.1e) into (3.19b), we have Tr{YlmW(t)}2 + Tr{Y¯lmW(t)}2≤ (Smaxlm )2.195Appendix B. Appendices for Chapter 3Its matrix form is(Smaxlm )2 Tr{YlmW(t)} Tr{Y¯lmW(t)}Tr{YlmW(t)} 1 0Tr{Y¯lmW(t)} 0 1 0, (l,m) ∈ L. (B.15)Let I12k (t) denote the entry in the first row and the second column of matrix Ik(t) in(B.2). We have Imaxk − 2I12k (t) = |Ik(t)|2 = (R2Ck +X2Ck)−1|Vk(t)− Vf (t)|2 in time slot t forac bus k ∈ N convac and filter bus f ∈ Nac in a VSC station. From (B.1d), we obtainI12k (t) =Tr{MkfW(t)}2(Imaxk − (R2Ck +X2Ck)) . (B.16)Let I11k (t) and I22k (t) denote the diagonal entries of matrix Ik(t) in (B.2). From (B.2),we haveI11k (t)+I22k (t)4=(Imaxk )2−I12k (t), k ∈ N convac , t ∈ T (B.17a)I11k (t) ≥(Imaxk )24, k ∈ N convac , t ∈ T . (B.17b)In Lemma B.1.1, we show that constraints (B.17a) and (B.17b) are sufficient to obtainthe solution matrices Ik(t), k ∈ N convac , t ∈ T with the form in (B.2).Lemma B.1.1 If matrix Ik(t), k ∈ N convac , t ∈ T is rank one and its entries satisfy con-straints (B.17b) and (B.17b), then there exists a nonnegative number |Ik(t)|, for whichmatrix Ik(t) can be written as (B.2).Proof : Consider a rank one matrix Ik(t), and its entries satisfy constraints (B.17b) and196Appendix B. Appendices for Chapter 3(B.17b). The determinant of the rank one matrix Ik(t) is zero. Hence, we haveI11k (t) I22k (t) =(I12k (t))2. (B.18)From constraint (B.17a), we haveI22k (t) = 4((Imaxk )2−I12k (t)− I11k (t)). (B.19)By substituting (B.19) into (B.18), we obtain4 I11k (t)((Imaxk )2−I12k (t)− I11k (t))=(I12k (t))2. (B.20)By rearranging (B.20), we have4 I11k (t) (Imaxk )2 =(I12k (t))2+ 4 I11k (t) I12k (t) +(2 I11k (t))2=(I12k (t) + 2 I11k (t))2. (B.21)Constraint (B.17b) implies that there exists a nonnegative number |Ik(t)| ≥ 0, for whichwe haveI11k (t) =(Imaxk + |Ik(t)|)24. (B.22)We substitute (B.22) into (B.21). Taking the square root of both sides, we obtainI12k (t) =(Imaxk )2 − |Ik(t)|22. (B.23)197Appendix B. Appendices for Chapter 3Substituting (B.22) and (B.23) into (B.18), we haveI22k (t) = (Imaxk − |Ik(t)|)2. (B.24)Equations (B.22), (B.23), and (B.24) imply that matrix Ik(t) has the form in (B.2). Thiscompletes the proof. Lemma B.1.1 implies that if we include constraints (B.16), (B.17b), and (B.17b) intothe constraint set of the SCUC problem and determine rank one solution matrices Ik(t), k ∈N convac , t ∈ T , then we can obtain the converters’ ac current.Transforming the Objective FunctionSubstituting (B.1a) into (3.18a) for k ∈ N , we have PGk(t) = Tr{YkW(t)} + PDk(t) −PBk(t). Let vector PB(t) = (PBk(t), k ∈ N ) denote the profile of injected active power fromthe energy storage systems into the grid in time slot t. The generation cost function (3.9)becomesCgen(W(t), PD(t), PB(t), u(t), s(t), d(t))=∑k∈N(ck2(Tr{YkW(t)}+ PDk(t)− PBk(t))2+ ck1(Tr{YkW(t)}+ PDk(t)− PBk(t))+ ck0uk(t) + csuk sk(t) + csdk dk(t)). (B.25)In the SDP form, the objective function must be linear. We introduce the auxiliaryvariables ϑk(t), k ∈ N , t ∈ T and replace Cgen(·) with∑k∈N ϑk(t) +ck0uk(t) + csuk sk(t) +csdk dk(t). We include the matrix form of inequality ck2 (Tr{YkW(t)}+ PDk(t)− PBk(t))2 +ck1(Tr{YkW(t)} +PDk(t)−PBk(t)) ≤ ϑk(t) into the constraints set for all generators. For198Appendix B. Appendices for Chapter 3k ∈ N and t ∈ T , we haveϑk(t)− ck1ωk(t) √ck2 ωk(t)√ck2 ωk(t) 1 0, (B.26)where ωk(t) = Tr{YkW(t)}+ PDk(t)− PBk(t) in time slot t.By introducing the auxiliary variables µj(t) for each sample j in time slot t and ηjk(t)for each bus k and sample j in time slot t, we can replace function Γβ(·) in the objectivefunction (3.23) with α(t)+ 11−β∑j∈J Pr{P jG(t), PjD(t)}µj(t). Then, we include the follow-ing inequalities for bus k ∈ N , sample j ∈ J , and time slot t ∈ T into the constraints set:∑k∈Ncres,kηjk(t) ≤ µj(t) + α(t), (B.27a)PGk(t)− PDk(t)− P jGk(t) + P jDk(t) ≤ ηjk(t). (B.27b)To represent the total system losses, we can substitute (B.1a) into (3.16). Thus, weobtainPloss(t) =∑k∈NTr{YkW(t)}. (B.28)The SDP form of the objective function in (3.23) can be expressed asfSDPobj =∑t∈T(∑k∈N(ϑk(t) + ck0uk(t) + csuk sk(t) + csdk dk(t))+ ωloss∑k∈NTr{YkW(t)}+ ωcvar(α(t)+11− β∑j∈JPr{P jG(t),PjD(t)}µj(t))). (B.29)Constraints (3.7), (3.8), (3.13a), (3.13b), (3.14), and (3.15b) in the main manuscript,199Appendix B. Appendices for Chapter 3as well as constraints (B.5)−(B.17b), and (B.26)−(B.27b) define the feasible set ΦSDPfor the decision variables φ = (ϑk(t), uk(t), sk(t), dk(t), PBk(t), µj(t), ηjk(t), j ∈ J , k ∈N , α(t),W(t), Ik(t), k ∈ N convac , t ∈ T ).B.2 Proof of Theorem 3.1We show that the solution matrices Ioptk,θ1,θ2(t), k ∈ N convac , t ∈ T to problem (3.30) are ranktwo to minimize the losses in the system. The losses on the VSC station with converterbus k ∈ N convac in time slot t ∈ T areP convloss,k(t) = Tr{SkIk(t)}= I11k (t) S11k + 2 I12k (t) S12k + I22k (t) S22k . (B.30)Equality constraint (B.16) implies that the value of the entry I12k (t) of matrix Ik(t)depends on the value of matrix W(t). However, the entries I11k (t) and I22k (t) of matrix Ik(t)can take any values that satisfy constraints (B.17a) and (B.17b). Consider the given vectorsθ1 and θ2. To minimize the converter losses in (B.30), problem (3.30) returns solutionmatrix Ioptk,θ1,θ2(t), k ∈ N convac , t ∈ T with minimum value for I11k,θ1,θ2(t) S11k + I22k,θ1,θ2(t) S22ksubject to constraints (B.17a) and (B.17b). From (B.17a), we haveI22k,θ1,θ2(t) = 4((Imaxk )2−I12k,θ1,θ2(t)− I11k,θ1,θ2(t)). (B.31)Substituting (B.31) into I11k,θ1,θ2(t) S11k + I22k,θ1,θ2(t) S22k and performing some algebraic200Appendix B. Appendices for Chapter 3manipulations, we obtainI11k,θ1,θ2(t) S11k + I22k,θ1,θ2(t) S22k = I11k,θ1,θ2(t) (S11k − 4 S22k ) + 4((Imaxk )2−I12k,θ1,θ2(t))S22k .(B.32)We have S11k =ak+bkImaxk +ck(Imaxk )2 and S22k =ak−bk+ck4(Imaxk )2 . Thus, we obtain S11k ≥ 4S22k . Tominimize the value of I11k,θ1,θ2(t) S11k + I22k,θ1,θ2(t) S22k in (B.32), the entry I11k,θ1,θ2(t) should beminimized. From constrain (B.17b), the minimum value of I11k,θ1,θ2(t) is(Imaxk )24. Therefore,the solution matrix Ioptk,θ1,θ2(t), k ∈ N convac , t ∈ T does not have the form in (B.2), and thusfrom Lemma B.1.1 in Appendix B.1, it is not rank one. This completes the proof. B.3 Proof of Theorem 3.2From the proof of Theorem 3.1, the solution matrices Ioptk,θ1,θ2(t), k ∈ N convac , t ∈ T toproblem (3.30) are all rank two because we have S11k ≥ 4S22k . We also have I11,optk,θ1,θ2(t) =(Imaxk )24, k ∈ N convac , t ∈ T . We introduce the trace norm εTr{Ik(t)}, for all k ∈ N convac , t ∈T to the objective function of problem (3.31) to make the solution matrices Ioptk,θ1,θ2(t), k ∈N convac , t ∈ T all rank one.Consider a penalty coefficient ε, for which the solution matrices Ioptk,θ1,θ2(t), k ∈ N convac , t ∈T to problem (3.31) are all rank one. Let ∆ I11,optk,θ1,θ2(t) and ∆ I22,optk,θ1,θ2(t) denote the differencein the optimal values of entries I11,optk,θ1,θ2(t) and I22,optk,θ1,θ2(t) of solution matrices Ioptk,θ1,θ2(t), k ∈N convac , t ∈ T to problems (3.30) and (3.31), respectively. When the rank two solution ma-trices Ioptk,θ1,θ2(t), k ∈ N convac , t ∈ T in problem (3.30) becomes rank one for problem (3.31),the total losses of the system increase by approximately∑t∈T∑k∈N convac ∆ I11,optk,θ1,θ2(t) S11k +∆ I22,optk,θ1,θ2(t) S22k . When the losses increase, the total generation level in the system willincrease as well. Let P 30Gk(t) and P31Gk(t) denote the output active power of the generator201Appendix B. Appendices for Chapter 3connected to bus k in time slot t by solving problems (3.30) and (3.31), respectively. Let∆PGk(t) = P31Gk(t)− P 30Gk(t) denote the change in the output power of the generator in busk in time slot t. In the worst case, the increase in the system losses is compensated bythe increase in the output power of the conventional generators. Hence, the CVaR term in(3.23) can be assumed to be unchanged in the worst-case scenario. Let ∆Cgen denote thechange in the total generation cost in (3.9). Using the first order approximation aroundP 29Gk(t), k ∈ N , t ∈ T , we have∆Cgen ≈∑t∈T∑k∈N(2ck2P30Gk(t) + ck1)∆PGk(t). (B.33)Substituting cmax1 , cmax2 , and PmaxG into (B.33) for all generators, we obtain∆Cgen ≤ K∑t∈T∑k∈N∆PGk(t), (B.34)where K = 2cmax2 PmaxG + cmax1 . The increase in the generation level of the generators isequal to the increase in the total losses. Hence, we have∑t∈T∑k∈N∆PGk(t) =∑t∈T∑k∈N convac∆I11,optk,θ1,θ2(t)S11k + ∆I22,optk,θ1,θ2(t)S22k . (B.35)Thus, we obtain∆Cgen ≤ K∑t∈T∑k∈N convac∆I11,optk,θ1,θ2(t)S11k +∆I22,optk,θ1,θ2(t)S22k . (B.36)Equation (B.36) provides an upper bound for the change in the total generation cost of thesystem when we solve problem (3.31). The change in the term associated with the system202Appendix B. Appendices for Chapter 3losses in the objective function of problem (3.31) is equal to∑t∈T∑k∈N convac(∆I11,optk,θ1,θ2(t)(ωlossS11k + ε)+ ∆I22,optk,θ1,θ2(t)(ωlossS22k + ε) ). (B.37)For an appropriate value of the penalty coefficient ε, the change in the objective functionof problem (3.31) is negative. Therefore, we have∑t∈T∑k∈N convac∆I11,optk,θ1,θ2(t)((K + ωloss) S11k + ε)+ ∆I22,optk,θ1,θ2(t)((K + ωloss) S22k + ε) ≤ 0.(B.38)Moreover, equation (B.32) implies that ∆I22,optk,θ1,θ2(t) = −4 ∆I11,optk,θ1,θ2(t). Hence, inequality(B.38) is equivalent to∑t∈T∑k∈N convac∆I11,optk,θ1,θ2(t)( ((K + ωloss) S11k + ε)− 4 ((K + ωloss) S22k + ε) ) ≤ 0. (B.39)If the coefficient ε is chosen such that ((K + ωloss) S11k + ε) − 4 ((K + ωloss) S22k + ε) isless than or equal to zero, then inequality (B.39) will hold as well. That is, it is sufficientto have (K + ωloss) (S11k − 4 S22k )− 3ε ≤ 0, which is equivalent toε ≥ (K + ωloss)3(S11k − 4 S22k ). (B.40)From S11k =ak + bkImaxk + ck(Imaxk )2and S22k =ak − bk + ck4(Imaxk )2, we obtain S11k −4 S22k =(Imaxk +1)bk(Imaxk )2.Thus, (B.40) is equivalent toε ≥ bk (K + ωloss) (Imaxk + 1)3(Imaxk )2. (B.41)Inequality (B.41) must hold for all k ∈ N convac . Thus, bk, ck1 and ck2, and PmaxGk are replaced203Appendix B. Appendices for Chapter 3with their maximum values, and Imaxk is relplaced with its minimum value in the underlyingsystem. The penalty coefficient ε can be approximated by (3.32). The proof is completed.B.4 Proof of Theorem 3.3Let φoptθ1,θ2 be an optimal solution to problem (3.31) under the given vectors θ1 and θ2.Consider time slot t ∈ T . We show that matrix Woptθ1,θ2(t) is at most rank two if forall set of operating generators, the SDP relaxation gap for the ac-dc OPF problem inthe underlying ac-dc grid is zero. We construct an ac-dc OPF problem from the SCUCproblem (3.31) by replacing all variables with their optimal values in problem (3.31) exceptfor matrices W(t) and Ik(t), k ∈ N convac , and variables ϑk(t), k ∈ N . Let fOPFobj denote theobjective function of the obtained ac-dc OPF problem. We havefOPFobj =∑k∈Nϑk(t) + ωloss∑k∈NTr{YkW(t)}+ ε∑k∈N convacTr{Ik(t)}. (B.42)Let φ˜ = (W(t), Ik(t), k ∈ N convac , ϑk(t), k ∈ N ) denote the decision variables in theobtained ac-dc OPF problem for time slot t. Let ΦOPF denote the feasible set for decisionvariables φ˜. It can be obtained from the feasible set ΦSDP for the SCUC problem (3.31)when all variables except decision variables φ˜ are replaced with their optimal values inproblem (3.31). Hence, the ac-dc OPF problem in time slot t obtained from problem(3.31) can be written asminimizeφ˜∈ΦOPFfOPFobj (B.43a)subject to Ik(t) 0, k ∈ N convac , (B.43b)W(t) 0. (B.43c)204Appendix B. Appendices for Chapter 3The feasible set of the OPF problem (B.43) is a subset of the feasible set of the SCUCproblem (3.31). Furthermore, the decision variables for the SCUC problem (3.31) otherthan those in φ˜ are set to their optimal values to obtain the OPF problem (B.43). Hence,the optimal solution matrix Wopt(t) to the OPF problem (B.43) is equal to the optimalsolution matrix Woptθ1,θ2(t) to the SCUC problem (3.31). If under the given operatinggenerators, the SDP relaxation gap for the ac-dc OPF problem (B.43) is zero, then problem(B.43) has a solution Wopt(t) with rank of at most two [140]. Thus, the solution matrixWoptθ1,θ2(t) to the SCUC problem (3.31) is at most rank two in time slot t. Using the sameapproach for all time slots t ∈ T completes the proof. B.5 Proof of Theorem 3.4fREG,29obj,θ1,θ2 is the optimal value of problem (3.29). Hence, the value of fREGobj,θ1,θ2is greater thanor equal to fREG,29obj,θ1,θ2 for any feasible solutions of problem (3.29). The optimal solution ofproblem (3.31) that satisfies the rank constraints (3.26c) and (3.26d) is a feasible solutionof problem (3.29). Thus, we havefREG,29obj,θ1,θ2 ≤ fREG,31obj,θ1,θ2 , (B.44)which proves the left-hand side of (3.33).Let Iopt,29k,θ1,θ2(t) and Iopt,31k,θ1,θ2(t), k ∈ N convac , t ∈ T denote the solution matrices to prob-lems (3.29) and (3.31), respectively. Since fREG,31obj,θ1,θ2 + ε∑t∈T∑k∈N convac Tr{Iopt,31k,θ1,θ2(t)} is theoptimal objective value of problem (3.31), we havefREG,31obj,θ1,θ2 + ε∑t∈T∑k∈N convacTr{Iopt,31k,θ1,θ2(t)} ≤ fREG,29obj,θ1,θ2 + ε∑t∈T∑k∈N convacTr{Iopt,29k,θ1,θ2(t)}. (B.45)205Appendix B. Appendices for Chapter 3After rearranging the terms, inequality (B.45) becomesfREG,31obj,θ1,θ2 − fREG,29obj,θ1,θ2 ≤ ε∑t∈T∑k∈N convac(Tr{Iopt,29k,θ1,θ2(t)} − Tr{Iopt,31k,θ1,θ2(t)}). (B.46)We determine the upper bound for Tr{Iopt,29k,θ1,θ2(t)} − Tr{Iopt,31k,θ1,θ2(t)}. According to The-orem 3.1, rank one matrices Iopt,29k,θ1,θ2(t) and Iopt,31k,θ1,θ2(t) have the form in (B.2). Therefore,there exist nonnegative numbers I29k (t) and I31k (t) such thatTr{Iopt,29k,θ1,θ2(t)} =(Imaxk + |I29k (t)|)24+ (Imaxk − |I29k (t)|)2, (B.47a)Tr{Iopt,31k,θ1,θ2(t)} =(Imaxk + |I31k (t)|)24+ (Imaxk − |I31k (t)|)2. (B.47b)We can show that I29k (t) = 0 maximizes Tr{Iopt,29k,θ1,θ2(t)} in (B.47a). We can also showthat I31k (t) =35Imaxk minimizes Tr{Iopt,31k,θ1,θ2(t)} in (B.47b). Hence, we haveTr{Iopt,29k,θ1,θ2(t)} − Tr{Iopt,31k,θ1,θ2(t)} ≤54(Imaxk )2 − 45(Imaxk )2= 0.45 (Imaxk )2. (B.48)By substituting the right-hand side of (B.48) into (B.46), we obtainfREG,31obj,θ1,θ2 − fREG,29obj,θ1,θ2 ≤ 0.45T ε∑k∈N convac(Imaxk )2. (B.49)The proof is completed. 206Appendix B. Appendices for Chapter 3B.5.1 Proof of Theorem 3.5Consider problem (3.27). We approximate ‖uk(t)‖0 bylog(1+‖uk(t)‖1σ)log(1+ 1σ ). We also approximate‖1− uk(t)‖0 bylog(1+‖1−uk(t)‖1σ)log(1+ 1σ ). For sufficiently small σ, the approximation is tight. Weuse the first order Taylor approximation of functions log(1+‖uk(t)‖1σ)and log(1+‖1−uk(t)‖1σ)near an arbitrary point ûk(t) as follows:log(1 +‖uk(t)‖1σ)≈ log(1 +‖ûk(t)‖1σ)+‖uk(t)‖1 − ‖ûk(t)‖1‖ûk(t)‖1 + σ, (B.50a)log(1 +‖1− uk(t)‖1σ)≈ log(1 +‖1− ûk(t)‖1σ)+‖1− uk(t)‖1 − ‖1− ûk(t)‖1‖1− ûk(t)‖1 + σ. (B.50b)We substitute (B.50a) and (B.50b) into the objective function of problem (3.27) andremove the constant terms. We have the following optimization problem.minimizeφ∈ΦSDPfSDPobj +∑t∈T∑k∈Nς ‖uk(t)‖1‖ûk(t)‖1 + σ+ς ‖1− uk(t)‖1‖1− ûk(t)‖1 + σ(B.51)subject to constraints (3.26c)−(3.26f).The coefficients ς‖ûk(t)‖1+σ andς‖1−ûk(t)‖1+σ in the objective function of problem (B.51) arethe same as coefficients θk1(t) and θk2(t) in the objective function of problem (3.29), respec-tively. That is, under the given vector û = (ûk(t), k ∈ N , t ∈ T ), we obtain coefficientsθk1(t) and θk2(t) in problem (3.29) as follows:θk1(t) =ς‖ûk(t)‖1 + σ, k ∈ N , t ∈ T , (B.52a)θk2(t) =ς‖1− ûk(t)‖1 + σ, k ∈ N , t ∈ T . (B.52b)In Theorem 3.4, we show that the objective values of problems (3.29) and (3.31) areapproximately equal and the approximation is tight. Furthermore, the linear approxima-207Appendix B. Appendices for Chapter 3tion in the right-hand side of (B.50a) and (B.50b) are tangent majorant of the logarithmicfunctions in the left-hand side of (B.50a) and (B.50b), respectively. The right-hand sidesmajorize the left-hand sides with equality at ûk(t). Hence, we can apply the majorization-minimization (MM) algorithm [178] to obtain vector û in an iterative fashion. Lines 1 to6 of Algorithm 3.1 correspond to the iterations in the MM algorithm. We start with anarbitrary initial vector û1, which corresponds to the initial value for coefficients θ11 andθ12. Then, in iteration i, we solve the convex optimization problem (3.31) under the givenvector ûik (which corresponds to coefficients θi1 and θi2) and obtain the updated vectorûi+1 as ûi+1 = (uopt,ik,θi1,θi2(t), k ∈ N , t ∈ T ). This procedure corresponds to the loop withinLines 2 to 6 in Algorithm 3.1. The objective function in each iteration of Algorithm 3.1is convex and continuously differentiable and the feasible set is closed and convex. Hence,if the optimal solution to problem (3.31) can be obtained in each iteration (i.e., matri-ces Wopt,iθi1,θi2(t), t ∈ T are at most rank two), then for sufficiently small σ and large ς, theproposed algorithm converges to local optimal solution [178]. The proof is completed. 208Appendix CAppendices for Chapter 4C.1 Proof of Equation (4.1)Consider appliance a ∈ Ai in household i. According to Definition 4.1, δa,i,t for t ∈ T is thenumber of time slots since the most recent time slot that appliance a becomes awake withthe most recent new task. In other words, appliance a has not become awake with a newtask again in time slots t−δa,i,t+1, . . . , t since it became awake in time slot t−δa,i,t+1. Thevalue of Pa,i(δa,i,t) is the probability that the difference between two sequential wake-uptimes for appliance a is δa,i,t. Given the current time slot t, the probability Pa,i,t+1 thatappliance a ∈ Ai becomes awake with a new task in the next time slot t + 1 ∈ T can beobtained from the Bayes’ rule asPa,i,t+1 =Prob{E1 |E2}Prob{E2}Prob{E1} , (C.1)where E1 is the event that appliance a has not become awake with a new task until timeslot t, and E2 is the event that appliance a becomes awake in time slot t+1 after δa,i,t timeslots since it became awake with the most recent task. With probability Prob{E1 |E2} =1, appliance a has not become awake with a new task until time slot t conditioned onthe event that it becomes awake with a new task in time slot t + 1. With probabilityProb{E2} = Pa,i(δa,i,t), appliance a becomes awake in time slot t + 1 after δa,i,t time slotssince it became awake with the most recent task. Appliance a has not become awake in209Appendix C. Appendices for Chapter 4time slots t− δa,i,t + 1, . . . , t with probability Prob{E1} = 1−∑δa,i,t−1∆=1 Pa,i(∆). Therefore,Pa,i,t+1 can be obtained as (4.1). This completes the proof. C.2 Proof of Theorem 4.1Game 4.2 is the fixed point solution of every household’s best response policy. Householdi solves the Bellman equations (4.20) for all approximate observation profiles oˆ ∈ O whenother households’ policies are fixed. We construct a Bayesian game from the underlyingfully observable game with incomplete information as follows:Game C.2.1 Bayesian Game Among Virtual Households:Players: The set of virtual households, where each virtual household (i, oˆ) correspondsto each real household i ∈ N and observation profile oˆ ∈ O.Types: The type of each virtual household (i, oˆ) is the observation oi ∈ Oi of house-hold i. Pi(oi|oˆ) is the probability that virtual household (i, oˆ) has type oi.Strategies: The strategy for virtual household (i, oˆ) is the probability distribution pii(oˆ)over the actions xi(oˆ) ∈ Xˆi(oˆ).Costs: The cost of each virtual household (i, oˆ) with strategy profile pii(oˆ) is equal toEpii(oˆ){Qpi−ii (oˆ,xi(oˆ))}, where Qpi−ii (oˆ,xi(oˆ)) is defined in (4.18).We consider the Bayesian Nash equilibrium (BNE) solution concept for the underlyingBayesian game among virtual households. We show that the BNE corresponds to the MPEof Game 4.2 among households i ∈ N . In Game C.2.1, each virtual household (i, oˆ) aims todetermine its BNE strategy piBNEi (oˆ) to minimize EpiBNEi (oˆ){QpiBNE−ii (oˆ,xi(oˆ))}when othervirtual households’ strategies are fixed. Therefore, in the BNE all virtual households solvethe Bellman equations in (4.20). Consequently, the BNE of the Game C.2.1 among virtualhouseholds corresponds to the MPE of Game 4.2 among real households. A BNE always210Appendix C. Appendices for Chapter 4exists for the Bayesian games with a finite number of players and actions [184, Ch. 6].Thus, an MPE exists for the fully observable game with incomplete information amonghouseholds. This completes the proof. C.3 Proof of Theorem 4.2We use an approach similar to [141, Theorem 3.8.2] to show that the joint policy pi isan MPE if and only if f obji (Vpii ,pii) = 0 for all households i ∈ N . Then, we obtain thecondition in (4.24) for the policy in an MPE. Our proof involves two steps.Step (a) Consider the joint policy pi and value functions V pii (oˆ), i ∈ N , in the feasibleset of problem (4.23), for which we have f obji (Vpii ,pii) = 0 for i ∈ N . We show that thepolicy pi is an MPE. According to the constraint set of problem (4.23), the Bellman errorsfor the actions in an approximate observation profile oˆ are nonnegative. Since f obji (Vpii ,pii)is the expectation over the Bellman errors, its value is nonnegative for all feasible policiesand value functions. If f obji (Vpii ,pii) = 0 for all i ∈ N , then the policy pi and the valuefunctions V pii (oˆ), i ∈ N are the global optimum of problem (4.23) for all households.Hence, no household has the incentive to unilaterally change its policy, in order to furtherreduce its objective value f obji (Vpii ,pii). In other words, the policy pi is an MPE.Next, we show that for an MPE policy piMPE, we can determine a value functionV piMPEi (oˆ) such that fobji (VpiMPEi ,piMPEi ) = 0. From (4.22), fobji (VpiMPEi ,piMPEi ) = 0 is equiv-alent to∑oˆ∈OEpiMPEi (oˆ){Bi(V piMPEi , oˆ,xi(oˆ))}= 0, ∀ i ∈ N . (C.2)According to the constraint set of problem (4.23), the Bellman errors for the actions inan observation profile oˆ are nonnegative in the MPE. Thus, each term of the summation211Appendix C. Appendices for Chapter 4in (C.2) should be zero. That isEpiMPEi (oˆ){Bi(V piMPEi , oˆ,xi(oˆ))}= 0, ∀ oˆ ∈ O, ∀ i ∈ N , (C.3)which is equivalent toEpiMPEi (oˆ){QpiMPE−ii (oˆ,xi(oˆ))−V piMPEi (oˆ)}= 0, ∀ oˆ ∈ O, ∀ i ∈ N . (C.4)piMPEi (oˆ) is a randomized policy. Hence, we have EpiMPEi (oˆ){V piMPEi (oˆ)}= V piMPEi (oˆ).Hence, for all approximate observation profile oˆ ∈ O, (C.4) can be rewritten asV piMPEi (oˆ)=EpiMPEi (oˆ){QpiMPE−ii (oˆ,xi(oˆ))}, ∀ i ∈ N . (C.5)For household i, we define the average cost in approximate observation profile oˆ asc¯i(oˆ) = EpiMPE(oˆ) {ci(oˆ,x(oˆ))}. We define the average transition probability from observa-tion oˆ to oˆ′ as P¯ (oˆ′ | oˆ) = EpiMPE(oˆ) {P (oˆ′ | oˆ,x(oˆ))}. We define vectors c¯i=(c¯i(oˆ), oˆ ∈ O)and V piMPEi =(V piMPEi (oˆ), oˆ ∈ O), and define the transition matrix P¯ =[P¯ (oˆ′ | oˆ), oˆ, oˆ′ ∈ O].By substituting (4.18) into (C.5), we haveV piMPEi = (1− β)c¯i + βP¯ V piMPEi . (C.6)By rearranging the terms in (C.6), we obtain(I − βP¯ )V piMPEi = (1− β)c¯i, (C.7)where I is the identity matrix. Matrix P¯ is a stochastic matrix (i.e., each of its entries is anonnegative real number representing a probability), and thus its eigenvalues are less than212Appendix C. Appendices for Chapter 4or equal to one. Besides, the discount factor β is less than one. Hence, the eigenvalues ofmatrix I − βP¯ are positive, and thereby it is invertible (or nonsingular). From (C.7), wecan obtain V piMPEi asV piMPEi = (1− β)(I − βP¯ )−1 c¯i. (C.8)Therefore, for the MPE policy piMPE, we obtain the value function V piMPEi (oˆ) in (C.8)such that f obji (VpiMPEi ,piMPEi ) = 0 for all households i ∈ N .Step (b) We obtain the condition in (4.24) for the policy in an MPE. For each householdi ∈ N , the objective function f obji (V piMPEi ,piMPEi ) in (4.22) can be expressed asf obji (VpiMPEi ,piMPEi ) =∑oˆ∈O∑xi(oˆ)∈Xˆi(oˆ)piMPEi (oˆ,xi(oˆ))Bi(V piMPEi , oˆ,xi(oˆ)). (C.9)The Bellman error Bi(VpiMPEi , oˆ,xi(oˆ)) is nonnegative. Hence, from equation (C.9),f obji (VpiMPEi ,piMPEi ) = 0 is equivalent to piMPEi (oˆ,xi(oˆ))Bi(V piMPEi , oˆ,xi(oˆ))= 0 for allhouseholds i ∈ N with action xi(oˆ)∈ Xˆi(oˆ) in observation profile oˆ. This completes theproof. C.4 Bellman Error ApproximationThe basis function in (4.32) is equal to the expectation over the Bellman errors for allfeasible actions xi(oˆk−1) ∈ Xi(ok−1i ). ECC i knows the observation ok−1i , the approximateobservation profile oˆk−1, and the cost ci(oˆk−1,xk−1i (oˆk−1),xk−1−i (oˆk−1))for the chosen ac-tion xk−1i (oˆk−1) in iteration k−1, as well as the current observation oki and the approximateobservation profile oˆk. ECC i needs to use these available information to approximate theBellman error for an arbitrary feasible action xi(oˆk−1) ∈ Xi(ok−1i ). We use the TD error213Appendix C. Appendices for Chapter 4as an estimation for the Bellman error [123, Lemma 3]. We haveBk−1i(V pi,k−1i , oˆk−1,xi(oˆk−1)) ≈ (1− β) ci (oˆk−1,xi(oˆk−1),xk−1−i (oˆk−1))+ β V pi,k−1i(oˆ(xi(oˆk−1)), θk−1i)−V pi,k−1i (oˆk−1, θk−1i ) ,(C.10)where oˆ(xi(oˆk−1))is the approximate observation profile if household i chooses actionxi(oˆk−1) in the time slot k− 1. ECC i determines oˆ (xi(oˆk−1)) in the following two steps:Step (a) ECC i knows observations ok−1i and oki . It can determine the set of appliancesthat become awake with a new task in time slot k. ECC i can also determine the state ofother operating appliances for a feasible action xi(oˆk−1). Therefore, ECC i can determinethe state of its own household for an arbitrary feasible action xi(oˆk−1).Step (b) The states of other households are fixed. Furthermore, ECC i knows theobservation oˆki for the chosen action xk−1i (oˆk−1). Using the result of Step (a), ECC i cancompute the average aggregate load demands for the feasible actions of all households for anarbitrary feasible action xi(oˆk−1) ∈ Xi(ok−1i ), and thus it can determine the approximateobservation profile oˆ(xi(oˆk−1)) for all households for an arbitrary feasible xi(oˆk−1).ECC i also needs to compute the cost ci(oˆk−1,xi(oˆk−1),xk−1−i (oˆk−1))for feasible actionxi(oˆk−1) ∈ Xi(ok−1i ). ECC i knows the payment to the utility company for the chosenaction xk−1i (oˆk−1). Since the load demand of one household is much smaller than theaggregate load demand of all households, we can assume that the price value is unchangedwhen household i unilaterally changes its load demand. Thus, ECC i can estimate itspayment for an arbitrary feasible action xi(oˆk−1) ∈ Xi(ok−1i ). ECC i can also determinethe discomfort cost for action xi(oˆk−1) ∈ Xi(ok−1i ). Therefore, it can compute the costci(oˆk−1,xi(oˆk−1),xk−1−i (oˆk−1))for an arbitrary feasible action xi(oˆk−1). Finally, ECC i isable to compute the approximate Bellman error in (C.10).214Appendix DAppendices for Chapter 5D.1 Multiclass M/M/1 Queuing System ModelIn the first step, we model the arrival and execution of the incoming workloads to datacenter d as a time-dependent multiclass M/M/1 queuing system. In the second step, weuse the transient behaviour of the underlying queuing system to show that a workload withservice class c in data center d experiences the maximum expected execution time eitherat the beginning or at the end of each time slot t.a) Consider data center d ∈ D in time slot t ∈ T . In the underlying multiclass queuingsystem, the proportion of time that the servers are busy to execute the workloads of theservice classes other than c is ρd(t)−ρc,d(t). Hence, the proportion of time that the serversare busy to execute the workloads of service class c is 1 − (ρd(t) − ρc,d(t)). Thus, theexecution rate of the workloads of service class c is µc,d(t) = (1−ρd(t)+ρc,d(t))µc,d(t). Theprocess of the incoming workloads of service class c to data center d can be modeled as atime-dependent M/M/1 queuing system with workloads’ arrival rate and execution rateλc,d(t) = pc,d(t, t)λc,d(t) and µc,d(t) = (1− ρd(t) + ρc,d(t))µc,d(t), respectively.b) To determine the maximum expected execution time of a workload, we use thetransient behaviour of the M/M/1 queuing system corresponding to the workloads ofservice class c. We consider time slot t and use the continuous parameter τ to representthe time passed from the beginning of time slot t. Hence, τ varies from 0 to one time slot.We drop time index t from the parameters in the rest of the proof to avoid the potential215Appendix D. Appendices for Chapter 5of confusion. The state of the queue represents the number of workloads in the system intime τ . Consider the corresponding M/M/1 queuing system for the workloads of serviceclass c in time slot t. Let qc,d(k1, k2, τ) denote the probability of being in state k2 in timeτ when the initial state of the queue in time τ = 0 is k1. For data center d, let Wc,d(k1, τ)denote the expected execution time of an incoming workload of service class c in time τwhen the initial state is k1. We haveWc,d(k1, τ) =∞∑k=0(k + 1µc,d)qc,d(k1, k, τ). (D.1)We can rewrite (D.1) asWc,d(k1, τ) =∞∑k=0(kµc,d)qc,d(k1, k, τ) +∞∑k=0(1µc,d)qc,d(k1, k, τ). (D.2)The value of∑∞k=0 k qc,d(k1, k, τ) in the first summation of (D.2) is equal to the expectednumber of workloads of service class c in time τ for a system with initial state k1. Wedenote this summation by Qc,d(k1, τ). For the summation in the second term, we have∑∞k=0 qc,d(k1, k, τ) = 1. We can rewrite (D.2) asWc,d(k1, τ) =Qc,d(k1, τ) + 1µc,d. (D.3)We now show that a workload of service class c in data center d experiences the maximumexpected execution time either at the beginning or at the end of each time slot t. That is,function Wc,d(k1, τ) is maximized when either τ = 0 or τ is equal to one time slot.From (D.3), it is sufficient to determine time τ that maximizes function Qc,d(k1, τ). Wecan compute the derivative of function Qc,d(k1, τ) with respect to τ as [200]Q′c,d(k1, τ) = λc,d − µc,d (1− qc,d(k1, 0, τ)). (D.4)216Appendix D. Appendices for Chapter 5We now consider the following three scenarios for k1, the initial number of workloads ofservice class c.1) k1 = 0: In this scenario, the queuing system corresponded to service class c isinitially empty. Thus, with probability of one, no workload is initially in the system. Wehave qc,d(k1, 0, 0) = 1. From (D.4), we have Q′c,d(k1, 0) = λc,d, which is nonnegative.By increasing τ from 0 to one time slot, qc,d(k1, 0, τ) decreases from one to its steadystate value, and hence, Q′c,d(k1, τ) decreases from λc,d and converges to zero gradually.Thus, the value of Qc,d(k1, τ) increases gradually and converges to its steady state value.Therefore, Qc,d(k1, τ) and Wc,d(k1, τ) are maximized when τ is equal to one time slot. Themaximum value of Wc,d(k1, τ) is1µc,d−λc,d.2) 0 < k1 ≤ λc,dµc,d−λc,d : In this scenario, there initially exist some workloads with serviceclass c, but the number of initial workloads is less than or equal toλc,dµc,d−λc,d. Thus, withzero probability, no workload is initially in the system. We have qc,d(k1, 0, 0) = 0. From(D.4), we have Q′(k1, 0) = λc,d − µc,d, which is negative.By increasing τ from 0 to one time slot, qc,d(k1, 0, τ) increases from zero to its steadystate value. The value of Q′c,d(k1, τ) increases from its initial negative value, i.e., λc,d−µc,dgradually, becomes positive, and then converges to zero. Hence, the value of Qc,d(k1, τ)decreases at the beginning, and then increases gradually to converges to its steady statevalue. Therefore, Qc,d(k1, τ) and Wc,d(k1, τ) are maximized when τ is equal to one timeslot. The maximum value of Wc,d(k1, τ) is1µc,d−λc,d.3) k1 >λc,dµc,d−λc,d: In this scenario, the number of initial workloads is greater thanλc,dµc,d−λc,d. We have qc,d(k1, 0, 0) = 0. From (D.4), we have Q′(k1, 0) = λc,d − µc,d, which isnegative.By increasing τ from 0 to one time slot, qc,d(k1, 0, τ) increases from zero to its steadystate value. The value of Q′c,d(k1, τ) increases from its initial negative value, i.e., λc,d−µc,d217Appendix D. Appendices for Chapter 50 50 100 150 200 250 300 350 40000.511.522.5Figure D.1: The average waiting time Wc,d(k1, τ) versus τ for different values of k1 =Ic,d(pc,d, t). The workloads average arrival and execution rates are λc,d = 4 workloads persecond and µc,d = 5 workloads per second.and converges to zero. Thus, the value of Qc,d(k1, τ) decreases gradually and converges toits steady state value. Hence, in this scenario, the values of Qc,d(k1, τ) and Wc,d(k1, τ) aremaximized when τ is equal to zero.To completes the discussion, we provide an illustrative example. Fig. D.1 shows theaverage waiting time in an M/M/1 queuing system with λc,d = 4 workloads per second,µc,d = 5 workloads per second, and k1 = Ic,d(pc,d, t) = 0, 2, . . . , 10 of workloads are inthe queue in time τ = 0. If Ic,d(pc,d, t) is less than 4, then Wc,d(k1, τ) is maximized whenτ →∞. If Ic,d(pc,d, t) is greater than 4, then W (k1, τ) is maximized in τ = 0. D.2 Proof of Theorem 5.1To prove Theorem 5.1, we substitute (5.26) into the right-hand side of (5.25) and substitute(5.19) into the left-hand side of (5.25) for strategy profiles s and s˜, and show that theyare the same.By substituting (5.26) into the right-hand side of (5.25) for s = (sd, s−d) and s˜ =218Appendix D. Appendices for Chapter 5(s˜d, s−d), we obtainP (sd, s−d)− P (s˜d, s−d) =∑t∈T((pw(t) + κu(t) eotheru (t))ed(t) + κu(t) e2d(t)+ κu(t)∑d′∈m(u)\ded(t) ed′(t)− (pw(t) + κu˜(t) eotheru˜ (t)) e˜d(t)− κu˜(t) e˜2d(t)− κu˜(t)∑d′∈m(u˜)\de˜d(t) ed′(t))+ ωcvard(Γd,βd(αd,θd)− Γd,βd(α˜d, θ˜d)). (D.5)In (D.5), the terms related to the data centers other than data center d cancel each other,since the strategy of other data centers are unchanged in s˜ = (s˜d, s−d).By substituting (5.19) into the left-hand side of (5.25) for s = (sd, s−d) and s˜ =(s˜d, s−d), we havecd(sd, s−d)− cd(s˜d, s−d) =∑t∈T(ed(t) pru(eu(t),m)− e˜d(t) pru˜(eu˜(t), m˜))+ ωcvard(Γd,βd(αd,θd)− Γd,βd(α˜d, θ˜d)), (D.6)By substituting the retail price (5.1) into (D.6), the cost change for data center d willbe equal to the potential function change in (D.5). This completes the proof. D.3 Proof of Theorem 5.2We first show that the global minimum of the potential function (5.26) is a stable outcome.Let (am?,m?) be the global minimum of P (·). Thus, if data center d changes its actionprofile to ad or its utility company to m(d) unilaterally, then the value of the potential219Appendix D. Appendices for Chapter 5function increases. The change in the exact potential function is equal to the change inthe cost of the deviating data center d. Hence, the cost of data center d increases as well.Consequently, no unilateral deviation from (a?,m?) can reduce the cost of any data center,and (a?,m?) is a stable outcome of the matching game. As the exact potential function(5.26) has at least one global minimum, we know that the matching game has at least onestable outcome.Next, we show that a stable outcome (a,m) is in setM by the contradiction. Supposethat a stable outcome (a,m) is not in set M. Hence, we have a 6= am. By definition, amis the global minimum of P (·) under matching m. We know that P (·) is a convex functionof a. Therefore, a unilateral change of ad for any data center d in the opposite directionof the gradient ∇ad P (a,m) will reduce the potential function, and thus the cost of thatdata center. It contradicts the supposition that (a,m) is a stable outcome. Hence, (a,m)is in set M. D.4 Proof of Theorem 5.3Since the potential function (5.26) is lower bounded (it is always positive), it is sufficientto show that the potential function decreases in each iteration of Algorithm 5.1. Line 17 ofAlgorithm 5.1 guarantees that if the algorithm converges, the result is a stable outcome.Our proof involves two steps:Step a) We show that the potential function decreases when the data centers updatetheir utility company choices in Line 12 of Algorithm 5.1. Each data center updates itsutility company choice to reduce its cost. Nevertheless, we cannot directly use (5.25) inDefinition 5.3 to conclude that the potential function decreases as a result of reductionin the cost of a data center. In fact, the equality in (5.25) holds if a data center changesits utility company choice unilaterally. However, in Line 12 of Algorithm 5.1, several data220Appendix D. Appendices for Chapter 5centers may update their utility company choices simultaneously.Consider iteration i of Algorithm 5.1. We prove by induction that the potential functiondecreases when k data centers update their utility company choices simultaneously, wherek ≥ 1 is an arbitrary number.Base case: Consider k = 1. If one data center updates its utility company choice inLine 12 of Algorithm 5.1, then it can be considered as a unilateral change in the strategy ofone data center to reduce its cost. Hence, from (5.25) in Definition 5.3, potential functionP (si) decreases, when the cost of a data center decreases.Induction step: Consider k = d. Suppose that the potential function decreases, whend data centers change their utility company choices in Line 12 of Algorithm 5.1. We provethat potential function P (si) decreases, when there are k = d+ 1 data centers that changetheir utility company choices simultaneously. Without loss of generality, we assume that{1, . . . , d + 1} is the set of d + 1 data centers that change their utility company choices.The simultaneous changes in the utility company choices of d+ 1 data centers 1, . . . , d+ 1can be interpreted as the simultaneous changes in the utility company choices of datacenters 1, . . . , d, and the change of utility company choice for data center d+ 1. From oursupposition, when data centers 1, . . . , d change their utility company choices, the potentialfunction decreases. In the following, we show that the potential function further decreaseswhen data center d+ 1 changes its matched utility company.Assume that data center d+ 1 decides to leave utility company mi(d+ 1) in iteration iand connects to utility company mi+1(d+ 1) to reduce its bill payment. In Algorithm 5.1,a utility company accepts at most one termination request from data centers per iteration.Thus, data center d+1 is the only one leaving utility company mi(d+1). After updating thematching of data centers 1, . . . , d, utility company mi(d+ 1) may accept a new connectionrequest from a data center, which increases the total demand for utility company mi(d).221Appendix D. Appendices for Chapter 5Therefore, the payment of data center d+1 to utility company mi(d+1) will increase afterupdating the matching of data centers 1, . . . , d.On the other hand, in Algorithm 5.1, a utility company accepts at most one connectionrequest from data centers per iteration. Hence, data center d+1 is the only one connectingto utility company mi+1(d + 1). Utility company mi+1(d + 1) may accept a terminationrequest from a data center, which decreases the total demand for utility company mi+1(d).Hence, after updating the matching of data centers 1, . . . , d, the payment of data center dto utility company mi+1(d) will decrease. Therefore, the cost of data center d+1 decreaseseven by changing the utility company choice of data centers 1, . . . , d simultaneously. Theexact potential function decreases when the cost of data center d + 1 decreases. Conse-quently, the potential function decreases, when k = d+ 1 data centers change their utilitycompany choices simultaneously in Line 12 of Algorithm 5.1.By the principle of induction, the potential function decreases when multiple datacenters change their utility company choices simultaneously.Step b) Under a given matching mi+1, (5.25) implies that ∇aid cd(aid,ai−d,mi+1)=∇aid P(aid,ai−d,mi+1). If all data centers use (5.27) for the update, the potential functionvaries in the opposite direction of its gradient, ∇ai P(aid,ai−d,mi+1). Under a given match-ing mi+1, P (·) is a convex function of ai and has a Lipschitz continuous derivative. Thus,for sufficiently small step size, the opposite gradient direction is a decreasing direction.We note that the step sizes γid are not required to be equal for all data centers d ∈ D.Different but sufficiently small step sizes for data centers lead to updating the decisionvector ai in the opposite subgradient direction of the potential function. 222
- Library Home /
- Search Collections /
- Open Collections /
- Browse Collections /
- UBC Theses and Dissertations /
- Algorithm design for optimal power flow, security-constrained...
Open Collections
UBC Theses and Dissertations
Featured Collection
UBC Theses and Dissertations
Algorithm design for optimal power flow, security-constrained unit commitment, and demand response in… Bahrami, Shahab 2017
pdf
Page Metadata
Item Metadata
Title | Algorithm design for optimal power flow, security-constrained unit commitment, and demand response in energy systems |
Creator |
Bahrami, Shahab |
Publisher | University of British Columbia |
Date Issued | 2017 |
Description | Energy management is of prime importance for power system operators to enhance the use of the existing and new facilities, while maintaining a high level of reliability. In this thesis, we develop analytical models and efficient algorithms for energy management programs in transmission and distribution networks. First, we study the optimal power flow (OPF) in ac-dc grids, which is a non-convex optimization problem. We use convex relaxation techniques and transform the problem into a semidefinite program (SDP). We derive the sufficient conditions for zero relaxation gap and design an algorithm to obtain the global optimal solution. Subsequently, we study the security-constrained unit commitment (SCUC) problem in ac-dc grids with generation and load uncertainty. We introduce the concept of conditional value-at risk to limit the net power supply shortage. The SCUC is a nonlinear mixed-integer optimization problem. We use ℓ₁-norm approximation and convex relaxation techniques to transform the problem into an SDP. We develop an algorithm to determine a near-optimal solution. Next, we target the role of end-users in energy management activities. We study demand response programs for residential users and data centers. For residential users, we capture their coupled decision making in a demand response program with real-time pricing as a partially observable stochastic game. To make the problem tractable, we approximate the optimal scheduling policy of the residential users by the Markov perfect equilibrium (MPE) of a fully observable stochastic game with incomplete information. We develop an online load scheduling learning algorithm to determine the users’ MPE policy. Last but not least, we focus on the demand response program for data centers in deregulated electricity markets, where each data center can choose a utility company from multiple available suppliers. We model the data centers’ coupled decisions of utility company choices and workload scheduling as a many-to-one matching game with externalities. We characterize the stable outcome of the game, where no data center has an incentive to unilaterally change its strategy. We develop a distributed algorithm that is guaranteed to converge to a stable outcome. |
Genre |
Thesis/Dissertation |
Type |
Text |
Language | eng |
Date Available | 2017-08-21 |
Provider | Vancouver : University of British Columbia Library |
Rights | Attribution-NonCommercial-NoDerivatives 4.0 International |
DOI | 10.14288/1.0354557 |
URI | http://hdl.handle.net/2429/62754 |
Degree |
Doctor of Philosophy - PhD |
Program |
Electrical and Computer Engineering |
Affiliation |
Applied Science, Faculty of Electrical and Computer Engineering, Department of |
Degree Grantor | University of British Columbia |
Graduation Date | 2017-09 |
Campus |
UBCV |
Scholarly Level | Graduate |
Rights URI | http://creativecommons.org/licenses/by-nc-nd/4.0/ |
Aggregated Source Repository | DSpace |
Download
- Media
- 24-ubc_2017_september_bahrami_shahab.pdf [ 14.06MB ]
- Metadata
- JSON: 24-1.0354557.json
- JSON-LD: 24-1.0354557-ld.json
- RDF/XML (Pretty): 24-1.0354557-rdf.xml
- RDF/JSON: 24-1.0354557-rdf.json
- Turtle: 24-1.0354557-turtle.txt
- N-Triples: 24-1.0354557-rdf-ntriples.txt
- Original Record: 24-1.0354557-source.json
- Full Text
- 24-1.0354557-fulltext.txt
- Citation
- 24-1.0354557.ris
Full Text
Cite
Citation Scheme:
Usage Statistics
Share
Embed
Customize your widget with the following options, then copy and paste the code below into the HTML
of your page to embed this item in your website.
<div id="ubcOpenCollectionsWidgetDisplay">
<script id="ubcOpenCollectionsWidget"
src="{[{embed.src}]}"
data-item="{[{embed.item}]}"
data-collection="{[{embed.collection}]}"
data-metadata="{[{embed.showMetadata}]}"
data-width="{[{embed.width}]}"
async >
</script>
</div>
Our image viewer uses the IIIF 2.0 standard.
To load this item in other compatible viewers, use this url:
http://iiif.library.ubc.ca/presentation/dsp.24.1-0354557/manifest