An Approximate Dynamic Programming Approach for Coordinated Charging Control at Vehicle-to-Grid Aggregator by Jinbiao Xu B.Sc., Computer Science and Technology, Fudan University, China, 2009 A THESIS SUBMITTED IN PARTIAL FULFILMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF APPLIED SCIENCE in THE FACULTY OF GRADUATE STUDIES (Electrical and Computer Engineering) THE UNIVERSITY OF BRITISH COLUMBIA (Vancouver) August 2011 c© Jinbiao Xu, 2011 ii Abstract A vehicle-to-grid (V2G) aggregator is an agent between the power grid and the plug-in hybrid electrical vehicles (PHEVs). In this thesis, we study the coordinated charging control at a V2G aggregator. The coordinated charging control brings the advantages of minimizing the charging cost and reducing the power losses, by coordinating the control sequences of a group of PHEVs. On one hand, the lower cost of charging gives the users of PHEVs an incentive to cooperate. On the other hand, with an increasing popularity of PHEVs, the impact on the power distribution grid such as power losses should be of concern to the aggregator. To this end, we investigate the tradeoffs between reducing the charging cost and the power losses. We formulate the coordinated charging control as a dynamic programming problem, given the planned schedules of all the vehicles at an aggregator. As an inherent property of a V2G aggregator, we enable bidirectional electric power flows between PHEVs and the power grid. Due to the curse of dimensionality, we apply an approximate dynamic programming approach to decrease the dimensionality of both the state space and control space. Simulation results show that coordinated charging control can reduce both the total charging cost and the aggregated power losses significantly, compared with the uncoordinated control where every vehicle starts charging Abstract iii as soon as it is plugged in. We also show that the charging control with bidirectional power flows outperforms remarkably the one with unidirectional flows. iv Preface I am the first author of all chapters. All chapters are co-authored with Dr. Vincent W.S. Wong, who supervised the research. A version of Chapter 2 has been accepted for pub- lication. Jinbiao Xu and Vincent W.S. Wong, “An approximate dynamic programming approach for coordinated charging control at vehicle-to-grid aggregator,” in Proc. IEEE SmartGridComm, Brussels, Belgium, October 2011. I formulated the optimization prob- lem as a dynamic programming problem and solved it using an approximate dynamic programming approach. I conducted simulations and prepared the original draft, which was further revised by Dr. Wong. The IEEE holds the copyright for the publication above. However, I retain the right to extract content from the work for the publication of my thesis. vTable of Contents Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ii Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv Table of Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v List of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix List of Acronyms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . x Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.1.1 Advantages of PHEVs . . . . . . . . . . . . . . . . . . . . . . . . 4 1.1.2 Disadvantages of PHEVs . . . . . . . . . . . . . . . . . . . . . . . 5 1.2 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 1.3 Motivations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Table of Contents vi 1.4 Preliminary Dynamic Programming . . . . . . . . . . . . . . . . . . . . . 10 1.4.1 Dynamic Programming Problem . . . . . . . . . . . . . . . . . . . 11 1.4.2 The Three Curses of Dimensionality . . . . . . . . . . . . . . . . . 13 1.4.3 Approximate Dynamic Programming . . . . . . . . . . . . . . . . 13 1.5 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 1.6 List of Publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 1.7 Structure of the Thesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 2 Coordinated Charging Control at V2G Aggregator . . . . . . . . . . . 18 2.1 System Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 2.1.1 State Transition . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 2.1.2 Charging Cost ct(rt) . . . . . . . . . . . . . . . . . . . . . . . . . 21 2.1.3 Power Losses lt(rt) . . . . . . . . . . . . . . . . . . . . . . . . . . 22 2.1.4 Departure Penalty gt(st) . . . . . . . . . . . . . . . . . . . . . . . 23 2.1.5 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . 23 2.2 An Approximate Dynamic Program Approach . . . . . . . . . . . . . . . 24 2.2.1 State Aggregation . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 2.2.2 Sub-States . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 2.2.3 Approximate Policy Iteration Using Monte Carlo Simulation . . . 28 2.3 Performance Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 2.3.1 Simulation Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 2.3.2 Simulation Results . . . . . . . . . . . . . . . . . . . . . . . . . . 34 Table of Contents vii 3 Conclusions and Future Work . . . . . . . . . . . . . . . . . . . . . . . . . 41 3.1 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 3.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 3.2.1 Incorporating Renewable Energy Sources and Battery Storage . . 42 3.2.2 Studying the Effects of Unanticipated Schedules . . . . . . . . . . 49 Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 viii List of Tables 1.1 Estimated Future PHEV Incremental Manufacture Costs [1] . . . . . . . 5 2.1 List of Simulation Parameters . . . . . . . . . . . . . . . . . . . . . . . . 33 ix List of Figures 2.1 The system model of a V2G aggregator. . . . . . . . . . . . . . . . . . . 19 2.2 An example schedule of vehicle i. . . . . . . . . . . . . . . . . . . . . . . 20 2.3 The transition diagram of sub-states. . . . . . . . . . . . . . . . . . . . . 27 2.4 One-line diagram of the 5-bus radial primary distribution system (base line voltage at 12.47 kV). . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 2.5 The day-ahead locational marginal price used in simulation. . . . . . . . 32 2.6 The distribution of available time slots of all PHEVs. . . . . . . . . . . . 34 2.7 The total charging cost versus the importance weight λ1. . . . . . . . . . 35 2.8 The total power losses versus the importance weight λ1. . . . . . . . . . . 36 2.9 The total departure penalty versus the importance weight λ1. . . . . . . 37 2.10 The sum of objective functions from time slot 1 to T versus the importance weight λ1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 2.11 Total cost of charging and power losses varying with different λ1 (λ2 = 10). 39 3.1 One-line diagram of the IEEE 8-bus radial primary distribution grid. . . 45 xList of Acronyms ACSR Aluminum Conductor Steel Reinforced AMI Advanced Metering Infrastructure DP Dynamic Programming EPRI Electric Power Research Institute ISO Independent System Operator LMP Locational Marginal Price NIST National Institute of Standards and Technology PHEV Plug-in Hybrid Electric Vehicle PV Photovoltaic SOC State of Charge V2G Vehicle-to-Grid xi Acknowledgements I would like to express my profound gratitude to my graduate supervisor Dr. Vincent Wong for his guidance and valuable help throughout my graduate studies. I deeply appreciate the amount of time and effort he invested in guiding me in research. His valuable advice is indispensable for the completion of this thesis. I would also like to thank my beloved parents for bringing me up and supporting me forever. Special thanks to my computer science teacher Mr. Yan Kaiming, for teaching me computer programming and leading me to the fascinating world of science and engineering in my junior middle school. This research is supported by the Natural Sciences and Engineering Research Council (NSERC) of Canada. 1Chapter 1 Introduction In this chapter, we first present the background information and related works of the charging control at vehicle-to-grid aggregator. We provide preliminary background of a classic approximate dynamic programming approach to tackle the curses of dimensional- ity. We also discuss the motivation and our contributions. The structure of the thesis is given at the end of this chapter. 1.1 Background Earlier attempts on using electric control, metering, and monitoring build up the fun- damentals of smart power grid. For example, automatic meter reading [2, 3] was used for monitoring loads for industrial customers, and has been evolved into the Advanced Metering Infrastructure (AMI) in smart grid. According to the definition of smart grid by the Electric Power Research Institute (EPRI), a smart grid is a power grid with the mod- ernization of the country’s electricity transmission and distribution systems to maintain a reliable and secure electricity infrastructure that can meet future demand growth and to achieve power reliability and power quality, energy efficiency benefits, environmental Chapter 1. Introduction 2 and conservation benefits and direct financial benefits [4]. Smart grid technologies can help reducing CO2 emission and producing a cleaner environment. Recently, much attention in the power industry has been drawn to research on incorporating renewable energy into the power grid, and on increasing energy efficiency through demand response and smart control [5, 6, 7]. For instance, plug-in hybrid electric vehicles (PHEVs) have been used as a tool to shift demand within the transportation sector, from gasoline to electricity. A typical PHEV can cruise a long distance using only electric power with an all-electric range of forty or more miles [8], due to the ever- increasing battery size. PHEVs have become much more popular in developed countries since they provide substantial fuel savings as a result of reduced gasoline consumption. As an increasing number of PHEVs are being plugged into the power grid, the control or management issue may arise. To this end, a vehicle-to-grid (V2G) aggregator is required to decide the control sequences of a group of PHEVs based on technical constraints (e.g., the state of charge of a PHEV) and specific objectives (e.g., minimizing the cost of charging). One important constraint of PHEVs is the state of charge (SOC) in percentage. Since different types of battery have different ranges of SOC (e.g., a lead-acid battery installed on a hybrid-electric vehicle has an SOC range [16%, . . . , 80%] [9]), without loss of generality, we assume the SOC range in this thesis is [0%, 1%, . . . , 100%]. The charging of PHEVs usually has multiple objectives. On the customer side, the users may be interested in reducing their charging cost. On the other hand, the convenience may also Chapter 1. Introduction 3 play an important role. The users park their vehicles in one of the sites managed by an aggregator, in order to charge the vehicles for their next travel. A conventional and uncoordinated charging scheme would give the users the great convenience as it begins to charge at the full charging rate until the anticipated target SOC is reached. However, just a few percents lower than the target SOC usually does not affect the convenience for the next travel. Hence, the tradeoff between satisfying the target SOC and minimizing the charging cost is possible. On the operator side, the V2G aggregator may be concerned about the charging impact on the local distribution grid. Furthermore, uncoordinated charging of PHEVs may lead to problems in the distribution grid (e.g., power losses). Coordinated charging control [10, 11, 12, 13] manages the charging sequences of PHEVs in real-time. By disabling charging of vehicles at peak time periods when the demand is high, charging control helps smoothing the load of the power grid and thus reduces power losses in the distribution grid. The studies in [10, 13] indicated that the uncoordinated charging can result in 40% power losses more than coordinated charging. The work in [11] showed that the coordinated charging control can maximize the load factor and minimize the load variance other than minimizing power losses. Parks et al. [12] conducted experiments on coordinated charging control. They analyzed the real data from Xcel Energy Colorado, and concluded that there is a huge coincidence between the normal power grid peak demand hours and the time when significant charging would occur, and that a smart charging control is required to shave the charging profile of PHEVs. They also showed that coordinated charging control can significantly reduce the Chapter 1. Introduction 4 cost of charging and also the fuel costs [12]. In order to make coordinated charging control feasible, an aggregator needs to have real-time demand-side monitoring and the ability to communicate with end-users. The National Institute of Standards and Technology (NIST) smart grid conceptual model [4] defined seven important smart grid domains: bulk generation, transmission, distribu- tion, customers, operations, markets and service providers. Since a V2G aggregator is in the operations domain, it can interact with the distribution grid and end-users conve- niently. Hence, the emergence of smart grid paves the way for the charging control of an aggregator. In addition to the technical challenge, an aggregator still has to confront market challenges. One of the market challenges is that a V2G aggregator has to find a large number of PHEVs within a local grid for contracts. According to [14, 15], however, PHEVs will be widely available across the North America. Hence, the rapid expansion of PHEVs paves the way for the commercial V2G aggregators. 1.1.1 Advantages of PHEVs One distinctive advantage of PHEVs is to reduce carbon emission, compared with those conventional vehicles. Even when we charge the PHEVs using electricity that comes from a CO2 emitting source (e.g., a coal or gas fired power plant), the net CO2 production from a PHEV is typically one half to one third of that from a comparable conventional vehicle [16]. Chapter 1. Introduction 5 Table 1.1: Estimated Future PHEV Incremental Manufacture Costs [1] Type Year 2011 Year 2015 Year 2020 Year 2030 PHEV 40 $14,100-$18,100 $11,200-$14,200 $9,600-$12,200 $8,800-$11,000 PHEV 10 $5,500-$6,300 $4,600-$5,200 $4,100-$4,500 $3,700-$4,100 Other than the environmental benefits, the other advantage of a PHEV is its V2G capability. As a PHEV can supply energy to the grid, it could be used in backup power supply, ancillary services, peak shaving, load shifting and shaving the output of intermit- tent generation. 1.1.2 Disadvantages of PHEVs Despite the promising future of PHEVs, one of their disadvantages is that PHEVs have a much higher cost than the conventional vehicles. Table 1.1 [1] shows the incremen- tal manufacture costs of future PHEVs, compared with the conventional ones. These are only manufacturing costs that do not include engineering, marketing, or other costs. For example, PHEV-40 (a PHEV with 40-mile all-electric range) are estimated to cost $14, 100− $18, 100 more than the conventional vehicles in manufacturing by 2011. How- ever, as shown in Table 1.1, those additional costs of PHEVs over conventional vehicles decrease year by year. Hence, it is worthwhile to study the applications of PHEVs. Another shortcoming of PHEVs is that the penetration of PHEVs on the power grid decreases the grid capacity. As the work in [17] indicated, the current electric Chapter 1. Introduction 6 grid infrastructure has enough spare generating capacity to support PHEV penetration levels from 30% to 70% when being considered nation-wide in the United States. Yet this penetration level is only for the whole country, and in small local power grid, the penetration of PHEVs may result in an increase in the peak demand for electric power and cause the lack of grid capacity. It will also incur a high investment cost for grid operators to upgrade the power grid capacity. Rather than upgrading the power grid, a smart coordinated charging control of PHEVs can help in alleviating the lack of grid capacity. 1.2 Related Work In this section, we discuss the related work on V2G charging control and recent appli- cations of PHEVs. With the deployment of smart grid technologies, the control and schedule of V2G power flows (and thus bidirectional flows) becomes possible [18]. Recently, the growing body of literature [4, 8, 19, 20, 21] paid much attention to the use of PHEVs in energy markets. The Electric Power Research Institute (EPRI) has managed and run several user cases of using electric vehicles as a distributed energy resource in smart grids [4]. Kempton et al. [8, 22] retrofitted a Toyota Scion xB and pre- sented a demonstration project to use electric vehicles in frequency regulation. Saber et al. proposed the coordinated scheduling of PHEVs for unit commitment [19]. They used the heuristic method particle swarm optimization to solve the optimal control PHEVs. However, the solution quality is hard to guarentee as the heuristic method itself is not Chapter 1. Introduction 7 stable. Han et al. proposed the scheduling of a V2G aggregator for frequency regula- tion [20]. By deciding when to charge and when to regulate, the aggregator maximizes each PHEV’s compensation from frequency regulation against its charging cost. How- ever, their model may not be suitable to frequency regulation, as the ISO (Independent System Operator) will view all the vehicles of an aggregator as a whole and not split those vehicles into two groups (i.e., charging and regulating). Sortomme et al. proposed smart charging algorithms to obtain an optimal operation point of the aggregate load of all PHEVs [21]. When the aggregator operates below the optimal point, it is considered as regulating up, and vice versa. However, they assumed that V2G power flows and dynamic arrivals and departures of vehicles are disabled. Several charging controls have been studied in previous literature [12, 23, 24, 25]. In [23], Shao et al. examined the impact of charging PHEVs on a distribution trans- former under charging scenarios of all PHEVs starts normal-charging at 6 pm, all PHEVs starts quick-charging at 6 pm, all PHEVs starts normal-charging at off-peak hours and all PHEVs starts quick-charging at off-peak hours. They showed that under the third scenarios, the transformer has the highest efficiency as the peak demand of the third scenario is the smallest. Parks et al. [12] investigated three charging controls: uncoor- dinated charging (begins to charging at the full charging rate once plugged in), delayed charging (delays charging until 10 pm when PHEVs are at home) and off-peak charging (only charges at off-peak hours.) They showed that the uncoordinated charging maxi- mizes electric operation and minimizes carbon emissions, while off-peak charging fills the Chapter 1. Introduction 8 valley of overnight demand and smooths the long-term load. Letendre and Watts [25] experimented with a fleet of 50, 000 PHEVs in the Vermont state. They implemented uncoordinated night charging, delayed and optimal night charging. In the delayed charg- ing control, they used financial incentives such as off-peak rates to shift the peak load. In the optimal nighttime charging, the vehicles are charged only at the night hours with lowest demand. They found that the uncoordinated charging could lead to an increase in the peak demand even at a low penetration rate, while delayed and optimal charging can accommodate up to twice PHEVs than the uncoordinated charging. Tate and Boyd [26] presented a charging control problem of hybrid electric vehicles. They formulated the problem of maximizing the fuel efficiency or minimizing the fuel usage as a nonlinear convex optimization problem that is approximated by a linear pro- gram. The formulation of the linear program simplifies finding a global optimal solution. However, the minimization of fuel usage may not become an interest of both vehicle users and the aggregator. Recent studies [10, 11] presented charging control with respect to power losses of the power grid. In [10], Clement et al. proposed an optimal charging control that minimizes the power losses of the distribution grid. They used stochastic programming to model the unknown household load at each grid node. By shifting the load of PHEVs to non-peak time periods of household loads, the system peak load and the power losses are reduced. In [11], Sortomme et al. formulated the PHEV charging problem as a quadratic program. They studied the relationship among power losses, load factor and load variance. The Chapter 1. Introduction 9 works in both [10] and [11] allow only power flows from the power grid to vehicles, while a V2G aggregator allows power flows in both directions. Moreover, Clement and Sortomme et al. considered the coordinated charging control with the sole purpose of minimizing the power losses. This will clearly not be practical in a V2G aggregator as the users will not be satisfied, given that only minimizing the power losses may lead to an increase of charging cost. 1.3 Motivations In this thesis, we consider the coordinated charging control of a V2G aggregator. The coordinated charging control has to satisfy the customers’ needs. In other words, the users of PHEVs may require the aggregator to achieve an SOC that is higher or equal to the target SOC at departure. We propose a departure penalty function to address this issue. As of the vehicle users’ interest, the charging cost has to be minimized. To achieve this goal, a PHEV has to discharge at peak time and charge at non-peak time. However, since different PHEVs have different schedules, a portion of PHEVs may coincide with peak demand and need help from other PHEVs. Hence, the charging control of the V2G aggregator should not focus on a single PHEV. The coordinated charging aims at the cooperation among different PHEVs. It allows some PHEVs to discharge while others are in need of energy. Moreover, as a large penetration of PHEVs would impose an increased pressure on the distribution grid, the grid capacity may run out and lead to blackouts in the local power grid. The coordinated charging control should have the ability to Chapter 1. Introduction 10 reduce congestion on transmission lines and provide transmission loading relief. Last but not least, as power supply often comes from remote places, the power losses on the distribution grid may be huge and increase in proportion to the square of the load. The coordinated charging control should aim at reducing the quadratically increasing power losses. In order to address all the above issues, we propose a weighted objective function that combines departure penalty, charging cost, and power losses [27]. The objectives of this thesis are three-fold: minimizing the total charging cost, min- imizing the power losses on the distribution grid; and increasing power grid capacity at peak time through V2G power flows. On one hand, a lower cost of charging gives the owners of PHEVs an incentive to cooperate. On the other hand, uncoordinated charging of PHEVs may impose great impact on the power grid, such as the decreased voltage, in- creased peak load and power losses. It is the aggregator’s interest to reduce the resultant power losses, by coordinating a group of PHEVs with dynamic schedules. Considering the dynamics of user behavior, PHEVs can have different time of arrivals and departures. Given the schedules of all PHEVs at the aggregator, we optimize the weighted objective function using approximate dynamic programming. 1.4 Preliminary Dynamic Programming In this section, we provide some background information of preliminary dynamic pro- gramming. First, we introduce a basic dynamic program. Then we explain the three curses of dimensionality. Finally, we discuss the approximate dynamic programming Chapter 1. Introduction 11 approach that is used in the Chapter 2 of this thesis. 1.4.1 Dynamic Programming Problem Dynamic programming is a mathmatical tool to study sequential decision problem [28, 29, 30, 31]. We present a finite-horizon dynamic programming problem in this section. We consider a system with a total of T time slots. Let Sk denote the state space, which is a given finite set. Let the xk denote the state at the kth time slot. xk is an element of the state space Sk. Similarly, let uk denote the control or action at the kth time slot. Let Ok denote the control space, which is a given finite set. uk is an element of the control space Ok. Let Dk denote the disturbance space, which is a given finite set. ωk is the random disturbance at the kth time slot and it is an element of the disturbance space Dk. The control uk is constrained to take values in a given nonempty subset ∅ 6= U(xk) ⊂ Ok. And we have uk ∈ U(xk), ∀xk ∈ Sk. The random disturbance ωk subjects to a probability distribution and is not dependent prior disturbances ωk−1, . . . , ω1. Let fk (k = 1, . . . , T ) denote the given transition functions. The discrete-time system is xk+1 = fk(xk,uk, ωk), k = 1, . . . , T − 1, (1.1) The cost incurred at the kth time slot is denoted by gk(xk,uk, ωk). Chapter 1. Introduction 12 We consider a vector π that consist of a sequence of policies π = (π1, π2, . . . , πT ) , (1.2) where πk maps state xk into control uk = πk(xk) ∈ U(xk). We define the value functions Jpik , k = 1, . . . , T , associated with a given policy π, as the sum of cost starting from xk. Jpik (xk) = E { T∑ i=k gk (xi,ui, ωi) } = E { T∑ i=k gk (xi, πi(xi), ωi) } , (1.3) The optimization objective is to obtain the closed-loop optimal policy π∗, a feasible policy that minimizes the sum of cost starting from the initial state x1. π∗ = argmin pi Jpi1 (x1) = argmin pi E { T∑ k=1 gk (xk, πk(xk), ωk) } . (1.4) The number of mappings from the state space to the control space is finite. And thus the enumeration of π is finite. The Dynamic Programming Algorithm The optimal value (or cost-to-go) functions Jpi ∗ k (xk) satisfy the following recursion of dynamic programming Jpi ∗ k (xk) = min uk∈U(xk) E { gk (xk,uk, ωk) + J pi∗ k+1(fk(xk,uk, ωk)) } , k = 1, . . . , T − 1. (1.5) Chapter 1. Introduction 13 The dynamic programming algorithm is essentially the backward induction until Jpi ∗ 1 (x1). First we initialize J pi∗ T (xT ). Second we derive J pi∗ T−1(xT−1) from (1.5), so on and so forth. After T − 1 steps, we finally obtain Jpi ∗ 1 (x1). 1.4.2 The Three Curses of Dimensionality Let us consider the situation when xk, uk and ωk are all vectors. The explosions in the state, control and outcome space (the space for next states, explodes due to the disturbance ωk) are refer to as the well-known the three curses of dimensionality. The key problem is that the state, control and outcome spaces are all vector spaces. It is indeed impossible to store an vector space in a modern computer. Moreover, it is even impossible to enumerate all the possible states or controls for any fixed time slot. In (1.5), the curse of the state space results in the difficulty to enumerate value functions Jpi ∗ k (xk). The curse of the control space results in the difficulty to take the minimum operation. And the curse of the outcome space results in the difficulty to enumerate the value functions of the next state Jpi ∗ k+1(fk(xk,uk, ωk)). 1.4.3 Approximate Dynamic Programming In this thesis, we have two curses of dimensionality. One is from the state space and the other is from the control space. We use state aggregation [28] to tackle the the curse of the state space Sk, and introduce sub states [28] to tackle the the curse of the control space Ok. Chapter 1. Introduction 14 Aggregation Method Let H(xk) denote an aggregation function. The aggregation method basically gets the hash of the state vector xk and maps the original multi-dimensional state space to a smaller aggregated space, which can have one dimension or multi dimensions. Eqn. (1.5) can be converted into (1.6). Jpi ∗ k (H(xk)) takes average according to its original states. Jpi ∗ k (H(xk)) = ∑ v |H(v)=H(xk) θ(v) min uk∈Uk(v) E { gk (v,uk, ωk) + J pi∗ k+1(H(fk(v,uk, ωk))) } , k = 1, . . . , T − 1. (1.6) where θ(v) denotes the disaggregation probability. Sub States The minimization operation in (1.5) suffers from the dimensionality of the control space. One way to solve this issue is to introduce sub states. This method basically divides the multi-dimensional control vector uk into several subsets of dimensions. Let M denote the number of sub states. We now introduce sub states x1k,x 2 k, . . . ,x M k between xk and xk+1. Let u j k denote the control vector of the sub-state x j k. Each sub state xjk (j = 1, . . . ,M) matches with the corresponding control vector of x j k. In (1.5), we take the minimization over uk. After introducing the sub states, we instead take the minimization over u1k, and then take the minimization over u 2 k, so on and so forth. We obtain a solution for uik between sub state x i k and x i+1 k . A solution for Chapter 1. Introduction 15 uk can be obtained by combining the solutions u 1 k,u 2 k, . . . ,u M k . The new solution for uk may not be a minimized solution in (1.5). However, we trade the quality of the solution for a feasible time complexity of the dynamic programming algorithm. 1.5 Contributions The contributions of this thesis are as follows: • We formulate the coordinated charging control problem as a dynamic program. The model captures the tradeoff between the total cost of charging, the power losses, and the departure penalty. • We apply approximate dynamic programming [28], a state-of-the-art approach that tackles the curse of dimensionality [29], to solve the problem. • We propose a coordinated charging control algorithm at a V2G aggregator. • Simulation results show that the coordinated charging control can reduce both the total cost of charging and power losses significantly, compared with uncoordinated charging. • The coordinated charging control with bidirectional power flows incurs much less power losses, when its total cost of charging is the same as that of the counterpart scheme with unidirectional power flows (i.e., V2G flows disabled). It also incurs less charging cost than the scheme with unidirectional power flows, when the power Chapter 1. Introduction 16 losses of both schemes are the same. Our work differs from [20] in that we consider a coordinated charging control of a V2G aggregator while Han et al. considered each PHEV individually, although both works studied the tradeoff between the cost of charging and satisfying the target SOC.. In [20], each PHEV is optimized with its own schedule and interests. In this case, the charging control in [20] is uncoordinated. The work [20] did not address the cooperation among PHEVs and not consider the impact of PHEV charging on the power grid. Prior work [10, 11] considered the coordinated charging control with the sole purpose of minimizing the power losses. Our work differs in two aspects. First, the objectives we used in this thesis are more practical since the owners of PHEVs can hardly benefit from the sole objective of minimizing the power losses. Moreover, simulation results show that the cash value of the power losses is much less than that of total charging cost. Therefore, the charging cost should not be neglected. Second, the problem formulation is more flexible. In other words, we enable bidirectional power flows between PHEVs and the power grid whereas [10, 11] disabled the V2G power flows in order to reduce the problem size and the number of constraints. We also take into account the the dynamics of PHEVs. With our formulation, PHEVs can have different time of arrivals and departures. 1.6 List of Publications The following publication has been completed based on the work in this thesis. Chapter 1. Introduction 17 • Jinbiao Xu and Vincent W.S. Wong, “An approximate dynamic programming ap- proach for coordinated charging control at vehicle-to-grid aggregator,” accepted for publication in Proc. of IEEE SmartGridComm, Brussels, Belgium, October 2011. 1.7 Structure of the Thesis The rest of this thesis is organized as follows. In Chapter 2, we present our system model and problem formulation. We then describe the approximate dynamic programming approach to solve the optimization problem. Furthermore, we present our simulation setup and comparison results. Conclusions and future work are given in Chapter 3. 18 Chapter 2 Coordinated Charging Control at V2G Aggregator In this chapter, we present the problem formulation of the coordinated charging control of a vehicle-to-grid aggregator. We also develop an approximate dynamic programming approach, which reduces the dimensionality of both the state space and control space, to obtain the optimal control sequences. Finally, we present simulation results of the coordinated charging control at an aggregator and give our analysis. 2.1 System Model The system model of a V2G aggregator is shown in Fig. 2.1. An aggregator manages multiple participating sites, such as residential and public sites. When a PHEV arrives at a participating site, it can be plugged into the power grid. The aggregator operates all those present PHEVs simultaneously. The scheduling period is from time slot 1 to T . The length of each time slot is ∆. In this thesis, we assume that an initial schedule of each PHEV during the scheduling Chapter 2. Coordinated Charging Control at V2G Aggregator 19 Aggregator Vehicle User Terminal Figure 2.1: The system model of a V2G aggregator. period is known before time slot 1. Since the schedules may be volatile, the owners of PHEVs can update their schedules with the aggregator via wireless networks. If any user does not comply with the planned schedule, the aggregator can re-run the scheduling algorithm when the schedule is updated. A planned schedule of a PHEV includes the expected arrival and departing time slots, and the expected initial/target SOC at each arrival/departure time slot. Note that each PHEV can arrive at and depart from the grid multiple times. Let ai,t denote the expected initial SOC of vehicle i at arrival time slot t (valid only at the arrival time slots). Similarly, di,t denotes the expected target SOC of vehicle i at departure time slot t (valid only at the departure time slots). An example schedule of vehicle i is shown in Fig. 2.2. We assume the V2G aggregator has control of a total number of N vehicles. Let si,t denote the state of charge (SOC) of vehicle i, at the beginning of time slot t. We Chapter 2. Coordinated Charging Control at V2G Aggregator 20 Time Slot t 1 2 100 150 200 T=240... ...   Present            Absent                 Present Absent ... ... = 40% = 90% = 70%= 50% Figure 2.2: An example schedule of vehicle i. discretize si,t, i.e., si,t ∈ {0%, 1%, . . . , 100%}. Vehicle i has a maximum charging and discharging power with a magnitude of mi. Let µi denote the number of time slots that takes vehicle i to charge from empty to full capacity using full power mi. The normalized charging or discharging rate of vehicle i at time slot t is ri,t. In this thesis, ri,t can either take −1 (discharging at full power mi), 0 (idle), or 1 (charging at full power mi). We define the state variable as st = (s1,t, s2,t, . . . , sN,t), t = 1, . . . , T. (2.1) The control of time slot t rt = (r1,t, r2,t, . . . , rN,t) ∈ U(st), where the control space U(st) is U(st) = { (r1,t, . . . , rN,t) | ri,t ∈ {−1, 0, 1}, ∀ i = 1, . . . , N ; (2.2) ri,t = 0, if vehicle i is absent at time t; 0% ≤ ⌊ si,t + ri,t µi 100% ⌋ ≤ 100%, if vehicle i is present in the power grid at time t } . where ⌊ ⌋ is the floor function. Chapter 2. Coordinated Charging Control at V2G Aggregator 21 2.1.1 State Transition Both st and rt are vectors. The state transition function f has the form st+1 = f(st, rt) (2.3) = (s1,t+1, s2,t+1, . . . , sN,t+1). For i = 1, . . . , N , si,t+1 =   ai,t+1, if vehicle i arrives at time t+ 1,⌊ si,t + ri,t µi 100% ⌋ , if vehicle i is present in power grid at time t, 0, otherwise. (2.4) When vehicle i is absent from the power grid at time slot t and does not arrive at time t+ 1, we let si,t+1 = 0 by default (2.4). 2.1.2 Charging Cost ct(rt) The total charging cost of the aggregator at time t is ct(rt) = pt N∑ i=1 ri,tmi∆, (2.5) where pt is the price of electricity in time t. In this thesis, we obtain pt from the day- ahead market [32]. Our formulation can be extended when pt is modeled as a random variable with certain probability distribution. Chapter 2. Coordinated Charging Control at V2G Aggregator 22 2.1.3 Power Losses lt(rt) The resistance of transmission lines causes the real power losses, while the reactive power losses attribute to reactive elements such as inductors. The real power losses can reduce the efficiency of transmitted energy to customers, whereas the reactive power losses may decrease the reactive power of the distribution system which is required to maintain at a certain amount for sufficient level of voltage [33]. We consider the real power losses in this thesis. Let X denote the number of transmission lines and lt(rt) denote the total power losses of an aggregator at time slot t lt(rt) = X∑ j=1 I2j (rt)Ωj∆, (2.6) where Ij(rt) and Ωj are the current and the resistance of transmission line j, respectively. In this thesis, we assume the penetration level of PHEVs is 100%. In other words, we consider that all the loads of the aggregator are PHEV loads. However, other loads such as random household loads can be incorporated into our problem formulation. In that case, the problem still remains as a dynamic program. The current Ij is the sum of power flows that pass by line j. Let Φj,t denote the set of vehicles, whose current passes by line j in time slot t. Ij(rt) = ∑ i∈Φj,t ri,tmi V , (2.7) where V is the base line voltage. Chapter 2. Coordinated Charging Control at V2G Aggregator 23 2.1.4 Departure Penalty gt(st) When vehicle i departs at time slot t with SOC si,t, a penalty gi,t(si,t) will occur. We assume that the departure penalty is greater than zero if and only if the departure SOC si,t does not reach the target SOC di,t. There are a variety of penalty functions gi,t(si,t) we can use. In this thesis, we choose the quadratic function gi,t(si,t) =   (di,t − si,t) 2, if vehicle i departs at time t, and si,t < di,t, 0, otherwise. (2.8) The total departure penalty of all PHEVs at time slot t is gt(st) = N∑ i=1 gi,t (si,t) . (2.9) 2.1.5 Problem Formulation The objective function ot(st, rt) in time slot t is ot(st, rt) = ct(rt) + λ1lt(rt) + λ2gt(st), (2.10) where λ1 and λ2 are two positive control variables. λ1 represents how much power losses (in W-h) are worth $1 charging cost. λ2 represents how much departure penalty is worth $1 charging cost. In real applications, the value of the control variables λ1 and λ2 will depend on the specific application requirement (e.g., if the first priority is minimizing the power losses, a large λ1 should be appropriate.) We consider the class of policies (also referred to as control laws [28]) that consist of Chapter 2. Coordinated Charging Control at V2G Aggregator 24 a sequence of policies π = (π1, π2, . . . , πT ) , (2.11) where πt maps state st into control rt = πt(st). We define the value functions Jpit , t = 1, . . . , T , associated with a given policy π, as the sum of cost starting from st. Jpit (st) = T∑ k=t ok (sk, rk) = T∑ k=t ok (sk, πk(sk)) . (2.12) The optimization objective is to obtain the closed-loop optimal policy π∗, a feasible policy that minimizes the sum of cost starting from the initial state s1. π∗ = argmin pi Jpi1 (s1) = argmin pi T∑ k=1 ok (sk, πk(sk)) . (2.13) 2.2 An Approximate Dynamic Program Approach The main issue of solving problem (2.13) is that both st and rt are vectors. Both the state and control space will increase exponentially as the number of vehicles increases. It is also computationally intractable to enumerate all the possible states and controls at any one time slot. In this section, we use the approximate dynamic programming [28] to obtain a sub-optimal solution of problem (2.13). State aggregation is used to reduce the state space, whereas sub-states are introduced to reduce the control space. Chapter 2. Coordinated Charging Control at V2G Aggregator 25 2.2.1 State Aggregation We use an aggregate state space to approximate the value function of the original problem. In this thesis, the aggregation function is H(st) = N∑ i=1 si,t. (2.14) Let Ĵpi ∗ t (H(st)) (t = 1, . . . , T ) denote the optimal value functions of the aggregate problem. We can obtain Ĵpi ∗ t from the unique solution of Bellman’s equation [29]. Ĵpi ∗ t (H(st)) = ∑ s |H(s)=H(st) δs { ot(s, π ∗ t (s)) + Ĵ pi∗ t+1 (H(f(s, π ∗ t (s)))) } , (2.15) where δs is the disaggregation probability [28]. δs are the same for all s such that H(s) = H(st). And ∑ s |H(s)=H(st) δs = 1. Since it takes exponential time to enumerate ∀s : H(s) = H(st), we can only approx- imate Ĵpi ∗ t (H(st)) using Monte Carlo simulations. As the number of samples that we generated increases, the approximation for Ĵpi ∗ t (H(st)), denoted by J̃ pi∗ t (H(st)), will be close to optimal. In this thesis, we solve the optimal policy π∗ by approximating the corresponding value functions Ĵpi ∗ t by J̃ pi∗ t , t = 1, . . . , T . Once we obtain J̃ pi∗ t , the optimal policy π ∗ can be computed as follows. For t = 1, . . . , T , π∗t (st) = arg min rt∈U(st) { ot ( st, rt ) + J̃pi ∗ t+1 ( H(f(st, rt)) )} . (2.16) Note that in (2.16), it takes exponential time to enumerate all the feasible control vari- ables in U(st) since rt is a vector. The vector space of rt results in the curse of dimen- Chapter 2. Coordinated Charging Control at V2G Aggregator 26 sionality [29]. To tackle this issue, we use sub-states [28] to reduce the dimensionality of the control space. 2.2.2 Sub-States We first define the controllable set of time slot t, Ψt as Ψt = {ri,t | vehicle i (1 ≤ i ≤ N) is present in the power grid at time t}. (2.17) For all vehicles that are absent at time slot t (ri,t /∈ Ψt), we have ri,t = 0 from (2.2). We partition Ψt equally into the sequence of M groups of control variables Ψ 1 t , . . . ,Ψ M t (Ψt = ⋃M j=1Ψ j t). Each group Ψ j t (j = 1, . . . ,M − 1) contains the same number of control variables except that the last group ΨMt may possibly contain fewer control variables (e.g., Ψ1t = {r1,t, . . . , r5,t}, . . . ,Ψ M t = {rN−4,t, . . . , rN,t}). In this thesis, we let each group consist of no more than ν = 5 control variables. Thus, it takes at most O(3ν) to enumerate all feasible controls of each group Ψjt , j = 1, . . . ,M . We now introduce sub-states s1t , s 2 t , . . . , s M t between st and st+1. Each sub-state s j t (j = 1, . . . ,M) matches with the corresponding group of control variables Ψjt . Let r j t denote the control vector of the sub-state sjt . Since r j t is associated with a group of control variables Ψjt , we have r j t ∈ U j(st), where the control space U j(st) is U j(st) = { (rj1,t, . . . , r j N,t) | r j i,t ∈ {−1, 0, 1}, ∀ i = 1, . . . , N ; (2.18) rji,t = 0, if r j i,t /∈ Ψ j t ; 0% ≤ ⌊ si,t + r j i,t µi 100% ⌋ ≤ 100%, if rji,t ∈ Ψ j t } . Chapter 2. Coordinated Charging Control at V2G Aggregator 27 ... Figure 2.3: The transition diagram of sub-states. Sub-State Transition The transition of sub-states is s j+1 t = f ( s j t , r j t ) , j = 1, . . . ,M − 1, (2.19) and st+1 = f ( sMt , r M t ) . (2.20) For s1t , we have s1t = ( s11,t, s 1 2,t, . . . , s 1 N,t ) . (2.21) For i = 1, . . . , N , s1i,t =   ai,t+1, if vehicle i arrives at time t+ 1, si,t, if vehicle i is present at time t, 0, otherwise. (2.22) The transition diagram of sub-states is shown in Fig. 2.3. The dashed line is the regular state transition without sub states, while the solid lines are equivalent state transition with sub states. We obtain a solution for rit between sub state s i t and s i+1 t . We Chapter 2. Coordinated Charging Control at V2G Aggregator 28 simply take the minimization over rit, given that r 1 t , . . . , r i−1 t are known since we already obtained the solutions. When we take the minimization over rit, we are in the sub state sit. Therefore, r i+1 t , . . . , r M t can be ignored (i.e., we simply let them be zero) while we minimize rit. After M transitions among sub states, a solution for rt is obtained, which simply combines r1t , r 2 t , . . . , r M t . Now, we can obtain π∗t (st) from (2.16) by decomposing it into a sequence of optimiza- tion problems after introducing sub-states. For t = 1, . . . , T and j = 1, . . . ,M , π∗jt (s j t ) = arg min r j t∈U j(st) { ot ( s j t , r j t ) + J̃pi ∗ t+1 ( H(f(sjt , r j t)) )} , (2.23) where π∗jt (s j t ) contains a valid solution for all ri,t ∈ Ψ j t . Although the combination of r1t , r 2 t , . . . , r M t may not be the optimal solution for rt, we can, by this way, trade the quality of solution for the time complexity of the approximate approach. 2.2.3 Approximate Policy Iteration Using Monte Carlo Simulation In this section, we present a variant of approximate policy iteration [28] based on Monte Carlo simulation. A complete policy iteration includes policy evaluation (steps 1−20) and policy improvement (steps 21− 22) as shown in Algorithm 1. Suppose that the current policy is π. We evaluate and approximate the value functions of π by J̃kt , k = 1, . . . , K, t = 1, . . . , T , using K evaluation iterations. Chapter 2. Coordinated Charging Control at V2G Aggregator 29 In steps 5−12, we explore some random control with probability of ǫ, where 0 < ǫ < 1, otherwise we obtain the control rt driven by current policy π. In steps 16 − 19, we update the value function approximation for the aggregate state space J̃kt (H(st)) with the accumulated path cost Z. Here, γk is the step size for the kth iteration. After we run the evaluation steps for the current policy π by K iterations, we obtain J̃Kt , t = 1, . . . , T . Based on these approximated value functions, we generate an improved policy π̄, which will be used in the next policy iteration, using the decomposition of control variables (2.23) with sub-states in step 22. Chapter 2. Coordinated Charging Control at V2G Aggregator 30 Algorithm 1 Approximate policy iteration using Monte Carlo simulation 1: Initialize J̃0t (t = 1, . . . , T ) 2: for k = 1 to K do {Evaluate current policy π} 3: s1 ← initial state 4: for t = 1 to T do 5: if rand(0, 1) < ǫ then 6: Generate a random exploratory control rt 7: else {Obtain control rt driven by current policy π} 8: rt ← 0 9: for j = 1 to M do 10: rt ← rt + argminrjt∈Uj(st) { ot ( s j t , r j t ) + J̃pit+1 ( H ( f ( s j t , r j t )))} 11: end for 12: end if 13: st+1 ← f(st, rt) 14: end for 15: Z ← 0 16: for t = T downto 1 do 17: Accumulate the path cost Z ← Z + ot(st, rt) 18: Update the value functions J̃kt (H(st))← (1− γk)J̃ k−1 t (H(st)) + γkZ J̃kt (x)← J̃ k−1 t (x) ∀x : x 6= H(st) 19: end for 20: end for 21: J̃ p̄it ← J̃ K t (t = 1, . . . , T ) 22: Generate an improved policy π̄ = (π̄1, . . . , π̄T ) by decomposing (2.16) into (2.23) Chapter 2. Coordinated Charging Control at V2G Aggregator 31 138 kV 12.47 kV 1 2 3 1' 2' 4 5 3' 4' Figure 2.4: One-line diagram of the 5-bus radial primary distribution system (base line voltage at 12.47 kV). 2.3 Performance Evaluation In this section, we provide numerical results of the coordinated charging control at an aggregator with N = 100 PHEVs for the duration of 24 hours. We also compare with the uncoordinated charging control and the unidirectional-power-flow scheme. 2.3.1 Simulation Setup We consider an IEEE 5-bus radial primary distribution system in Fig. 2.4. Here, only buses 2 to 5 are load buses. We assign N 4 = 25 PHEVs equally to each load bus. That is, vehicles 1-25 are parked at bus 2, vehicles 26-50 are parked at bus 3, so on and so forth. There are four transmission lines (1′, 2′, 3′, 4′). We assume the transmission lines are aluminum conductor, steel reinforced (ACSR) 26/7 with 1-mile length. The resistance of each transmission line is 0.1859 Ohm. The base line voltage V = 12.47 kV. Since a PHEV may arrive or depart at any minute, we select a small granularity ∆ Chapter 2. Coordinated Charging Control at V2G Aggregator 32 2 4 6 8 10 12 14 16 18 20 22 24 20 40 60 80 100 120 140 160 180 Hour Pr ic e ($ / M W h) Figure 2.5: The day-ahead locational marginal price used in simulation. which is 6 minutes. The scheduling time period is from 12 PM to the next 12 PM. In other words, T = 240. We employ the day-ahead locational marginal price (LMP) for pt from California ISO [32]. The price information is shown in Fig. 2.5. We assume that all N vehicles have a maximum power mi = 1 kW, and their battery capacity is 10 kWh. Hence, µi = 100 for i = 1, . . . , N . In Algorithm 1, we choose K = 10 9. We use a step size of γk = 1 k [28]. Table 2.1 gives a summary of simulation parameters. We generate the arrivals and departures of each vehicle (i = 1, . . . , N) as follows. Each vehicle has two disjoint available periods that it is present in the power grid. The length of each available period is randomly distributed among {40, . . . , 80} time slots, namely 4 hours to 8 hours. The time slots of arrival/departure and the corresponding initial/target SOC ai,t/di,t are generated at random. And we also guarantee that di,t is Chapter 2. Coordinated Charging Control at V2G Aggregator 33 Table 2.1: List of Simulation Parameters Symbol Definition Value N The number of vehicles. 100 Ωj The resistant of transmission line j. 0.1859 Ohm V The base line voltage. 12.47 kV ∆ The length of each time slot. 6 minutes T The number of time slots. 240 mi The maximum power of vehicle i. 1 kW µi The number of time slots that takes vehicle i from empty battery capacity to full capacity. 100 K The number of evaluation iteration. 109 γk The step size of the kth iteration. 1 k possible to reached at departure. Fig. 2.6 shows the distribution of available time slots of a total number of N = 100 PHEVs. In Fig. 2.6, each line segment of vehicle i represents a time duration when vehicle i is plugged in at the grid. For instance, the first line segment of vehicle i = 1 represents that vehicle i = 1 arrives in the grid at time slot 42 and departs at time slot 109. Chapter 2. Coordinated Charging Control at V2G Aggregator 34 Figure 2.6: The distribution of available time slots of all PHEVs. 2.3.2 Simulation Results In this section, we provide comparison results of the bidirectional-flow coordinated charg- ing control and other counterpart schemes. Coordinated v.s. Uncoordinated To show the advantages of coordinated charging control, we first compare our proposed coordinated charging control with the uncoordinated charging control scheme. The unco- ordinated charging control simply let each vehicle start charging as soon as it is plugged into the grid. Each vehicle keeps charging until either the target SOC is reached or it departs from the power grid. Uncoordinated charging control is a myopic scheme that does not consider the aggregate load of all PHEVs. However, a distinct advantage is that it minimizes the departure penalty gt. In our simulations, the total departure penalty of Chapter 2. Coordinated Charging Control at V2G Aggregator 35 0.01 0.05 0.2 0.3 0.4 0.5 0.7 1 10 15 20 25 30 35 40 45 λ1 To ta l C os t o f C ha rg in g ($ ) Uncoordinated charging control Coordinated charging control, λ2 = 0.05 Coordinated charging control, λ2 = 0.005 Coordinated charging control, λ2 = 0.001 Figure 2.7: The total charging cost versus the importance weight λ1. the uncoordinated scheme is always zero. Fig. 2.7 shows the total cost of charging of all PHEVs. The accumulated power losses of the power distribution grid are shown in Fig. 2.8. Results show that increasing the departure penalty will essentially reduce the total charging cost and accumulated power losses. However, in practice, the charging requirement of PHEVs (i.e., meeting the target SOC) should be given a high priority. In this thesis, we let λ2 = 0.001, 0.005 and 0.05. When λ2 = 0.05, the total departure penalty of the coordinated scheme is always zero, implying that all target SOCs have been reached (Fig. 2.9). Although a larger departure penalty implies further away from the target SOCs, in the worst case when λ2 = 0.001, the average gap between the target SOC and the actual SOC, is at most 5%. When Chapter 2. Coordinated Charging Control at V2G Aggregator 36 0.01 0.05 0.2 0.3 0.4 0.5 0.7 1 0.005 0.01 0.015 0.02 0.025 0.03 0.035 0.04 λ1 To ta l P ow er L os se s (k −W h) Uncoordinated charging control Coordinated charging control, λ2 = 0.05 Coordinated charging control, λ2 = 0.005 Coordinated charging control, λ2 = 0.001 Figure 2.8: The total power losses versus the importance weight λ1. λ2 = 0.005, the average gap is at most 1%. We also investigate the largest gap between the target SOC and the actual SOC. When λ2 = 0.001, the largest gap is 8%. When λ2 = 0.005, the largest gap is only 2% Coordinated charging control has basically two advantages. First, it is price-aware. Fig. 2.7 shows that the coordinated scheme can reduce as much as 70% total cost of charging compared with the uncoordinated scheme when λ1 = 0.01 and λ2 = 0.001. Sec- ond, it enables the cooperation among different PHEVs, in order to smooth the aggregate load over time and thus reduce the power losses. Fig. 2.8 shows that the coordinated charging control can reduce as much as 77% accumulated power losses when λ1 = 1 and λ2 = 0.001. Also, results in Fig. 2.8 show that the total power losses decrease as λ1 Chapter 2. Coordinated Charging Control at V2G Aggregator 37 0.01 0.05 0.2 0.3 0.4 0.5 0.7 1 0 1000 2000 3000 4000 5000 6000 7000 8000 λ1 To ta l D ep ar tu re P en al ty Uncoordinated charging control Coordinated charging control, λ2 = 0.05 Coordinated charging control, λ2 = 0.005 Coordinated charging control, λ2 = 0.001 Figure 2.9: The total departure penalty versus the importance weight λ1. increases. An interesting fact is that in Fig. 2.7, the lowest cost of charging is $13 when λ1 = 0.01 and λ2 = 0.001, whereas the highest power losses of the coordinated scheme is 0.035 kWh in Fig. 2.8. Since the price of electricity that we used for the simulations is always less than $0.2/kWh, the cash value of the accumulated power losses is smaller than the cost of charging. The owners of PHEVs may be reluctant to sacrifice the charging cost to reduce the power losses and thus may prefer a smaller λ1. On the other hand, the power losses are concern to the V2G aggregator. As more power losses impose greater impact to the power grid, the aggregator may prefer a larger λ1 in the interest of reducing the power losses. Chapter 2. Coordinated Charging Control at V2G Aggregator 38 0.01 0.05 0.2 0.3 0.4 0.5 0.7 1 10 20 30 40 50 60 70 80 90 Th e Su m o f O bje cti ve Fu nc tio ns λ1 Uncoordinated charging control Coordinated charging control, λ2 = 0.05 Coordinated charging control, λ2 = 0.005 Coordinated charging control, λ2 = 0.001 Figure 2.10: The sum of objective functions from time slot 1 to T versus the importance weight λ1. Fig. 2.10 shows the sum of objective functions (i.e., eqn. (2.12)), starting from time 1 with initial state s1. The coordinated charging control outperforms the uncoordinated one under all scenarios we use. The reason is that the coordinated charging control reduces both the total cost of charging ∑T t=1 ct(rt) and total power losses ∑T t=1 lt(rt), while keeping the weighted departure penalty λ2 ∑T t=1 gt(st) relatively low. Bidirectional Flows v.s. Unidirectional Flows To show the benefits of enabling V2G power flows, we then compare the coordinated scheme with bidirectional power flows with the coordinated scheme with unidirectional flows (i.e., ri,t ∈ {0, 1}). For a fair comparison, we choose the total departure penalty of Chapter 2. Coordinated Charging Control at V2G Aggregator 39 27 28 29 30 31 32 33 34 35 36 0.01 0.011 0.012 0.013 0.014 0.015 0.016 0.017 Total Cost of Charging ($) To ta l P ow er L os se s (k −W h) Coordinated charging control with unidirectional flows Coordinated charging control with bidirectional flows λ1 = 0.5 λ1 = 0 λ1 = 5.5 λ1 = 20 Figure 2.11: Total cost of charging and power losses varying with different λ1 (λ2 = 10). both schemes to be zero by setting the importance weight λ2 at a large value (λ2 = 10). In other words, all the target SOCs in both schemes are reached. Fig. 2.11 shows the total cost of charging and the power losses of both schemes varying with different λ1. The way we choose λ1 is to make the ranges of power losses of both schemes corresponding to each other. And the minimum λ1 for the unidirectional-flow scheme is 0 (i.e., only minimize the charging cost), while the λ1 for the bidirectional scheme can be set further smaller than 0.5. The results show that coordinated charging control with bidirectional power flows incurs much lower power losses, when its total cost of charging is the same as that of the counterpart scheme with unidirectional power flows (i.e., V2G flows disabled). On the other hand, it also incurs less charging cost than the scheme with unidirectional power Chapter 2. Coordinated Charging Control at V2G Aggregator 40 flows, when the aggregated power losses of both schemes are the same. Note that the coordinated scheme with unidirectional flows has a minimum cost of charging of $31.5 when λ1 = 0, while the one with bidirectional flows has a total cost of charging for only $27 with the same amount of power losses where λ1 = 0.5. Another interesting fact is that, as the power losses of both schemes become larger and larger, the gap between the charging cost of both schemes are increasing. The reason can be found in the X-axis of Fig. 2.11: enabling V2G flows provides the possibility of reduce more charging cost. In Fig. 2.11, the charging cost of the bidirectional-flow scheme has decreased $11 (38− 27) while the charging cost of the unidirectional one has decreased only $6.5. 41 Chapter 3 Conclusions and Future Work In this chapter, we conclude this thesis by summarizing the research results and contri- butions. We also suggest directions for future work. 3.1 Conclusions In this thesis, we developed the coordinated charging control of a V2G aggregator, which reduces both the total cost of charging and the aggregated power losses. Given the day-ahead electricity prices and schedules of PHEVs, we formulated the problem as a dynamic program. We solved the problem using approximate dynamic programming, which reduces the dimensionality of both the state and control spaces. Simulation results of an aggregator with 100 PHEVs are presented to validate the benefits of the coordinated charging control. Simulation results show that coordinated charging control can reduce both the total charging cost and the aggregated power losses significantly, compared with the uncoordinated control where every vehicle starts charging as soon as it is plugged in. We also showed that the charging control with bidirectional power flows outperforms remarkably the one with unidirectional flows. Chapter 3. Conclusions and Future Work 42 3.2 Future Work In this section, we suggest two future extensions to improve the quality of our present work. 3.2.1 Incorporating Renewable Energy Sources and Battery Storage Since promoting green environment is an inherent property of smart grid, it would be interesting to incorporate renewable energy sources and battery storage into the problem formulation. Renewable energy sources [34, 35, 36] such as wind turbines, photovoltaic (PV), are easy to deploy in the smart grid. Other than that, battery storage [37, 38, 39, 40] has been studied to integrate into the power grid. Battery storage is a group of electrochemical cells and often comes in the form of battery arrays. The battery storage may serve as an electric buffer. Battery storage would usually have high initial cost, but can be recharged very cheaply and used many times. Renewable Energy Sources We can assume that the aggregator owns a total number of H renewable energy sources. The output power of each renewable energy source is denoted by ωi,t(i = 1, . . . , H). Let ωt denote a vector of output power ωt = (ω1,t, ω2,t, . . . , ωH,t) . (3.1) Chapter 3. Conclusions and Future Work 43 Since most renewable energy sources are predictable power supply such as hydro power, wind power and photovoltaic, we assume that each random variable ωi,t has a given distribution. Battery Storage A battery storage can be regarded as a special PHEV that is always online (i.e., present in the grid). There is no target SOC requirement for battery storage. We assume that the aggregator owns a total number of G battery storage. Let bi,t denote the state of charge of storage i at the beginning of time slot t. We define the state vector bt as bt = (b1,t, . . . , bG,t). (3.2) Correspondingly, let ui,t denote the control variable. We then define the control vector ut as ut = (u1,t, . . . , uG,t). (3.3) The domain of the control variable ui,t is ui,t ∈ {−1, 0, 1}. (3.4) Storage i has a maximum charging and discharging power with a magnitude of m̂i. Let µ̂i denote the number of time slots that takes storage i to charge from empty to full capacity. Chapter 3. Conclusions and Future Work 44 Hence, we have 0% ≤ ⌊ bi,t + ui,t µ̂i 100% ⌋ ≤ 100% (3.5) Let ŝt denote the new state variable. We have ŝt = (st,bt) (3.6) Similarly, let r̂t denote the new control variable. And we also have r̂t = (rt,ut) (3.7) The new state transition function f̂ has the form ŝt+1 = f̂ (̂st, r̂t) = (f(st, rt), f(bt,ut)) (3.8) Let us assume that storage i has an initial battery capacity βi at the beginning of time slot 1. For i = 1, . . . , G, bi,t =   βi, t = 1,⌊ bi,t−1 + ui,t−1 µ̂i 100% ⌋ , otherwise. (3.9) In (3.9), the state transition of battery storage i is given. At the beginning of time slot 1, bi,1 is initialized as βi (a known constant). Since the battery storage is a special PHEV that is always online, the following states can be calculated by recursions. State Reduction for Battery Storage Recall that the time complexity of dynamic programming algorithms depends on the number of variables in the state vector ŝt = [stbt] (i.e., N + G). Thus it is absolutely Chapter 3. Conclusions and Future Work 45 1 2 3 4 6 5 7 8 Figure 3.1: One-line diagram of the IEEE 8-bus radial primary distribution grid. necessary to reduce the state vector whenever possible. Fortunately, we know that all the battery storage arrive at time slot 1 and never depart. There is no requirement of departure SOC for all battery storage. Hence, we can reduce the bt from G dimensions to reduced dimensions. The reason why we can preform state reduction for the battery storage lies in the fact that a number of battery storage have the same location in the power distribution grid and schedule. Moreover, unlike electric vehicles, battery storage does not have target SOC requirement. The way we reduce bt is simply that we view all the battery storage within the same bus of the distribution grid as a whole. For example, Fig. 3.1 gives an example 8-bus distribution grid. In Fig. 3.1, bus 8 may connect with four buildings. And we can combine all the battery storage within those four buildings. Assume that the maximum charging and discharging power of the original four battery storage are Chapter 3. Conclusions and Future Work 46 m̂1, . . . , m̂4 and the number of time slots that takes storage 1− 4 to charge from empty to full capacity are µ̂1, . . . , µ̂4 and the initial battery SOC are β1, . . . , β4 at the beginning of time slot 1. The combined battery storage for those battery storage within bus 8 has a maximum charging and discharging power of min4i=1{m̂i}, and the number of time slots that takes it to charge from empty to full capacity is ∑ 4 i=1 m̂iµ̂i min4i=1{m̂i} . The initial SOC of the combined battery storage is ∑ 4 i=1 m̂iµ̂iβi∑ 4 i=1 m̂iµ̂i . Similarly, we can preform state reduction for all other buses in Fig. 3.1. Although we can obtain performance gain for the algorithm through state reduction, the shortcoming of this approach is that only one battery storage can be active at one time. Since we take all battery storage as a whole, if one of the battery is charged to full state or discharged to empty state, the other will continue the former’s operation. The New Problem Formulation The total charging cost of the aggregator at time t is ct(r̂t, ωt) = pt∆ ( N∑ i=1 ri,tmi + G∑ i=1 ui,tm̂i − H∑ i=1 ωi,t ) . (3.10) The total power losses of an aggregator at time slot t is lt(r̂t, ωt) = X∑ j=1 I2j (r̂t, ωt)Ωj∆. (3.11) The current Ij is the quotient of the sum of power and the voltage. To obtain the total power across transmission line j, we let Φj,t denote the set of vehicles, which constitute Chapter 3. Conclusions and Future Work 47 a portion of the total power across transmission line j. Let φj,t denote the set of battery storage, which constitute a portion of the total power across transmission line j. Let ϕj,t denote the set of renewable energy sources, which constitute a portion of the total power across transmission line j. Ij(r̂t, ωt) = ∑ i∈Φj,t ri,tmi + ∑ i∈φj,t ui,tm̂i − ∑ i∈ϕj,t ωi,t V . (3.12) Since there is no requirement of departure SOC for all battery storage, the total departure penalty at time slot t is gt(̂st) = N∑ i=1 gi,t (si,t) . (3.13) The objective function ot(̂st, r̂t, ωt) in time slot t is ot(̂st, r̂t, ωt) = ct(r̂t, ωt) + λ1lt(r̂t, ωt) + λ2gt(̂st). (3.14) From the objective function in (3.14), we can conclude that the scheduling problem of the aggregator that incorporates with renewable energy sources and battery storage, remains as a dynamic program. The difference between the new objective function (3.14) and the original one in (2.10) is that the new one includes random variables ωt. Further- more, the new problem can be readily solved using approximate dynamic programming. Algorithm 2 shows the updated approximate policy iteration using Monte Carlo sim- ulation. Steps 1 − 21 is the policy evaluation for the current policy, and steps 22 − 23 obtains an improved policy. The only change is the step 4, where we generate random samples for ω1, . . . , ωT from their given distribution. Chapter 3. Conclusions and Future Work 48 Algorithm 2 Updated approximate policy iteration using Monte Carlo simulation 1: Initialize J̃0t (t = 1, . . . , T ) 2: for k = 1 to K do {Evaluate current policy π} 3: ŝ1 ← initial state 4: Generate samples of random variables ω1, . . . , ωT 5: for t = 1 to T do 6: if rand(0, 1) < ǫ then 7: Generate a random exploratory control r̂t 8: else {Obtain control r̂t driven by current policy π} 9: r̂t ← 0 10: for j = 1 to M do 11: r̂t ← r̂t + argminr̂jt∈Uj (̂st) { ot ( ŝ j t , r̂ j t , ωt ) + J̃pit+1 ( H ( f̂ ( ŝ j t , r̂ j t )))} 12: end for 13: end if 14: ŝt+1 ← f̂ (̂st, r̂t) 15: end for 16: Z ← 0 17: for t = T downto 1 do 18: Accumulate the path cost Z ← Z + ot(̂st, r̂t, ωt) 19: Update the value functions J̃kt (H(̂st))← (1− γk)J̃ k−1 t (H(̂st)) + γkZ J̃kt (x)← J̃ k−1 t (x) ∀x : x 6= H(̂st) 20: end for 21: end for 22: J̃ p̄it ← J̃ K t (t = 1, . . . , T ) 23: Generate an improved policy π̄ = (π̄1, . . . , π̄T ) Chapter 3. Conclusions and Future Work 49 3.2.2 Studying the Effects of Unanticipated Schedules Throughout this thesis, we have assumed that the initial schedules of PHEVs are known before the scheduling periods. Recall that a planned schedule of a PHEV includes the arrival and departure time, the initial and target SOC. In practice, people tend to change their travel schedules based on last-minute decisions. Hence the expected arrival and departure time may change. Moreover, the expected initial SOC may also change due to different driving styles and various climate conditions. For example, a college student may be expected to drive home after school at 4 pm. However, he may change his mind at 3 pm and decide to drive to the football court first. Then he will drive home after a 2-hour football match with friends. The student can notify the aggregator his change of schedule via his mobile phone. The aggregator will notice the change of the following time of arrival and also calculate the change of initial SOC from the driving distances between school and the football court, home and the football court. Although the aggregator can re-run the scheduling algorithm once any information of the schedules has changed, the effects of unanticipated schedules on performance are not within the scope of this thesis. One important thing to address is that, given the fact that the uncertainty cannot be predicted (i.e., we do not know the probability of when and how any schedule will change, and the probability distribution of how one schedule changes to another), the best effort we can try is to re-schedule given the maximum information. Hence, the further improvement of this study should focus on modeling the when and how the schedules change, as we cannot derive an adaptive algorithm without the knowledge of Chapter 3. Conclusions and Future Work 50 the uncertainty we want to adapt to. For future work, it would be interesting to study the effects of unanticipated schedules of PHEVs. First, investigate the performance gap between the case where the actual arrival and departure times are known beforehand and the case where a portion of the arrival and departure times are changing during the scheduling period. Second, study the performance gap between the case where the actual initial SOCs are known beforehand and the case where some of the initial SOCs are changing during the scheduling period. 51 Bibliography [1] “Transitions to alternative transportation technologies: Plug-in hybrid electric vehicles,” National Research Council, The National Academies Press, Washington, D.C., 2010. [Online]. Available: http://books.nap.edu/openbook.php?record id= 12826&page=2 [2] B. Russell, “Communication alternatives for distribution metering and load manage- ment,” IEEE Trans. on Power Apparatus and Systems, vol. 99, no. 4, pp. 1448–1455, July 1980. [3] N. Miura, H. Sato, H. Narita, and M. Takaki, “Automatic meter-reading system by power line carrier communications,” IEE Proceedings, Generation, Transmission and Distribution, vol. 137, no. 1, pp. 25–31, Jan. 1990. [4] D. V. Dollen, “Report to NIST on the smart grid interoperability standards roadmap,” Electric Power Research Institute, June 2006. [Online]. Available: http:// www.nist.gov/smartgrid/upload/InterimSmartGridRoadmapNISTRestructure.pdf [5] A. H. Mohsenian-Rad, V. W. S. Wong, J. Jatskevich, R. Schober, and A. Leon- Garcia, “Autonomous demand side management based on game-theoretic energy Bibliography 52 consumption scheduling for the future smart grid,” IEEE Trans. on Smart Grid, vol. 1, no. 3, pp. 320–331, Dec. 2010. [6] P. Samadi, H. Mohsenian-Rad, R. Schober, V. W. S. Wong, and J. Jatskevich, “Optimal real-time pricing algorithm based on utility maximization for smart grid,” in Proc. of IEEE SmartGridComm, Gaithersburg, MD, Oct. 2010. [7] J. Lopes, F. Soares, and P. Almeida, “Integration of electric vehicles in the electric power system,” Proceedings of the IEEE, vol. 99, no. 1, pp. 168–183, Jan. 2011. [8] W. Kempton, V. Udo, K. Huber, K. Komara, S. Letendre, S. Baker, D. Brunner, and N. Pearre, “A test of vehicle-to-grid (V2G) for energy storage and frequency regulation in the PJM system,” University of Delaware, Pepco Holdings Inc., PJM Interconnect and Green Mountain College, Jan. 2009. [9] B. Bhangu, P. Bentley, D. Stone, and C. Bingham, “Nonlinear observers for pre- dicting state-of-charge and state-of-health of lead-acid batteries for hybrid-electric vehicles,” IEEE Trans. on Vehicular Technology, vol. 54, no. 3, pp. 783–794, May 2005. [10] K. Clement, E. Haesen, and J. Driesen, “The impact of charging plug-in hybrid electric vehicles on a residential distribution grid,” IEEE Trans. on Power Systems, vol. 25, no. 1, pp. 371–380, Feb. 2010. Bibliography 53 [11] E. Sortomme, M. Hindi, S. MacPherson, and S. Venkata, “Coordinated charging of plug-in hybrid electric vehicles to minimize distribution system losses,” IEEE Trans. on Smart Grid, vol. 2, no. 1, pp. 198–205, Mar. 2011. [12] K. Parks, P. Denholm, and T. Markel, “Costs and emissions associated with plug- in hybrid electric vehicle charging in the Xcel energy Colorado service territory,” Technical report, National Renewable Energy Laboratory (NREL), May 2007. [13] S. Acha, T. C. Green, and N. Shah, “Effects of optimised plug-in hybrid vehicle charging strategies on electric distribution network losses,” in Proc. IEEE PES Transmission and Distribution Conf. and Exposition, New Orleans, LA, Apr. 2010. [14] J. Axsen and K. Kurani, “The early U.S. market for PHEVs: Anticipating consumer awareness, recharge potential, design priorities and energy impacts,” Institute of Transportation Studies, University of California, Davis, July 2008. [Online]. Available: http://pubs.its.ucdavis.edu/download pdf.php?id=1191 [15] “Global market analysis of plug in hybrid electric vehicles,” Frost and Sullivan, Dec. 2007. [Online]. Available: http://www.frost.com/prod/servlet/report-toc.pag? repid=M12D-01-00-00-00 [16] C. Gribben, “Debunking the myth of EVs and smokestacks,” Electric Vehicle Association of Greater Washington, D.C, 2007. [Online]. Available: http://www.electroauto.com/info/pollmyth.shtml Bibliography 54 [17] S. W. Hadley, “Impact of plug-in hybrid vehicles on the electric grid,” Oak Ridge National Laboratory, Tech. Rep., Oct. 2006. [Online]. Available: http://www.ornl.net/info/ornlreview/v40 2 07/2007 plug-in paper.pdf [18] A. Ipakchi and F. Albuyeh, “Grid of the future,” IEEE Power and Energy Magazine, vol. 7, no. 2, pp. 52–62, Mar. 2009. [19] A. Saber and G. Venayagamoorthy, “Optimization of vehicle-to-grid scheduling in constrained parking lots,” in Proc. IEEE PES Power and Energy Society General Meeting, New Orleans, LA, July 2009. [20] S. Han, S. Han, and K. Sezaki, “Development of an optimal vehicle-to-grid aggrega- tor for frequency regulation,” IEEE Trans. on Smart Grid, vol. 1, no. 1, pp. 65–72, June 2010. [21] E. Sortomme and M. El-Sharkawi, “Optimal charging strategies for unidirectional vehicle-to-grid,” IEEE Trans. on Smart Grid, vol. 2, no. 1, pp. 131–138, Mar. 2011. [22] W. Kempton and J. Tomic, “Vehicle-to-grid power fundamentals: Calculating ca- pacity and net revenue,” Journal of Power Sources, vol. 144, no. 1, pp. 268–279, June 2005. [23] S. Shao, M. Pipattanasomporn, and S. Rahman, “Challenges of PHEV penetration to the residential distribution network,” in Proc. IEEE Power and Energy Society General Meeting, Calgary, Canada, July 2009. Bibliography 55 [24] S. Shao, T. Zhang, M. Pipattanasomporn, and S. Rahman, “Impact of TOU rates on distribution load shapes in a smart grid with PHEV penetration,” in Proc. IEEE Transmission and Distribution Conference and Exposition, Orlando, FL, Apr. 2010. [25] S. Letendre and R. A. Watts, “Effects of plug-in hybrid electric vehicles on the Vermont electric transmission system,” in Proc. Transportation Research Board 88th Annual Meeting, Washington, DC, Jan. 2009. [26] E. Tate and S. Boyd, “Finding ultimate limits of performance for hybrid electric ve- hicles,” in Proc. SAE Future Transportation Technology Conference and Exposition, Costa Mesa, CA, Aug. 2000. [27] J. Xu and V. W. S. Wong, “An approximate dynamic programming approach for coordinated charging control at vehicle-to-grid aggregator,” in Proc. of IEEE Smart- GridComm, Brussels, Belgium, Oct. 2011. [28] D. P. Bertsekas, Dynamic Programming and Optimal Control, vol. II, 3rd ed. Athena Scientific, 2007. [29] R. E. Bellman, Dynamic Programming, 1st ed. Princeton University Press, 1957. [30] R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction, 1st ed. MIT Press, Cambridge, MA, 1998. [31] D. P. Bertsekas and J. N. Tsitsiklis, Neuro-Dynamic Programming (Optimization and Neural Computation Series 3), 1st ed. Athena Scientific, 1996. Bibliography 56 [32] California ISO Open Access Same-time Information System. [Online]. Available: http://oasis.caiso.com [33] M. Kashem, A. Le, M. Negnevitsky, and G. Ledwich, “Distributed generation for minimization of power losses in distribution systems,” in Proc. IEEE Power Engi- neering Society General Meeting 2006, Montreal, Canada, June 2006. [34] C. Cecati, C. Citro, and P. Siano, “Combined operations of renewable energy systems and responsive demand in a smart grid,” IEEE Trans. on Sustainable Energy, July 2011. [35] A. Saber and G. Venayagamoorthy, “Plug-in vehicles and renewable energy sources for cost and emission reductions,” IEEE Trans. on Industrial Electronics, vol. 58, no. 4, pp. 1229–1238, Apr. 2011. [36] G. Shafiullah, A. Oo, D. Jarvis, A. Ali, and P. Wolfs, “Prospects of solar energy in Australia,” in Proc. International Conference on Electrical and Computer Engineer- ing (ICECE), Dhaka, Bangladesh, Dec. 2010. [37] M. Glavin, P. Chan, S. Armstrong, and W. Hurley, “A stand-alone photovoltaic su- percapacitor battery hybrid energy storage system,” in Proc. 13th Power Electronics and Motion Control Conference, Poznan, Poland, Sept. 2008. [38] Q. Wang and S. Choi, “The design of battery energy storage system in a unified power-flow control scheme,” IEEE Trans. on Power Delivery, vol. 23, no. 2, pp. 1015–1024, Apr. 2008. Bibliography 57 [39] S. Wakao and N. Ueda, “Investigation of the storage battery station for effective utilization of electric power generated by PV clusters,” in Proc. IEEE 4th World Conference on Photovoltaic Energy Conversion, Waikoloa, HI, May 2006. [40] H. Qian, J. Zhang, J.-S. Lai, and W. Yu, “A high-efficiency grid-tie battery energy storage system,” IEEE Trans. on Power Electronics, vol. 26, no. 3, pp. 886–896, Mar. 2011.