Open Collections

UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

An approximate dynamic programming approach for coordinated charging control at vehicle-to-grid aggregator Xu, Jinbiao 2011

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
24-ubc_2011_fall_xu_jinbiao.pdf [ 456.3kB ]
Metadata
JSON: 24-1.0072073.json
JSON-LD: 24-1.0072073-ld.json
RDF/XML (Pretty): 24-1.0072073-rdf.xml
RDF/JSON: 24-1.0072073-rdf.json
Turtle: 24-1.0072073-turtle.txt
N-Triples: 24-1.0072073-rdf-ntriples.txt
Original Record: 24-1.0072073-source.json
Full Text
24-1.0072073-fulltext.txt
Citation
24-1.0072073.ris

Full Text

An Approximate Dynamic Programming Approach for Coordinated Charging Control at Vehicle-to-Grid Aggregator by Jinbiao Xu B.Sc., Computer Science and Technology, Fudan University, China, 2009  A THESIS SUBMITTED IN PARTIAL FULFILMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF APPLIED SCIENCE  in  THE FACULTY OF GRADUATE STUDIES (Electrical and Computer Engineering)  THE UNIVERSITY OF BRITISH COLUMBIA (Vancouver)  August 2011  c Jinbiao Xu, 2011  ii  Abstract A vehicle-to-grid (V2G) aggregator is an agent between the power grid and the plug-in hybrid electrical vehicles (PHEVs). In this thesis, we study the coordinated charging control at a V2G aggregator. The coordinated charging control brings the advantages of minimizing the charging cost and reducing the power losses, by coordinating the control sequences of a group of PHEVs. On one hand, the lower cost of charging gives the users of PHEVs an incentive to cooperate. On the other hand, with an increasing popularity of PHEVs, the impact on the power distribution grid such as power losses should be of concern to the aggregator. To this end, we investigate the tradeoffs between reducing the charging cost and the power losses. We formulate the coordinated charging control as a dynamic programming problem, given the planned schedules of all the vehicles at an aggregator. As an inherent property of a V2G aggregator, we enable bidirectional electric power flows between PHEVs and the power grid. Due to the curse of dimensionality, we apply an approximate dynamic programming approach to decrease the dimensionality of both the state space and control space. Simulation results show that coordinated charging control can reduce both the total charging cost and the aggregated power losses significantly, compared with the uncoordinated control where every vehicle starts charging  Abstract  iii  as soon as it is plugged in. We also show that the charging control with bidirectional power flows outperforms remarkably the one with unidirectional flows.  iv  Preface I am the first author of all chapters. All chapters are co-authored with Dr. Vincent W.S. Wong, who supervised the research. A version of Chapter 2 has been accepted for publication. Jinbiao Xu and Vincent W.S. Wong, “An approximate dynamic programming approach for coordinated charging control at vehicle-to-grid aggregator,” in Proc. IEEE SmartGridComm, Brussels, Belgium, October 2011. I formulated the optimization problem as a dynamic programming problem and solved it using an approximate dynamic programming approach. I conducted simulations and prepared the original draft, which was further revised by Dr. Wong. The IEEE holds the copyright for the publication above. However, I retain the right to extract content from the work for the publication of my thesis.  v  Table of Contents Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  ii  Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  iv  Table of Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  v  List of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  viii  List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  ix  List of Acronyms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  x  Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  xi  1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  1  1.1  Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  1  1.1.1  Advantages of PHEVs . . . . . . . . . . . . . . . . . . . . . . . .  4  1.1.2  Disadvantages of PHEVs . . . . . . . . . . . . . . . . . . . . . . .  5  1.2  Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  6  1.3  Motivations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  9  Table of Contents 1.4  vi  Preliminary Dynamic Programming . . . . . . . . . . . . . . . . . . . . .  10  1.4.1  Dynamic Programming Problem . . . . . . . . . . . . . . . . . . .  11  1.4.2  The Three Curses of Dimensionality . . . . . . . . . . . . . . . . .  13  1.4.3  Approximate Dynamic Programming . . . . . . . . . . . . . . . .  13  1.5  Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  15  1.6  List of Publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  16  1.7  Structure of the Thesis . . . . . . . . . . . . . . . . . . . . . . . . . . . .  17  2 Coordinated Charging Control at V2G Aggregator . . . . . . . . . . .  18  2.1  2.2  2.3  System Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  18  2.1.1  State Transition . . . . . . . . . . . . . . . . . . . . . . . . . . . .  21  2.1.2  Charging Cost ct (rt ) . . . . . . . . . . . . . . . . . . . . . . . . .  21  2.1.3  Power Losses lt (rt ) . . . . . . . . . . . . . . . . . . . . . . . . . .  22  2.1.4  Departure Penalty gt (st ) . . . . . . . . . . . . . . . . . . . . . . .  23  2.1.5  Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . .  23  An Approximate Dynamic Program Approach . . . . . . . . . . . . . . .  24  2.2.1  State Aggregation . . . . . . . . . . . . . . . . . . . . . . . . . . .  25  2.2.2  Sub-States . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  26  2.2.3  Approximate Policy Iteration Using Monte Carlo Simulation . . .  28  Performance Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . .  31  2.3.1  Simulation Setup . . . . . . . . . . . . . . . . . . . . . . . . . . .  31  2.3.2  Simulation Results . . . . . . . . . . . . . . . . . . . . . . . . . .  34  Table of Contents 3 Conclusions and Future Work . . . . . . . . . . . . . . . . . . . . . . . . .  vii 41  3.1  Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  41  3.2  Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  42  3.2.1  Incorporating Renewable Energy Sources and Battery Storage . .  42  3.2.2  Studying the Effects of Unanticipated Schedules . . . . . . . . . .  49  Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  51  viii  List of Tables 1.1  Estimated Future PHEV Incremental Manufacture Costs [1] . . . . . . .  5  2.1  List of Simulation Parameters . . . . . . . . . . . . . . . . . . . . . . . .  33  ix  List of Figures 2.1  The system model of a V2G aggregator. . . . . . . . . . . . . . . . . . .  19  2.2  An example schedule of vehicle i. . . . . . . . . . . . . . . . . . . . . . .  20  2.3  The transition diagram of sub-states. . . . . . . . . . . . . . . . . . . . .  27  2.4  One-line diagram of the 5-bus radial primary distribution system (base line voltage at 12.47 kV). . . . . . . . . . . . . . . . . . . . . . . . . . . .  31  2.5  The day-ahead locational marginal price used in simulation. . . . . . . .  32  2.6  The distribution of available time slots of all PHEVs. . . . . . . . . . . .  34  2.7  The total charging cost versus the importance weight λ1 . . . . . . . . . .  35  2.8  The total power losses versus the importance weight λ1 . . . . . . . . . . .  36  2.9  The total departure penalty versus the importance weight λ1 . . . . . . .  37  2.10 The sum of objective functions from time slot 1 to T versus the importance weight λ1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  38  2.11 Total cost of charging and power losses varying with different λ1 (λ2 = 10). 39 3.1  One-line diagram of the IEEE 8-bus radial primary distribution grid. . .  45  x  List of Acronyms ACSR  Aluminum Conductor Steel Reinforced  AMI  Advanced Metering Infrastructure  DP  Dynamic Programming  EPRI  Electric Power Research Institute  ISO  Independent System Operator  LMP  Locational Marginal Price  NIST  National Institute of Standards and Technology  PHEV  Plug-in Hybrid Electric Vehicle  PV  Photovoltaic  SOC  State of Charge  V2G  Vehicle-to-Grid  xi  Acknowledgements I would like to express my profound gratitude to my graduate supervisor Dr. Vincent Wong for his guidance and valuable help throughout my graduate studies. I deeply appreciate the amount of time and effort he invested in guiding me in research. His valuable advice is indispensable for the completion of this thesis. I would also like to thank my beloved parents for bringing me up and supporting me forever. Special thanks to my computer science teacher Mr. Yan Kaiming, for teaching me computer programming and leading me to the fascinating world of science and engineering in my junior middle school. This research is supported by the Natural Sciences and Engineering Research Council (NSERC) of Canada.  1  Chapter 1 Introduction In this chapter, we first present the background information and related works of the charging control at vehicle-to-grid aggregator. We provide preliminary background of a classic approximate dynamic programming approach to tackle the curses of dimensionality. We also discuss the motivation and our contributions. The structure of the thesis is given at the end of this chapter.  1.1  Background  Earlier attempts on using electric control, metering, and monitoring build up the fundamentals of smart power grid. For example, automatic meter reading [2, 3] was used for monitoring loads for industrial customers, and has been evolved into the Advanced Metering Infrastructure (AMI) in smart grid. According to the definition of smart grid by the Electric Power Research Institute (EPRI), a smart grid is a power grid with the modernization of the country’s electricity transmission and distribution systems to maintain a reliable and secure electricity infrastructure that can meet future demand growth and to achieve power reliability and power quality, energy efficiency benefits, environmental  Chapter 1. Introduction  2  and conservation benefits and direct financial benefits [4]. Smart grid technologies can help reducing CO2 emission and producing a cleaner environment. Recently, much attention in the power industry has been drawn to research on incorporating renewable energy into the power grid, and on increasing energy efficiency through demand response and smart control [5, 6, 7]. For instance, plug-in hybrid electric vehicles (PHEVs) have been used as a tool to shift demand within the transportation sector, from gasoline to electricity. A typical PHEV can cruise a long distance using only electric power with an all-electric range of forty or more miles [8], due to the everincreasing battery size. PHEVs have become much more popular in developed countries since they provide substantial fuel savings as a result of reduced gasoline consumption. As an increasing number of PHEVs are being plugged into the power grid, the control or management issue may arise. To this end, a vehicle-to-grid (V2G) aggregator is required to decide the control sequences of a group of PHEVs based on technical constraints (e.g., the state of charge of a PHEV) and specific objectives (e.g., minimizing the cost of charging). One important constraint of PHEVs is the state of charge (SOC) in percentage. Since different types of battery have different ranges of SOC (e.g., a lead-acid battery installed on a hybrid-electric vehicle has an SOC range [16%, . . . , 80%] [9]), without loss of generality, we assume the SOC range in this thesis is [0%, 1%, . . . , 100%]. The charging of PHEVs usually has multiple objectives. On the customer side, the users may be interested in reducing their charging cost. On the other hand, the convenience may also  Chapter 1. Introduction  3  play an important role. The users park their vehicles in one of the sites managed by an aggregator, in order to charge the vehicles for their next travel. A conventional and uncoordinated charging scheme would give the users the great convenience as it begins to charge at the full charging rate until the anticipated target SOC is reached. However, just a few percents lower than the target SOC usually does not affect the convenience for the next travel. Hence, the tradeoff between satisfying the target SOC and minimizing the charging cost is possible. On the operator side, the V2G aggregator may be concerned about the charging impact on the local distribution grid. Furthermore, uncoordinated charging of PHEVs may lead to problems in the distribution grid (e.g., power losses). Coordinated charging control [10, 11, 12, 13] manages the charging sequences of PHEVs in real-time. By disabling charging of vehicles at peak time periods when the demand is high, charging control helps smoothing the load of the power grid and thus reduces power losses in the distribution grid. The studies in [10, 13] indicated that the uncoordinated charging can result in 40% power losses more than coordinated charging. The work in [11] showed that the coordinated charging control can maximize the load factor and minimize the load variance other than minimizing power losses. Parks et al. [12] conducted experiments on coordinated charging control. They analyzed the real data from Xcel Energy Colorado, and concluded that there is a huge coincidence between the normal power grid peak demand hours and the time when significant charging would occur, and that a smart charging control is required to shave the charging profile of PHEVs. They also showed that coordinated charging control can significantly reduce the  Chapter 1. Introduction  4  cost of charging and also the fuel costs [12]. In order to make coordinated charging control feasible, an aggregator needs to have real-time demand-side monitoring and the ability to communicate with end-users. The National Institute of Standards and Technology (NIST) smart grid conceptual model [4] defined seven important smart grid domains: bulk generation, transmission, distribution, customers, operations, markets and service providers. Since a V2G aggregator is in the operations domain, it can interact with the distribution grid and end-users conveniently. Hence, the emergence of smart grid paves the way for the charging control of an aggregator. In addition to the technical challenge, an aggregator still has to confront market challenges. One of the market challenges is that a V2G aggregator has to find a large number of PHEVs within a local grid for contracts. According to [14, 15], however, PHEVs will be widely available across the North America. Hence, the rapid expansion of PHEVs paves the way for the commercial V2G aggregators.  1.1.1  Advantages of PHEVs  One distinctive advantage of PHEVs is to reduce carbon emission, compared with those conventional vehicles. Even when we charge the PHEVs using electricity that comes from a CO2 emitting source (e.g., a coal or gas fired power plant), the net CO2 production from a PHEV is typically one half to one third of that from a comparable conventional vehicle [16].  5  Chapter 1. Introduction Table 1.1: Estimated Future PHEV Incremental Manufacture Costs [1] Type  Year 2011  Year 2015  Year 2020  Year 2030  PHEV 40  $14,100-$18,100  $11,200-$14,200  $9,600-$12,200  $8,800-$11,000  PHEV 10  $5,500-$6,300  $4,600-$5,200  $4,100-$4,500  $3,700-$4,100  Other than the environmental benefits, the other advantage of a PHEV is its V2G capability. As a PHEV can supply energy to the grid, it could be used in backup power supply, ancillary services, peak shaving, load shifting and shaving the output of intermittent generation.  1.1.2  Disadvantages of PHEVs  Despite the promising future of PHEVs, one of their disadvantages is that PHEVs have a much higher cost than the conventional vehicles. Table 1.1 [1] shows the incremental manufacture costs of future PHEVs, compared with the conventional ones. These are only manufacturing costs that do not include engineering, marketing, or other costs. For example, PHEV-40 (a PHEV with 40-mile all-electric range) are estimated to cost $14, 100 − $18, 100 more than the conventional vehicles in manufacturing by 2011. However, as shown in Table 1.1, those additional costs of PHEVs over conventional vehicles decrease year by year. Hence, it is worthwhile to study the applications of PHEVs. Another shortcoming of PHEVs is that the penetration of PHEVs on the power grid decreases the grid capacity. As the work in [17] indicated, the current electric  Chapter 1. Introduction  6  grid infrastructure has enough spare generating capacity to support PHEV penetration levels from 30% to 70% when being considered nation-wide in the United States. Yet this penetration level is only for the whole country, and in small local power grid, the penetration of PHEVs may result in an increase in the peak demand for electric power and cause the lack of grid capacity. It will also incur a high investment cost for grid operators to upgrade the power grid capacity. Rather than upgrading the power grid, a smart coordinated charging control of PHEVs can help in alleviating the lack of grid capacity.  1.2  Related Work  In this section, we discuss the related work on V2G charging control and recent applications of PHEVs. With the deployment of smart grid technologies, the control and schedule of V2G power flows (and thus bidirectional flows) becomes possible [18]. Recently, the growing body of literature [4, 8, 19, 20, 21] paid much attention to the use of PHEVs in energy markets. The Electric Power Research Institute (EPRI) has managed and run several user cases of using electric vehicles as a distributed energy resource in smart grids [4]. Kempton et al. [8, 22] retrofitted a Toyota Scion xB and presented a demonstration project to use electric vehicles in frequency regulation. Saber et al. proposed the coordinated scheduling of PHEVs for unit commitment [19]. They used the heuristic method particle swarm optimization to solve the optimal control PHEVs. However, the solution quality is hard to guarentee as the heuristic method itself is not  Chapter 1. Introduction  7  stable. Han et al. proposed the scheduling of a V2G aggregator for frequency regulation [20]. By deciding when to charge and when to regulate, the aggregator maximizes each PHEV’s compensation from frequency regulation against its charging cost. However, their model may not be suitable to frequency regulation, as the ISO (Independent System Operator) will view all the vehicles of an aggregator as a whole and not split those vehicles into two groups (i.e., charging and regulating). Sortomme et al. proposed smart charging algorithms to obtain an optimal operation point of the aggregate load of all PHEVs [21]. When the aggregator operates below the optimal point, it is considered as regulating up, and vice versa. However, they assumed that V2G power flows and dynamic arrivals and departures of vehicles are disabled. Several charging controls have been studied in previous literature [12, 23, 24, 25]. In [23], Shao et al. examined the impact of charging PHEVs on a distribution transformer under charging scenarios of all PHEVs starts normal-charging at 6 pm, all PHEVs starts quick-charging at 6 pm, all PHEVs starts normal-charging at off-peak hours and all PHEVs starts quick-charging at off-peak hours. They showed that under the third scenarios, the transformer has the highest efficiency as the peak demand of the third scenario is the smallest. Parks et al. [12] investigated three charging controls: uncoordinated charging (begins to charging at the full charging rate once plugged in), delayed charging (delays charging until 10 pm when PHEVs are at home) and off-peak charging (only charges at off-peak hours.) They showed that the uncoordinated charging maximizes electric operation and minimizes carbon emissions, while off-peak charging fills the  Chapter 1. Introduction  8  valley of overnight demand and smooths the long-term load. Letendre and Watts [25] experimented with a fleet of 50, 000 PHEVs in the Vermont state. They implemented uncoordinated night charging, delayed and optimal night charging. In the delayed charging control, they used financial incentives such as off-peak rates to shift the peak load. In the optimal nighttime charging, the vehicles are charged only at the night hours with lowest demand. They found that the uncoordinated charging could lead to an increase in the peak demand even at a low penetration rate, while delayed and optimal charging can accommodate up to twice PHEVs than the uncoordinated charging. Tate and Boyd [26] presented a charging control problem of hybrid electric vehicles. They formulated the problem of maximizing the fuel efficiency or minimizing the fuel usage as a nonlinear convex optimization problem that is approximated by a linear program. The formulation of the linear program simplifies finding a global optimal solution. However, the minimization of fuel usage may not become an interest of both vehicle users and the aggregator. Recent studies [10, 11] presented charging control with respect to power losses of the power grid. In [10], Clement et al. proposed an optimal charging control that minimizes the power losses of the distribution grid. They used stochastic programming to model the unknown household load at each grid node. By shifting the load of PHEVs to non-peak time periods of household loads, the system peak load and the power losses are reduced. In [11], Sortomme et al. formulated the PHEV charging problem as a quadratic program. They studied the relationship among power losses, load factor and load variance. The  Chapter 1. Introduction  9  works in both [10] and [11] allow only power flows from the power grid to vehicles, while a V2G aggregator allows power flows in both directions. Moreover, Clement and Sortomme et al. considered the coordinated charging control with the sole purpose of minimizing the power losses. This will clearly not be practical in a V2G aggregator as the users will not be satisfied, given that only minimizing the power losses may lead to an increase of charging cost.  1.3  Motivations  In this thesis, we consider the coordinated charging control of a V2G aggregator. The coordinated charging control has to satisfy the customers’ needs. In other words, the users of PHEVs may require the aggregator to achieve an SOC that is higher or equal to the target SOC at departure. We propose a departure penalty function to address this issue. As of the vehicle users’ interest, the charging cost has to be minimized. To achieve this goal, a PHEV has to discharge at peak time and charge at non-peak time. However, since different PHEVs have different schedules, a portion of PHEVs may coincide with peak demand and need help from other PHEVs. Hence, the charging control of the V2G aggregator should not focus on a single PHEV. The coordinated charging aims at the cooperation among different PHEVs. It allows some PHEVs to discharge while others are in need of energy. Moreover, as a large penetration of PHEVs would impose an increased pressure on the distribution grid, the grid capacity may run out and lead to blackouts in the local power grid. The coordinated charging control should have the ability to  Chapter 1. Introduction  10  reduce congestion on transmission lines and provide transmission loading relief. Last but not least, as power supply often comes from remote places, the power losses on the distribution grid may be huge and increase in proportion to the square of the load. The coordinated charging control should aim at reducing the quadratically increasing power losses. In order to address all the above issues, we propose a weighted objective function that combines departure penalty, charging cost, and power losses [27]. The objectives of this thesis are three-fold: minimizing the total charging cost, minimizing the power losses on the distribution grid; and increasing power grid capacity at peak time through V2G power flows. On one hand, a lower cost of charging gives the owners of PHEVs an incentive to cooperate. On the other hand, uncoordinated charging of PHEVs may impose great impact on the power grid, such as the decreased voltage, increased peak load and power losses. It is the aggregator’s interest to reduce the resultant power losses, by coordinating a group of PHEVs with dynamic schedules. Considering the dynamics of user behavior, PHEVs can have different time of arrivals and departures. Given the schedules of all PHEVs at the aggregator, we optimize the weighted objective function using approximate dynamic programming.  1.4  Preliminary Dynamic Programming  In this section, we provide some background information of preliminary dynamic programming. First, we introduce a basic dynamic program. Then we explain the three curses of dimensionality. Finally, we discuss the approximate dynamic programming  Chapter 1. Introduction  11  approach that is used in the Chapter 2 of this thesis.  1.4.1  Dynamic Programming Problem  Dynamic programming is a mathmatical tool to study sequential decision problem [28, 29, 30, 31]. We present a finite-horizon dynamic programming problem in this section. We consider a system with a total of T time slots. Let Sk denote the state space, which is a given finite set. Let the xk denote the state at the kth time slot. xk is an element of the state space Sk . Similarly, let uk denote the control or action at the kth time slot. Let Ok denote the control space, which is a given finite set. uk is an element of the control space Ok . Let Dk denote the disturbance space, which is a given finite set. ωk is the random disturbance at the kth time slot and it is an element of the disturbance space Dk . The control uk is constrained to take values in a given nonempty subset ∅ = U(xk ) ⊂ Ok . And we have uk ∈ U(xk ), ∀xk ∈ Sk . The random disturbance ωk subjects to a probability distribution and is not dependent prior disturbances ωk−1 , . . . , ω1 . Let fk (k = 1, . . . , T ) denote the given transition functions. The discrete-time system is  xk+1 = fk (xk , uk , ωk ),  k = 1, . . . , T − 1,  The cost incurred at the kth time slot is denoted by gk (xk , uk , ωk ).  (1.1)  12  Chapter 1. Introduction We consider a vector π that consist of a sequence of policies  (1.2)  π = (π1 , π2 , . . . , πT ) , where πk maps state xk into control uk = πk (xk ) ∈ U(xk ).  We define the value functions Jkπ , k = 1, . . . , T , associated with a given policy π, as the sum of cost starting from xk . T  Jkπ (xk )  =E  gk (xi , ui , ωi) i=k T  =E  (1.3) gk (xi , πi (xi ), ωi) ,  i=k  The optimization objective is to obtain the closed-loop optimal policy π ∗ , a feasible policy that minimizes the sum of cost starting from the initial state x1 . π ∗ = arg min J1π (x1 ) π  (1.4)  T  = arg min E π  gk (xk , πk (xk ), ωk ) . k=1  The number of mappings from the state space to the control space is finite. And thus the enumeration of π is finite.  The Dynamic Programming Algorithm The optimal value (or cost-to-go) functions Jkπ (xk ) satisfy the following recursion of ∗  dynamic programming ∗  Jkπ (xk ) =  ∗  π min E gk (xk , uk , ωk ) + Jk+1 (fk (xk , uk , ωk )) ,  uk ∈U (xk )  k = 1, . . . , T − 1. (1.5)  13  Chapter 1. Introduction  The dynamic programming algorithm is essentially the backward induction until J1π (x1 ). First we initialize JTπ (xT ). Second we derive JTπ−1 (xT −1 ) from (1.5), so on ∗  ∗  ∗  and so forth. After T − 1 steps, we finally obtain J1π (x1 ). ∗  1.4.2  The Three Curses of Dimensionality  Let us consider the situation when xk , uk and ωk are all vectors. The explosions in the state, control and outcome space (the space for next states, explodes due to the disturbance ωk ) are refer to as the well-known the three curses of dimensionality. The key problem is that the state, control and outcome spaces are all vector spaces. It is indeed impossible to store an vector space in a modern computer. Moreover, it is even impossible to enumerate all the possible states or controls for any fixed time slot. In (1.5), the curse of the state space results in the difficulty to enumerate value ∗  functions Jkπ (xk ). The curse of the control space results in the difficulty to take the minimum operation. And the curse of the outcome space results in the difficulty to π enumerate the value functions of the next state Jk+1 (fk (xk , uk , ωk )). ∗  1.4.3  Approximate Dynamic Programming  In this thesis, we have two curses of dimensionality. One is from the state space and the other is from the control space. We use state aggregation [28] to tackle the the curse of the state space Sk , and introduce sub states [28] to tackle the the curse of the control space Ok .  14  Chapter 1. Introduction Aggregation Method  Let H(xk ) denote an aggregation function. The aggregation method basically gets the hash of the state vector xk and maps the original multi-dimensional state space to a smaller aggregated space, which can have one dimension or multi dimensions. Eqn. (1.5) can be converted into (1.6). Jkπ (H(xk )) takes average according to its ∗  original states. ∗  ∗  Jkπ (H(xk )) =  π θ(v) min E gk (v, uk , ωk ) + Jk+1 (H(fk (v, uk , ωk ))) , v |H(v)=H(xk )  uk ∈Uk (v)  k = 1, . . . , T − 1. (1.6) where θ(v) denotes the disaggregation probability.  Sub States The minimization operation in (1.5) suffers from the dimensionality of the control space. One way to solve this issue is to introduce sub states. This method basically divides the multi-dimensional control vector uk into several subsets of dimensions. Let M denote the number of sub states. We now introduce sub states x1k , x2k , . . . , xM k between xk and xk+1 . Let ujk denote the control vector of the sub-state xjk . Each sub state xjk (j = 1, . . . , M) matches with the corresponding control vector of xjk . In (1.5), we take the minimization over uk . After introducing the sub states, we instead take the minimization over u1k , and then take the minimization over u2k , so on and so forth. We obtain a solution for uik between sub state xik and xi+1 k . A solution for  Chapter 1. Introduction  15  uk can be obtained by combining the solutions u1k , u2k , . . . , uM k . The new solution for uk may not be a minimized solution in (1.5). However, we trade the quality of the solution for a feasible time complexity of the dynamic programming algorithm.  1.5  Contributions  The contributions of this thesis are as follows: • We formulate the coordinated charging control problem as a dynamic program. The model captures the tradeoff between the total cost of charging, the power losses, and the departure penalty. • We apply approximate dynamic programming [28], a state-of-the-art approach that tackles the curse of dimensionality [29], to solve the problem. • We propose a coordinated charging control algorithm at a V2G aggregator. • Simulation results show that the coordinated charging control can reduce both the total cost of charging and power losses significantly, compared with uncoordinated charging. • The coordinated charging control with bidirectional power flows incurs much less power losses, when its total cost of charging is the same as that of the counterpart scheme with unidirectional power flows (i.e., V2G flows disabled). It also incurs less charging cost than the scheme with unidirectional power flows, when the power  Chapter 1. Introduction  16  losses of both schemes are the same. Our work differs from [20] in that we consider a coordinated charging control of a V2G aggregator while Han et al. considered each PHEV individually, although both works studied the tradeoff between the cost of charging and satisfying the target SOC.. In [20], each PHEV is optimized with its own schedule and interests. In this case, the charging control in [20] is uncoordinated. The work [20] did not address the cooperation among PHEVs and not consider the impact of PHEV charging on the power grid. Prior work [10, 11] considered the coordinated charging control with the sole purpose of minimizing the power losses. Our work differs in two aspects. First, the objectives we used in this thesis are more practical since the owners of PHEVs can hardly benefit from the sole objective of minimizing the power losses. Moreover, simulation results show that the cash value of the power losses is much less than that of total charging cost. Therefore, the charging cost should not be neglected. Second, the problem formulation is more flexible. In other words, we enable bidirectional power flows between PHEVs and the power grid whereas [10, 11] disabled the V2G power flows in order to reduce the problem size and the number of constraints. We also take into account the the dynamics of PHEVs. With our formulation, PHEVs can have different time of arrivals and departures.  1.6  List of Publications  The following publication has been completed based on the work in this thesis.  Chapter 1. Introduction  17  • Jinbiao Xu and Vincent W.S. Wong, “An approximate dynamic programming approach for coordinated charging control at vehicle-to-grid aggregator,” accepted for publication in Proc. of IEEE SmartGridComm, Brussels, Belgium, October 2011.  1.7  Structure of the Thesis  The rest of this thesis is organized as follows. In Chapter 2, we present our system model and problem formulation. We then describe the approximate dynamic programming approach to solve the optimization problem. Furthermore, we present our simulation setup and comparison results. Conclusions and future work are given in Chapter 3.  18  Chapter 2 Coordinated Charging Control at V2G Aggregator In this chapter, we present the problem formulation of the coordinated charging control of a vehicle-to-grid aggregator. We also develop an approximate dynamic programming approach, which reduces the dimensionality of both the state space and control space, to obtain the optimal control sequences. Finally, we present simulation results of the coordinated charging control at an aggregator and give our analysis.  2.1  System Model  The system model of a V2G aggregator is shown in Fig. 2.1. An aggregator manages multiple participating sites, such as residential and public sites. When a PHEV arrives at a participating site, it can be plugged into the power grid. The aggregator operates all those present PHEVs simultaneously. The scheduling period is from time slot 1 to T . The length of each time slot is ∆. In this thesis, we assume that an initial schedule of each PHEV during the scheduling  Chapter 2. Coordinated Charging Control at V2G Aggregator  19  Aggregator Vehicle User Terminal  Figure 2.1: The system model of a V2G aggregator. period is known before time slot 1. Since the schedules may be volatile, the owners of PHEVs can update their schedules with the aggregator via wireless networks. If any user does not comply with the planned schedule, the aggregator can re-run the scheduling algorithm when the schedule is updated. A planned schedule of a PHEV includes the expected arrival and departing time slots, and the expected initial/target SOC at each arrival/departure time slot. Note that each PHEV can arrive at and depart from the grid multiple times. Let ai,t denote the expected initial SOC of vehicle i at arrival time slot t (valid only at the arrival time slots). Similarly, di,t denotes the expected target SOC of vehicle i at departure time slot t (valid only at the departure time slots). An example schedule of vehicle i is shown in Fig. 2.2. We assume the V2G aggregator has control of a total number of N vehicles. Let si,t denote the state of charge (SOC) of vehicle i, at the beginning of time slot t. We  Chapter 2. Coordinated Charging Control at V2G Aggregator ☎✡✝ ✟✄  ✁✂✂☎✠✝✡✟ ✝✡✟ ✟☎✡✝ ✟✄  ✁✂✂☎✠ ✁✂✂  ✁✂✂☎✆✝ ✞✟✄  ✁✂✂☎✠✝✂  ✁✂✂ Absent Present Absent   ✁✂✂✄  ✁✂✂ Present  = 40% 1  = 90% 2  20  ...  100  = 70%  = 50%  ...  150  ...  200  ...  T=240  Time Slot t  Figure 2.2: An example schedule of vehicle i. discretize si,t , i.e., si,t ∈ {0%, 1%, . . . , 100%}. Vehicle i has a maximum charging and discharging power with a magnitude of mi . Let µi denote the number of time slots that takes vehicle i to charge from empty to full capacity using full power mi . The normalized charging or discharging rate of vehicle i at time slot t is ri,t . In this thesis, ri,t can either take −1 (discharging at full power mi ), 0 (idle), or 1 (charging at full power mi ). We define the state variable as st = (s1,t , s2,t , . . . , sN,t ),  t = 1, . . . , T.  (2.1)  The control of time slot t rt = (r1,t , r2,t , . . . , rN,t ) ∈ U(st ), where the control space U(st ) is U(st ) =  (r1,t , . . . , rN,t ) | ri,t ∈ {−1, 0, 1}, ∀ i = 1, . . . , N; ri,t = 0, if vehicle i is absent at time t; 0% ≤ si,t +  ri,t 100% µi  ≤ 100%,  if vehicle i is present in the power grid at time t . where ⌊ ⌋ is the floor function.  (2.2)  Chapter 2. Coordinated Charging Control at V2G Aggregator  2.1.1  21  State Transition  Both st and rt are vectors. The state transition function f has the form st+1 = f (st , rt )  (2.3)  = (s1,t+1 , s2,t+1 , . . . , sN,t+1 ). For i = 1, . . . , N,     ai,t+1 ,     si,t+1 = si,t +        0,  if vehicle i arrives at time t + 1, ri,t 100% µi  , if vehicle i is present in power grid at time t,  (2.4)  otherwise.  When vehicle i is absent from the power grid at time slot t and does not arrive at time t + 1, we let si,t+1 = 0 by default (2.4).  2.1.2  Charging Cost ct (rt)  The total charging cost of the aggregator at time t is N  ct (rt ) = pt  ri,t mi ∆,  (2.5)  i=1  where pt is the price of electricity in time t. In this thesis, we obtain pt from the dayahead market [32]. Our formulation can be extended when pt is modeled as a random variable with certain probability distribution.  Chapter 2. Coordinated Charging Control at V2G Aggregator  2.1.3  22  Power Losses lt (rt )  The resistance of transmission lines causes the real power losses, while the reactive power losses attribute to reactive elements such as inductors. The real power losses can reduce the efficiency of transmitted energy to customers, whereas the reactive power losses may decrease the reactive power of the distribution system which is required to maintain at a certain amount for sufficient level of voltage [33]. We consider the real power losses in this thesis. Let X denote the number of transmission lines and lt (rt ) denote the total power losses of an aggregator at time slot t X  Ij2 (rt )Ωj ∆,  lt (rt ) =  (2.6)  j=1  where Ij (rt ) and Ωj are the current and the resistance of transmission line j, respectively. In this thesis, we assume the penetration level of PHEVs is 100%. In other words, we consider that all the loads of the aggregator are PHEV loads. However, other loads such as random household loads can be incorporated into our problem formulation. In that case, the problem still remains as a dynamic program. The current Ij is the sum of power flows that pass by line j. Let Φj,t denote the set of vehicles, whose current passes by line j in time slot t. Ij (rt ) = i∈Φj,t  where V is the base line voltage.  ri,t mi , V  (2.7)  Chapter 2. Coordinated Charging Control at V2G Aggregator  2.1.4  23  Departure Penalty gt (st )  When vehicle i departs at time slot t with SOC si,t , a penalty gi,t (si,t ) will occur. We assume that the departure penalty is greater than zero if and only if the departure SOC si,t does not reach the target SOC di,t . There are a variety of penalty functions gi,t (si,t ) we can use. In this thesis, we choose the quadratic function     (di,t − si,t )2 , if vehicle i departs at time t, and si,t < di,t , gi,t (si,t ) =    0, otherwise.  (2.8)  The total departure penalty of all PHEVs at time slot t is N  gt (st ) =  gi,t (si,t ) .  (2.9)  i=1  2.1.5  Problem Formulation  The objective function ot (st , rt ) in time slot t is  ot (st , rt ) = ct (rt ) + λ1 lt (rt ) + λ2 gt (st ),  (2.10)  where λ1 and λ2 are two positive control variables. λ1 represents how much power losses (in W-h) are worth $1 charging cost. λ2 represents how much departure penalty is worth $1 charging cost. In real applications, the value of the control variables λ1 and λ2 will depend on the specific application requirement (e.g., if the first priority is minimizing the power losses, a large λ1 should be appropriate.) We consider the class of policies (also referred to as control laws [28]) that consist of  24  Chapter 2. Coordinated Charging Control at V2G Aggregator a sequence of policies π = (π1 , π2 , . . . , πT ) ,  (2.11)  where πt maps state st into control rt = πt (st ). We define the value functions Jtπ , t = 1, . . . , T , associated with a given policy π, as the sum of cost starting from st . T  Jtπ (st ) =  ok (sk , rk ) k=t T  =  (2.12) ok (sk , πk (sk )) .  k=t  The optimization objective is to obtain the closed-loop optimal policy π ∗ , a feasible policy that minimizes the sum of cost starting from the initial state s1 . π ∗ = arg min J1π (s1 ) π  (2.13)  T  = arg min π  2.2  ok (sk , πk (sk )) . k=1  An Approximate Dynamic Program Approach  The main issue of solving problem (2.13) is that both st and rt are vectors. Both the state and control space will increase exponentially as the number of vehicles increases. It is also computationally intractable to enumerate all the possible states and controls at any one time slot. In this section, we use the approximate dynamic programming [28] to obtain a sub-optimal solution of problem (2.13). State aggregation is used to reduce the state space, whereas sub-states are introduced to reduce the control space.  Chapter 2. Coordinated Charging Control at V2G Aggregator  2.2.1  25  State Aggregation  We use an aggregate state space to approximate the value function of the original problem. In this thesis, the aggregation function is N  H(st ) =  si,t .  (2.14)  i=1 ∗ Let Jˆtπ (H(st )) (t = 1, . . . , T ) denote the optimal value functions of the aggregate ∗ problem. We can obtain Jˆtπ from the unique solution of Bellman’s equation [29].  ∗ Jˆtπ (H(st )) =  π∗ δs ot (s, πt∗ (s)) + Jˆt+1 (H(f (s, πt∗ (s)))) ,  (2.15)  s | H(s)=H(st )  where δs is the disaggregation probability [28]. δs are the same for all s such that H(s) = H(st ). And  δs = 1. s | H(s)=H(st )  Since it takes exponential time to enumerate ∀s : H(s) = H(st ), we can only approx∗ imate Jˆtπ (H(st )) using Monte Carlo simulations. As the number of samples that we ∗ ∗ generated increases, the approximation for Jˆtπ (H(st )), denoted by J˜tπ (H(st )), will be  close to optimal. In this thesis, we solve the optimal policy π ∗ by approximating the corresponding ∗ ∗ ∗ value functions Jˆtπ by J˜tπ , t = 1, . . . , T . Once we obtain J˜tπ , the optimal policy π ∗ can  be computed as follows. For t = 1, . . . , T , πt∗ (st ) = arg min  rt ∈U (st )  π∗ ot st , rt + J˜t+1 H(f (st , rt ))  .  (2.16)  Note that in (2.16), it takes exponential time to enumerate all the feasible control variables in U(st ) since rt is a vector. The vector space of rt results in the curse of dimen-  Chapter 2. Coordinated Charging Control at V2G Aggregator  26  sionality [29]. To tackle this issue, we use sub-states [28] to reduce the dimensionality of the control space.  2.2.2  Sub-States  We first define the controllable set of time slot t, Ψt as Ψt = {ri,t | vehicle i (1 ≤ i ≤ N) is present in the power grid at time t}.  (2.17)  For all vehicles that are absent at time slot t (ri,t ∈ / Ψt ), we have ri,t = 0 from (2.2). We partition Ψt equally into the sequence of M groups of control variables Ψ1t , . . . , ΨM t (Ψt =  M j j=1 Ψt ).  Each group Ψjt (j = 1, . . . , M − 1) contains the same number of control  variables except that the last group ΨM t may possibly contain fewer control variables (e.g., Ψ1t = {r1,t , . . . , r5,t }, . . . , ΨM t = {rN −4,t , . . . , rN,t }). In this thesis, we let each group consist of no more than ν = 5 control variables. Thus, it takes at most O(3ν ) to enumerate all feasible controls of each group Ψjt , j = 1, . . . , M. j We now introduce sub-states s1t , s2t , . . . , sM t between st and st+1 . Each sub-state st  (j = 1, . . . , M) matches with the corresponding group of control variables Ψjt . Let rjt denote the control vector of the sub-state sjt . Since rjt is associated with a group of control variables Ψjt , we have rjt ∈ U j (st ), where the control space U j (st ) is U j (st ) =  j j j (r1,t , . . . , rN,t ) | ri,t ∈ {−1, 0, 1}, ∀ i = 1, . . . , N; j j ri,t = 0, if ri,t ∈ / Ψjt ;  0% ≤ si,t +  j ri,t 100% µi  j ≤ 100%, if ri,t ∈ Ψjt .  (2.18)  Chapter 2. Coordinated Charging Control at V2G Aggregator  27  ...  Figure 2.3: The transition diagram of sub-states. Sub-State Transition The transition of sub-states is sj+1 = f sjt , rjt , t  j = 1, . . . , M − 1,  (2.19)  and M st+1 = f sM . t , rt  (2.20)  s1t = s11,t , s12,t , . . . , s1N,t .  (2.21)  For s1t , we have  For i = 1, . . . , N,  s1i,t      ai,t+1 , if vehicle i arrives at time t + 1,     = si,t , if vehicle i is present at time t,        0, otherwise.  (2.22)  The transition diagram of sub-states is shown in Fig. 2.3. The dashed line is the regular state transition without sub states, while the solid lines are equivalent state transition with sub states. We obtain a solution for rit between sub state sit and si+1 t . We  Chapter 2. Coordinated Charging Control at V2G Aggregator  28  simply take the minimization over rit , given that r1t , . . . , ri−1 are known since we already t obtained the solutions. When we take the minimization over rit , we are in the sub state M sit . Therefore, ri+1 t , . . . , rt can be ignored (i.e., we simply let them be zero) while we  minimize rit . After M transitions among sub states, a solution for rt is obtained, which simply combines r1t , r2t , . . . , rM t . Now, we can obtain πt∗ (st ) from (2.16) by decomposing it into a sequence of optimization problems after introducing sub-states. For t = 1, . . . , T and j = 1, . . . , M, πt∗j (sjt ) = arg min  rjt ∈U j (st )  π∗ ot sjt , rjt + J˜t+1 H(f (sjt , rjt ))  ,  (2.23)  where πt∗j (sjt ) contains a valid solution for all ri,t ∈ Ψjt . Although the combination of r1t , r2t , . . . , rM t may not be the optimal solution for rt , we can, by this way, trade the quality of solution for the time complexity of the approximate approach.  2.2.3  Approximate Policy Iteration Using Monte Carlo Simulation  In this section, we present a variant of approximate policy iteration [28] based on Monte Carlo simulation. A complete policy iteration includes policy evaluation (steps 1−20) and policy improvement (steps 21 − 22) as shown in Algorithm 1. Suppose that the current policy is π. We evaluate and approximate the value functions of π by J˜tk , k = 1, . . . , K, t = 1, . . . , T , using K evaluation iterations.  Chapter 2. Coordinated Charging Control at V2G Aggregator  29  In steps 5−12, we explore some random control with probability of ǫ, where 0 < ǫ < 1, otherwise we obtain the control rt driven by current policy π. In steps 16 − 19, we update the value function approximation for the aggregate state space J˜tk (H(st )) with the accumulated path cost Z. Here, γk is the step size for the kth iteration. After we run the evaluation steps for the current policy π by K iterations, we obtain J˜tK , t = 1, . . . , T . Based on these approximated value functions, we generate an improved policy π ¯ , which will be used in the next policy iteration, using the decomposition of control variables (2.23) with sub-states in step 22.  Chapter 2. Coordinated Charging Control at V2G Aggregator Algorithm 1 Approximate policy iteration using Monte Carlo simulation 1: Initialize J˜t0 (t = 1, . . . , T ) 2: for k = 1 to K do {Evaluate current policy π} 3:  s1 ← initial state  4:  for t = 1 to T do  5: 6: 7:  if rand(0, 1) < ǫ then Generate a random exploratory control rt else {Obtain control rt driven by current policy π}  8:  rt ← 0  9:  for j = 1 to M do π rt ← rt + arg minrj ∈U j (st ) ot sjt , rjt + J˜t+1 H f sjt , rjt  10: 11:  t  end for  12:  end if  13:  st+1 ← f (st , rt )  14:  end for  15:  Z←0  16:  for t = T downto 1 do  17:  Accumulate the path cost Z ← Z + ot (st , rt )  18:  Update the value functions J˜tk (H(st )) ← (1 − γk )J˜tk−1 (H(st )) + γk Z J˜tk (x) ← J˜tk−1 (x)  19:  ∀x : x = H(st )  end for  20: end for 21: J˜tπ¯ ← J˜tK (t = 1, . . . , T ) 22: Generate an improved policy π ¯ = (¯ π1 , . . . , π ¯T ) by decomposing (2.16) into (2.23)  30  Chapter 2. Coordinated Charging Control at V2G Aggregator  31  5 4' 4 3'  138 kV  12.47 kV  1  1'  2  2'  3  Figure 2.4: One-line diagram of the 5-bus radial primary distribution system (base line voltage at 12.47 kV).  2.3  Performance Evaluation  In this section, we provide numerical results of the coordinated charging control at an aggregator with N = 100 PHEVs for the duration of 24 hours. We also compare with the uncoordinated charging control and the unidirectional-power-flow scheme.  2.3.1  Simulation Setup  We consider an IEEE 5-bus radial primary distribution system in Fig. 2.4. Here, only buses 2 to 5 are load buses. We assign  N 4  = 25 PHEVs equally to each load bus. That is,  vehicles 1-25 are parked at bus 2, vehicles 26-50 are parked at bus 3, so on and so forth. There are four transmission lines (1′ , 2′ , 3′ , 4′ ). We assume the transmission lines are aluminum conductor, steel reinforced (ACSR) 26/7 with 1-mile length. The resistance of each transmission line is 0.1859 Ohm. The base line voltage V = 12.47 kV. Since a PHEV may arrive or depart at any minute, we select a small granularity ∆  Chapter 2. Coordinated Charging Control at V2G Aggregator  32  180  160  Price ($ / MWh)  140  120  100  80  60  40  20  2  4  6  8  10  12  14  16  18  20  22  24  Hour  Figure 2.5: The day-ahead locational marginal price used in simulation. which is 6 minutes. The scheduling time period is from 12 PM to the next 12 PM. In other words, T = 240. We employ the day-ahead locational marginal price (LMP) for pt from California ISO [32]. The price information is shown in Fig. 2.5. We assume that all N vehicles have a maximum power mi = 1 kW, and their battery capacity is 10 kWh. Hence, µi = 100 for i = 1, . . . , N. In Algorithm 1, we choose K = 109 . We use a step size of γk =  1 k  [28]. Table 2.1 gives a summary of simulation parameters.  We generate the arrivals and departures of each vehicle (i = 1, . . . , N) as follows. Each vehicle has two disjoint available periods that it is present in the power grid. The length of each available period is randomly distributed among {40, . . . , 80} time slots, namely 4 hours to 8 hours. The time slots of arrival/departure and the corresponding initial/target SOC ai,t /di,t are generated at random. And we also guarantee that di,t is  Chapter 2. Coordinated Charging Control at V2G Aggregator  33  Table 2.1: List of Simulation Parameters Symbol  Definition  Value  N  The number of vehicles.  100  Ωj  The resistant of transmission line j.  V  The base line voltage.  12.47 kV  ∆  The length of each time slot.  6 minutes  T  The number of time slots.  mi  The maximum power of vehicle i.  µi  The number of time slots that takes vehicle i  0.1859 Ohm  240 1 kW  from empty battery capacity to full capacity.  100  K  The number of evaluation iteration.  109  γk  The step size of the kth iteration.  1 k  possible to reached at departure. Fig. 2.6 shows the distribution of available time slots of a total number of N = 100 PHEVs. In Fig. 2.6, each line segment of vehicle i represents a time duration when vehicle i is plugged in at the grid. For instance, the first line segment of vehicle i = 1 represents that vehicle i = 1 arrives in the grid at time slot 42 and departs at time slot 109.  Chapter 2. Coordinated Charging Control at V2G Aggregator  34  Figure 2.6: The distribution of available time slots of all PHEVs.  2.3.2  Simulation Results  In this section, we provide comparison results of the bidirectional-flow coordinated charging control and other counterpart schemes.  Coordinated v.s. Uncoordinated To show the advantages of coordinated charging control, we first compare our proposed coordinated charging control with the uncoordinated charging control scheme. The uncoordinated charging control simply let each vehicle start charging as soon as it is plugged into the grid. Each vehicle keeps charging until either the target SOC is reached or it departs from the power grid. Uncoordinated charging control is a myopic scheme that does not consider the aggregate load of all PHEVs. However, a distinct advantage is that it minimizes the departure penalty gt . In our simulations, the total departure penalty of  Chapter 2. Coordinated Charging Control at V2G Aggregator  35  45  Uncoordinated charging control Coordinated charging control, λ = 0.05 2 Coordinated charging control, λ2 = 0.005 Coordinated charging control, λ = 0.001 2  Total Cost of Charging ($)  40  35  30  25  20  15  10 0.01 0.05  0.2  0.3  0.4  0.5  λ  0.7  1  1  Figure 2.7: The total charging cost versus the importance weight λ1 . the uncoordinated scheme is always zero. Fig. 2.7 shows the total cost of charging of all PHEVs. The accumulated power losses of the power distribution grid are shown in Fig. 2.8. Results show that increasing the departure penalty will essentially reduce the total charging cost and accumulated power losses. However, in practice, the charging requirement of PHEVs (i.e., meeting the target SOC) should be given a high priority. In this thesis, we let λ2 = 0.001, 0.005 and 0.05. When λ2 = 0.05, the total departure penalty of the coordinated scheme is always zero, implying that all target SOCs have been reached (Fig. 2.9). Although a larger departure penalty implies further away from the target SOCs, in the worst case when λ2 = 0.001, the average gap between the target SOC and the actual SOC, is at most 5%. When  Chapter 2. Coordinated Charging Control at V2G Aggregator  36  0.04  Uncoordinated charging control Coordinated charging control, λ = 0.05 2 Coordinated charging control, λ = 0.005 2 Coordinated charging control, λ2 = 0.001  Total Power Losses (k−Wh)  0.035  0.03  0.025  0.02  0.015  0.01  0.005 0.01 0.05  0.2  0.3  0.4  0.5  λ  0.7  1  1  Figure 2.8: The total power losses versus the importance weight λ1 . λ2 = 0.005, the average gap is at most 1%. We also investigate the largest gap between the target SOC and the actual SOC. When λ2 = 0.001, the largest gap is 8%. When λ2 = 0.005, the largest gap is only 2% Coordinated charging control has basically two advantages. First, it is price-aware. Fig. 2.7 shows that the coordinated scheme can reduce as much as 70% total cost of charging compared with the uncoordinated scheme when λ1 = 0.01 and λ2 = 0.001. Second, it enables the cooperation among different PHEVs, in order to smooth the aggregate load over time and thus reduce the power losses. Fig. 2.8 shows that the coordinated charging control can reduce as much as 77% accumulated power losses when λ1 = 1 and λ2 = 0.001. Also, results in Fig. 2.8 show that the total power losses decrease as λ1  Chapter 2. Coordinated Charging Control at V2G Aggregator  37  8000  7000  Total Departure Penalty  6000  5000  4000  3000  Uncoordinated charging control Coordinated charging control, λ2 = 0.05 Coordinated charging control, λ = 0.005 2 Coordinated charging control, λ = 0.001 2  2000  1000  0 0.01 0.05  0.2  0.3  0.4  0.5  λ  0.7  1  1  Figure 2.9: The total departure penalty versus the importance weight λ1 . increases. An interesting fact is that in Fig. 2.7, the lowest cost of charging is $13 when λ1 = 0.01 and λ2 = 0.001, whereas the highest power losses of the coordinated scheme is 0.035 kWh in Fig. 2.8. Since the price of electricity that we used for the simulations is always less than $0.2/kWh, the cash value of the accumulated power losses is smaller than the cost of charging. The owners of PHEVs may be reluctant to sacrifice the charging cost to reduce the power losses and thus may prefer a smaller λ1 . On the other hand, the power losses are concern to the V2G aggregator. As more power losses impose greater impact to the power grid, the aggregator may prefer a larger λ1 in the interest of reducing the power losses.  Chapter 2. Coordinated Charging Control at V2G Aggregator  38  90  The Sum of Objective Functions  80  Uncoordinated charging control Coordinated charging control, λ2 = 0.05 Coordinated charging control, λ2 = 0.005 Coordinated charging control, λ = 0.001 2  70  60  50  40  30  20  10 0.01 0.05  0.2  0.3  0.4  0.5  0.7  λ  1  1  Figure 2.10: The sum of objective functions from time slot 1 to T versus the importance weight λ1 . Fig. 2.10 shows the sum of objective functions (i.e., eqn. (2.12)), starting from time 1 with initial state s1 . The coordinated charging control outperforms the uncoordinated one under all scenarios we use. The reason is that the coordinated charging control reduces both the total cost of charging  T t=1 ct (rt )  while keeping the weighted departure penalty λ2  and total power losses  T t=1  T t=1 lt (rt ),  gt (st ) relatively low.  Bidirectional Flows v.s. Unidirectional Flows To show the benefits of enabling V2G power flows, we then compare the coordinated scheme with bidirectional power flows with the coordinated scheme with unidirectional flows (i.e., ri,t ∈ {0, 1}). For a fair comparison, we choose the total departure penalty of  Chapter 2. Coordinated Charging Control at V2G Aggregator  λ1 = 0.5  39  λ1 = 0  Total Power Losses (k−Wh)  0.017  0.016  0.015  0.014  0.013  λ = 5.5 1 0.012  0.011  Coordinated charging control with unidirectional flows Coordinated charging control with bidirectional flows 0.01 27  28  29  30  31  32  33  λ = 20 1 34  35  36  Total Cost of Charging ($)  Figure 2.11: Total cost of charging and power losses varying with different λ1 (λ2 = 10). both schemes to be zero by setting the importance weight λ2 at a large value (λ2 = 10). In other words, all the target SOCs in both schemes are reached. Fig. 2.11 shows the total cost of charging and the power losses of both schemes varying with different λ1 . The way we choose λ1 is to make the ranges of power losses of both schemes corresponding to each other. And the minimum λ1 for the unidirectional-flow scheme is 0 (i.e., only minimize the charging cost), while the λ1 for the bidirectional scheme can be set further smaller than 0.5. The results show that coordinated charging control with bidirectional power flows incurs much lower power losses, when its total cost of charging is the same as that of the counterpart scheme with unidirectional power flows (i.e., V2G flows disabled). On the other hand, it also incurs less charging cost than the scheme with unidirectional power  Chapter 2. Coordinated Charging Control at V2G Aggregator  40  flows, when the aggregated power losses of both schemes are the same. Note that the coordinated scheme with unidirectional flows has a minimum cost of charging of $31.5 when λ1 = 0, while the one with bidirectional flows has a total cost of charging for only $27 with the same amount of power losses where λ1 = 0.5. Another interesting fact is that, as the power losses of both schemes become larger and larger, the gap between the charging cost of both schemes are increasing. The reason can be found in the X-axis of Fig. 2.11: enabling V2G flows provides the possibility of reduce more charging cost. In Fig. 2.11, the charging cost of the bidirectional-flow scheme has decreased $11 (38 − 27) while the charging cost of the unidirectional one has decreased only $6.5.  41  Chapter 3 Conclusions and Future Work In this chapter, we conclude this thesis by summarizing the research results and contributions. We also suggest directions for future work.  3.1  Conclusions  In this thesis, we developed the coordinated charging control of a V2G aggregator, which reduces both the total cost of charging and the aggregated power losses. Given the day-ahead electricity prices and schedules of PHEVs, we formulated the problem as a dynamic program. We solved the problem using approximate dynamic programming, which reduces the dimensionality of both the state and control spaces. Simulation results of an aggregator with 100 PHEVs are presented to validate the benefits of the coordinated charging control. Simulation results show that coordinated charging control can reduce both the total charging cost and the aggregated power losses significantly, compared with the uncoordinated control where every vehicle starts charging as soon as it is plugged in. We also showed that the charging control with bidirectional power flows outperforms remarkably the one with unidirectional flows.  42  Chapter 3. Conclusions and Future Work  3.2  Future Work  In this section, we suggest two future extensions to improve the quality of our present work.  3.2.1  Incorporating Renewable Energy Sources and Battery Storage  Since promoting green environment is an inherent property of smart grid, it would be interesting to incorporate renewable energy sources and battery storage into the problem formulation. Renewable energy sources [34, 35, 36] such as wind turbines, photovoltaic (PV), are easy to deploy in the smart grid. Other than that, battery storage [37, 38, 39, 40] has been studied to integrate into the power grid. Battery storage is a group of electrochemical cells and often comes in the form of battery arrays. The battery storage may serve as an electric buffer. Battery storage would usually have high initial cost, but can be recharged very cheaply and used many times.  Renewable Energy Sources We can assume that the aggregator owns a total number of H renewable energy sources. The output power of each renewable energy source is denoted by ωi,t (i = 1, . . . , H). Let ωt denote a vector of output power  ωt = (ω1,t , ω2,t , . . . , ωH,t ) .  (3.1)  Chapter 3. Conclusions and Future Work  43  Since most renewable energy sources are predictable power supply such as hydro power, wind power and photovoltaic, we assume that each random variable ωi,t has a given distribution.  Battery Storage A battery storage can be regarded as a special PHEV that is always online (i.e., present in the grid). There is no target SOC requirement for battery storage. We assume that the aggregator owns a total number of G battery storage. Let bi,t denote the state of charge of storage i at the beginning of time slot t. We define the state vector bt as  bt = (b1,t , . . . , bG,t ).  (3.2)  Correspondingly, let ui,t denote the control variable. We then define the control vector ut as ut = (u1,t , . . . , uG,t ).  (3.3)  The domain of the control variable ui,t is  ui,t ∈ {−1, 0, 1}.  (3.4)  Storage i has a maximum charging and discharging power with a magnitude of m ˆ i. Let µ ˆi denote the number of time slots that takes storage i to charge from empty to full capacity.  Chapter 3. Conclusions and Future Work  44  Hence, we have 0% ≤ bi,t +  ui,t 100% ≤ 100% µ ˆi  (3.5)  Let ˆ st denote the new state variable. We have (3.6)  ˆ st = (st , bt ) Similarly, let ˆ rt denote the new control variable. And we also have  (3.7)  ˆ rt = (rt , ut ) The new state transition function fˆ has the form ˆ st+1 = fˆ(ˆ st , ˆ rt )  (3.8) = (f (st , rt ), f (bt , ut )) Let us assume that storage i has an initial battery capacity βi at the beginning of time slot 1. For i = 1, . . . , G, bi,t =      βi ,     bi,t−1 +  t = 1, ui,t−1 100% µ ˆi  (3.9)  , otherwise.  In (3.9), the state transition of battery storage i is given. At the beginning of time slot 1, bi,1 is initialized as βi (a known constant). Since the battery storage is a special PHEV that is always online, the following states can be calculated by recursions.  State Reduction for Battery Storage Recall that the time complexity of dynamic programming algorithms depends on the number of variables in the state vector ˆ st = [st bt ] (i.e., N + G). Thus it is absolutely  Chapter 3. Conclusions and Future Work  45  8  7  6  1  2  3  4  5  Figure 3.1: One-line diagram of the IEEE 8-bus radial primary distribution grid. necessary to reduce the state vector whenever possible. Fortunately, we know that all the battery storage arrive at time slot 1 and never depart. There is no requirement of departure SOC for all battery storage. Hence, we can reduce the bt from G dimensions to reduced dimensions. The reason why we can preform state reduction for the battery storage lies in the fact that a number of battery storage have the same location in the power distribution grid and schedule. Moreover, unlike electric vehicles, battery storage does not have target SOC requirement. The way we reduce bt is simply that we view all the battery storage within the same bus of the distribution grid as a whole. For example, Fig. 3.1 gives an example 8-bus distribution grid. In Fig. 3.1, bus 8 may connect with four buildings. And we can combine all the battery storage within those four buildings. Assume that the maximum charging and discharging power of the original four battery storage are  Chapter 3. Conclusions and Future Work  46  m ˆ 1, . . . , m ˆ 4 and the number of time slots that takes storage 1 − 4 to charge from empty to full capacity are µ ˆ1 , . . . , µ ˆ4 and the initial battery SOC are β1 , . . . , β4 at the beginning of time slot 1. The combined battery storage for those battery storage within bus 8 has a maximum charging and discharging power of min4i=1 {m ˆ i }, and the number of time slots that takes it to charge from empty to full capacity is battery storage is  4 ˆ iµ ˆ i βi i=1 m 4 m ˆ ˆi iµ i=1  4 ˆ iµ ˆi i=1 m min4i=1 {m ˆ i}  . The initial SOC of the combined  . Similarly, we can preform state reduction for all other buses  in Fig. 3.1. Although we can obtain performance gain for the algorithm through state reduction, the shortcoming of this approach is that only one battery storage can be active at one time. Since we take all battery storage as a whole, if one of the battery is charged to full state or discharged to empty state, the other will continue the former’s operation.  The New Problem Formulation The total charging cost of the aggregator at time t is N  ct (ˆ r t , ω t ) = pt ∆  G  ui,t m ˆi −  ri,t mi + i=1  H  i=1  ωi,t .  (3.10)  i=1  The total power losses of an aggregator at time slot t is X  Ij2 (ˆ rt , ωt )Ωj ∆.  lt (ˆ rt , ωt ) =  (3.11)  j=1  The current Ij is the quotient of the sum of power and the voltage. To obtain the total power across transmission line j, we let Φj,t denote the set of vehicles, which constitute  47  Chapter 3. Conclusions and Future Work  a portion of the total power across transmission line j. Let φj,t denote the set of battery storage, which constitute a portion of the total power across transmission line j. Let ϕj,t denote the set of renewable energy sources, which constitute a portion of the total power across transmission line j.  Ij (ˆ rt , ωt ) =  i∈Φj,t  ri,t mi +  i∈φj,t  ui,t m ˆi −  i∈ϕj,t  ωi,t  V  .  (3.12)  Since there is no requirement of departure SOC for all battery storage, the total departure penalty at time slot t is N  gt (ˆ st ) =  gi,t (si,t ) .  (3.13)  i=1  The objective function ot (ˆ st , ˆ rt , ωt ) in time slot t is  ot (ˆ st , ˆ rt , ωt ) = ct (ˆ rt , ωt ) + λ1 lt (ˆ rt , ωt ) + λ2 gt (ˆ st ).  (3.14)  From the objective function in (3.14), we can conclude that the scheduling problem of the aggregator that incorporates with renewable energy sources and battery storage, remains as a dynamic program. The difference between the new objective function (3.14) and the original one in (2.10) is that the new one includes random variables ωt . Furthermore, the new problem can be readily solved using approximate dynamic programming. Algorithm 2 shows the updated approximate policy iteration using Monte Carlo simulation. Steps 1 − 21 is the policy evaluation for the current policy, and steps 22 − 23 obtains an improved policy. The only change is the step 4, where we generate random samples for ω1 , . . . , ωT from their given distribution.  Chapter 3. Conclusions and Future Work Algorithm 2 Updated approximate policy iteration using Monte Carlo simulation 1: Initialize J˜t0 (t = 1, . . . , T ) 2: for k = 1 to K do {Evaluate current policy π} 3:  ˆ s1 ← initial state  4:  Generate samples of random variables ω1 , . . . , ωT  5:  for t = 1 to T do  6: 7: 8: 9: 10:  if rand(0, 1) < ǫ then Generate a random exploratory control ˆ rt else {Obtain control ˆ rt driven by current policy π} ˆ rt ← 0 for j = 1 to M do π ˆ rt ← ˆ rt + arg minˆrj ∈U j (ˆst ) ot ˆ rjt , ωt + J˜t+1 rjt sjt , ˆ H fˆ ˆ sjt , ˆ  11: 12:  t  end for  13:  end if  14:  ˆ st+1 ← fˆ(ˆ st , ˆ rt )  15:  end for  16:  Z←0  17:  for t = T downto 1 do  18:  Accumulate the path cost Z ← Z + ot (ˆ st , ˆ rt , ωt )  19:  Update the value functions J˜tk (H(ˆ st )) ← (1 − γk )J˜tk−1 (H(ˆ st )) + γk Z J˜tk (x) ← J˜tk−1 (x)  20:  ∀x : x = H(ˆ st )  end for  21: end for 22: J˜tπ¯ ← J˜tK (t = 1, . . . , T ) 23: Generate an improved policy π ¯ = (¯ π1 , . . . , π ¯T )  48  Chapter 3. Conclusions and Future Work  3.2.2  49  Studying the Effects of Unanticipated Schedules  Throughout this thesis, we have assumed that the initial schedules of PHEVs are known before the scheduling periods. Recall that a planned schedule of a PHEV includes the arrival and departure time, the initial and target SOC. In practice, people tend to change their travel schedules based on last-minute decisions. Hence the expected arrival and departure time may change. Moreover, the expected initial SOC may also change due to different driving styles and various climate conditions. For example, a college student may be expected to drive home after school at 4 pm. However, he may change his mind at 3 pm and decide to drive to the football court first. Then he will drive home after a 2-hour football match with friends. The student can notify the aggregator his change of schedule via his mobile phone. The aggregator will notice the change of the following time of arrival and also calculate the change of initial SOC from the driving distances between school and the football court, home and the football court. Although the aggregator can re-run the scheduling algorithm once any information of the schedules has changed, the effects of unanticipated schedules on performance are not within the scope of this thesis. One important thing to address is that, given the fact that the uncertainty cannot be predicted (i.e., we do not know the probability of when and how any schedule will change, and the probability distribution of how one schedule changes to another), the best effort we can try is to re-schedule given the maximum information. Hence, the further improvement of this study should focus on modeling the when and how the schedules change, as we cannot derive an adaptive algorithm without the knowledge of  Chapter 3. Conclusions and Future Work  50  the uncertainty we want to adapt to. For future work, it would be interesting to study the effects of unanticipated schedules of PHEVs. First, investigate the performance gap between the case where the actual arrival and departure times are known beforehand and the case where a portion of the arrival and departure times are changing during the scheduling period. Second, study the performance gap between the case where the actual initial SOCs are known beforehand and the case where some of the initial SOCs are changing during the scheduling period.  51  Bibliography [1] “Transitions to alternative transportation technologies: Plug-in hybrid electric vehicles,” National Research Council, The National Academies Press, Washington, D.C., 2010. [Online]. Available: http://books.nap.edu/openbook.php?record id= 12826&page=2 [2] B. Russell, “Communication alternatives for distribution metering and load management,” IEEE Trans. on Power Apparatus and Systems, vol. 99, no. 4, pp. 1448–1455, July 1980. [3] N. Miura, H. Sato, H. Narita, and M. Takaki, “Automatic meter-reading system by power line carrier communications,” IEE Proceedings, Generation, Transmission and Distribution, vol. 137, no. 1, pp. 25–31, Jan. 1990. [4] D. V. Dollen, “Report to NIST on the smart grid interoperability standards roadmap,” Electric Power Research Institute, June 2006. [Online]. Available: http:// www.nist.gov/smartgrid/upload/InterimSmartGridRoadmapNISTRestructure.pdf [5] A. H. Mohsenian-Rad, V. W. S. Wong, J. Jatskevich, R. Schober, and A. LeonGarcia, “Autonomous demand side management based on game-theoretic energy  Bibliography  52  consumption scheduling for the future smart grid,” IEEE Trans. on Smart Grid, vol. 1, no. 3, pp. 320–331, Dec. 2010. [6] P. Samadi, H. Mohsenian-Rad, R. Schober, V. W. S. Wong, and J. Jatskevich, “Optimal real-time pricing algorithm based on utility maximization for smart grid,” in Proc. of IEEE SmartGridComm, Gaithersburg, MD, Oct. 2010. [7] J. Lopes, F. Soares, and P. Almeida, “Integration of electric vehicles in the electric power system,” Proceedings of the IEEE, vol. 99, no. 1, pp. 168–183, Jan. 2011. [8] W. Kempton, V. Udo, K. Huber, K. Komara, S. Letendre, S. Baker, D. Brunner, and N. Pearre, “A test of vehicle-to-grid (V2G) for energy storage and frequency regulation in the PJM system,” University of Delaware, Pepco Holdings Inc., PJM Interconnect and Green Mountain College, Jan. 2009. [9] B. Bhangu, P. Bentley, D. Stone, and C. Bingham, “Nonlinear observers for predicting state-of-charge and state-of-health of lead-acid batteries for hybrid-electric vehicles,” IEEE Trans. on Vehicular Technology, vol. 54, no. 3, pp. 783–794, May 2005. [10] K. Clement, E. Haesen, and J. Driesen, “The impact of charging plug-in hybrid electric vehicles on a residential distribution grid,” IEEE Trans. on Power Systems, vol. 25, no. 1, pp. 371–380, Feb. 2010.  53  Bibliography  [11] E. Sortomme, M. Hindi, S. MacPherson, and S. Venkata, “Coordinated charging of plug-in hybrid electric vehicles to minimize distribution system losses,” IEEE Trans. on Smart Grid, vol. 2, no. 1, pp. 198–205, Mar. 2011. [12] K. Parks, P. Denholm, and T. Markel, “Costs and emissions associated with plugin hybrid electric vehicle charging in the Xcel energy Colorado service territory,” Technical report, National Renewable Energy Laboratory (NREL), May 2007. [13] S. Acha, T. C. Green, and N. Shah, “Effects of optimised plug-in hybrid vehicle charging strategies on electric distribution network losses,” in Proc. IEEE PES Transmission and Distribution Conf. and Exposition, New Orleans, LA, Apr. 2010. [14] J. Axsen and K. Kurani, “The early U.S. market for PHEVs:  Anticipating  consumer awareness, recharge potential, design priorities and energy impacts,” Institute of Transportation Studies, University of California, Davis, July 2008. [Online]. Available: http://pubs.its.ucdavis.edu/download pdf.php?id=1191 [15] “Global market analysis of plug in hybrid electric vehicles,” Frost and Sullivan, Dec. 2007. [Online]. Available: http://www.frost.com/prod/servlet/report-toc.pag? repid=M12D-01-00-00-00 [16] C. Gribben,  “Debunking the myth of EVs and smokestacks,”  Electric  Vehicle Association of Greater Washington, D.C, 2007. [Online]. Available: http://www.electroauto.com/info/pollmyth.shtml  Bibliography  54  [17] S. W. Hadley, “Impact of plug-in hybrid vehicles on the electric grid,” Oak Ridge National Laboratory, Tech. Rep., Oct. 2006. [Online]. Available: http://www.ornl.net/info/ornlreview/v40 2 07/2007 plug-in paper.pdf [18] A. Ipakchi and F. Albuyeh, “Grid of the future,” IEEE Power and Energy Magazine, vol. 7, no. 2, pp. 52–62, Mar. 2009. [19] A. Saber and G. Venayagamoorthy, “Optimization of vehicle-to-grid scheduling in constrained parking lots,” in Proc. IEEE PES Power and Energy Society General Meeting, New Orleans, LA, July 2009. [20] S. Han, S. Han, and K. Sezaki, “Development of an optimal vehicle-to-grid aggregator for frequency regulation,” IEEE Trans. on Smart Grid, vol. 1, no. 1, pp. 65–72, June 2010. [21] E. Sortomme and M. El-Sharkawi, “Optimal charging strategies for unidirectional vehicle-to-grid,” IEEE Trans. on Smart Grid, vol. 2, no. 1, pp. 131–138, Mar. 2011. [22] W. Kempton and J. Tomic, “Vehicle-to-grid power fundamentals: Calculating capacity and net revenue,” Journal of Power Sources, vol. 144, no. 1, pp. 268–279, June 2005. [23] S. Shao, M. Pipattanasomporn, and S. Rahman, “Challenges of PHEV penetration to the residential distribution network,” in Proc. IEEE Power and Energy Society General Meeting, Calgary, Canada, July 2009.  Bibliography  55  [24] S. Shao, T. Zhang, M. Pipattanasomporn, and S. Rahman, “Impact of TOU rates on distribution load shapes in a smart grid with PHEV penetration,” in Proc. IEEE Transmission and Distribution Conference and Exposition, Orlando, FL, Apr. 2010. [25] S. Letendre and R. A. Watts, “Effects of plug-in hybrid electric vehicles on the Vermont electric transmission system,” in Proc. Transportation Research Board 88th Annual Meeting, Washington, DC, Jan. 2009. [26] E. Tate and S. Boyd, “Finding ultimate limits of performance for hybrid electric vehicles,” in Proc. SAE Future Transportation Technology Conference and Exposition, Costa Mesa, CA, Aug. 2000. [27] J. Xu and V. W. S. Wong, “An approximate dynamic programming approach for coordinated charging control at vehicle-to-grid aggregator,” in Proc. of IEEE SmartGridComm, Brussels, Belgium, Oct. 2011. [28] D. P. Bertsekas, Dynamic Programming and Optimal Control, vol. II, 3rd ed. Athena Scientific, 2007. [29] R. E. Bellman, Dynamic Programming, 1st ed. Princeton University Press, 1957. [30] R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction, 1st ed. MIT Press, Cambridge, MA, 1998. [31] D. P. Bertsekas and J. N. Tsitsiklis, Neuro-Dynamic Programming (Optimization and Neural Computation Series 3), 1st ed. Athena Scientific, 1996.  Bibliography  56  [32] California ISO Open Access Same-time Information System. [Online]. Available: http://oasis.caiso.com [33] M. Kashem, A. Le, M. Negnevitsky, and G. Ledwich, “Distributed generation for minimization of power losses in distribution systems,” in Proc. IEEE Power Engineering Society General Meeting 2006, Montreal, Canada, June 2006. [34] C. Cecati, C. Citro, and P. Siano, “Combined operations of renewable energy systems and responsive demand in a smart grid,” IEEE Trans. on Sustainable Energy, July 2011. [35] A. Saber and G. Venayagamoorthy, “Plug-in vehicles and renewable energy sources for cost and emission reductions,” IEEE Trans. on Industrial Electronics, vol. 58, no. 4, pp. 1229–1238, Apr. 2011. [36] G. Shafiullah, A. Oo, D. Jarvis, A. Ali, and P. Wolfs, “Prospects of solar energy in Australia,” in Proc. International Conference on Electrical and Computer Engineering (ICECE), Dhaka, Bangladesh, Dec. 2010. [37] M. Glavin, P. Chan, S. Armstrong, and W. Hurley, “A stand-alone photovoltaic supercapacitor battery hybrid energy storage system,” in Proc. 13th Power Electronics and Motion Control Conference, Poznan, Poland, Sept. 2008. [38] Q. Wang and S. Choi, “The design of battery energy storage system in a unified power-flow control scheme,” IEEE Trans. on Power Delivery, vol. 23, no. 2, pp. 1015–1024, Apr. 2008.  Bibliography  57  [39] S. Wakao and N. Ueda, “Investigation of the storage battery station for effective utilization of electric power generated by PV clusters,” in Proc. IEEE 4th World Conference on Photovoltaic Energy Conversion, Waikoloa, HI, May 2006. [40] H. Qian, J. Zhang, J.-S. Lai, and W. Yu, “A high-efficiency grid-tie battery energy storage system,” IEEE Trans. on Power Electronics, vol. 26, no. 3, pp. 886–896, Mar. 2011.  

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
http://iiif.library.ubc.ca/presentation/dsp.24.1-0072073/manifest

Comment

Related Items