Applied Science, Faculty of
Engineering, School of (Okanagan)
DSpace
UBCO
Dueck, Kevin
Master of Applied Science - MASc
University of British Columbia
Electric vehicles (EVs) are becoming a more popular alternative to internal combustion engine vehicles, however a concern among EV manufacturers and customers is the longevity of the EV's energy source, the battery. The battery is a large contributor to the cost of an EV and is susceptible to wear due to charge/discharge cycles and heat. This wear is a main depreciator to the EV's worth. Batteries are considered a low power density energy storage device, however a hybrid energy storage system (HESS) can be formed with an improved power density by interfacing batteries and super-capacitors (SCs). An HESS utilizes the high energy density of batteries and the high power density of SCs; this lowers the wear of the batteries by directing high current transients to the capacitors and lowering the heat generated by the batteries. Proper control is imperative to developing an effective HESS that will extend the life of the batteries. This thesis presents a novel control for a typical EV with a battery size of 24kWh coupled with a minimally sized HESS comprised of a SC bank with a 94.5kJ or 26.3Wh capacity using a neural network (NN) trained by a genetic algorithm. This method uses a NN to find patterns in simulated driving profiles to optimize the SCs' state of charge and SCs' current in a way that reduces the RMS current delivered by the batteries by up to 15% and reduces the peak currents by up to 52.5%. A 15% reduction in battery RMS current correlates to a 28% reduction the thermal energy produced by the battery due to its internal resistance. This reduction in heat reduces wear on the battery and simplifies thermal management strategies for the battery. This thesis will discus the construction of such a control system, the code of the genetic algorithm, parameter selection, and the effectiveness of this solution in controlling an HESS for practical use in consumer EVs.
Neural Network Trained via Reinforced Learning asControl for a Battery Super-Capacitor Hybrid EnergyStorage System in Electric VehiclesbyKevin DueckB.A.Sc. Hons., The University of British Columbia, 2018A THESIS SUBMITTED IN PARTIAL FULFILLMENT OFTHE REQUIREMENTS FOR THE DEGREE OFMASTER OF APPLIED SCIENCEinTHE COLLEGE OF GRADUATE STUDIES(Electrical Engineering)THE UNIVERSITY OF BRITISH COLUMBIA(Okanagan)August 2020© Kevin Dueck, 2020The following individuals certify that they have read, and recommend to the College ofGraduate Studies for acceptance, a thesis/dissertation entitled:Neural Network trained via Reinforced Learning as Control for a BatterySuper-Capacitor Hybrid Energy Storage System in Electric VehiclesSubmitted by Kevin Dueck in partial fulfilment of the requirements of the degree ofMaster of Applied ScienceDr. Wilson Eberle, Faculty of Applied Science, UBC OkanaganSupervisorDr. Liwei Wang, Faculty of Applied Science, UBC OkanaganSupervisory Committee MemberDr. Morad Abdelaziz, Faculty of Applied Science, UBC OkanaganSupervisory Committee MemberDr. Zheng Liu, Faculty of Applied Science, UBC OkanaganUniversity ExamineriiAbstractElectric vehicles (EVs) are becoming a more popular alternative to internal combustionengine vehicles, however a concern among EV manufacturers and customers is the longevityof the EV’s energy source, the battery. The battery is a large contributor to the cost of anEV and is susceptible to wear due to charge/discharge cycles and heat. This wear is a maindepreciator to the EV’s worth. Batteries are considered a low power density energy storagedevice, however a hybrid energy storage system (HESS) can be formed with an improvedpower density by interfacing batteries and super-capacitors (SCs). An HESS utilizes thehigh energy density of batteries and the high power density of SCs; this lowers the wear ofthe batteries by directing high current transients to the capacitors and lowering the heatgenerated by the batteries. Proper control is imperative to developing an effective HESSthat will extend the life of the batteries.This thesis presents a novel control for a typical EV with a battery size of 24kWh coupledwith a minimally sized HESS comprised of a SC bank with a 94.5kJ or 26.3Wh capacityusing a neural network (NN) trained by a genetic algorithm. This method uses a NN to findpatterns in simulated driving profiles to optimize the SCs’ state of charge and SCs’ currentin a way that reduces the RMS current delivered by the batteries by up to 15% and reducesthe peak currents by up to 52.5%. A 15% reduction in battery RMS current correlates toiiiAbstracta 28% reduction the thermal energy produced by the battery due to its internal resistance.This reduction in heat reduces wear on the battery and simplifies thermal managementstrategies for the battery. This thesis will discus the construction of such a control system,the code of the genetic algorithm, parameter selection, and the effectiveness of this solutionin controlling an HESS for practical use in consumer EVs.ivLay SummaryElectric vehicles (EVs) offer many benefits over vehicles with internal combustion engines,however the battery of an EV looses energy capacity over time, resulting in a loss of drivingrange. This loss in capacity is due to heat generated by the battery, which degrades thebattery’s internals. A solution to this problem is to supplement the battery with super-capacitors which generate less heat than batteries when delivering power; combining theseenergy sources forms a hybrid energy storage system (HESS). To lower the battery’s heat,the HESS must be properly controlled. A neural network (NN) is a computer algorithmwhich mimics the function of a brain and is an effective tool to recognize patterns in datathat, once trained, provides an appropriate output. This thesis uses a NN trained by agenetic algorithm to control power flow in an HESS and extend the life of an EVs battery.vTable of ContentsAbstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iiiLay Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vTable of Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viList of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xList of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiList of Acronyms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xviiList of Symbols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xixAcknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxiiChapter 1: Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 Control for Active Hybrid Energy Storage Systems . . . . . . . . . . . . . . 21.3 Operation of an Artificial Neural Network . . . . . . . . . . . . . . . . . . . 31.4 Training a Neural Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61.4.1 Supervised Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . 7viTABLE OF CONTENTS1.4.2 Unsupervised Learning . . . . . . . . . . . . . . . . . . . . . . . . . . 71.4.3 Reinforced Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . 71.5 Thesis Organization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8Chapter 2: Literature Review . . . . . . . . . . . . . . . . . . . . . . . . . . . 102.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102.2 Purpose of a Hybrid Energy Storage System . . . . . . . . . . . . . . . . . . 102.3 Construction of a Hybrid Energy Storage System . . . . . . . . . . . . . . . 142.3.1 Passive Hybrid Energy Storage Systems . . . . . . . . . . . . . . . . 142.3.2 Active Hybrid Energy Storage Systems . . . . . . . . . . . . . . . . . 162.4 Control of a Hybrid Energy Storage System . . . . . . . . . . . . . . . . . . 192.4.1 Rule-Based Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222.4.2 Optimization-Based Control . . . . . . . . . . . . . . . . . . . . . . . 242.4.3 Neural Network Control . . . . . . . . . . . . . . . . . . . . . . . . . 292.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32Chapter 3: Neural Network Control and Genetic Algorithm . . . . . . . . . 333.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333.2 Modelled Hybrid Energy Storage System Parameters . . . . . . . . . . . . . 333.3 Neural Network Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . 363.3.1 Inputs to the Neural Network . . . . . . . . . . . . . . . . . . . . . . 363.3.2 Number of Layers and Neurons of Neural Network . . . . . . . . . . . 413.3.3 Construction of Neural Network . . . . . . . . . . . . . . . . . . . . . 453.4 External Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46viiTABLE OF CONTENTS3.5 Genetic Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 503.5.1 Genetic Algorithm Parameters . . . . . . . . . . . . . . . . . . . . . . 503.5.2 Genetic Algorithm Operation . . . . . . . . . . . . . . . . . . . . . . 543.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57Chapter 4: Simulation Results . . . . . . . . . . . . . . . . . . . . . . . . . . . 584.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 584.2 Progression of Genetic Algorithm . . . . . . . . . . . . . . . . . . . . . . . . 584.3 Results of Neural Network Control for Hybrid Energy Storage System . . . . 594.4 Results Compared to Other Control Strategies . . . . . . . . . . . . . . . . . 654.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66Chapter 5: Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 675.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 675.2 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 675.3 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70Appendices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75Appendix A: Driving Profile Waveforms . . . . . . . . . . . . . . . . . . . . . . . 75Appendix B: Matlab Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79B.1 Main code for training a NN . . . . . . . . . . . . . . . . . . . . . . . . . . . 79B.2 Randomization function. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85B.3 Mutate function. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85viiiTABLE OF CONTENTSB.4 Map fitness function. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85B.5 Mating pool function. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86B.6 Offspring function. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86B.7 Save function. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87B.8 Weight evaluator function. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87B.9 Generation progression and NN weight analyser code. . . . . . . . . . . . . . 87Appendix C: Simulink Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92ixList of TablesTable 2.1 Specification of Li-ion battery cell model components used in equation2.2 [1]. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12Table 3.1 Parameters of the EV and modelled HESS. . . . . . . . . . . . . . . . 34Table 3.2 Neuron combinations of the hidden layers of a 3 layer NN. . . . . . . 43Table 3.3 Parameters initialized at the beginning of the genetic algorithm. . . . 51Table 4.1 Percent RMS current reduction, percent max peak current reduction,and percent PAV R/PAPR reduction of a NN controlled HESS trainedthe UDDS profile. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63Table 4.2 Percent RMS current reduction, percent max peak current reduction,and percent PAV R/PAPR reduction of a NN controlled HESS trainedthe US06 profile. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64Table 4.3 Results of the proposed control compared with HESSs designed withthe similar function of reducing battery RMS and peak currents andreducing PAPR/PAVR ratios. . . . . . . . . . . . . . . . . . . . . . . 65xList of FiguresFigure 1.1 Block diagram of an active HESS where the SC bank is decoupledfrom a the EV powertrain components via a DC/DC converter. . . . 2Figure 1.2 Frequency spectral analysis of an EVs load current. Effective HESScontrol will split into high-frequency components from low-frequencycomponents and directed these components to the SC bank and bat-tery bank respectively. . . . . . . . . . . . . . . . . . . . . . . . . . 3Figure 1.3 A single neuron of a NN with two inputs (x1 and x2) and one output(y). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4Figure 1.4 Architecture of a 3 layer ANN. . . . . . . . . . . . . . . . . . . . . . 5Figure 2.1 Lithium-ion battery stress factor as a function of temperature basedon equation 2.1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11Figure 2.2 Equivalent Thevenin battery cell model. . . . . . . . . . . . . . . . . 12Figure 2.3 Equivalent circuit of SC cells. . . . . . . . . . . . . . . . . . . . . . . 13Figure 2.4 Ragone plot of different energy storage devices. . . . . . . . . . . . . 13xiLIST OF FIGURESFigure 2.5 Different configurations of interfacing the battery and SC bank to theDC bus in a drive train: (a) Direct connection of battery and SCbank to the DC bus (passive control), (b) Partially-decoupled con-figuration, type I, (c) Partially-decoupled configuration, type II, (d)Fully-decoupled configuration of parallel-connected battery and SCbank, (e) Cascaded configuration type I, (f) Cascaded configuration,type II, (g) Parallel converter configuration, and (h) integrated con-figuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15Figure 2.6 Schematic representation of an NPC used in an integrated HESStopology. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18Figure 2.7 Schematic representation of an MSI used in an integrated HESS topol-ogy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19Figure 2.8 Power split between components of an HESS (top graph: time analysisof load current profile, lower graph: FFT applied to the top graphwith a period of the simulation length). The split frequencies of thebottom graph represent frequency the sharing plan for LPF controlwith high-frequency waveforms directed to the SCs and low-frequencywaveforms directed to the battery. . . . . . . . . . . . . . . . . . . . 20Figure 2.9 Semilog plot comparing the computational complexity ofO(n), O(nlogn),and O(n2) with input size. . . . . . . . . . . . . . . . . . . . . . . . . 21xiiLIST OF FIGURESFigure 2.10 Control diagram of HESS control using an LPF with a variable cor-ner frequency to directed load current frequency components to thebatteries or SCs. Battery and SC current signals are further modi-fied by the battery current charge loop control to consider hardwareconstraints (cited from [2]). . . . . . . . . . . . . . . . . . . . . . . . 27Figure 2.11 Average model of a boost converter. . . . . . . . . . . . . . . . . . . 28Figure 2.12 Average model of a buck converter. . . . . . . . . . . . . . . . . . . . 28Figure 2.13 Structure of the Hammerstein-type neural network. . . . . . . . . . . 30Figure 3.1 Block diagram of the modelled HESS used to develop NN based control 34Figure 3.2 Time taken of a 3.1F SC bank to discharge from 250V to 50V basedon converter limit. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36Figure 3.3 Percent reduction in RMS battery current of a trained NN controllinga converter of ranging current limits. . . . . . . . . . . . . . . . . . . 37Figure 3.4 Block diagram of the modelled converter which considers operatingefficiency. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38Figure 3.5 UDDS profiles for speed, moving average, and local max, used asinputs to the NN. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39Figure 3.6 UDDS profiles for torque and acceleration, used as inputs to the NN. 40Figure 3.7 UDDS profiles for motor current and capacitor voltage, used as inputsto the NN. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40Figure 3.8 Control diagram which produces the local max and average profilesused as inputs to the NN. . . . . . . . . . . . . . . . . . . . . . . . . 41xiiiLIST OF FIGURESFigure 3.9 RMS value of the first layer weight vectors applied to the input signalsof the NN, used to evaluate the effectiveness of a particular inputagainst other inputs. . . . . . . . . . . . . . . . . . . . . . . . . . . . 41Figure 3.10 Genetic algorithm progression of a 4 layer network with hidden layersconsisting of 50 neurons per layer. . . . . . . . . . . . . . . . . . . . 42Figure 3.11 Genetic algorithm progression of a 3 layer network with hidden layersconsisting of 50 neurons per layer. . . . . . . . . . . . . . . . . . . . 43Figure 3.12 Genetic algorithm progression of a 2 layer network with hidden layersconsisting of 50 neurons per layer. . . . . . . . . . . . . . . . . . . . 44Figure 3.13 Battery RMS current of a trained NN with various combinations ofhidden layer neurons based on table 3.2. . . . . . . . . . . . . . . . 44Figure 3.14 NN simulated using Simulink and used as control. (a) Architecture ofthe NN constructed in Simulink with 12 neurons in the first hiddenlayer and 9 in the second. (b) The block diagram of the same NNimplemented in Simulink. . . . . . . . . . . . . . . . . . . . . . . . . 45Figure 3.15 Control diagram used to normalize an input signal to the NN. . . . . 46Figure 3.16 Diagrams of control used to ensure the SC bank does not exceed itsvoltage operation limits (top) and diagram of control used to ensurethat the converter does not exceed its current operation limits (bottom). 47Figure 3.17 The effect of various population sizes on the progression of a geneticalgorithm. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52Figure 3.18 An RMS frequency spectrum analysis of the NN input signals gener-ated in ADVISOR. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52xivLIST OF FIGURESFigure 3.19 The effect of various mutation rates on the progression of a geneticalgorithm. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54Figure 4.1 Progression of the genetic algorithm initialized with the parametersof table 3.3 used to train a 3 layer NN with 8 inputs, 12 neurons inthe second layer, 9 neurons in the 3rd layer, and 1 output. . . . . . . 59Figure 4.2 Graph showing the reduction a trained NN controlled HESS will haveon an EVs load current. The load current represents the batterycurrent of an EV with no added HESS, the battery current representsthe reduction in battery current an HESS can offer, and the convertercurrent is the difference of the two currents that is directed to the SCbank. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60Figure 4.3 An urban driving section of the currents shown in figure 4.2. . . . . . 61Figure 4.4 Graph showing the voltages and current sourced from the SC bank ofthe HESS controlled by a NN trained on the UDDS profile. . . . . . 62Figure 4.5 Graph showing the voltages and current sourced from the SC bank ofthe HESS controlled by a NN trained on the US06 profile. . . . . . . 64Figure A.1 Speed input of UDDS driving profile. . . . . . . . . . . . . . . . . . . 75Figure A.2 Load current and torque inputs of UDDS driving profile. . . . . . . . 75Figure A.3 Speed input of US06 driving profile. . . . . . . . . . . . . . . . . . . 76Figure A.4 Load current and torque inputs of US06 driving profile. . . . . . . . 76Figure A.5 Speed input of LA92 driving profile. . . . . . . . . . . . . . . . . . . 77Figure A.6 Load current and torque inputs of LA92 driving profile. . . . . . . . 77xvLIST OF FIGURESFigure A.7 Speed input of NEDC driving profile. . . . . . . . . . . . . . . . . . . 78Figure A.8 Load current and torque inputs of NEDC driving profile. . . . . . . . 78Figure C.1 Main Simulink model (HESS Layout with converter subsystem). . . . 92Figure C.2 Main Simulink model (inputs from Matlab workspace and Neural Net-work and Control subsystems.) . . . . . . . . . . . . . . . . . . . . . 93Figure C.3 Main Simulink model (output signals to workspace). . . . . . . . . . 94Figure C.4 Converter subsystem. . . . . . . . . . . . . . . . . . . . . . . . . . . 95Figure C.5 Neural network subsystem input normalizing and first layer bus. . . . 96Figure C.6 Neural network subsystem internal layers, output, and local max gen-erator control. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97Figure C.7 SC voltage and converter current control subsystem. . . . . . . . . . 98xviList of AcronymsAC Alternating CurrentADVISOR Advanced Vehicle SimulatorANN Artificial Neural NetworkDC Direct CurrentEV Electric VehicleFC Fuel CellFFT Fast Fourier TransformFIR Finite Impulse ResponseFTP-72 Federal Test Procedure-72HESS Hybrid Energy Storage SystemHNN Hammerstein-type Neural NetworkICE Internal Combustion VehicleIIR Infinite Impulse ResponseKCL Kirchoff’s Current LawLPF Low-Pass FilterMSI Multi-Sourced InverterNEDC New European Driving CyclexviiList of AcronymsNN Neural NetworkPAPR Peak to Average Power RatioPAVR Peak to Average Velocity RatioPBC Passivity-Based ControllerSC Super-CapacitorSoC State of ChargeUDDS Urban Dynamometer Driving ScheduleUC Ultra-CapacitorxviiiList of SymbolsA Function ConstantAn×nx(t)n×1 State Space State Matrixα Feedback GainB Function ConstantBn×mu(t)m×1 State Space Input Matrixβ Neural Network GainC CapacitanceCp×nx(t)n×1 State Space Output MatrixCPn Capacitance of pole nCSC or CUC Super-Capacitor or Ultra-Capacitor CapacitanceD Multiplication ConstantDp×mu(t)m×1 State Space Direct Transition Matrix∆ibat,max Change in Maximum Instantaneous Battery Current∆V Voltage Rangeεinitial Randomization Factorγ Input Gainibat Instantaneous Battery CurrentxixList of Symbolsibat,max Maximum Instantaneous Battery CurrentiL Instantaneous Inductor CurrentIbat Average Battery CurrentIhigh side Current Sourced from High Voltage Side of ConverterIlimit DC-DC Converter Current LimitIlow side Current Sourced from Low Voltage Side of ConverterILoad Average Load CurrentIref Reference Current from Neural NetworkIRMS Root Mean Square Current of SimulationISC or IUC Average Super-Capacitor or Ultra-Capacitor CurrentkT Temperature ConstantL Inductanceη Converter EfficiencyPmax Maximum Powerr ResistanceRPn Resistance of pole nRSC or RUC Super-Capacitor or Ultra-Capacitor ResistanceRSeries Series ResistanceT TemperatureTref Reference TemperatureΘm,n Neural Network Weight MatrixVbuffer Buffer Voltage When Super-Capacitor Voltage Limit Control is ActivexxList of SymbolsVgain Gain Applied to Voltage Control SignalVLoad Load VoltageVmin Minimum Voltage of Super-capacitor BankVOC(SOC) Battery Cell VoltageVSC or VUC Super-Capacitor or Ultra-Capacitor Voltagexn Inputx˙(t)n×1 State Space State Vectorxˆ Normalized Inputy(t)p×1 State Space Output Vectoryn OutputZ(S)Batt Impedance of Battery CellZ(S)SC or Z(S)UC Impedance of Super-Capacitor or Ultra-CapacitorxxiAcknowledgementsI would like to thank my partner, Jesuina McDonald for her continued support during mystudies. I would like to thank my professor Dr. Wilson Eberle for his support and guidancethroughout this project. This project is funded by Ford Motor Co., without this funding theproject would not be possible, so I would also like to thank their contributions. AdditionallyI would like to thank my colleague Tobias Lindsay for his work in starting this project ofwhich I had the opportunity to continue. Lastly I would like to thank my family for theirsupport, both emotional and financial throughout my education leading to and includingthis project.xxiiChapter 1Introduction1.1 MotivationAs climate change becomes a more recognized problem, vehicle manufactures are invest-ing into electric vehicles (EVs) as a clean alternative to vehicles using internal combustionengines (ICEs). However, EVs are not without their drawbacks, which include chargingrates, vehicle range, battery power density, and battery longevity. The later is of concern asEVs are gaining popularity among the public; lithium-ion batteries, the prominent batteryused in EVs, are susceptible to heat that is generated by rapid discharging or charging of thebatteries, as is the case when an EV accelerates or decelerates. Heat accelerates the degra-dation of batteries and as the battery ages in a vehicle, driving range is reduced. A solutionto mitigate this problem is to supplement the battery of an EV with a super-capacitor (SC),or ultra-capacitor (UC) bank to form a hybrid energy storage system (HESS). The blockdiagram describing a SC/battery HESS is shown in figure 1.1, here the SC bank is linked tothe EV’s traditional powertrain components via a bi-directional DC/DC converter. Formingan HESS by adding a SC pack to a EV’s battery pack can extend the life of a battery bydirecting potentially harmful current transients while driving to the SC bank, leaving thebatteries to supply a baseline DC current. The SC pack can also aid the performance of11.2. Control for Active Hybrid Energy Storage SystemsDC busDC/AC MotorSC bank BatteryDC/DCAdditional HESS Hardware EV Powertrain ComponentsFigure 1.1: Block diagram of an active HESS where the SC bank is decoupled from a the EV powertraincomponents via a DC/DC converter.an EV by increasing the available power to the motor(s) of the EV. Proper control of theDC/DC converter is critical to create an effective HESS. This thesis discusses the use of ma-chine learning and neural networks (NNs) for control of power flow, reviews previous HESScontrol strategies and develops novel control utilizing a NN trained with reinforced learningby a genetic algorithm.1.2 Control for Active Hybrid Energy Storage SystemsEach energy source of an HESS has its own advantages and disadvantages, the currentsourced from either energy source must be controlled to fully utilize the advantages offeredby each energy source. Effective control splits a driving current profile into subsequent cur-rent profiles to direct these profiles to either energy storage component. An ideal HESSwill separate and divert high and low frequency components of the driving power profile tothe energy storage element with a high-power density and the storage with the high energydensity respectively, as shown in figure 1.2. Previous methods are discussed in section 2.4and include rule-based and optimization-based control. Rule-based control is more com-21.3. Operation of an Artificial Neural Networkputationally simple, yet may not offer the same improvements as a more computationallytaxing optimization-based control technique. This thesis will focus on optimization-basedcontrol utilizing a NN to control current flow in the HESS. The NN will solve the problemof determining when to discharge and charge the SCs and how much current to source fromeach energy storage component based on current vehicle and HESS conditions to obtain aSC current profile and battery current profile like those shown in figure 1.2.0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18 0.2f (Hz)0246810121416Amplitude (A)IBattery ISuper-capacitorBattery CurrentSuper-capacitor CurrentTimeTimeAmplitude (A)Amplitude (A)Figure 1.2: Frequency spectral analysis of an EVs load current. Effective HESS control will split intohigh-frequency components from low-frequency components and directed these components tothe SC bank and battery bank respectively.1.3 Operation of an Artificial Neural NetworkNNs are meant to address computing problems not easily solved by conventional controlsolutions, such as classifying data or pattern recognition [3]. The concept of an artificialneural network (ANN), or NN, mimics the progression of a signal through neurons in abiological neural network like the brain. A biological neural network is a intricate web of31.3. Operation of an Artificial Neural Networkinterconnections that allows a series of electrical signals to be transmitted and influenceeach other to produce an output. In order to construct an ANN, one must understand theoperation of a biological NN. A neuron is comprised of dendrites, which receive an electricalsignal, and an axon, which transmits an output signal based on the inputs the dendritesreceive [3]. The actual operation of the human brain is of course much more complicatedand considers many more elements, but for the sake of analogy, this understanding canbe translated to ANNs. NNs can be referred to as a ”connectionist” computational system,which follows no linear path when executing a problem. This is contrary to traditional linearprograms which execute one line of code followed by another. Given the unpredictable natureof vehicle loading when driving, NNs are a good solution to HESS control for electric vehiclesbecause of their ability to recognize patterns in datasets. A NN operates by modifying ax1x2Neuron(processor)yFigure 1.3: A single neuron of a NN with two inputs (x1 and x2) and one output (y).signal as it passes through rows of interconnected ”neurons”. Four individual steps arecompleted as a signal progresses through each neuron for each time step of an ANN. Thefollowing steps will reference a two input neuron, as shown in figure 1.3. The first step receivesboth inputs to the neuron, illustrated as x1 and x2. Each signal input to the neuron has aweight applied to it referred to as Θ for the second step. The third step sums the weightedinputs and the fourth step generates an output by passing the summed signal through an41.3. Operation of an Artificial Neural Networkactivation function. These steps are represented by the function shown in equation 1.1.y = tanh(Θ1x1 + Θ2x2) (1.1)Typically an activation function for a discrete NN is the signum function which classifiesthe output discretely as either negative, or positive 1 given the inputs, however for the caseof a linear NN, the activation function is the hyperbolic tan function; this allows a linearrepresentation of the weights but limits the signal to ±1. Mathematically, the inputs (x) are1xnx2x11a1(1)an(1)a2(1) hΘ(x)Layer 1 Layer 2 Layer 3 Outputa1(3)Figure 1.4: Architecture of a 3 layer ANN.represented as a column vector and the weight function (Θ) of the neurons is represented asa matrix, shown in equation 1.2; the output of one neuron is the sigmoid (g(z)) function ofthe dot product of the inputs and the corresponding row of the weight matrix as shown inequation 1.3 and figure 1.4. Here a(m)n represents the operations of one neuron, m signifiesthe layer and n signifies the neuron of that layer. Each arrow represents a scaling factorapplied between an input and a neuron. The product is a column vector that acts as inputsto the next layer of neurons. In the column vectors, the subscript refers to the number of a51.4. Training a Neural Networkneuron in a layer and the superscript refers to layer number. The column vector result forthe first layer of a network is shown in equation 1.3. Inputs to the NN include driving profilevariables such as speed, acceleration, power, and parameters of energy sources like state ofcharge (SoC).xn×1 =x1x2...xn Θm×n =Θ1,1 Θ1,2 . . . Θ1,nΘ2,1 Θ2,2 . . . Θ2,n...Θm,1 Θm,2 . . . Θm,n (1.2)a(2)1 = g(Θ(1)10 x0 + Θ(1)11 x1 + ...+ Θ(1)1nxn)a(2)2 = g(Θ(1)20 x0 + Θ(1)21 x1 + ...+ Θ(1)2nxn)...a(2)n = g(Θ(1)n0x0 + Θ(1)n1x1 + ...+ Θ(1)nnxn)hΘ(x) = a(3)1 = g(Θ(2)10 a(2)0 + Θ(2)11 a(2)1 + ...+ Θ(2)1n a(2)3)(1.3)1.4 Training a Neural NetworkNNs are traditionally used in conjunction with machine learning to solve classificationproblems. Examples of this include determining spam in an email inbox or recognizinghandwriting. The output of an ANN can be made linear instead of discrete to make anANN usable as control for an HESS. A step to building an effective NN is training the valueof the weight matrices (Θ). Training a NN means adjusting the values of weights so theNN provides an effective output. These weights are trained by either supervised learning,unsupervised learning, or reinforced learning.61.4. Training a Neural Network1.4.1 Supervised LearningSupervised learning is a process where an ideal output for the NN is already known, andthe weights are trained such that the output of the NN matches the ideal output given aset of inputs. The trained network is then used for control in environments similar to thetraining data set. The first step is to randomize the weights of the Θ matrices, then applythe input signals. The signal propagates through the NN and produces an output and a costfunction evaluates the error of the signal, that is the difference between the desired signal andthe NN’s output, and produces a value. This value is then used to back-propagate throughthe NN and adjust the weights of the Θ matrices. This is done repetitively until the costfunction is at the desired level, the trained NN is then tested on other datasets to measureits effectiveness.1.4.2 Unsupervised LearningUnsupervised learning is used in discrete classification NNs to determine like clusters ofdata. Here, the number of categories a data set can be categorized into is selected, and thealgorithm groups data that are related into one of these categories. Unsupervised learningis a technique for discrete NNs is not applicable for linear NNs and HESS applications.1.4.3 Reinforced LearningThe final learning method for NNs is reinforced learning. The optimal output is notneeded for this technique, instead a genetic algorithm and Darwinian theory is used todetermine weights (in this case referred to as the genes) of a NN. Reinforced learning is71.5. Thesis Organizationcomprised of three main components: heredity, variation, and selection. NN genes are testedin generations and the algorithm allows better performing genes to continue to subsequentgenerations while eliminating poor performing genes. The genetic algorithm operates overfour main steps; the first step of a genetic algorithm is to generate a randomized populationof genes. Each member of the population is run and the effectiveness of each member isrecorded as a fitness value. The second step is to select members with a high fitness whosegenes will transfer to subsequent generations. A matting pool is created from the population,where members with a higher fitness occupy a larger portion of the matting pool. Those witha larger portion of the mating pool have a higher probability of being selected for the nextgeneration. The third step is reproduction, where a number of members from the matingpool are selected and their genes are combined to produce a new gene matrix. In the laststep, some of the newly generated members’ genes are selected at random and are givena new value in a process referred to as mutation; this prevents the algorithm from beingconstrained by its initial genes and converging at a local minima. This process is repeateduntil the fitness from one generation to the next no longer improves.1.5 Thesis OrganizationThis thesis is organized into 5 chapters. Chapter 1 discusses the need for an HESSin electric vehicles, different methods of developing control for an HESS, the fundamentaloperation of a NN, and methods of training a NN. Chapter 2 explores previous strategies ofHESS control and the use of NN for control of power flow. It compares the computationaleffort of other methods using the big O notation, a method to compare computational81.5. Thesis Organizationcomplexity. Chapter 3 presents the design process and structure of the NN developed inthis thesis, discussing the different components of the control and the method of which itis trained. Chapter 4 presents the results of the genetic algorithm training the NN and theresults of the NN control on the HESS. Chapter 5 presents the conclusions, summarizes thethesis results and discusses opportunities for future work.9Chapter 2Literature Review2.1 OverviewThis chapter reviews and analyses previous works of HESS construction and control andcompares their benefits and drawbacks. This chapter’s first section 2.2 discusses the purposeof an HESS as studied by others. Section 2.3 explores how an HESS is implemented withpassive and various active topologies and discusses the advantages and drawbacks of each.Section 2.4 discusses various methods of control of an HESS and compares the computationaleffort of these types of control.2.2 Purpose of a Hybrid Energy Storage SystemAn HESS is used to combine the benefits of two different energy storage devices. This is asolution not needed with ICE vehicles, as the energy and power density of fossil fuels sourcesis 2 orders of magnitude larger than that of batteries making it unnecessary to supplementfossil fuel with other energy sources [4]. To make EVs a compelling option compared totraditional ICE vehicles, EV manufactures are focusing on increasing the performance andlongevity of their vehicles. With the increase of demanded performance comes increasedstress on the batteries of an EV. Batteries produce current from an electrochemical reaction102.2. Purpose of a Hybrid Energy Storage Systemwhich has inherent problems like a slow transient response to current demands and a highinternal resistance as compared to SCs. The heat generated in the lithium-ion cell causesaccelerated wear if not removed or reduced. A study done by [5] shows a lithium-ion batterywill reduce its capacity from 90% to 60% over a period of 5 years if the operating temper-ature is increased from 15◦C to 55◦C. The battery temperature stress model is shown inequation 2.1 [5] where KT is the temperature stress coefficient, obtained by experimentallytesting specific cells, Tref is a reference temperature, typically 293K, and T is the operatingtemperature. This equation models the temperature dependence of the rate of a chemicalreaction.ST (T ) = ekT (T−Tref )TrefT (2.1)Figure 2.1 plots the stress factor between 280K and 330K, which is approximately between15◦C and 55◦C.280 285 290 295 300 305 310 315 320 325 330Tempurature (Kelvin)012345678910Stress FactorFigure 2.1: Lithium-ion battery stress factor as a function of temperature based on equation 2.1.112.2. Purpose of a Hybrid Energy Storage SystemTo compare parasitic heat loss between lithium-ion batteries and SCs, an RMS currentvalue of 23.48ARMS, taken from the Federal Test Procedure-72 (FTP-72) driving profile [6]using an EV with a mass of 912kg to analyse I2R losses. This is based on a battery packwith an operating voltage of 350V like that of the Nissan Leaf® which has a 24kWh lithiumion battery pack with 96 cells in series and 2 in parallel [7]. To quantify the heat producedby a lithium ion cell, the model of a lithium-ion battery bank can be used, shown in figure2.2 and the formula for impedance of the battery is given by equation 2.2 [8].+-VOC (SOC)RSeriesRP1CP1RP2CP2 VBatt+-Figure 2.2: Equivalent Thevenin battery cell model.Z(s)Batt = RSeries +RP11 +RP1CP1s+RP21 +RP2C2s(2.2)Table 2.1: Specification of Li-ion battery cell model components used in equation 2.2 [1].Parameter RSeries RP1 CP1 RP2 CP2Value 0.07 Ω 0.05 Ω 703.6 F 0.5 Ω 4475 FIn this configuration of two parallel banks of lithium-ion cells, each cell with an internalresistance of 0.62Ω [1] will produce an excess of 171W of heat when the battery bank supplies23.48ARMS of current. For comparison an SC cell with a internal resistance of RUC =0.0022Ω, such as Maxwell Technologies’® BCAP0310 cell, will generate 1.2W of heat given122.2. Purpose of a Hybrid Energy Storage Systemthe same current demand. A model of such a cell is shown in figure 2.3 with the impedancerepresented in equation 2.3 and CUC = 310F .CUCRUCVUC+-Figure 2.3: Equivalent circuit of SC cells.Z(s)UC = RUC +1CUCs(2.3)One can see the benefits of using SCs to supply current, however because SCs store theirenergy in electrostatic fields they are limited by their relatively low energy density (5Wh/kg)compared to lithium ion batteries (200− 250Wh/kg) [9]. The relationship of power densityand energy density of different energy storage devices is shown in figure 2.4. The motivation101 102 103 104Gravimetric Power Density (W/kg)10-210-1100101102103Gravemetric Energy Denstiy (Wh/kg)Super-CapacitorsFuel CellsBatteriesLi-ionDouble-LayerCapacitorsElectrolyticCapacitorsFigure 2.4: Ragone plot of different energy storage devices.132.3. Construction of a Hybrid Energy Storage Systemfor an HESS is to combine the benefits of two energy storage devices while eliminating theirdrawbacks, such as combining SCs and lithium-ion batteries. Proper control ensures thatharmful current transients are diverted away from the batteries to the SCs to maximiselongevity of the batteries. The power density of a SC cell is much higher (5000W/kg) thanthat of a lithium-ion battery (400− 500W/kg) [9] and an effective HESS utilizes the powerdensity of an SC bank. An HESS can also aid the performance of a vehicle by lowering RMScurrent demanded from the battery and offering more power when accelerating, makingan EV a more appealing alternative to an ICE vehicle. An HESS provides the ability tobetter utilize the advantages available from electric motors in EVs, whose performance hashistorically been limited by battery technology.2.3 Construction of a Hybrid Energy Storage SystemThe topology of an HESS falls within two main categories: passive and active [10]. ActiveHESS topologies include subcategories with different configurations of the DC/AC inverter,DC/DC converter(s), and energy sources. Figure 2.5 illustrates these various configurations.Each topology will impact the cost of the HESS, the effectiveness of the HESS, and theperformance of the EV.2.3.1 Passive Hybrid Energy Storage SystemsThe most simple HESS topology is a passive system, which is made by connecting bothenergy sources of the HESS to the same DC bus with no use of intermediate DC/DC con-verters, as represented by figure 2.5(a). This topology removes the need for an intermediate142.3. Construction of a Hybrid Energy Storage SystemDC busDC/AC MotorSC bankBatteryDC busDC/AC MotorSC bankBattery DC/DCDC busDC/AC MotorSC bank BatteryDC/DCDC busDC/AC MotorSC bankBattery DC/DCDC busDC/AC MotorSC bankBattery DC/DCDC/DCDC busDC/AC MotorSC bank Battery DC/DCDC/DC(a)(b)(c)(d)(e)(f )(g)DC busDC/AC MotorSC bank Battery DC/DCDC/DCDC/AC MotorSC bankBattery(h)Figure 2.5: Different configurations of interfacing the battery and SC bank to the DC bus in a drive train:(a) Direct connection of battery and SC bank to the DC bus (passive control), (b)Partially-decoupled configuration, type I, (c) Partially-decoupled configuration, type II, (d)Fully-decoupled configuration of parallel-connected battery and SC bank, (e) Cascadedconfiguration type I, (f) Cascaded configuration, type II, (g) Parallel converter configuration,and (h) integrated configuration.converter(s), control software and hardware, and the costs associated with these components.A passive HESS relies on the inherent impedances of each source to control power flow [10].In the case of a SC–lithium-ion battery HESS, the relatively higher impedance of the bat-teries compared to the SCs, as shown in formula 2.2 and 2.3, will direct high frequencyload currents to the SC bank, and low frequency components to the battery. However this152.3. Construction of a Hybrid Energy Storage Systemtechnique does have drawbacks which make it unideal for use. In this topology the capacityof the SC bank is not fully utilized because the ∆V of the SC bank is constrained by thevoltage drop of the impedance of the batteries.2.3.2 Active Hybrid Energy Storage SystemsA more effective HESS system is an active HESS which uses DC/DC converters to directpower flow. In the case of an active HESS, power flow from each energy source is controlled byeither one or two converters. This allows the SC bank to fully cycle between its minimum andmaximum SoC, fully utilizing the energy available in the SC bank. Active HESS topologiescan be categorized as either partially decoupled, fully decoupled, or integrated.Partially Decoupled TopologiesA partially decoupled HESS has one energy source connected to the DC bus of theinverter and one energy source connected to the DC bus via a DC/DC converter. In thisway, power from both energy sources can be controlled independently; the source connectedto the converter is controlled directly and the other energy source is controlled via Kirchhoff’scurrent law (KCL), see formula 2.4.Iload = ISC + Ibat (2.4)Examples of partially decoupled systems are shown in figure 2.5 (b) and (c). An HESS wherethe battery is decoupled from the DC bus will make the battery immune to large fluctuationsin DC bus voltage, however the DC bus voltage must vary to charge and discharge the SC162.3. Construction of a Hybrid Energy Storage Systembank. This is a disadvantage, as this will vary the available peak voltage that can beapplied to the motor through the inverter, unless a inverter with boost capabilities is usedor the voltage rating of the motor is less than that of the minimum voltage of the SC bank[10]. If the SCs are decoupled, the SCs can be fully cycled without affecting the inverterspeak voltage, however the batteries are exposed to small voltage fluctuations on the DCbus. SC decoupled HESSs are a popular topology, and are done by [8, 11–16]. Partiallydecoupled systems greatly improve the effectiveness of an HESS however some elements arenot controlled.Fully Decoupled TopologiesA fully decoupled system separates both energy sources from the DC bus through use ofDC/DC converters. Figure 2.5 (d) shows a fully decoupled HESS topology that provides a DCbus voltage with no fluctuations, however the battery is exposed to voltage fluctuations of theSCs. While this provides the advantages of fully decoupling the energy sources, this topologyshares disadvantages of the passive HESS and as such is not a common implementation ofan HESS. Cascaded topologies is another fully decoupled topology option where convertersseparate each energy source from the DC bus and each other, represented in figures 2.5(e) and (f). These topologies offer the ability to directly control current from each source,however they require control to consider the power flow from each source, and are susceptibleto instabilities “as they can represent a DC/DC converter with a constant power load, thusrequiring significant amount of care” [10]. The voltage of the energy sources must be designedso that the diodes parallel to each converter are reverse biased; that is, the energy sourcefurthest removed from the inverter must have a voltage lower than the intermediate energy172.3. Construction of a Hybrid Energy Storage Systemsource. Another fully decoupled system connects both energy sources in parallel to theDC bus, as represented in figure 2.5 (g). In this, the voltage of the battery or SC is notconstrained to any value, however similar to cascaded topologies, control considering currentflow from each energy source must be developed to ensure a stable system. Fully decoupledtopologies are chosen by the authors of the papers: [2, 15].Integrated TopologiesBatteryTo motorNPCSC bankCHCLFigure 2.6: Schematic representation of an NPC used in an integrated HESS topology.A different approach to a HESS is to control power flow from each energy source withthe inverter used to supply the EVs drive motor. An example of this is done by [17], wherea neutral point clamped converter is used, and the capacitors are connected to the midpointof the converter, as shown in figure 2.6. This configuration allows a wide range of voltagesfor the SC, offering the advantages of an active HESS without the need of an additionalconverter [17].A multi-sourced inverter (MSI) topology, demonstrated by [18], controls power flow from182.4. Control of a Hybrid Energy Storage SystemBatterySupercapacitor BankTo motorSaSbScRa Rb RcTcTbTaFigure 2.7: Schematic representation of an MSI used in an integrated HESS topology.either SCs or a lithium-ion battery. The MSI topology is shown in figure 2.7. In this topology,the two sources are connected to their own respective high side switches, and both share lowside switches. The MSI powers the induction motor which drives the EV. The motivationfor integrated HESS systems is to reduce cost compared to other active HESS topologies.2.4 Control of a Hybrid Energy Storage SystemEffective control is needed to fully utilize the advantages provided by active HESSs. Thefundamental concept of an HESS is to split the load current profile into high-frequency tran-sients and a low-frequency baseline power. High-frequency load components will be directedto the energy source with higher specific power density, and the higher specific energy densityenergy source will supply the baseline power. Figure 2.8 describes such an energy split foran HESS, where the top figure shows the load current over the simulation length in the timedomain, and the bottom figure shows the results of a fast Fourier transform (FFT) doneover the length of the top figure. High-frequency and low-frequency current waveforms aredirected to the SCs and battery respectively through use of an ideal low-pass filter (LPF).192.4. Control of a Hybrid Energy Storage SystemLoad Current (A)0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18 0.2f (Hz)0246810121416Amplitude (A)IBattery ISuper-capacitor0 200 400 600 800 1000 1200 1400Simulation Time (s)-40-20020406080100120Figure 2.8: Power split between components of an HESS (top graph: time analysis of load current profile,lower graph: FFT applied to the top graph with a period of the simulation length). The splitfrequencies of the bottom graph represent frequency the sharing plan for LPF control withhigh-frequency waveforms directed to the SCs and low-frequency waveforms directed to thebattery.Control for an HESS can be categorized as rule-based or optimization-based. Rule-basedrequires less computational effort than optimization-based at the cost of effectiveness. In202.4. Control of a Hybrid Energy Storage Systemcomputer science, computational load of an algorithm is represented in big O notation and isreferred to as complexity. This notation will be used to compare computational requirementsof discussed control solutions; O(1) represents a problem where execution time is constant,O(n) represents a problem where execution time is linear with input size, O(n log n) rep-resents a problem where complexity is mostly linear with a logarithmic element, and O(n2)represents a problem where execution time has a quadratic relationship with input size.The complexity of the control solutions explored in this chapter can be described by one ofthese big O notations, and a figure comparing these notations is shown in figure 2.9, wherecomplexity is a function of input size for each big O representation.0 100 200 300 400 500 600 700 800 900 1000Input size (n)101102103104105106Compexity O(n)O(n)O(nlogn)O(n2)Figure 2.9: Semilog plot comparing the computational complexity of O(n), O(n log n), and O(n2) withinput size.212.4. Control of a Hybrid Energy Storage System2.4.1 Rule-Based ControlRule-based control is one where an algorithm has a specific output for given inputsand approaches are either “deterministic or fuzzy type” [10]. Rule-based control “offer(s)feasibility of real-time implementation as well as robustness to the system uncertainties”[8]. An advantage of this control option is the requirement for little computational effort,making it an appealing choice for low-cost consumer vehicles. Rule-based control is usedwith the NPC converter, as explored by [17] which consisted of three control loops. Thefunction of the 3 control loops are as follows: the outer most loop controls DC-link energyused to optimize the energy in the SCs based on the vehicles speed; the middle loop createsa reference DC-link voltage based on a reference battery current; and the inner loop createsan optimized DC-link current profile, which indirectly controls the SoC of the SCs. Thisprovides effective control, however it is unique to the NPC topology used in this instance.Another method of rule-based control is to maintain a constant battery current andvary the SCs’ current based on the vehicles needs. This was implemented with a cascadedconverter HESS comprised of SCs and lithium-ion batteries (figure 2.5 (f)), and the parallelconverter with fuel cells (FCs) and SCs (figure 2.5 (g)) [15]. In this configuration a substantialsized SC pack is needed to compensate for the large amount of energy needed to fullyaccelerate or decelerate in order to keep the battery voltage constant.In another rule-based control strategy, a two part framework is used to control an HESScomprised of batteries and SCs, developed by [13]. The first part considers load elementssuch as vehicle dynamics, motor characteristics, and regenerative braking to generate a SCSoC reference point. The goal of which is to predict future driving profiles and optimize the222.4. Control of a Hybrid Energy Storage SystemSC bank to hand future power loads. The second framework optimizes power flow in theHESS by minimizing the magnitude and variation of battery power and power loss.Rule-based control is also implemented by authors [11], where control is designed arounda minimally sized SC bank. The control algorithm consists of two separate layers: thetop layer “determines the real-time optimal energy to be shared between the two energysources, while maintaining maximum EV battery SoC, and adjust SC SoC to react to futurepower requests” [11], the lower layer regulates power between the two energy sources andthe inverter, based on the power flow determined by the upper layer. The author claims aRMS battery current reduction of 19% and a max peak battery current reduction of 50%with an SC bank with an energy capacity of 64.7kJ or 18.0Wh.Another example of rule-based control is done by [19], where an algorithm is developedaround a flow chart considering power required by the electric motor, voltage of the SCs,and power available from the SCs. The output is one of four scenarios which either chargethe SCs, discharge the SCs with assistance from the battery, use the SCs to fully supply theDC motors current needs, or disconnect the SCs from the DC bus. This HESS uses a largeSC bank with a capacity of 176kJ or 48.9Wh which reduces battery current by 56.25%.Authors [20] develop a fully decoupled HESS for use in trains with FCs, batteries, andSCs and use simple feedback loops to determine the power flow from each component. Thefeedback loop uses PI controllers to manage power flow, which is determined from a operationmode controller. This controller operates in three modes including discharge mode, chargemode, and fast charge mode. Since FCs have a very slow response, the battery and SCsabsorb driving transients.Rule-based control is effective control for an HESS using low computational power; typi-232.4. Control of a Hybrid Energy Storage Systemcally rule-based control requires comparisons or subtractions to determine error and progressthrough an algorithm. The complexity for a subtraction operations is linear or O(n). Rule-based control is simple to implement and execute but may not produce the best outcome dueto unpredictable nature of a vehicles driving profile and limitations in algorithm flexibilityas compared to optimization-based control.2.4.2 Optimization-Based ControlOptimization-based control is typically more computational taxing than rule-based con-trol. Previously, optimization-based control may not have been feasible, but as computa-tional hardware has grown more powerful and cheaper, optimization-based control is becom-ing more researched. Optimization systems are constructed by developing control around aknown drive cycle and developing or optimising the control for the given set of data. Thedriving cycle is chosen such that it will represent the expected range of environments a ve-hicle is subject too. A number of various control strategies for optimization-based controlhave been explored, one example is the wavelet-fuzzy power allocation strategy to controlan HESS, implemented by [21]. This control consists of two parts; the first part, wavelet,refers to wavelet transform which is a relatively new tool in signal processing and the novelidea presented by the authors. Wavelet transform is the sum of signals that are scaled andshifted versions of the mother wavelet over time, this paper uses a method known as dis-crete wavelet transform and the Mallat algorithm to split the load power between SCs andbatteries. Secondary control is a rule-based fuzzy logic controller. The authors claim thatthis control is not as computationally heavy as previous optimization control strategies dueto the use of the Mallat algorithm.242.4. Control of a Hybrid Energy Storage SystemAnother optimization-based control method is a passivity-based controller (PBC). A PBCcontroller is a state space controller that achieves closed loop stability “with respect to thestorage function which has a minimum at the desired equilibrium point” [22]. While thismethod has been implemented in the past, the authors use a Kalman filter to reduce thenumber of required sensors, simplifying the system.Chaos-synchronization is a more elaborate method of optimization-based control, em-ployed by [23]. The control is developed by modelling the SC, battery, and converter asa state space system, then a robust adaptive controller is developed which automaticallyregulates and synchronizes the chaotic systems. The systems are referred to as chaotic dueto the fact that future driving patterns are not cyclic and follow no predictive behaviour.The above optimization-based methods use state space systems to provide control. BigO notation can be used to estimate the computational effort of these systems: complexity ofmatrix multiplication between matrices of size n×m and n×p is O(nmp) and the complexityof matrix addition is O(n) where n is the number of elements in the matrices. So for a statespace system such as:x˙(t)n×1 = An×nx(t)n×1 +Bn×mu(t)m×1y(t)p×1 = Cp×nx(t)n×1 +Dp×mu(t)m×1(2.5)where n is the number of states, m is the number of inputs, and p is the number of outputs,252.4. Control of a Hybrid Energy Storage Systemthe complexity is represented as:O(n2) +O(n) +O(nm) +O(pn) +O(p) +O(pm)=O(n2 + n+ nm+ pn+ p+ pm)=O(n2 + nm+ pn+ pm)≈O(4n2)=O(n2)(2.6)Here computation time is dominated by matrix multiplication, and because big O notation isupper-bound, matrix addition can be neglected. Similarly, additional quadratic terms (nm,pn, and pm) can be dropped as they are all of the same power as n2. A state space controllerlike the one used by [23] will then have a complexity of O(n2).A common implementation of optimization-based control is to use a LPF to separatepower components of an HESS. Figure 2.8 shows an optimal outcome of LPF based controllike that shown in figure 2.10, where a filter is used to generate current profiles for the high(SC) and low (battery) power dense components of the HESS, and outer-loop control isused to ensure SoC and currents of each source stay within their limits. Examples of LPFcontrol for a HESS are demonstrated by [2, 8, 14–16]. The corner frequency of the LPF maybe fixed or made variable to increase effectiveness. The authors of [16] and [2] determinetwo corner frequencies for either urban or highway driving, and select either based on thevehicle’s speed. A fully variable corner frequency is done by [14] where the authors developan algorithm to determine the optimal corner frequency based on driving conditions of avehicle. The control diagram for LPF control similar to that used by [2] is shown in figure262.4. Control of a Hybrid Energy Storage System2.10. This control is for a fully-decoupled system; the top control elements include the LPFBattery current charge loopLow pass lterIscIBatteryILoadBattery SOC-+++-+CornerFrequencyAlgorithmFigure 2.10: Control diagram of HESS control using an LPF with a variable corner frequency to directedload current frequency components to the batteries or SCs. Battery and SC current signalsare further modified by the battery current charge loop control to consider hardwareconstraints (cited from [2]).which splits the current reference between the two sources and the bottom control adjuststhe reference to ensure that both currents are within their limits. The corner frequency ofthe LPF can be optimized for a given driving profile or based on the impedances of the SCbank, converter, and battery. Authors [8, 13] developed mathematical models of the lithium-ion battery banks, SCs, and DC/DC converter and analysing their respective impedancesand determine the optimal corner frequency for LPF control. The transfer function for thebattery and SC bank is shown previously in equations 2.2 and 2.3. The final component tomodel is the DC/DC converter that links the SCs to the DC bus. These converters operate intwo different modes, buck or boost, depending on the direction of power flow in the HESS,as such two different models are required as shown in figures 2.11 and 2.12. The inputimpedance of these two models is represented by the equations 2.7 and 2.8 [8].Zboostin (s) =CCSCLs2 + (RSC + r1)CCSCs+ [CSC(1−D)2 + C]CCSCs(2.7)Zbuckin (s) =CSCLs2 + (RSC+r1)CSCs+ 1CCSCLs3 + (RSC + r1)CSCCs2 + (CSCD2 + C)s(2.8)272.4. Control of a Hybrid Energy Storage SystemCSCRSCL r1DiLDVLoadC VLoad ILoadiLFigure 2.11: Average model of a boost converter.CSCRSCCVLoadDVLoadILoad DiLL r1iLFigure 2.12: Average model of a buck converter.The above mathematical models can then be used with the previous transfer functionsrepresented by equations 2.2 and 2.3 to analyse the frequency response of the batteries andthe SC bank/DC/DC converter pair. Once the bode plots are generated, an optimal cornerfrequency for the LPF can be selected for the control. LPFs can be categorized as eitherdigital or analog, and digital filters are either finite impulse response (FIR) or infinite impulseresponse (IIR) filters. IIR filters offer flexibility in design with the presence of both poles andzeros in the mathematical formulation. Filters can be tested in simulation or with prototypesto determine their effectiveness and stability. Most filters are based on the FFT operation,which has a complexity of O(n log n).282.4. Control of a Hybrid Energy Storage System2.4.3 Neural Network ControlAnother form of optimization-based control is the NN. In recent years NNs have made alarger presence in control. As discussed in section 1.3, NN control algorithm loosely mimicsneurons in the brain, where a signal is taken from inputs and propagates through one ormore layers of neurons before generating an output signal. A NN developed by [18] wherethe current is controlled by a multi source inverter, similar to the integrated HESS topologydemonstrated by [17]. The NN is trained with supervised learning and optimal duty cyclesfor a given driving profile are obtained by a ”dynamic programming algorithm” [18]. Thestructure of the NN used has 7 inputs, 1 output, and two hidden layers, the first with 10neurons and the second with 5. The optimization problem is discovered with the formulae2.9 - 2.11.f1(k) = (i2bat(k)Rbat + i2UC(k)RUC)/Pmax (2.9)f2(k) = i2bat/i2bat,max (2.10)f3(k) = (ibat(k)− ibat(k − 1))2/(∆i2bat,max) (2.11)Here RUC is the internal resistance of the UCs, Rbat is the internal resistance of the battery,iUC is the UC’s current, and ibat is the batteries’ current. A weighted sum of equations 2.9- 2.11 is applied to the optimization problem, where the weights were chosen so that thebatteries current magnitude and fluctuations will have the biggest influence. The resultingequation is 2.12.N∑k=1Af1(k) +Bf2(k) + Cf3(k) (2.12)292.4. Control of a Hybrid Energy Storage SystemUsing the optimal driving profile generated, the NN weights are then trained using the scaledgradient method and the performance is measured by the mean squared error. The algorithmis left to run until the error is within specified acceptable margins, then the NN weights aresaved to by used by the NN in practice. The authors claim a RMS battery current reductionof 50% with a SC bank having an energy capacity of 1440kJ or 400Wh.Authors [24] develop NN control using a Hammerstein-type neural-network (HNN). Thisis implemented to control power flow in a large scale electrical grid HESS rather than a EV.A Hammerstein-type neural-network is a network “which formulates the Hammerstein modelwith a non-linear static gain in cascade with a linear dynamic block” [24]. This paper usesFCs and SCs to increase stability with load frequency control for a multi-area interconnectedpower system. The block diagram of a Hammerstein-type NN is shown in figure 2.13. This•1•n•3•2u(k)γ1γnγ3γ2∑x(k)z-1 z-1z-1ß0 ßnß2ß1∑z-1z-1z-1an a1a2y(k)Figure 2.13: Structure of the Hammerstein-type neural network.structure is ”composed of a single dynamic node with two tapped delay lines, and these delaylines form the non-linear static element and linear dynamic element” [24]. In this scenario,the weights are trained online through reinforced learning based on a sample power profile302.4. Control of a Hybrid Energy Storage Systemprovided by the controller. The output of the HNN can be represented by the formula 2.13,yˆ(k) =− αˆ1(ˆy)(k − 1)− αˆ2yˆ(k − 2)− ...− αˆnα yˆ(k − nα)+ β0xˆ(k) + βˆ1xˆ(k − 1) + ...+ βˆnβ xˆ(k − nβ)=−nα∑i=1αˆiyˆ(k − i) +nβ∑j=0βˆjxˆ(k − j)(2.13)Where the hidden layer is represented by the equation 2.14 [24].xˆ(k) =nγ∑l=1γˆlul(k) (2.14)The negative gradient method is applied for adjusting the HNN weights. This exampledemonstrates the improvements available from a NN over traditional PID control. This NNdiffers from [18] in that no optimal duty cycle is needed to achieve the proper output, insteadthe NN continually improves with use. This makes the control easier to implement, and moreadaptable to various scenarios.NNs are a compelling solution to HESS control because of their ability to generate non-linear outputs. This makes them ideal to find a solution to the abstract inputs a drivingprofile has. The big O notation can be used to determine the complexity of a NN by knowingthe number of layers and their sizes. A 3 layer NN with n inputs, m neurons in the firstlayer, o neurons in the second layer, and p outputs can be represented as follows:y(t)p×1 = Θ3(p×o)(Θ2(o×m)(Θ1(m×n+1)xn+1×1)+ 10,0)+ 10,0 (2.15)312.5. Summarywhere +10,0 represents an inserted bias unit. The complexity of this NN is then:O(m(n+ 1) +O(o(m+ 1)) +O(p(o+ 1))=O(m(n+ 1) + o(m+ 1) + p(o+ 1))≈O(3n2)=O(n2)(2.16)Similar to state-space controllers, the complexity of a NN is O(n2).2.5 SummaryThis chapter reviewed the work of other researchers’ HESS topologies and control tech-niques and discussed their respective advantages and disadvantages, including computationaleffort. Each work offers effective control of an HESS, however some are more suited to larger,more robust systems and some smaller, more practical systems. This thesis aims to explorethe potential of a low cost HESS for use in consumer vehicles. The system most suited forthis situation is that of an active, partially decouple topology, where the battery is directlyconnected to the DC bus of the vehicle, and the capacitor’s power contribution is controlledvia a bi-directional DC/DC converter. Having a larger capacitor bank adds weight and costto a vehicle, so it is more practical to design a smaller SC bank and develop effective con-trol to utilize the bank. Optimization-based control using a NN offers effective control andallows the opportunity to be trained on unique driving profiles. This combination of HESStopology and control lowers battery stress and extends the battery life of an EV, ultimatelyincreasing its value.32Chapter 3Neural Network Control and GeneticAlgorithm3.1 OverviewThis section discusses the control developed for an active partially-decoupled HESS.Section 3.1 presents the parameters of the vehicle and elements of the HESS of which theNN will be developed around. Section 3.3 discusses the inputs to the NN, the numberof layers and the number of neurons per layer (known as hyper-parameters) of the NN,and the construction of the NN. Section 3.4 discusses the control that considers SC andconverter limits and conditions the signal produced by the NN. Section 3.5 discusses thegenetic algorithm used to train the NN and the effect various parameters of the geneticalgorithm have on the learning rate.3.2 Modelled Hybrid Energy Storage System ParametersThe modelled HESS is based off a typical small economic 4 seat EV. The parametersof the vehicle are listed in table 3.1. The HESS topology is a partially decoupled, wherethe SC bank is connected to the DC bus via a bi-directional boost converter. The SCBank is assembled from 100 series connected, 310F SCs like that of Maxwell’s® BCAP0310P270 T10. The result is an SC bank capacity of 3.1F and a total weight of 6kg. The333.2. Modelled Hybrid Energy Storage System ParametersTable 3.1: Parameters of the EV and modelled HESS.Component ValueVehicle Weight 912kgLi-ion Battery Pack Capacity 24kWhLi-ion Battery Pack Nominal Voltage 350VSC Bank Capacitance 3.1FSC Bank Weight 6kgSC Bank Rated Voltage 270VSC Bank Voltage Operating Range 50V − 250VSC Bank Energy Capacity 94.5kJ or 26.3WhSC Bank Equivalent DC Series Resistance 220mΩConverter Efficiency 95%Converter Limit 15kW or 60A at 250Vdiagram of the HESS is shown in figure 3.1. Load current is determined by demands fromthe motor and inverter, the battery supplies the majority of the load current with the SCbank supplementing current via the converter. The control developed in this thesis aimsDC/DCConverterC+-Battery MotorInverterDC-BusIref (x)IC IrefIbatIloadFigure 3.1: Block diagram of the modelled HESS used to develop NN based controlto minimize the RMS current of the batteries and is optimized around the FTP-72 drivingprofile, also called the urban dynamometer driving schedule (UDDS) profile, like that of[6]. The control must also ensure that the components of the HESS operate within theirrespective power limits. This includes preventing over or under charging the SCs, and343.2. Modelled Hybrid Energy Storage System Parametersensuring the current through the bi-directional converter does not exceed its 60A limit. Thecurrent limit for the converter is determined by two methods, the first is by determining thetime taken to fully charge or discharge the SC bank. The intention is that the SC bank andconverter be sized such that SC bank will be fully discharged when accelerating the vehiclefrom 0kph to 100kph in a reasonable time. The formula is shown below which calculatesthe total available charge of the SC bank and determines the time required to discharge orcharge the bank based on the maximum available current:t =CSC bank ·∆VSC bankIlimit(3.1)The results are shown in figure 3.2 The other method is to train the NN on the UDDSprofile with various converter limits and observe the point at which increasing the converterlimit offers a diminishing increases in HESS effectiveness. The results of this method areshown in figure 3.3. Figure 3.3 shows that increasing the converter limit past approximately40A offers little improvement in performance when trained on the FTP-72 driving profile,however 40A offers a discharge cycle time of 15.5s. A converter limit of 60A was chosen whichoffers a discharge cycle time of 10.3s; this is a more reasonable 0kph to 100kph time for apractical vehicle. The parameters for the vehicle and HESS system listed in table 3.1 areused to develop the NN control in the following sections. The HESS system and control aremodelled in Simulink, a block diagram of the average model of the converter that considersefficiency losses is shown in figure 3.4. This control circuit models losses in the converter bylimiting output current to 95% depending on the direction of current flow.353.3. Neural Network Parameters10 20 30 40 50 60 70 80Converter Current Limit (A)020406080100120140Charge/Discharge Time of SC Bank (s)Figure 3.2: Time taken of a 3.1F SC bank to discharge from 250V to 50V based on converter limit.3.3 Neural Network Parameters3.3.1 Inputs to the Neural NetworkThe driving current profile of an EV is determined by unrelated external conditions suchas traffic speed, road incline, wind speed, and vehicle weight; and the control for a HESSsystem used by an EV must consider the influences of all these conditions. As previouslydiscussed, a trained NN offers the ability to find patterns in the inputs to optimize thereference current for future inputs. If the full driving profile is known, the current profilesof the EV’s battery and SC bank can be optimized by a dynamic programming algorithmknowing future demands and the NN trained by supervised learning, like that done by [18].Another option is to use reinforced learning and allow a genetic algorithm to optimize theweights of a NN. Reinforced learning introduces the opportunity to find an optimal solution363.3. Neural Network Parameters10 20 30 40 50 60 70 80Converter Current Limit (A)4681012141618Percent Reduction in RMS Current (%)Figure 3.3: Percent reduction in RMS battery current of a trained NN controlling a converter of rangingcurrent limits.that may not be apparent when designing a current profile for the HESS externally.The NN used in the HESS control takes eight inputs to produce a reference signal. TheUDDS driving profile is used to train and test the NN control for the HESS. The NationalRenewable Energy Laboratory’s advanced vehicle simulator (ADVISOR) uses the UDDSdriving profile, along with the vehicle parameters listed in table 3.1 to generate the inputprofiles used by the NN. ADVISOR generates three separate profiles: load current, vehiclespeed, and motor torque. Other NN inputs are generated in Simulink from the ADVISORprofiles, these inputs include vehicle acceleration, a moving average of speed, and localmaximum of speed. The NN also monitors the voltage of the capacitor bank and the finalinput is a bias unit with a constant value of one. The goal of the inputs, apart from thebias unit, is to provide as much information of the vehicles current situation that would373.3. Neural Network ParametersICapsIDC BusIref (z) z -1z -1Vbat(z)Vcaps(z)< 0≥ 01 - ɳConverter -+11 - ɳConverter -+1Figure 3.4: Block diagram of the modelled converter which considers operating efficiency.affect or be affected by the current demanded by the motor and the SoC of the capacitors,whether that be an urban environment or a highway. The NN can recognize patterns indriving profiles from these inputs and adjust the reference HESS current appropriately. Thetorque, and acceleration profiles are shown in figure 3.6, and the load current profile andthe resulting capacitor voltage profiles from a trained NN are shown in figure 3.7. Thespeed, moving average of speed, and local max of speed profiles are shown in figure 3.5.The block diagram of the control used to generate the moving average and local max areshown in figure 3.8. The window length for the moving average was chosen to be 1200s383.3. Neural Network Parametersby analysing the speed profile and adjusting so that the moving average profile depicts thedriving circumstances the vehicle is currently in, such as highway driving or city traffic. Thegoal is to provide a fast enough response to adjust for a current driving environment, yetwith enough memory to provide a useful input of the vehicles current driving environment.The local max is generated by analysing the acceleration of the vehicle and storing themaximum speed when acceleration changes from positive to negative. The objective for alocal maximum input is that in urban, stop and go driving environments, the vehicle’s topspeed will remain relatively the same, as the speed limit of a section of road will likely beconsistent. The effectiveness of each input can be analysed after the NN weights have0 200 400 600 800 1000 1200 1400Simulation Time (s)051015202530Speed (m/s)Vehicle SpeedLocal Max of SpeedMoving Average of SpeedFigure 3.5: UDDS profiles for speed, moving average, and local max, used as inputs to the NN.been trained by the genetic algorithm. This is done by evaluating the RMS value of the firstlayer weights corresponding to each input. Consider figure 1.4, to evaluate the effectivenessof input x1, the RMS value of all the weights represented by all the arrows exiting x1 would393.3. Neural Network Parameters0 200 400 600 800 1000 1200 1400Simulation Time (s)-2-1.5-1-0.500.511.52Acceleration (m/s2 )-100-80-60-40-20020406080100Torque (N/m)AccelerationTorqueFigure 3.6: UDDS profiles for torque and acceleration, used as inputs to the NN.0 200 400 600 800 1000 1200 1400Simulation Time (s)-40-20020406080100120Current (A)050100150200250300Voltage (V)Load CurrentCapacitor VoltageFigure 3.7: UDDS profiles for motor current and capacitor voltage, used as inputs to the NN.be evaluated. Figure 3.9 shows RMS value of the layer 1 weights of a NN trained on theUDDS profile. The effectiveness of each input can be compared to one another to determine403.3. Neural Network Parameters≤ 0.5≥ 0.5xacceleration(z)z -1SetResetQSet-ResetFlip-FlopHoldu yResetxspeed(z)xlocal max(z)Moving Average xmoving average(z)Figure 3.8: Control diagram which produces the local max and average profiles used as inputs to the NN.which inputs provide more influence on the performance of the NN compared to others.BiasLoad CurrentSC VoltageSpeedAccelerationLocal Max of SpeedTorqueMoving Average of SpeedFirst Layer Inputs of NN00.020.040.060.080.10.120.14RMS Value of Weight VectorFigure 3.9: RMS value of the first layer weight vectors applied to the input signals of the NN, used toevaluate the effectiveness of a particular input against other inputs.3.3.2 Number of Layers and Neurons of Neural NetworkTo develop a NN, the number of layers and number of neurons (known as hyper-parameters)are chosen such that an effective output will be produced yet not over designed such that413.3. Neural Network Parametersthe NN will not require an unnecessary amount of computational hardware. Before pro-ceeding with development of NN control for an HESS, it was necessary to prove that a NNcould provide effective control for an HESS. To prove the functionality of NN control for anHESS, a network saturated with neurons was built; this was an over designed solution thatrequired more computational power than necessary, however it would be a starting point tolater optimize the NN for HESS control. Initially a four layer network was chosen, with 50neurons per each of the three hidden layers. The genetic algorithm was allowed to train thenetwork with the results shown in figure 3.10 and a minimum battery RMS current value of19.82A. Firstly the number of layers were reduced to observe an impact on HESS perfor-0 20 40 60 80 100 120 140 160 180 200Generation Number19.52020.52121.52222.52323.5Battery RMS Current (A)Figure 3.10: Genetic algorithm progression of a 4 layer network with hidden layers consisting of 50 neuronsper layer.mance. A layer was removed from the four layer NN and the genetic algorithm was againallowed to train the NN. The trained network offered a battery RMS current of 19.01A andthe progression of the algorithm is shown in 3.11. This was repeated again with two layers,423.3. Neural Network Parameters0 20 40 60 80 100 120 140 160 180 200Generation Number1919.52020.52121.52222.52323.524Battery RMS Current (A)Figure 3.11: Genetic algorithm progression of a 3 layer network with hidden layers consisting of 50 neuronsper layer.with the results shown in figure 3.12 and a minimum battery RMS current value of 19.80A.From this, it is determined that a three layer network had enough layers to provide optimalcontrol for the HESS. To further optimize the NN the number of neurons needed for eachlayer is minimized. To determine this, the genetic algorithm repetitively trained the NN forvarious values of neurons for the first and second hidden layers. The value for the numberof neurons is listed in table 3.2 and the parameters of the genetic algorithm are the same asthat listed in table 3.3 with the exception of the simulation time step, which was changedto 1s in the interest of achieving quicker simulations. The genetic algorithm trained everyTable 3.2: Neuron combinations of the hidden layers of a 3 layer NN.layer 2 neurons: 4 5 6 7 8 9 10 12 15 20 25 30layer 3 neurons: 4 5 6 7 8 9 10 12 15 20 25 30combination of neurons shown in table 3.2 and the results of the minimum battery RMS433.3. Neural Network Parameters0 20 40 60 80 100 120 140 160 180 200Generation Number19.619.82020.220.420.620.82121.221.421.6Battery RMS Current (A)Figure 3.12: Genetic algorithm progression of a 2 layer network with hidden layers consisting of 50 neuronsper layer.current are shown in figure 3.13. The RMS current data represented in figure 3.13 has a1801910Battery RMS Current (A)Neurons in 2nd Hidden Layer2020 0521Neurons in 1st Hidden Layer10152030 2530Figure 3.13: Battery RMS current of a trained NN with various combinations of hidden layer neuronsbased on table 3.2.443.3. Neural Network Parametersminima when the number of neurons in the first layer is 12 and the number of neurons in thesecond layer is 9. The combination was chosen such that the number of neurons per layer isthe lowest without effecting the RMS current reduction of the trained network.3.3.3 Construction of Neural Network1x7x2x11a1(1)a2(1)a12(1)hΘ(x)Input Layer Layer 2 Output Layer Output1a1(2)a2(2)a9(2)Layer 3a1(3)(a)xmoving avg of speed(z)xlocal max of speed(z)xvehicle torque(z)xvehicle acceleration(z)xvehicle speed(z)xV capacitors(z)xI load(z)bias unit (1)(8 x 1)Θ1(12 x 8)tanh(x)(12 x 1)bias unit (1)(13 x 1)Θ2(9 x 13)(9 x 1)bias unit (1)(10 x 1)tanh(x)Θ3(1 x 10)saturation(x) xref(z)(b)Figure 3.14: NN simulated using Simulink and used as control. (a) Architecture of the NN constructed inSimulink with 12 neurons in the first hidden layer and 9 in the second. (b) The block diagramof the same NN implemented in Simulink.453.4. External ControlThe NN and control for the HESS is developed in Simulink. Figure 3.14 shows thestructure of the NN. Each signal input to the NN has varying magnitude and DC bias, toensure that the NN is not dominated by a single input, the inputs are normalized by acontrol circuit like that shown in figure 3.15. Signals from all the inputs pass through theNN as described in section 1.3. The inputs are normalized then made into a vector usinga multiplexor, then multiplied by the matrix of layer 1 weights. The output is a columnvector which is conditioned by the hyperbolic tan block. A bias unit is added to the vectorand the process repeats for the remaining layers. The output of the NN is not conditionedby a hyperbolic tan block, but rather limited by a saturation block. The NN generates anunconditioned reference current signal for the converter of the HESS which is later modifiedby control as described in section 3.4.-+x(z)xstandard deviationxmeanx(z)Figure 3.15: Control diagram used to normalize an input signal to the NN.3.4 External ControlThe NN of provides the optimal reference current for the converter to source from theSCs, however the NN does not consider the limits of the hardware of the HESS. To ensurethe converter is not damaged due to over power and the SCs are not damaged due to overvoltage situations, additional control is needed to condition the current reference signal463.4. External Control>Vcaps maxVcaps(z) Vgain max-+-++VbufferVcaps max<Vcaps min -++VbufferVcaps minVgain minVbuffer(z)Vbat(z)Ilim converter-Ilim converterminmaxIref (z)-+ Iref (z)SC Voltage Limit ControlDC/DC Converter Current Limit ControlFigure 3.16: Diagrams of control used to ensure the SC bank does not exceed its voltage operation limits(top) and diagram of control used to ensure that the converter does not exceed its currentoperation limits (bottom).provided from the NN to the converter. The reference current generated by the NN is thecurrent requested from the converter on the side that is connected to the batteries which isthe high voltage side of the converter. This is necessary so that the NN’s generated referencesignal is directly controlling the current that supplements the batteries, if the reference were473.4. External Controlapplied to the low side, the current injected into the DC bus of the HESS would be scaledbased on the voltage of the SCs and the battery. The converter is rated at a current of 60A,however this is from the low side of the converter, this means the available current to themotor of the vehicle depends on the SoC of the SCs and is determined by formula 3.2.Ihigh side =(VbattVSC bank)Ilow side (3.2)The first consideration of the external control is to ensure that the converter operates withinits current limits, which are ±60A on the low side. Controlling the current is not as simple asadding a saturation block to the output of the NN, because with a fixed current limit on thelow side of the converter, the current limit of the high side of the converter changes throughthe operating range of the SC bank. Instead the limits must change as the SC bank’s SoCchanges. The DC/DC converter current limit control of figure 3.16 shows the control for theconverter current limits. The minimum converter current is determined in formula 3.3 andthe maximum converter current is determined by the formula 3.4.Imin =(VbattVSC bank)Iconverter limit (3.3)Imax = −(VbattVSC bank)Iconverter limit (3.4)While the HESS is operating, the reference signal will be the minimum of either equation 3.3or the NN’s reference current, or the maximum of either equation 3.4 or the NN’s referencecurrent. This ensures the current from the low side of the converter does not exceed ±60A.To ensure that the SCs operate within their limits, SC voltage limit control block shown483.4. External Controlin figure 3.16 is implemented. This is a proportional control meant to counter the referencegenerated from the NN. When the voltage of the SCs reach within 10V of their specified limit,the control is active, and a buffer signal whose polarity is opposite the current reference signalis added to the NN’s signal to negate it and prevent further current transfer in a directionthat would damage the SCs. The formula showing this control when the SCs approach theirminimum voltage is shown in equation 3.5 and the formula showing the control when theSCs approach their maximum voltage is shown in equation 3.6. Here, Vmax and Vmin are thespecified limits of the SC bank, Vbuffer is the 10V buffer where the control becomes active,and Vgain is the gain applied to the signal so that it will match the maximum of the currentreference signal when the SC bank’s voltage reaches its limit.F (z)max = (Vmin + VBuffer − V (z)SC)VgainVgain =(VSC maxVbatt)(IlimitVbuffer) (3.5)F (z)min = (V (z)SC + VBuffer − Vmax)VgainVgain =(VSC minVbatt)(IlimitVbuffer) (3.6)The gain values differ because, similar to the current limit control, the current referencesignal changes as the SoC of the SC bank changes. The current and voltage control ensurethat the HESS operates within its limits, and no damage occurs to the hardware of theHESS. The NN is then allowed to learn and generate a signal without a need to considerthese constraints.493.5. Genetic Algorithm3.5 Genetic AlgorithmFor the NN to have an effective output for the HESS, it must first be trained. Training isthe process of determining a weighting value, referred to as Θ, to each input of each neuronas discussed in section 1.4. The process used to train the NN used in the control for thisHESS is reinforced learning which makes use of a genetic algorithm. A previous NN controluses supervised learning, where an optimal SC current profile must first be generated by a”dynamic programming algorithm” [18] and the NN is trained so that it’s output matchesthat of this algorithm’s. In this scenario, the effectiveness of the NN is limited by that ofthe algorithm, and care must be made to ensure the algorithm produces a SC profile thatis most effective at reducing the battery RMS current. Alternatively, reinforced learningdoes not require a predetermined SC current waveform for the NN to learn from. Instead agenetic algorithm will continue to adjust the weights of the NN to produce an SC currentprofile that reduces the batteries’ delivered RMS current. This presents the opportunity forthe NN to find solutions that may not be generated by an external algorithm. While thesystem model and control is constructed in Simulink, Matlab code is used to implement thegenetic algorithm. The code used for this thesis is shown in appendix section B and theparameters initialized by the genetic algorithm are listed in table 3.3.3.5.1 Genetic Algorithm ParametersThe rate at which a NN is trained depends on the parameters of the genetic algorithm andthe properties of the NN. The input layer size is determined by the number of inputs to theNN, as discussed in section 3.3.1. The hidden layer sizes are the size of the hidden layers of the503.5. Genetic AlgorithmTable 3.3: Parameters initialized at the beginning of the genetic algorithm.Parameter Parameter Name ValueInput Layer Size input layer size 7Hidden Layer 1 Size hidden layer 1 size 12Hidden Layer 2 Size hidden layer 2 size 9Output Layer Size output layer size 1Population Size population size 20Simulation Time Step T step 0.1sConverter Limit converter limit 60AConverter Efficiency converter efficiency 95%Number of Parents number of parents 2Mutation Rate mutation rate 0.3%Randomization Factor epsilon ini 0.15NN, and the size of these layers are discussed in section 3.3.2. The output layer size for thisNN is 1, and is the reference current to be used by the converter. This output signal is laterconditioned by additional control to ensure that the capacitor and converter do not exceedtheir respective operation parameters as discussed in section 3.4. Population size refers tothe number of vectors of NN weights that are to be run for each generation (one weight vectorincludes all the genes necessary for the NN to operate). This value needs to be large enoughto allow sufficient variation among members of the population, however not to large as tounnecessarily slow the progression of the genetic algorithm. Figure 3.17 shows the differencea population size will make on the learning rate of the genetic algorithm. Simulation timestep is the incremental step made by Simulink when the simulation is running. A time stepof 0.1 second is used for the genetic algorithm which translates to a simulation samplingfrequency of 10Hz. According to the Nyquist Sampling Theorem, if a sampling rate of 10Hzis used, a signal below 5Hz can be accurately interpreted without loss of information. Figure3.18 shows an RMS frequency analysis of the input signals generated from ADVISOR, abovea frequency 1Hz the RMS value of the signals are near zero and contain no useful information.513.5. Genetic Algorithm0 20 40 60 80 100 120 140 160 180 200Generation Number18192021222324Battery RMS Current40 Members20 Members10 MembersFigure 3.17: The effect of various population sizes on the progression of a genetic algorithm.This justifies a time step of 0.1Hz as more than sufficient when running the simulation as0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5Frequency (Hz)024681012 x(f) rmsLoad CurrentSpeedTorqueFigure 3.18: An RMS frequency spectrum analysis of the NN input signals generated in ADVISOR.523.5. Genetic Algorithmdecreasing the time step does not provide any improvements in simulation quality. Theconverter limit is the hardware constraints of the converter, this value is discussed in section3.1. The converter efficiency is the model’s losses in the converter, this value is chosen tobe 95% which is the typical efficiency of a conventional buck/boost converter [13]. Thenumber of parents refer to the number of members of a population that will contribute genesto a new member. A value of 2 is chosen as higher values limit progression of the geneticalgorithm. If more parents are used to reproduce NN genes, poorly performing memberswill influence subsequent generations and inhibit improvements of the genetic algorithm.The mutation rate is the probability that a gene is mutated and the randomization factoris range that the new value can fall between. A higher mutation rate will allow the geneticalgorithm to train the NN more quickly, however the outcome of the genetic algorithm hasrelatively high fluctuations from one generation to the next. A lower mutation rate resultsin a slower progression of the genetic algorithm, but lower fluctuations in the result as thegenetic algorithm progresses. Figure 3.19 compares the progression of a genetic algorithmwith only the mutation rate changed. A mutation rate to low slows or even stops geneticalgorithm progression, while a mutation rate too high results highly varying generationprogression. The randomization factor is the range of values which new and mutated geneswill fall between. The inputs to the NN are normalized so the mean of each input is 0 andthe standard deviation is 1, and the output is scaled by 200. A randomization factor of 0.15allows a single input to the NN to saturate the output, if the weights associated with that533.5. Genetic Algorithm0 20 40 60 80 100 120 140 160 180 200Generation Number18.51919.52020.52121.52222.523Battery RMS Current (A)mutation rate 0.001mutation rate 0.003mutation rate 0.010mutation rate 0.030Figure 3.19: The effect of various mutation rates on the progression of a genetic algorithm.input are all approximately 0.15, as shown in equation 3.7.a(2)1 = g(Θ(1)10 x0 + Θ(1)11 x1 + ...+ Θ(1)17 x7)a(2)1 = g (0.15× 1.0 + 0.15× 1.0 + ...+ 0.15× 1.0)a(2)1 = 1.05(3.7)It is important that a poorly performing NN results in an undesirable output, this ensures apoor fitness is assigned to that member, and the member’s genes have a low chance of beingselected for subsequent generations.3.5.2 Genetic Algorithm OperationThis section describes the operation of the genetic algorithm. When training a NN’sweights, the genetic algorithm first loads the driving profiles created by the ADVISOR543.5. Genetic Algorithmprogram. Next the parameters of the NN, simulation, and genetic algorithm are initialized;these parameters are listed in table 3.3.The program either loads parameters from a previous run to continue to develop, or uses afunction to randomize the parameters of the NN based on the mutation rate specified shownin equation 3.8, where rand is a randomized matrix whose size is the size of the input layer bythe size of the output layer plus 1, and εinit is the randomization factor. The program usesthese matrices to build a population to run for the first generation of the genetic algorithm.Θn = 2Random(Lout×lin+1)εinitial − εinitial (3.8)The total parameters are deconstructed to form column vectors and are combined to form amatrix, where each column is a member of the total population for that generation. Whenone member of the population is to be simulated, the parameters are organized into Θ arraysfor each layer of the NN and the simulation is run. Information of each run is stored including:battery current, capacitor current, simulation time, the RMS value of battery current, andthe NN parameters of each layer. After each member of the population is run, a function usesthe battery RMS results to determine the fitness of each population member. The operationof the function is described by equation 3.9, where IˆRMS is a matrix of the normalized RMSbattery currents, iRMS min is the minimum RMS value of the current generation, iRMS maxis the maximum RMS value of the current generation, and Ifitness is a scaled and invertedIˆRMS matrix. This equation produces a higher fitness value from a lower RMS current value.IˆRMS =IRMS − iRMS miniRMS max − iRMS min (3.9)553.5. Genetic AlgorithmIfitness = (1− IˆRMS)100 (3.10)After the fitness of each population member has been determined, a function uses the fitnessvalues to create a matting pool where population members with a higher fitness occupy alarger portion of the mating pool. The mating pool is a matrix where columns are occupiedby the NN parameters, the number of columns occupied be each population member is thefitness value of that member determined by the previous generation, shown in equation 3.11.Here Θmating pool notates the mating pool and Θn(m×fitness) is a matrix comprised of identicalcolumns which are the NN parameters of length m of a specific member and the number ofcolumns is the fitness value of that member.Θmating pool = Θ1(m×fitness) + Θ2(m×fitness) + ...+ Θn(m×fitness) (3.11)Next, a function produces members for the next generation by selecting members of themating pool determined by the number of parents specified. Random genes of the NNparameters are combined to create a new population member for the next generation. Thisis described by equation 3.12 which shows the next generation of NN parameters, labelledΘ2, whose elements are comprised of elements of 3 parents, Θ1a, Θ1b , and Θ1c . The Θ columnsmultiply with columns containing 1s and 0s, which signify the random selection of genes toproduce the next generation’s weight matrix.563.6. SummaryΘ21aΘ22cΘ23b...Θ2nb =Θ11aΘ12aΘ13a...Θ1na100...0ᵀ+Θ11bΘ12bΘ13b...Θ1nb001...1ᵀ+Θ11cΘ12cΘ13c...Θ1nc010...0ᵀ(3.12)The final step is to mutate the new population members by changing random genes basedon the mutation rate and the randomization factor. The output of this function is similar tothat of equation 3.8, however a percentage of random genes are changed which is dictatedby the specified mutation rate. At this point, a new generation has been created and thenext generation is ready to be run. The simulation is re-run repeatedly until the RMS valueof battery current no longer improves. The NN parameters are then saved to be used in theNN control.3.6 SummaryThis chapter discusses the Simulink model of the HESS and how it is controlled by theNN and the external control. Once trained, the NN generates the fundamental currentreference for the converter to supplement battery current and lower the overall RMS currentsupplied by the battery. The external control is meant to adjust the reference currentsignal to ensure that the converter and capacitor bank do not exceed their respective ratedoperational limits. This chapter then discussed the process in which parameters for the NNand genetic algorithm are selected and compares the effect that various parameters have onthe learning rate and the operation of the NN.57Chapter 4Simulation Results4.1 OverviewThis chapter will cover the results of the genetic algorithm used to train a NN used ina HESS, and the effect of a NN controlled HESS on battery current. Section 4.2 coversin detail the advancements in the HESS’s effectiveness as the NN is trained on a standarddriving profile. Section 4.3 shows the battery and load current waveforms and discusses theeffectiveness of the HESS in reducing the battery’s RMS current. Section 4.4 compares theeffectiveness of the NN control with other works discussed in section 2.4.2 Progression of Genetic AlgorithmAs the genetic algorithm runs, the genes of the NN are adjusted to continually improvethe performance of the HESS, as described in section 3.5. The progression of the geneticalgorithm is shown in figure 4.1, where the battery RMS current of best performing memberof a population for each generation is recorded and plotted with its generation number.Initially, substantial improvements are made to the battery RMS current genes randomlyfind their way to positions providing great improvements. With a mutation rate of 0.01%,the majority of change within the NN comes from rearranging existing genes, which show584.3. Results of Neural Network Control for Hybrid Energy Storage Systemthe biggest improvements at the beginning of the simulation. Values of genes which aid theperformance of the NN are likely to propagate into subsequent generations, and genes thatdo not aid performance occupy a much smaller section of the mating pool. Later genes aremutated which provide slight improvements until the genes are optimized around the givenprofile.0 50 100 150 200 250 300 350 400Generation Number18.51919.52020.52121.52222.52323.5Battery RMS Current (A)Figure 4.1: Progression of the genetic algorithm initialized with the parameters of table 3.3 used to train a3 layer NN with 8 inputs, 12 neurons in the second layer, 9 neurons in the 3rd layer, and 1output.4.3 Results of Neural Network Control for Hybrid Energy StorageSystemThe objective of the trained NN and external control is to reduce the RMS currentthat the battery bank must source to the load. Lowering the RMS current of the batteryreduces the thermal stress of the battery and prolongs the battery’s life. Training the NN594.3. Results of Neural Network Control for Hybrid Energy Storage Systemconditions the NN to provide a reference signal based on patterns found in the input signals,these patterns may not be easily discovered by manually creating an HESS control. TheNN aims to optimize the SC bank’s voltage to prepare for unforeseen driving circumstancesand source the appropriate amount of current from the converter. The results of the HESScontrolled by the NN trained on the UDDS driving profile are shown in figure 4.2. TheNN is effective at lowering battery current, shown in green, throughout the simulation withthe exception of when the vehicle accelerates to highway speeds (approximately 27m/s or100km/h) between 200s and 300s. Figure A.1 in the appendix shows the speed of the vehiclethroughout the UDDS profile. A figure showing the results of an urban driving scenario0 200 400 600 800 1000 1200 1400Simulation Time (s)-40-20020406080100120Current (A)Load CurrentConverter CurrentBattery CurrentFigure 4.2: Graph showing the reduction a trained NN controlled HESS will have on an EVs load current.The load current represents the battery current of an EV with no added HESS, the batterycurrent represents the reduction in battery current an HESS can offer, and the convertercurrent is the difference of the two currents that is directed to the SC bank.between 600s and 1000s is show in figure 4.3, where the speed of the vehicle does not exceed15m/s or approximately 50km/h, as would be the case of a central urban environment. The604.3. Results of Neural Network Control for Hybrid Energy Storage System600 650 700 750 800 850 900 950 1000Simulation Time (s)-30-20-100102030405060Current (A)Load CurrentConverter CurrentBattery CurrentFigure 4.3: An urban driving section of the currents shown in figure 4.2.other objective of the control is to ensure that the SC bank does not exceed its safe workingvoltage parameters and the converter does not exceed its safe working current parameters.These current and voltage values are shown in figure 4.4 which plots the SC bank’s voltageand current values throughout the simulation. It is noted that the NN fully utilized thecurrent and voltage limits of the converter and SC bank without exceeding these limits. Theuse of this HESS control reduces the battery RMS current value from 23.48A to 19.92A onthe UDDS driving profile, this is a reduction of 15.15%. This reduces the heat produced bya battery cell from 171W to 123W based on equation 2.2 and battery values of table 2.1 likethat of [1]. This is a 28.1% reduction in heat energy. The addition of an HESS also reducespeak current draw from the battery, as would be the case when the vehicle is accelerating.Other measures of the HESS effectiveness are: maximum peak battery current reduction,where the maximum reduction of the batteries’ peak current is used to evaluate the control,614.3. Results of Neural Network Control for Hybrid Energy Storage System0 200 400 600 800 1000 1200 1400Simulation Time (s)-100-80-60-40-20020406080100Current (A)050100150200250300Voltage (V)Capacitor CurrentCapacitor VoltageFigure 4.4: Graph showing the voltages and current sourced from the SC bank of the HESS controlled by aNN trained on the UDDS profile.and peak to average power ratio (PAPR) over peak to average velocity ratio (PAVR). Thelater method is done by [14], where the performance of an HESS is evaluated by calculatingPAPR and normalizing it with respect to velocity by dividing the PAPR by PAVR, as shownin formula 4.1.PAPR/PAV R %reduction =(Ppeak/Pmeanvpeak/vmean)100 (4.1)An effective HESS will decrease peaks in current. The reduction in RMS current and thereduction of PAPR/PAV R ratios is used to determine the effectiveness of a NN that istrained on the UDDS profile and simulated over the UDDS profile. Three other drivingprofiles like those used by [14] are used to test the effectiveness of the NN trained on theUDDS profile. The other profiles include US06, a high speed driving profile, LA92, anotherlow speed urban driving profile, and the new European driving cycle (NEDC), a standardizeddriving test to test emissions of passenger vehicles at a variety of fixed speeds. The speed, load624.3. Results of Neural Network Control for Hybrid Energy Storage Systemcurrent, and torque waveforms of each profile are shown in appendix section A. Battery RMSreduction, max peak battery current reduction, and PAPR/PAV R reduction evaluate theeffectiveness of the HESS in various ways. Measuring battery RMS current reduction does notreflect the impact peak current draws will have on the HESS, because when optimizing theNN weights, the genetic algorithm favours a reduction in battery RMS over the whole drivingcycle. PAPR/PAV R reduction and maximum peak battery current reduction howeverreflects a reduction of peak battery currents and is less influenced by a reduction in batteryRMS current over a driving profile. Maximum peak battery current reduction is a value ofthe largest reduction between a peak in battery current before the addition of an HESS,and the battery current after the addition of an HESS. The PAPR/PAV R will be largelyinfluenced by one large peak power value, regardless of the length of the simulation. Theresults of the HESS over these driving profiles are listed in table 4.1. These results showTable 4.1: Percent RMS current reduction, percent max peak current reduction, and percentPAV R/PAPR reduction of a NN controlled HESS trained the UDDS profile.Driving ProfileRMS Current (A) RMS CurrentReduction (%)Peak Battery Current (A) Max Peak BatteryCurrent Reduction (%)Before After Before AfterUDDS 23.48 19.92 15.15 36.89 17.52 52.51US06 62.55 59.68 4.59 100.67 75.84 24.68LA92 30.68 27.62 9.99 43.74 20.34 53.50NEDC 22.07 21.24 3.78 33.43 20.50 38.68Driving ProfilePeak Power (kW) Average Power (kW) PAPRPAVRPAPR/PAVR PAPR/PAVRReduction (%)Before After Before After Before After Before AfterUDDS 42.93 41.85 5.91 6.01 7.26 6.96 2.90 2.51 2.40 4.15US06 106.10 106.09 17.76 17.82 5.97 5.95 1.68 3.56 3.54 0.32LA92 47.40 43.72 7.52 7.61 6.31 5.75 2.70 2.34 2.13 8.86NEDC 42.53 42.53 5.26 5.32 8.08 8.00 3.61 2.34 2.21 1.03a NN that is most effective at lowering both RMS currents and PAPR/PAV R ratios inthe low speed urban profiles UDDS and LA92 where the voltage range of the SC bank iseffectively used. The control offers a reduction in the high-speed US06 profile and NEDC634.3. Results of Neural Network Control for Hybrid Energy Storage Systemprofile, however it does not fully optimize capacitor voltages and depletes the energy availablefrom the SC bank before reaching the maximum speed, shown in figure 4.5. The same results0 100 200 300 400 500 600Simulation Time (s)-80-60-40-20020406080Current (A)050100150200250300Voltage (V)Capacitor CurrentCapacitor VoltageFigure 4.5: Graph showing the voltages and current sourced from the SC bank of the HESS controlled by aNN trained on the US06 profile.are collected and compared with a NN trained on the US06 profile. When the NN is trainedTable 4.2: Percent RMS current reduction, percent max peak current reduction, and percentPAV R/PAPR reduction of a NN controlled HESS trained the US06 profile.Driving ProfileRMS Current (A) RMS CurrentReduction (%)Peak Battery Current (A) Max Peak BatteryCurrent Reduction (%)Before After Before AfterUDDS 23.48 22.69 3.39 33.52 16.97 49.38US06 62.55 57.75 7.67 100.67 75.57 24.94LA92 30.68 29.43 4.11 37.73 25.62 32.10NEDC 22.07 21.77 1.36 110.62 98.90 10.60Driving ProfilePeak Power (kW) Average Power (kW) PAPRPAVRPAPR/PAVR PAPR/PAVRReduction (%)Before After Before After Before After Before AfterUDDS 42.93 34.31 5.91 5.96 7.26 5.75 2.90 2.51 1.99 20.87US06 106.10 95.25 17.76 17.91 5.97 5.32 1.68 3.56 3.17 11.03LA92 47.42 43.66 7.51 7.58 6.31 5.76 2.70 2.34 2.13 8.77NEDC 42.56 38.05 5.26 5.28 8.09 7.21 3.61 2.24 1.99 10.92on a high speed profile, the US06 profile shows an improvement in battery RMS currentreduction and PAPR/PAVR reduction, however all other profiles have lower improvements644.4. Results Compared to Other Control Strategiesthan when compared to the NN trained on the more urban UDDS profile. NN control canoffer an effective solution provided it is supplied enough inputs to effectively determine thevehicle’s condition.4.4 Results Compared to Other Control StrategiesHESS control strategies are typically designed to either reduce battery RMS current andreduce battery peak current, or prolong the driving range of the vehicle. If the later is thecontrol objective, the SC bank is much larger than the SC bank used in this thesis and isa larger energy contributor to the vehicle. The HESS in this thesis and similar HESSs usea relatively smaller SC bank which has negligible impact on additional energy available tothe vehicle after charging. The results from the NN control developed in this thesis arecompared against control strategies that lower battery peak and RMS currents developedby authors [11, 14, 18, 19] in table 4.3. The energy capacity of the SC banks used in thisTable 4.3: Results of the proposed control compared with HESSs designed with the similar function ofreducing battery RMS and peak currents and reducing PAPR/PAVR ratios.System Driving ProfileSC Bank EnergyCapacity (Wh)Normalized SC BankEnergy CapacityProposed HESS and NN Control UDDS 26.3 1HESS and Rule-Based Control From [19] ”typical urban drive cycle” [19] 48.9 1.86HESS and Rule-Based Control From [11] UDDS 18.0 0.68HESS and NN Control From [18] NYC Drive Cycle 400 15.2HESS and LPF Control From [14] UDDS ∼55.6 2.12System ComplexityBattery RMSCurrent Reduction (%)Max Peak BatteryCurrent Reduction (%)PAPR/PAVRReduction (%)Proposed HESS and NN Control O(n2) 15 52.5 4.15HESS and Rule-Based Control From [19] O(n) NA 56.25 NAHESS and Rule-Based Control From [11] O(n) 19 50 NAHESS and NN Control From [18] O(n2) 50 NA NAHESS and LPF Control From [14] O(n log n) NA NA11.3 acceleration6.0 decelerationcontrol range from 18Wh to 400Wh, while a relatively small EV will have a battery with a654.5. Summarycapacity of 24kWh like the one mentioned in section 2.2. It is worth noting the difficultyin comparing results between control solutions as vehicle, HESS, and simulation parametersdiffer greatly from one experiment to the next. Additionally the same driving profile canresult in various load current waveforms depending on conditions such as vehicle weight andbattery capabilities. For example, reducing the vehicle weight from 906kg to 770kg willreduce the load current RMS from 23.72A to 16.46A; a reduction such as this will have adrastic impact on the performance of the HESS.4.5 SummaryThis chapter presented the progression of the genetic algorithm training the NN weightsused in the control. This chapter also presented the results that the trained NN had ona vehicle’s battery current, capacitor current, and capacitor voltage; where a trained NNoffered a 15.15% reduction in battery RMS current, a 52.5% maximum reduction in peakbattery current, and a 4.15% PAPR/PAVR reduction coupled with a 94.5kJ SC bank. Theeffectiveness of the NN trained on various profiles had over different driving profile simu-lations was evaluated and compared. It was found that a NN trained on a urban drivingprofile offered a better reduction in overall RMS currents, even if major spikes in current inhighway scenarios where not mitigated. This HESS and control developed in this thesis wasthen compared to other works using both rule-based and optimization-based control for theirdeveloped HESSs. However, direct comparison is difficult due to varying vehicle, simulation,and HESS parameters.66Chapter 5Conclusion5.1 OverviewThis chapter covers the work done in this thesis and present the final thoughts and find-ings of this thesis. Section 5.2 will discuss the results of the NN control and its effectiveness.Section 5.3 will suggest opportunity for future work and improvements to be made to thiscontrol to allow continual development of this control.5.2 SummaryChapter 1 discussed current stress problems associated with electric vehicle batteriesand introduced the HESS as a solution to reduce these stresses. The chapter discusses theneed for control of active HESS topologies and a brief explanation on the principals of NNswas given, including the theory of their operation and the mathematics on how a signalprogressed through a NN. The chapter then explained various strategies to assign values tothe weights of a NN, known as training the NN.Chapter 2 presented the contributions of other researchers on HESS and the control fortheir respective HESS. This included work on various topologies used on vehicles of varioussizes, or on HESSs used for power distribution. It discussed the advantages and drawbacks675.3. Future Workof various controls and computational effort of each type of control strategy.Chapter 3 presented the HESS for which control would be developed in this thesis. Thischapter then described the NN that would be used to control the HESS. It discussed theinputs to the NN, the size of the NN, the effect various sizes had on the final result of control,and the method in which the NN was implemented in the model. The chapter then explainedthe external control used to protect elements of the HESS. Finally the genetic algorithm usedto train the NN was introduced. The effect of varied parameters of the genetic algorithmwere compared, and the operation of the genetic algorithm was covered.Chapter 4 presented the progression of a NN as it was trained by a genetic algorithmand the results that the NN control had on the HESS when trained and tested on variousdriving profiles. Three methods to evaluate the effectiveness of control were presented andcompared over various scenarios, these methods were: RMS current reduction, maximumpeak battery current reduction, and PAPR/PAV R reduction. The effect of the NN controlover multiple aspects of the HESS was shown in various figures and the effectiveness overmixed driving profiles was compared.5.3 Future WorkThe NN control developed in this thesis provided effective control, yet there is opportunityfor further improvement. When designing the NN, the optimal hyper-parameters of the NNwere chosen through a repetitive process of training the NN with each combination of neuronsand evaluating its performance. This method is time consuming and lacks mathematicalvalidation. An improvement could be to derive a mathematical model to calculate the685.3. Future Workeffectiveness of a set of hyper parameters.The trained network will function best in settings similar to the profile which it is trained.If a NN is trained in an urban environment for example, the HESS will correctly utilize thecapacitor banks energy until the setting changes and the vehicle’s speed increases. Addingan input to better indicate what driving situation the vehicle is in would allow the NN totrain with this input and adjust the reference converter current accordingly. Future workcould involve determining the correct input to achieve this adding inputs to the NN thatwere not available from the ADVISOR simulator could aid in NN performance.Another potential for future improvement is providing better evaluation of a populationmember’s effectiveness in the genetic algorithm. Currently the genetic algorithm considersthe total battery RMS current value of a simulation run to determine effectiveness. Addingother evaluation factors could train a NN to perform better across various scenarios.Driving profiles to test on were limited by what was available on ADVISOR. Generatingnew driving profiles with more parameters on the vehicle and its environment could aid inNN training. Implementing this solution would be desirable to allow the NN to train on thescenario an individual vehicle is placed. If this solution is to be implemented into commercialvehicles, a method of continually adjusting NN weights while the vehicle is in operation couldbe developed.69Bibliography[1] M. Chen and G. A. Rinco´n-Mora, “Accurate electrical battery model capable of predict-ing runtime and I-V performance,” IEEE Transactions on Energy Conversion, vol. 21,no. 2, pp. 504–511, 2006. → pages x, 12, 61[2] A. Jaafar, C. Turpin, X. Roboam, E. Bru, and O. Rallieres, “Energy management of ahybrid system based on a fuel cell and a Lithium Ion battery: Experimental tests andintegrated optimal design,” Mathematics and Computers in Simulation, vol. 131, pp.21–37, 2017. [Online]. Available: http://dx.doi.org/10.1016/j.matcom.2016.01.007 →pages xiii, 18, 26, 27[3] D. Shiffman, S. Fry, and Z. Marsh, The Nature of Code. D. Shiffman, 2012. [Online].Available: https://books.google.ca/books?id=hoK6lgEACAAJ → pages 3, 4[4] M. Fischer, M. Werber, and P. V. Schwartz, “Batteries: Higher energy density thangasoline?” Energy Policy, vol. 37, no. 7, pp. 2639–2641, 2009. → pages 10[5] B. Xu, A. Oudalov, A. Ulbig, G. Andersson, and D. S. Kirschen, “Modeling of lithium-ion battery degradation for cell life assessment,” IEEE Transactions on Smart Grid,vol. 9, no. 2, pp. 1131–1140, 2018. → pages 11[6] N. T. Jeong, S. M. Yang, K. S. Kim, M. S. Wang, H. S. Kim, and M. W. Suh, “Urban70Chapter 5. Bibliographydriving cycle for performance evaluation of electric vehicles,” International Journalof Automotive Technology, vol. 17, no. 1, pp. 145–151, 2016. [Online]. Available:https://link.springer.com/article/10.1007{%}2Fs12239-016-0014-0 → pages 12, 34[7] T. Bruen and J. Marco, “Modelling and experimental evaluation of parallelconnected lithium ion cells for an electric vehicle battery system,” Journalof Power Sources, vol. 310, pp. 91–101, 2016. [Online]. Available: http://dx.doi.org/10.1016/j.jpowsour.2016.01.001 → pages 12[8] O. Salari, I. S. Member, K. H. Zaad, I. S. Member, A. Bakhshai, I. S. Member, P. Jain,and F. Ieee, “Filter Design for Energy Management Control of Hybrid Energy StorageSystems in Electric Vehicles,” 9th IEEE International Symposium on Power Electronicsfor Distributed Generation Systems (PEDG), pp. 1–7, 2018. → pages 12, 17, 22, 26, 27[9] I. Aharon and A. Kuperman, “Topological overview of powertrains for battery-poweredvehicles with range extenders,” IEEE Transactions on Power Electronics, vol. 26, no. 3,pp. 868–876, 2011. → pages 13, 14[10] A. Ostadi, M. Kazerani, and S. K. Chen, “Hybrid Energy Storage System (HESS) invehicular applications: A review on interfacing battery and ultra-capacitor units,” IEEETransportation Electrification Conference and Expo, pp. 1–7, 2013. → pages 14, 15, 17,22[11] B. Kerns, T. Lindsay, T. Williams, and W. Eberle, “A control algorithm to reduce elec-tric vehicle battery pack RMS currents enabling a minimally sized supercapacitor pack,”71Chapter 5. BibliographyIEEE Transportation and Electrification Conference and Expo, pp. 376–380, 2017. →pages 17, 23, 65[12] M. B. Camara, H. Gualous, F. Gustin, and A. Berthon, “Design and new control ofDC/DC converters to share energy between supercapacitors and batteries in hybridvehicles,” IEEE Transactions on Vehicular Technology, vol. 57, no. 5, pp. 2721–2735,2008. → pages[13] M. E. Choi, J. S. Lee, and S. W. Seo, “Real-time optimization for power managementsystems of a battery/supercapacitor hybrid energy storage system in electric vehicles,”IEEE Transactions on Vehicular Technology, vol. 63, no. 8, pp. 3600–3611, 2014. →pages 22, 27, 53[14] L. Sun, N. Zhang, M. Awadallah, and P. Walker, “An innovative control strategy for ahybrid energy storage system (HESS),” IEEE International Conference on Mechatron-ics, pp. 434–439, 2017. → pages 26, 62, 65[15] A. Tani, M. B. Camara, and B. Dakyo, “Energy management based on frequency ap-proach for hybrid electric vehicle applications: Fuel-cell/lithium-battery and ultraca-pacitors,” IEEE Transactions on Vehicular Technology, vol. 61, no. 8, pp. 3375–3386,2012. → pages 18, 22[16] H. Xiaoliang and H. Yoichi, International Battery, Hybrid and Fuel Cell Electric VehicleSymposium. → pages 17, 26[17] M. Porru, A. Serpi, I. Marongiu, and A. Damiano, “A novel hybrid energy storage72Chapter 5. Bibliographysystem for electric vehicles,” 41st Annual Conference of the IEEE Industrial ElectronicsSociety, vol. 2015, pp. 3732–3737, 2015. → pages 18, 22, 29[18] J. Ramoul, E. Chemali, L. Dorn-Gomba, and A. Emadi, “A Neural Network EnergyManagement Controller Applied to a Hybrid Energy Storage System using Multi-SourceInverter,” IEEE Energy Conversion Congress and Exposition, ECCE 2018, pp. 2741–2747, 2018. → pages 18, 29, 31, 36, 50, 65[19] R. Carter and A. Cruden, “Strategies for control of a battery/ supercapacitor system inan electric vehicle,” International Symposium on Power Electronics, Electrical Drives,Automation and Motion, pp. 727–732, 2008. → pages 23, 65[20] P. Garcia, J. P. Torreglosa, L. M. Fernandez, and F. Jurado, “Control strategies forhigh-power electric vehicles powered by hydrogen fuel cell, battery and supercapacitor,”Expert Systems with Applications, vol. 40, no. 12, pp. 4791–4804, 2013. → pages 23[21] B. Wei, S. Ding, J. Fang, J. Hang, and Q. Wang, “A wavelet-fuzzy power allocationstrategy of battery-supercapacitor hybrid system for electric vehicles,” IEEE Conferenceon Industrial Electronics and Applications, no. 2, pp. 2095–2099, 2018. → pages 24[22] I. R. Scola, M. Patrone, and D. Feroldi, “Observer-based Controller for an ElectricVehicle with Hybrid Energy Storage,” 2018 Argentine Conference on Automatic Control(AADECA), no. 1, pp. 1–6, 2018. → pages 25[23] X. Li, Z. Zhao, J. Zhang, and M. Zhu, “Robust Adaptive Output Feedback ControlScheme for Chaos Synchronization with Input Nonlinearity,” Journal of Control Scienceand Engineering, vol. 2016, 2016. → pages 25, 2673Chapter 5. Bibliography[24] D. Xu, J. Liu, X. G. Yan, and W. Yan, “A Novel Adaptive Neural Network ConstrainedControl for a Multi-Area Interconnected Power System with Hybrid Energy Storage,”IEEE Transactions on Industrial Electronics, vol. 65, no. 8, pp. 6625–6634, 2018. →pages 30, 3174Appendix ADriving Profile Waveforms0 200 400 600 800 1000 1200 1400Simulation Time (s)051015202530Speed (m/s)Figure A.1: Speed input of UDDS driving profile.0 200 400 600 800 1000 1200 1400Simulation Time (s)-40-20020406080100120Current (A)-50050100150Torque (Nm)Load CurrentTorqueFigure A.2: Load current and torque inputs of UDDS driving profile.75Appendix A. Driving Profile Waveforms0 100 200 300 400 500 600Simulation Time (s)0510152025303540Speed (m/s)Figure A.3: Speed input of US06 driving profile.0 100 200 300 400 500 600Simulation Time (s)-50050100150200250300Current (A)-100-50050100150200250300Torque (Nm)Load CurrentTorqueFigure A.4: Load current and torque inputs of US06 driving profile.76Appendix A. Driving Profile Waveforms0 500 1000 1500Simulation Time (s)051015202530Speed (m/s)Figure A.5: Speed input of LA92 driving profile.0 500 1000 1500Simulation Time (s)-100-50050100150Current (A)-100-50050100150Torque (Nm)Load CurrentTorqueFigure A.6: Load current and torque inputs of LA92 driving profile.77Appendix A. Driving Profile Waveforms0 200 400 600 800 1000 1200Simulation Time (s)05101520253035Speed (m/s)Figure A.7: Speed input of NEDC driving profile.0 200 400 600 800 1000 1200Simulation Time (s)-40-20020406080100120Current (A)-30-20-1001020304050Torque (Nm)Load CurrentTorqueFigure A.8: Load current and torque inputs of NEDC driving profile.78Appendix BMatlab CodeB.1 Main code for training a NN%% MainDirectNN4L% This code t r a i n s a NN through use o f a g e n e t i c g lgor i thm .Requires% ConverterDirectNN4L . s l x S imlu l ink model to run .c l c ; c l e a r a l l ; c l o s e a l l ;%% Load d r i v i n g p r o f i l e parameters or generate them from adv i so r .prompt = ’ Spec i f y input paremters to load , o the rw i s e h i t ente r torun advisor , then r e s t a r t main . [ input parameters XXXXX . mat ] :’ ;s t r = input ( prompt , ’ s ’ ) ;i f isempty ( s t r )run ( adv i so r )l o a d c u r r e n t = [ t , e s s c u r r e n t ] ; % Save load cur rentp r o f i l espeed = [ t , veh spd a ] ; % Sace speed p r o f i l etorque = [ t , mc trq out a ] ; % Save torque p r o f i l e% Save p r o f i l e s under s p e c i f i e d save namesave ( ’ input parameters NEDC ’ , ’ l oad cur r ent ’ , ’ speed ’ , ’ t ’ , ’torque ’ )e l s esave name = [ ’ input parameters ’ , s t r ] ;load ( save name ) ; % Load parameters from e x i s t i n gsave f i l eend%% Setup Parameters% I n i t i a l i z e the nea ra l network and g e n e t i c a lgor i thm parametersi n p u t l a y e r s i z e = 7 ;79B.1. Main code for training a NNh i d d e n l a y e r 1 s i z e = 12 ;h i d d e n l a y e r 2 s i z e = 9 ;o u t p u t l a y e r s i z e = 1 ;p o p u l a t i o n s i z e = 20 ;T step = 0 . 1 ;c o n v e r t e r l i m i t = 60 ; % Limit in amperesc o n v e r t e r e f f i c i e n c y = 0 . 9 5 ;g e n e r a t i o n l i m i t = 200 ; % Number o f g ene ra t i on s to runnumber of parents = 2 ;mutat ion rate = 0 . 0 1 ;e p s i l o n i n i t = 0 . 1 5 ; % I n i t i a l randomizat ion f a c t o r% Number o f t o t a l neurons f o r the NNparam size = ( i n p u t l a y e r s i z e + 1) * h i d d e n l a y e r 1 s i z e + . . .( h i d d e n l a y e r 1 s i z e + 1) * h i d d e n l a y e r 2 s i z e + . . .( h i d d e n l a y e r 2 s i z e + 1) * o u t p u t l a y e r s i z e ;s im length = length ( t ) ; %length o f Simulink s imu la t i on%% I n i t i a l i z a t i o n% Populates the neuron vec to r with weights , e i t h e r random orloaded from a% prev ious runprompt = ’ Randomize i n i t a l theta ? Y/N [Y ] : ’ ;s t r = input ( prompt , ’ s ’ ) ;i f isempty ( s t r )s t r = ’Y’ ;endnn params = ze ro s ( param size , p o p u l a t i o n s i z e ) ;% For loop c r e a t e s weight matr ixes f o r each l a y e r o f eachpopulat ion memberi f ( ( s t r == ’Y’ ) | | ( s t r == ’y ’ ) )f o r i = 1 : p o p u l a t i o n s i z ei n i t i a l T h e t a 1 = r a n d I n i t i a l i z e W e i g h t s ( i n p u t l a y e r s i z e ,h i d d e n l a y e r 1 s i z e , e p s i l o n i n i t ) ;i n i t i a l T h e t a 2 = r a n d I n i t i a l i z e W e i g h t s ( h i d d e n l a y e r 1 s i z e, h i d d e n l a y e r 2 s i z e , e p s i l o n i n i t ) ;i n i t i a l T h e t a 3 = r a n d I n i t i a l i z e W e i g h t s ( h i d d e n l a y e r 2 s i z e, o u t p u t l a y e r s i z e , e p s i l o n i n i t ) ;nn params ( : , i ) = [ i n i t i a l T h e t a 1 ( : ) ; i n i t i a l T h e t a 2 ( : ) ;i n i t i a l T h e t a 3 ( : ) ] ;80B.1. Main code for training a NNende l s e% Otherwise weight matr ixes are loaded from a prev iousgene ra t i onprompt = ’Which gene ra t i on number to load ? : ’ ;l oad gen = input ( prompt ) ;load ( [ ’ generat ionsDi rec t4L ’ , num2str ( load gen ) , ’ . mat ’ ] )f p r i n t f ( ’ Loaded f i l e : g ene ra t i onsDi r ec t4L%d . mat \n ’ , l oad gen )f o r i = 1 : p o p u l a t i o n s i z ei n i t i a l T h e t a 1 = Data . gene ra t i on ( end ) . Theta1 ;i n i t i a l T h e t a 2 = Data . gene ra t i on ( end ) . Theta2 ;i n i t i a l T h e t a 3 = Data . gene ra t i on ( end ) . Theta3 ;nn params ( : , i ) = [ i n i t i a l T h e t a 1 ( : ) ; i n i t i a l T h e t a 2 ( : ) ;i n i t i a l T h e t a 3 ( : ) ; ] ;endload mut rate = 0 . 0 1 ;% Neurons are mutated to cont inue a lgor i thm p r o g r e s s i o n froml a s t% gene ra t i onnn params = mutate ( nn params , load mut rate , e p s i l o n i n i t ) ;endgeneration num = 1 ;%% Run Simulat ionButtonHandle = u i c o n t r o l ( ’ Sty le ’ , ’ PushButton ’ , . . .’ Str ing ’ , ’ F in i sh sim ’ , . . .’ Callback ’ , ’ d e l e t e ( gcbf ) ’ ) ;whi l e ( generation num <= g e n e r a t i o n l i m i t )f p r i n t f ( ’ Generation : %d\n ’ , generation num )f i t n e s s b a t = ze ro s ( p o p u l a t i o n s i z e , 1) ; %c r e a t e f i t n e s sarray% Create gene ve c t o r s from nn param arrayf o r i = 1 : p o p u l a t i o n s i z eTheta1 = reshape ( nn params ( 1 : h i d d e n l a y e r 1 s i z e * (i n p u t l a y e r s i z e + 1) , i ) , . . .h i d d e n l a y e r 1 s i z e , ( i n p u t l a y e r s i z e + 1) ) ;Theta2 = reshape ( nn params ( (1 + ( h i d d e n l a y e r 1 s i z e * (i n p u t l a y e r s i z e + 1) ) ) : . . .( ( h i d d e n l a y e r 1 s i z e * ( i n p u t l a y e r s i z e + 1) ) ) + (h i d d e n l a y e r 2 s i z e * ( h i d d e n l a y e r 1 s i z e + 1) ) , i ), . . .h i d d e n l a y e r 2 s i z e , ( h i d d e n l a y e r 1 s i z e + 1) ) ;81B.1. Main code for training a NNTheta3 = reshape ( nn params ( ( ( 1 + h i d d e n l a y e r 1 s i z e * (i n p u t l a y e r s i z e + 1) ) ) + ( h i d d e n l a y e r 2 s i z e * (h i d d e n l a y e r 1 s i z e + 1) ) : . . .end , i ) , o u t p u t l a y e r s i z e , ( h i d d e n l a y e r 2 s i z e + 1) );% Run the s imu la t i on with s imul inksim ( ’ ConverterDirectNN4L ’ )f p r i n t f ( ’ The RMS current o f member %d i s : %5.2fA .\n ’ , i ,RMS current bat . data ( end ) )%Save va lue s from t h i s populat ion membercu r r en t s . i t t e r a t i o n ( i ) . ba t cu r r en t = C u r r e n t P r o f i l e s .s i g n a l s (1 ) . va lue s ; % batte ry cu r r en t scu r r en t s . i t t e r a t i o n ( i ) . cap cur r ent = C u r r e n t P r o f i l e s .s i g n a l s (2 ) . va lue s ; % c a p a c i t o r cu r r en t scu r r en t s . i t t e r a t i o n ( i ) . time = C u r r e n t P r o f i l e s . time ; %s imu la t i on timecu r r en t s . i t t e r a t i o n ( i ) . f i t n e s s = RMS current bat . data ( end ); %f i t n e s s o f each i t t e r a t i o ncu r r en t s . i t t e r a t i o n ( i ) . Theta1 = Theta1 ; %neura l networkparameters o f each i t t e r a t i o ncu r r en t s . i t t e r a t i o n ( i ) . Theta2 = Theta2 ; %neura l networkparameters o f each i t t e r a t i o ncu r r en t s . i t t e r a t i o n ( i ) . Theta3 = Theta3 ; %neura l networkparameters o f each i t t e r a t i o nf i t n e s s b a t ( i ) = RMS current bat . data ( end ) ; % save batte ryRMS cur rent to be used as f i t n e s sendindex = f i n d ( f i t n e s s b a t == min( f i t n e s s b a t ) ) ;index = index (1 ) ;b e s t o f . g ene ra t i on ( generation num ) = cur r en t s . i t t e r a t i o n ( index) ;%p lo t RMS value vs gene ra t i on numberRMS current = ze ro s ( s i z e ( b e s t o f . g ene ra t i on ) ) ;f o r i = 1 : l ength ( b e s t o f . g ene ra t i on )RMS current ( i ) = b e s t o f . g ene ra t i on ( i ) . f i t n e s s ;endf i g u r e (1 )p l o t ( [ 1 : l ength ( b e s t o f . g ene ra t i on ) ] , RMS current ) ;t i t l e ( ’ 4 Layer Direct ’ )82B.1. Main code for training a NN% map the f i t n e s s between 0 and 100no rm f i tn e s s = mapFitness ( f i t n e s s b a t ) ;f p r i n t f ( ’ Normalized f i t n e s s from RMS va lues . \n ’ )%c r e a t e mating poolmat ing pool = createMatingPool ( norm f i tne s s , nn params ) ;f p r i n t f ( ’ Created mating pool . \n ’ )%c r e a t e next gene ra t i onnew gen = produceOf f spr ing ( number of parents , mating pool ,p o p u l a t i o n s i z e ) ;f p r i n t f ( ’ Created next gene ra t i on . \n ’ )%mutate new genera t i onnn params = mutate ( new gen , mutat ion rate , e p s i l o n i n i t ) ;f p r i n t f ( ’ Mutated new genera t i on and updated nn parameters . \n’ )f p r i n t f ( ’ Generation completed . \n ’ )generation num = generation num + 1 ; %prog r e s s g e n e t i ca lgor i thm% Exit loop on keypres si f ˜ i shand l e ( ButtonHandle )d i sp ( ’ Loop stopped by user ’ ) ;break ;endpause ( 0 . 0 1 ) ; % A NEW LINEend% Save a l l data from e n t i r e g e n e t i c a lgor i thm runb e s t o f . metadata . i n p u t l a y e r s i z e = i n p u t l a y e r s i z e ;b e s t o f . metadata . h i d d e n l a y e r 1 s i z e = h i d d e n l a y e r 1 s i z e ;b e s t o f . metadata . h i d d e n l a y e r 2 s i z e = h i d d e n l a y e r 2 s i z e ;b e s t o f . metadata . o u t p u t l a y e r s i z e = o u t p u t l a y e r s i z e ;b e s t o f . metadata . p o p u l a t i o n s i z e = p o p u l a t i o n s i z e ;b e s t o f . metadata . T step = T step ;b e s t o f . metadata . c o n v e r t e r l i m i t = c o n v e r t e r l i m i t ;b e s t o f . metadata . number of parents = number of parents ;b e s t o f . metadata . mutat ion rate = mutat ion rate ;b e s t o f . metadata . e p s i l o n i n i t = e p s i l o n i n i t ;b e s t o f . metadata . save name = save name ;save num = SaveWithNumber ( ’ generat ionsDi rec t4L ’ , b e s t o f ) ;83B.1. Main code for training a NNf p r i n t f ( ’ Generat ions saved to f i l e : g ene ra t i onsDi r e c t4L%d . mat \n ’ ,save num )%% Find Best Generation% c l e a r unneeded v a r i a b l e sc l e a r v a r s =except save num T step save name c o n v e r t e r l i m i ts im length c o n v e r t e r e f f i c i e n c y ; c l o s e a l l ;load ( save name )% Rerun best gene ra t i onprompt = ’Run cur rent gene ra t i on ? Y/N [Y ] : ’ ;s t r = input ( prompt , ’ s ’ ) ;i f ( ( s t r == ’n ’ ) | | ( s t r == ’N’ ) )prompt = ’Which gene ra t i on to load ? ’ ;save num = input ( prompt ) ;endload ( [ ’ generat ionsDi rec t4L ’ , num2str ( save num ) , ’ . mat ’ ] )RMS current = ze ro s ( s i z e ( Data . gene ra t i on ) ) ;f o r i = 1 : l ength ( Data . gene ra t i on )RMS current ( i ) = Data . gene ra t i on ( i ) . f i t n e s s ;endc l o s e a l lp l o t ( [ 1 : l ength ( Data . gene ra t i on ) ] , RMS current ) ;t i t l e ( ’ Genetic Algorithm Progress ion ’ )x l a b e l ( ’ Generation Number ’ )y l a b e l ( ’ Battery RMS Current ’ )[ best RMS , index ] = min ( RMS current ) ;f p r i n t f ( ’ The lowest RMS bat cur r ent i s %5.2fA at gene ra t i on number%3. f .\n ’ , best RMS , index )Theta1 = Data . gene ra t i on ( index ) . Theta1 ;Theta2 = Data . gene ra t i on ( index ) . Theta2 ;Theta3 = Data . gene ra t i on ( index ) . Theta3 ;c o n v e r t e r l i m i t = Data . metadata . c o n v e r t e r l i m i t ;f p r i n t f ( ’ S imulat ion running . \n ’ )sim ( ’ ConverterDirectNN4L ’ )84B.2. Randomization function.l a y e r 1 w e i g h t s = Weight Evaluator ( Theta1 ) ;f p r i n t f ( ’ Layer one weights : %d \n ’ , l a y e r 1 w e i g h t s )f p r i n t f ( ’ S imulat ion f i n i s h e d .\n ’ )B.2 Randomization function.f unc t i on W = r a n d I n i t i a l i z e W e i g h t s ( L in , L out , e p s i l o n i n i t )%Randomly i n i t i a l i z e the weights o f a l a y e r with L in%incoming connec t i ons and L out outgoing connec t ionsW = ze ro s ( L out , 1 + L in ) ;W = rand ( L out , 1 + L in ) * 2 * e p s i l o n i n i t = e p s i l o n i n i t ;endB.3 Mutate function.f unc t i on nn params = mutate ( new gen , mutat ion rate , e p s i l o n i n i t )% mutate a s p e c i f i c gene o f the nuera l network parameters i f arandom value% i s l e s s than the mutation ra t e .nn params = ze ro s ( s i z e ( new gen ) ) ;f o r i = 1 : s i z e ( new gen , 2)th r e sho ld = rand ( s i z e ( new gen , 1) , 1) ;f o r j = 1 : s i z e ( new gen , 1)i f ( th r e sho ld ( j ) < mutat ion rate )nn params ( j , i ) = rand ( ) * 2 * e p s i l o n i n i t =e p s i l o n i n i t ;e l s enn params ( j , i ) = new gen ( j , i ) ;endendendB.4 Map fitness function.f unc t i on normal ized = mapFitness ( f i t n e s s b a t )% Maps f i t n e s s va lue s between 1 and 100max bat = max( f i t n e s s b a t ) ;min bat = min ( f i t n e s s b a t ) ;85B.5. Mating pool function.norm bat = ( f i t n e s s b a t = min bat ) . / ( max bat = min bat ) ;normal ized = 100*(1 = norm bat ) ;endB.5 Mating pool function.f unc t i on mating pool = createMatingPool ( norm f i tne s s , nn params )% Creates l a r g e array from neura l network parameters based onf i t n e s s .% Higher f i t n e s s parameters w i l l occupy a l a r g e r por t i on o f thearraymating pool = ze ro s ( s i z e ( nn params , 1) , 1) ;f o r i = 1 : s i z e ( norm f i tne s s , 1 )f o r j = 1 : no rm f i tn e s s ( i )mat ing pool = [ mating pool , nn params ( : , i ) ] ;endendmating pool = mating pool ( : , 2 : end ) ;endB.6 Offspring function.f unc t i on new gen = produceOf f spr ing ( num parents , mating pool ,p o p u l a t i o n s i z e )% Creates new gen o f populat ion s i z e based on a s p e c i f i e n d numbero f% parents f o r each c h i l dparent = ze ro s ( num parents , 1 ) ;new gen = ze ro s ( s i z e ( mating pool , 1) , p o p u l a t i o n s i z e ) ;f o r i = 1 : p o p u l a t i o n s i z e% determing three random parents from mating poolf o r k = 1 : num parentsparent ( k ) = f l o o r ( s i z e ( mating pool , 2) * rand ( ) + 1) ;end% c r e a t e c h i l d from parents , l oop ing through a l l i n d i c e s o f nnparametersf o r j = 1 : s i z e ( mating pool , 1)new gen ( j , i ) = mating pool ( j , parent ( f l o o r ( num parents* rand( ) + 1) ) ) ;end86B.7. Save function.endB.7 Save function.f unc t i on save num = SaveWithNumber ( FileName , Data )% Function saves the gene ra t i on run o f the g e n e t i c a lgor i thm[ fPath , fName , fExt ] = f i l e p a r t s ( FileName ) ;i f isempty ( fExt ) % No ’ . mat ’ in FileNamefExt = ’ . mat ’ ;FileName = f u l l f i l e ( fPath , [ fName , fExt ] ) ;endsave num = 1 ;i f e x i s t ( [ fName , num2str ( save num ) , fExt ] , ’ f i l e ’ )% Increment f i l e s t a r t i n g from base name .fD i r = d i r ( f u l l f i l e ( fPath , [ fName , num2str ( save num ) , fExt ] )) ;whi l e (˜ isempty ( fD i r ) )save num = save num + 1 ;fD i r = d i r ( f u l l f i l e ( fPath , [ fName , num2str ( save num ) ,fExt ] ) ) ;endendsave ( [ fName , num2str ( save num ) , fExt ] , ’ Data ’ ) ;endB.8 Weight evaluator function.f unc t i on input we ight s = Weight Evaluator ( Theta1 )% Find RMS value o f weight vec to r f o r each inputinput we ight s = rms ( Theta1 , 1 ) ;endB.9 Generation progression and NN weight analyser code.%% Generation p r o g r e s s i o n% This program loads NNs t ra ined on MainDirectNN4L .m and gene ra t e sf i g u r e s% to ana lyse the e f f e c t i v e n e s s . I t a l s o gene ra t e s PAPR/PAVR andRMS cur rent% reduct ion va lue s .c l e a r a l l ; c l c ;87B.9. Generation progression and NN weight analyser code.prompt = ’Which p r o f i l e to load ? ’ ;save name = input ( prompt , ’ s ’ ) ;prompt = ’Which gene ra t i on to load ? ’ ;save num = input ( prompt ) ;load ( [ ’ input parameters ’ , save name , ’ . mat ’ ] )load ( [ ’ generat ionsDi rec t4L ’ , num2str ( save num ) , ’ . mat ’ ] )RMS current = ze ro s ( s i z e ( Data . gene ra t i on ) ) ;f o r i = 1 : l ength ( Data . gene ra t i on )RMS current ( i ) = Data . gene ra t i on ( i ) . f i t n e s s ;endc l o s e a l lp l o t ( [ 1 : l ength ( Data . gene ra t i on ) ] , RMS current ) ;t i t l e ( ’ Genetic Algorithm Progress ion ’ )x l a b e l ( ’ Generation Number ’ )y l a b e l ( ’ Battery RMS Current (A) ’ )[ best RMS , index ] = min ( RMS current ) ;f p r i n t f ( ’ The lowest RMS bat cur r ent i s %5.2fA at gene ra t i on number%3. f .\n ’ , best RMS , index )Theta1 = Data . gene ra t i on ( index ) . Theta1 ;Theta2 = Data . gene ra t i on ( index ) . Theta2 ;Theta3 = Data . gene ra t i on ( index ) . Theta3 ;c o n v e r t e r l i m i t = Data . metadata . c o n v e r t e r l i m i t ;T step = Data . metadata . T step ;c o n v e r t e r e f f i c i e n c y = 0 . 9 5 ;s im length = length ( t ) ;f p r i n t f ( ’ S imulat ion running . \n ’ )sim ( ’ ConverterDirectNN4L ’ )f p r i n t f ( ’ S imulat ion f i n i s h e d . \n ’ )l a y e r 1 w e i g h t s = Weight Evaluator ( Theta1 ) ;f p r i n t f ( ’ Layer one weights : %d \n ’ , l a y e r 1 w e i g h t s )I l o a d = C u r r e n t P r o f i l e s . s i g n a l s (1 ) . va lue s ;I c o n v e r t e r = C u r r e n t P r o f i l e s . s i g n a l s (2 ) . va lue s ;I ba t = C u r r e n t P r o f i l e s . s i g n a l s (3 ) . va lue s ;time = C u r r e n t P r o f i l e s . time ;%%88B.9. Generation progression and NN weight analyser code.%Plot cur r ent p r o f i l e sp l o t ( time , I l oad , time , I c onve r t e r , time , I bat , ’ g ’ , ’ l inewidth ’ , 1 . 5 ) ;l egend ( ’ Load Current ’ , ’ Capacitor Current ’ , ’ Battery Current ’ )x l a b e l ( ’ S imulat ion Time ( s ) ’ )y l a b e l ( ’ Current (A) ’ )g r i d onV caps = CapacitorValues . s i g n a l s (1 ) . va lue s ;I c a p s = CapacitorValues . s i g n a l s (2 ) . va lue s ;time = CapacitorValues . time ;% Plot SC bank cur rent and vo l tagef i g u r e (2 )yyax i s l e f tp l o t ( time , I caps , ’ l inewidth ’ , 1 . 5 ) ;y l a b e l ( ’ Current (A) ’ )ylim ( [ =80 ,80 ] )yyax i s r i g h tp l o t ( time , V caps , ’ l inewidth ’ , 1 . 5 ) ;y l a b e l ( ’ Voltage (V) ’ )x l a b e l ( ’ S imulat ion Time ( s ) ’ )l egend ( ’ Capacitor Current ’ , ’ Capacitor Voltage ’ )xlim ( [ 0 , 600 ] )Speed = NNinputs . s i g n a l s (3 ) . va lue s ;loca l max = NNinputs . s i g n a l s (7 ) . va lue s ;moving avg = ze ro s ( s i z e ( Speed ) ) ;f o r i = 1 : l ength ( Speed )moving avg ( i ) = NNinputs . s i g n a l s (6 ) . va lue s (1 , 1 , i ) ;end%Plot speed p r o f i l e sf i g u r e (3 )p l o t ( time , Speed , time , local max , time , moving avg , ’ g ’ , ’ l inewidth’ , 1 . 5 )legend ( ’ Veh ic l e Speed ’ , ’ Local Max o f Speed ’ , ’ Moving Average o fSpeed ’ )y l a b e l ( ’ Speed (m/ s ) ’ )x l a b e l ( ’ S imulat ion Time ( s ) ’ )ylim ( [ 0 , 3 0 ] )torque nn = NNinputs . s i g n a l s (5 ) . va lue s ;a c c e l e r a t i o n = NNinputs . s i g n a l s (4 ) . va lue s ;89B.9. Generation progression and NN weight analyser code.%Plot Torque and Acce l e r a t i onf i g u r e (4 )yyax i s r i g h tp l o t ( time , torque nn , ’ l inewidth ’ , 1 . 5 ) ;y l a b e l ( ’ Torque (N/m) ’ )ylim ( [ =100 ,100 ] )yyax i s l e f tp l o t ( time , a c c e l e r a t i o n , ’ l inewidth ’ , 1 . 5 ) ;y l a b e l ( ’ Acc e l e r a t i on (m/ s ˆ2) ’ )ylim ( [ =2 ,2 ] )x l a b e l ( ’ S imulat ion Time ( s ) ’ )l egend ( ’ Torque ’ , ’ Acce l e ra t i on ’ )I l o a d = NNinputs . s i g n a l s (1 ) . va lue s ;% Plot load cur rent and c a p a c i t o r vo l tagef i g u r e (5 )yyax i s l e f tp l o t ( time , I l oad , ’ l inewidth ’ , 1 . 5 ) ;y l a b e l ( ’ Current (A) ’ )yyax i s r i g h tp l o t ( time , V caps , ’ l inewidth ’ , 1 . 5 ) ;y l a b e l ( ’ Voltage (V) ’ )x l a b e l ( ’ S imulat ion Time ( s ) ’ )l egend ( ’ Load Current ’ , ’ Capacitor Voltage ’ )% Plot load cur rent and torquef i g u r e (6 )yyax i s l e f tp l o t ( time , I l oad , ’ l inewidth ’ , 1 . 5 )y l a b e l ( ’ Current (A) ’ )yyax i s r i g h tp l o t ( time , torque nn , ’ l inewidth ’ , 1 . 5 ) ;y l a b e l ( ’ Torque (Nm) ’ )x l a b e l ( ’ S imulat ion Time ( s ) ’ )l egend ( ’ Load Current ’ , ’ Torque ’ )% Plot speedf i g u r e (7 )p l o t ( time , Speed , ’ l inewidth ’ , 1 . 5 )y l a b e l ( ’ Speed (m/ s ) ’ )x l a b e l ( ’ S imulat ion Time ( s ) ’ )%% Determine PAPR/PAVR and RMS reduct ion90B.9. Generation progression and NN weight analyser code.RMS current before = RMS current load . data ( end )RMS current after = RMS current bat . data ( end )c u r r e n t p e r r e d u c t i o n = ( RMS current load . data ( end ) =RMS current bat . data ( end ) ) / RMS current load . data ( end ) *100peak PWR before = max( load PWR) /1000peak PWR after = max( battery PWR ) /1000peak speed = max( Speed ) ;avg PWR before = mean( load PWR) /1000avg PWR after = mean( battery PWR ) /1000avg speed = mean( Speed ) ;PAPR before = peak PWR before/avg PWR beforePAPR after = peak PWR after/avg PWR afterPAVR = peak speed / avg speedPAPR PAVR before = PAPR before/PAVRPAPR PAVR after = PAPR after/PAVRPAPR PAVR per reduction = (PAPR PAVR before = PAPR PAVR after ) /PAPR PAVR before*10091Appendix CSimulink ModelFigure C.1: Main Simulink model (HESS Layout with converter subsystem).92Appendix C. Simulink ModelFigure C.2: Main Simulink model (inputs from Matlab workspace and Neural Network and Controlsubsystems.)93Appendix C. Simulink ModelFigure C.3: Main Simulink model (output signals to workspace).94Appendix C. Simulink ModelFigure C.4: Converter subsystem.95Appendix C. Simulink ModelFigure C.5: Neural network subsystem input normalizing and first layer bus.96Appendix C. Simulink ModelFigure C.6: Neural network subsystem internal layers, output, and local max generator control.97Appendix C. Simulink ModelFigure C.7: SC voltage and converter current control subsystem.98
Thesis/Dissertation
eng
Electrical Engineering
Vancouver : University of British Columbia Library
Attribution-NonCommercial-NoDerivatives 4.0 International
Graduate
Neural network trained via reinforced learning as control for a battery super-capacitor hybrid energy storage system in electric vehicles
Text