Applied Science, Faculty of
Mechanical Engineering, Department of
DSpace
UBCV
Zarei, Pegah
Master of Applied Science - MASc
University of British Columbia
This thesis proposes a deep learning framework for estimating the power use of electric vehicle drivetrains, using an artificial recurrent neural network architecture (long short-term memory), and techniques for data curation and analysis. Power samples of battery-electric buses operating in Victoria, B.C. were deployed to train, validate, and test the model. Microtrips were extracted from the dataset by taking out proportions of time spent at standstill. Anomalous trends of the dataset components were detected, marked as outliers, and excluded from the model input using Symbolic Aggregate approXimation (SAX) transformation filtering algorithm. A systematic optimization process is adopted to achieve the best hyperparameter configuration for this estimation task. The resulting fine-tuned deep LSTM model is then created, evaluated, and tested. More effort has been devoted to enhancing the efficacy of the model through investigating alternative model configurations and formulations. Among the several configurations tested, the best power consumption modeling approach yielded an error of about 5% (the RMSE of 0.05747) and a squared correlation coefficient of beyond 90% (R² of 0.92502). [This thesis was revised on October 25, 2021.]
i DATA-DRIVEN POWER ESTIMATION OF ELECTRIC BUSES USING DEEP LSTM NEURAL NETWORKS AND SYMBOLIC AGGREGATE APPROXIMATION FILTERING by Pegah Zarei B.A.Sc., Sharif University of Technology, 2017 A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF APPLIED SCIENCE in THE FACULTY OF GRADUATE AND POSTDOCTORAL STUDIES (Mechanical Engineering) THE UNIVERSITY OF BRITISH COLUMBIA (Vancouver) December 2020 © Pegah Zarei, 2020 ii The following individuals certify that they have read, and recommend to the Faculty of Graduate and Postdoctoral Studies for acceptance, the thesis entitled: Data-driven power estimation of electric buses using deep LSTM neural networks and symbolic aggregate approximation filterin Submitted by Pegah Zarei in partial fulfillment of the requirements for the degree of Master of Applied Science in Mechanical Engineering Examining Committee: Dr. Walter Mérida (Supervisor) Professor, Department of Mechanical Engineering, UBC Dr. Dana Grecov (Supervisory Committee Member) Associate Professor, Department of Mechanical Engineering, UBC Dr. Fariborz Taghipour (Supervisory Committee Member) Professor, Department of Chemical and Biological Engineering, UBC iii Abstract This thesis proposes a deep learning framework for estimating the power use of electric vehicle drivetrains, using an artificial recurrent neural network architecture (long short-term memory), and techniques for data curation and analysis. Power samples of battery-electric buses were deployed to train, validate, and test the model. Microtrips were extracted from the dataset by taking out proportions of time spent at standstill. Anomalous trends of the dataset components were detected, marked as outliers, and excluded from the model input using Symbolic Aggregate approXimation (SAX) transformation filtering algorithm. A systematic optimization process is adopted to achieve the best hyperparameter configuration for this estimation task. The resulting fine-tuned deep LSTM model is then created, evaluated, and tested. More effort has been devoted to enhancing the efficacy of the model through investigating alternative model configurations and formulations. Among the several configurations tested, the best power consumption modeling approach yielded an error of about 5% (the RMSE of 0.05747) and a squared correlation coefficient of beyond 90% (R2 of 0.92502). iv Lay summary The scientific consensus on global warming has urged all economic sectors, including the transportation sector, towards significant decarbonization initiatives. Battery electric vehicles have emerged as viable alternatives to conventional vehicles because they can reduce greenhouse gas emissions associated with urban transport. There is a tendency for cities around the world towards electrification, and electric buses have drawn a lot of attention due to their potential to reduce air pollution and greenhouse gas emissions. However, mass adoption is limited by larger up-front costs, limited driving range and the lack of recharging infrastructure. Vehicle power consumption estimation is a critical step toward addressing these challenges. Inspired by their environmental and economic gains, this thesis develops a framework to estimate power consumption of electric buses by integrating SAX filtering, and an artificial recurrent neural network architecture (Long short-term memory). v Preface This thesis was developed under the supervision of Professor Walter Mérida in the Department of Mechanical Engineering at the University of British Columbia. Walter’s kind and helpful advice has served me well. The entire material presented herein is an original work carried out and written by Pegah Zarei. My research questions were formulated together with Dr. Julián A. Fernández-Orjuela, without whose support this study would not have come to completion. Disclaimer: the intention behind the modeling framework developed here has been solely and exclusively to be able to estimate the instantaneous power required to run an arbitrary electric bus given the temporal measurements of the vehicle’s speed profile and the topography of the roads. Moreover, the data used in this study does not reflect the current state, performance, or capability of battery electric bus technology on the market today. vi Table of contents Abstract .................................................................................................................................................................... iii Lay summary .......................................................................................................................................................... iv Preface ....................................................................................................................................................................... v Table of contents ................................................................................................................................................... vi List of tables ............................................................................................................................................................. x List of figures ......................................................................................................................................................... xii List of abbreviations ..........................................................................................................................................xiii Acknowledgements ............................................................................................................................................ xiv Dedication .............................................................................................................................................................. xv Chapter 1 Introduction ........................................................................................................................................ 1 1.1 Climate crisis............................................................................................................................................... 1 1.2 Electric vehicles ......................................................................................................................................... 1 1.2.1. Overview ............................................................................................................................................. 1 1.2.2 Electric vehicles’ decarbonization strategy for transportation ...................................... 2 1.2.3 Challenges associated with the shift from ICE towards EV .............................................. 2 1.2.4 Regulatory and incentives environment ................................................................................. 3 1.3 Motivation for estimating vehicle’s power consumption .......................................................... 3 1.4 Research objective and outline ............................................................................................................ 4 1.4.1 Introduction to artificial neural networks and LSTM ........................................................ 5 1.4.2 Introduction to SAX filtering ........................................................................................................ 5 1.5 Research contributions ........................................................................................................................... 5 vii Chapter 2 Power consumption models: literature review ...................................................................... 7 2.1 Power consumption influencing factors .......................................................................................... 7 2.2 Power consumption modeling ............................................................................................................. 8 2.2.1 Physical approaches ............................................................................................................... 9 2.2.2 Data-driven approaches .................................................................................................... 10 2.3 Literature on vehicle power consumption neural networks models ................................ 10 2.3.1 Literature review ........................................................................................................................... 10 2.3.2 Research gap ................................................................................................................................... 14 Chapter 3 Dataset, data preprocessing and data processing .............................................................. 17 3.1 Dataset ....................................................................................................................................................... 17 3.2 Data processing ...................................................................................................................................... 17 3.2.1 Feature engineering ..................................................................................................................... 19 3.2.2 Grade filtering ............................................................................................................................ 19 3.3 Data processing: micro-trips identification ................................................................................. 21 Chapter 4 SAX filtering ..................................................................................................................................... 24 4.1 Methodology ............................................................................................................................................ 24 4.2 Motifs and discords ............................................................................................................................... 25 4.3 Implementation ...................................................................................................................................... 25 4.3.1 Binning the MTs ............................................................................................................................. 25 4.3.2 Selection of the length (ω) and the alphabet count (A) of patterns .......................... 27 4.3.3 Integration of bins and results ................................................................................................. 28 Chapter 5 Modeling approach: deep LSTM................................................................................................ 30 viii 5.1 ANN and DNN models .......................................................................................................................... 30 5.2 Overfitting and early stopping .......................................................................................................... 32 5.3 RNNs and LSTM for time series ........................................................................................................ 33 5.4 Computational tools .............................................................................................................................. 35 Chapter 6 Implementation of the system: ground-laying optimized DLSTM model .................. 37 6.1 Instantiating the model ....................................................................................................................... 37 6.2 Hyperparameter tuning....................................................................................................................... 37 6.3 Input preprocessing .............................................................................................................................. 38 6.3.1 Reshaping the timeseries data into 3-dimentioanl formatting ................................... 38 6.3.2 Split input to train, test, validation ......................................................................................... 40 6.3.3 Feature scaling ................................................................................................................................ 41 6.4 Figures of merit ...................................................................................................................................... 42 Chapter 7 Results of the ground-laying optimized DLSTM model and other model configurations and formulations ................................................................................................................... 43 7.1 Results of the ground-laying optimized DLSTM model .......................................................... 43 7.2 Model configurations and parametric study over the DLSTM model................................ 45 7.2.1 Basic model: least squares linear regression ...................................................................... 46 7.2.2 Ground-laying model without MT extraction and SAX filtering .................................. 46 7.2.3 Ground-laying model with different data feature configurations .............................. 47 7.2.4 Changing the rolling window size (W) .................................................................................. 48 7.3 Model improvement ............................................................................................................................. 49 Chapter 8 Conclusions, recommendations, and future research ....................................................... 50 ix 8.1 Conclusions .............................................................................................................................................. 50 8.2 Limitations and recommendations ................................................................................................. 52 References ............................................................................................................................................................. 54 Appendix ................................................................................................................................................................ 67 x List of tables TABLE 1 MODELING COMPONENTS OF THIS THESIS ALONG WITH PREVIOUSLY DEVELOPED VEHICLE POWER CONSUMPTION NEURAL NETWORKS MODELS .............................................................................................. 15 TABLE 2 FIRST TEN ROWS OF THE DATASET WITH SECOND-BY-SECOND SAMPLING DISTRIBUTION. ............. 18 TABLE 3 ILLUSTRATION OF THE DISTRIBUTION OF THE MTS ALONG THE 10 BINS. ........................................ 26 TABLE 4 THE SELECTED VALUES FOR Ω OR A FOR EACH OF THE 10 BINS. ........................................................ 27 TABLE 5 HYPERPARAMETER TUNING TEST CASES WITH SELECTED HYPERPARAMETERS. .............................. 38 TABLE 6 EXAMPLE OF A TIMESERIES WITH EIGHT ROWS OF DATA, THREE FEATURES, AND ONE OUTPUT ..... 39 TABLE 7 TIMESERIES DATA IN TABLE 6 RECONSTRUCTED INTO A TENSOR STRUCTURE WITH A ROLLING WINDOW SIZE W=2. ..................................................................................................................................... 39 TABLE 8 TIMESERIES DATA IN TABLE 6 RECONSTRUCTED INTO A TENSOR STRUCTURE WITH A ROLLING WINDOW SIZE W=3. ..................................................................................................................................... 40 TABLE 9 KEY PERFORMANCE INDICATORS OF THE GROUND-LAYING OPTIMIZED DLSTM MODEL. ............... 44 TABLE 10 QUANTITATIVE SUMMARY OF THE GROUND-LAYING OPTIMIZED DLSTM TRAINING AND FORECASTING RESULTS: THE MAXIMUM, THE MINIMUM, AND THE AVERAGE OF THE ACTUAL AND THE PREDICTED POWER VALUES. .......................................................................................................................... 44 TABLE 11 KEY PERFORMANCE INDICATORS OF THE LEAST SQUARE LINEAR REGRESSION MODEL ................. 46 TABLE 12 KEY PERFORMANCE INDICATORS OF THE MODELS WITHOUT MT EXTRACTION AND SAX FILTERING ......................................................................................................................................................................... 47 TABLE 13 KEY PERFORMANCE INDICATORS OF THE MODELS WITH DIFFERENT DATA FEATURE CONFIGURATIONS ........................................................................................................................................... 48 TABLE 14 KEY PERFORMANCE INDICATORS OF THE MODELS WITH DIFFERENT ROLLING WINDOW SIZE (W) ......................................................................................................................................................................... 49 TABLE 15 THE SELECTED TEST CASES USED TO CREATE THE OPTIMAL MODEL AND THE EVALUATION SCORES ......................................................................................................................................................................... 49 xi TABLE 16 HOW MT SELECTION FROM SPEED PROFILES CAN BE EXPANDED TO THE REST OF THE ATTRIBUTES. ......................................................................................................................................................................... 67 TABLE 17 HYPERPARAMETER JARGON TERMS DEFINITIONS. ............................................................................. 67 xii List of figures FIGURE 1 THE THREE STEPS TOWARDS ESTIMATING POWER CONSUMPTION: DATA PREPROCESSING, ANN MODEL DEVELOPMENT, AND TESTING ALTERNATIVE MODEL CONFIGURATIONS AND FORMULATIONS TO DEFINE THE BEST SUITED MODELING DESIGN. ................................................................................................ 4 FIGURE 2 A SNAPSHOT OF THE SPEED DATA ......................................................................................................... 18 FIGURE 3 A SNAPSHOT OF THE GRADE PROFILE, HIGHLIGHTING SOME THE FLAWED DATAPOINTS. ............... 20 FIGURE 4 THE SAVITZKY-GOLAY FILTERED GRADE PROFILES ALONGSIDE THE ORIGINAL SIGNAL AND THE CORRESPONDING TUNED PARAMETERS. ....................................................................................................... 21 FIGURE 5 A MT SPEED PROFILE. ........................................................................................................................... 22 FIGURE 6 MTS AND STOPS BETWEEN THEM FORM A TRIP. ................................................................................. 23 FIGURE 7 REPRESENTATION OF A MT BY A WORD USING SAX. ......................................................................... 25 FIGURE 8 EXTENDING THE LENGTH OF THE MTS AT EACH BATCH TO THE LONGEST MT IN THE SAME BATCH. ......................................................................................................................................................................... 26 FIGURE 9 A RANDOM OUTCOME FROM THE SAX ANALYSIS: A SET OF MOTIFS FROM BIN NUMBER 8. ............ 28 FIGURE 10 A RANDOM OUTCOME FROM THE SAX ANALYSIS: A SET OF DISCORDS FROM BIN NUMBER 8. ..... 29 FIGURE 11 SCHEMATIC REPRESENTATION OF A NEURON................................................................................... 31 FIGURE 12 STRUCTURE OF A DNN MODEL WITH 3 INPUTS, SEVERAL HIDDEN LAYERS AND ONE OUTPUT. .. 32 FIGURE 13 EXAMPLE OF A TRAIN AND TEST LEARNING CONVERGENCE CURVE, SHOWCASING THE EARLY STOPPING PRINCIPLE. ..................................................................................................................................... 33 FIGURE 14 RNN LOOP STRUCTURE FOR THE TTH INPUT. .................................................................................... 33 FIGURE 15 LSTM LOOP STRUCTURE. ................................................................................................................... 35 FIGURE 16 LOSS CURVES FOR THE GROUND-LAYING OPTIMIZED DLSTM MODEL........................................... 43 FIGURE 17 ACTUAL VERSUS THE ESTIMATED POWER VALUES FOR A RANDOM TWO-MINUTE TIME INTERVAL. ......................................................................................................................................................................... 45 xiii List of abbreviations API Application Programming Interface ANN Artificial Neural Networks BEV Battery Electric Vehicle C2ITS Data-driven Intelligent Transportation Systems DNN Deep Neural Networks EV Electric Vehicle FCEV Fuel-cell Electric Vehicle GHG Greenhouse Gas HEV Hybrid Electric Vehicle ICE Internal Combustion Engines MT Micro-trip PHEV Plug-in Hybrid Electric Vehicle RMSE Root Mean Squared Error R2 Squared Correlation Coefficient SAX Symbolic Aggregate Approximation xiv Acknowledgements The past two years at UBC have been quite a journey of struggles, tears, and learning. I would like to pay immense gratitude to those who supported me through thick and thin on this journey. I am indebted to my supervisor, Dr. Walter Mérida. He backed me up when I found my passion for data science more than mechanical simulations and encouraged me to pursue my dream career. I would like to extend my gratitude to my committee: Dr. Dana Grecov and Dr. Fariborz Taghipour. I express my appreciation to Dr. Julián A. Fernández-Orjuela. Julian was the first person who gave me the nod for a shift to a new project halfway in my master's program and kindly welcomed me to his research sub-group. He helped me come up with the main idea of my research and was always available and willing to answer my questions. My very special thanks to Nadia Adame for being exceptionally friendly and helpful with the research team. Her motherly care and attention towards everyone in our lab along with her amazing administrative skills relieved a lot of bureaucratic hurdles. xv Dedication I dedicate this work to my mom, my dad, and my best friend, Arman. You bring me love, care, and encouragement.1 Chapter 1 Introduction 1.1 Climate crisis Greenhouse gas (GHG) emissions are recognized as the main anthropogenic contributor of global climate change and the upward trend of global surface temperatures [1]. From 1998 to 2014, the global surface temperature has increased by 0.106°C per decade [2]. This Human-induced [3] climate change has had grave consequences for the planet, some of which include extreme weather events, higher wildlife extinction rates, coastal erosion, arctic erosion, and resurgence of some viruses and infectious diseases [4]. Air pollution is another contemporary problem linked to particulate matter and other tropospheric pollutants [5] that have a negative impact on human health and well-being [6]–[8]. The United Nations Member States have articulated the urge for combating climate change as one of the seventeen Sustainable Development Goals in 2015 for the year 2030 [9]. The Paris Agreement within the United Nations Framework Convention on Climate Change is aimed to cap average global temperatures below 2°C above pre-industrial levels [10]. The global transportation sector accounted for roughly a quarter of the Carbon Dioxide equivalent emissions from 1990 to 2015 in Canada [11]. The global passenger car fleet is projected to double by 2050, aggravating the environmental crisis [1]. National and international policy instruments have restricted tailpipe emissions by the road transportation sector to reverse its rising GHG emissions [12], [13]. Widespread electric vehicle adoption can reduce or eliminate the air pollution and GHG emissions associated with road transportation. 1.2 Electric vehicles 1.2.1. Overview An electric vehicle or EV in short, is a vehicle that is propelled entirely or partially by electricity and draws its required current from either storage batteries, power grids, or hydrogen powered fuel cells. EVs’ major types include Battery Electric Vehicle (BEV), Hybrid Electric Vehicle (HEV), Plug-In Hybrid Electric Vehicle (PHEV), and Fuel-cell Electric Vehicle (FCEV). BEVs’ or fully electric cars’ sole means of power supply are rechargeable battery packs, whereas HEVs and PHEVs use a combination of internal combustion engines (ICE) and electric motors as their propulsion source. The difference 2 between HEVs and PHEVs is that PHEVs, unlike HEVs, leverage batteries that may be recharged by power grids, allowing for greater driving ranges. FCEVs, on the other hand, incorporate on-board fuel cells into their powertrain to generate electricity. 1.2.2 Electric vehicles’ decarbonization strategy for transportation The electrification of mobility is a pathway towards decarbonized transportation if the electricity mix is from low- or zero-emission sources. While the GHG emissions from the supply of the electricity for EVs cannot be overlooked [14], [15], they can still reduce GHG emissions significantly over the course of their lifetimes [16]–[21]. This enables EVs to emit less carbon dioxide equivalent emissions per km travelled [22], [23]. Recent environmental impact studies on EV power consumption show that even EVs powered by the most carbon-intensive electricity fuel mix including coal-fired plants can still contribute to the decarbonization of transport by nearly 20% GHG emission reduction [24] and 6– 16% reduction if carbon capture sequestration is not employed [16]. Energy efficiency assessments of EVs highlight their competitive power consumption compared to the ICE vehicles [19], [25]. This is mostly due to electric motors high efficiency, enabling them to at least halve the use of power consumption compared to ICEs [26]. EVs produce no air pollutants (sulfur dioxide , oxides of nitrogen, and soot emissions), thus offer partial solution to local air pollution [27]. Compared to ICE vehicles, EVs make minimal heat and noise emissions, zero olfactory discomfort, and less motor vibrations allowing for greater comfort for passengers and pedestrians. 1.2.3 Challenges associated with the shift from ICE towards EV Increasing number of cities and countries around the world are transitioning from fossil-fuel towards electrified transportation. The global EV fleet exceeded 5 million in 2018 [28], reaching more than 2 million sales in 2018 solely, up from just a few thousand in 2010 [29]. Despite the rapid market penetration, EVs represent a small portion of the global fleet (under 3%) [29]. Their adoption is hindered mainly by higher upfront purchase prices compared to equivalent conventional vehicles. Additionally, some EVs have limited driving range. Limited mileage is considered a core barrier holding EVs back. It can lead to range anxiety, which refers to EV drivers’ fear about having insufficient battery charge to reach the destination. 3 1.2.4 Regulatory and incentives environment Various grants, incentives, and rebate programs are offered by national and international agencies to increase EV uptake. This support can accelerate widespread deployment of EVs by reducing the cost of ownership for both suppliers and end-customers. In Canada, the federal government provides Point-of-Sale purchase incentives of up to $5,000 per vehicle [30]. Furthermore, stringent laws have been promulgated on vehicles’ tailpipe emissions under the premise of GHG emissions reduction targets. For instance, The British Columbia (B.C.) Government passed the Zero-Emission Vehicles Act into law in 2019 [31]. Under this legislation, all new light-duty cars and trucks sold in B.C. must be zero-emission (i.e., propelled by either electricity or hydrogen) no later than 2040. 1.3 Motivation for estimating vehicle’s power consumption Power consumption estimations enable calculations on energy costs (or fuel costs in ICEs). The results of the fuel consumption1 models (with equivalent power consumption models for EVs) are also a key prerequisite for many GHG emission assessment models [32]–[34]. These quantitative evaluation metrics are useful for policy-makers [35]. Moreover, the development of energy estimation models can serve as a core foundation for a diverse set of Data-Driven Intelligent Transportation Systems (D2ITS) solutions. D2ITS represent the application of emerging technologies to create an interdisciplinary framework for achieving better safety, mobility, and efficiency of transportation systems. Examples of the current approaches include dynamic eco-driving techniques, energy saving routes or eco-route planning [36], [37], and smart traffic congestion control. 1 It is possible to relate power consumption with fuel consumption given the availability of the energy content of the fuel and the miles traveled data. 4 1.4 Research objective and outline The main target of this thesis was to develop a deep learning framework for estimating the power use of EV drivetrains. The framework uses an artificial recurrent neural network architecture called long short-term memory (LSTM) and combines different techniques for data curation and analysis. One of the main reasons to choose deep learning over a physical model was a desire to minimize the use of coefficients and parameters linked to the physical variables describing the motion dynamics of the vehicle (e.g., drag and rolling coefficients). In most situations, these coefficients are either not available or vary in time, making the models difficult to develop and validate. On the other hand, data-driven approaches use the data collected under normal operation, requiring more straightforward measurements (e.g., speed, elevation, time of the day). A more thorough comparison between these approaches can be found in section 2.2.1.2. The solution suggested in this thesis proceeds in three major parts, spanning over eight chapters as shown in Figure 1. First, data pre-processing was performed, which involved the process of identification and elimination of erroneous samples and SAX filtering (chapters 3 and 4). Second, and after reviewing power consumption model-development methodologies and their influential factors, a data-driven framework integrating ANN networks was developed. Subsequently, relevant hyper-parameters were tuned (chapters 5 and 6). Lastly, and because a suitable ANN training algorithm is not known a priori, a set of test cases were run to define the best suited modeling design (chapter 7). Figure 1 The three steps towards estimating power consumption: data preprocessing, ANN model development, and testing alternative model configurations and formulations to define the best suited modeling design. This thesis is organized as follows: Chapter 1 is an overview of EVs and their vital role on mitigating the enduring effects of climate change and local air pollutants. A broad review of studies that have shed light on most influential factors to power demands is provided in chapter 2 along with an overview of existing ANN modeling methods for estimating power consumption of vehicles. In 5 chapter 3, data preprocessing methods that are utilized to prepare the data for modeling are explained. SAX filtering technique is described in chapter 4. Chapter 5 elucidates the ground-laying model and deep learning methods utilized in this study. Implementations are done in chapter 6. Results of the ground-laying model are presented in chapter 7. A series of test cases are defined and evaluated against the ground-laying model in the same chapter. Final conclusions are drawn in chapter 8 together with limitations of the power consumption estimation framework proposed herein. Recommendations for the future works are proposed in the same chapter. 1.4.1 Introduction to artificial neural networks and LSTM The combined effects of massive database availability and fast calculation capabilities enable the use of data-driven models including artificial neural networks (ANN). ANNs are algorithms inspired by and loosely modeled after biological neural networks. ANNs endeavor to perform cognitive functions such as interpreting data and extracting non-trivial information from databases through iterative trial and error processes (dataset training), which are close to human reasoning. Long short-term memory (LSTM) methods are ANNs designed to process timeseries data. More details on this modeling technique are provided in chapter 5. 1.4.2 Introduction to SAX filtering Symbolic Aggregate approXimation or SAX was developed to allow for dimensionality reduction of a numerical series, e.g., a sequence of values indexed by time, namely time-series, into a shorter chain of characters which are meant to approximately represent patterns across a large dataset. This discrete symbolic method representation can find recurrent and anomalous patterns inside a dataset. Compared to other techniques such as discrete Fourier transformation, SAX filtering is a more simple and straightforward method to label unstructured (unlabeled) data, requiring low computing power. More details on this modeling technique are provided in chapter 4. 1.5 Research contributions The most significant contributions of this study are fourfold: 1. This study reports the first use of an ANN model on battery-electric bus powertrains. The model can be utilized for other types of electric vehicles. The focus of this study is to examine the 6 viability and advantages of making use of ANN models towards estimating power consumption for electric buses. 2. This study reports SAX filtering to clean driving-cycle data for the first time. A SAX filtering method is used to discard outliers (relatively infrequent behaviors of the system) from the most frequent patterns in numerical series, in this case temporal measurements of driving cycles. By employing SAX filtering, typical dynamics of the system can be extracted. 3. As a contribution to the analysis and design of the ANN model developed in this study, a parametric design process is applied on the model. The parametric design process developed in this study has a more extensive parameter pool compared to previous studies (section 6.2). A grid-search scheme was applied over a set of pre-defined system parameters and iteratively varied to achieve the best model architecture hence, the best performance. 4. This thesis contributes to the body of knowledge by finding a data-driven model with an acceptable accuracy (with an error of about 5% and a squared correlation coefficient of beyond 90%), compared to the other configurations tested. This model takes into account the effect of the system inputs and formulations (through extensive range of inputs and formulations) on the power consumption estimates based on the driving cycle data (sections 7.2 and 7.3). 7 Chapter 2 Power consumption models: literature review 2.1 Power consumption influencing factors Earlier efforts on power consumption modeling determined several power consumption factors. It is possible to relate the power consumption and fuel consumption upon the availability of the energy content of the fuel and the miles traveled data. Therefore, although the aim of this thesis is to estimate power consumption, the previous research on fuel consumption modeling for ICE vehicles is relevant. Hence, previous literature on both fuel consumption and power consumption have been summarized in this chapter. As summarized by Ahn et al., vehicle fuel consumption impact factors can be grouped into vehicular, traffic, roadway, surroundings, and driver related categories [38]. The first category includes factors such as the instantaneous efficiency of vehicle components and the overall powertrain efficiency, the weight of the vehicle, number of the cylinders, and most importantly [39], the manufacturing quality. Sullivan et al. investigated the impact of the total vehicle weight on the fuel consumption [40]. Their findings show that with a 10% weight reduction with constant payload mass, fuel consumption drops by 2% for trains and trucks and by 4% for buses. Predictably, increasing the share of light-weighted materials in vehicle components results in less fuel consumption rates [41]. Besides, Holdstock et al. examined a set of four vehicles to explore the role of vehicle drivetrain topology on fuel consumption [42]. Their results underline that adoption of multiple-speed mechanical transmissions enhances the performance and fuel consumption of vehicles compared to single or double speed variants. The second category includes traffic conditions which indirectly induce speed profiles of the vehicles, thus can affect their fuel consumption [43]. Tielert et al. assessed economic and environmental benefits of wirelessly receiving traffic-light-to-vehicle communications to schedule an optimum speed profile and avoid braking and acceleration maneuvers [44]. According to their results, utilizing this technique can reduce fuel usage by approximately 22%. Asadi and Vahidi formulated an optimization-based control algorithm which enables a predictive cruise control system to achieve better fuel economy for the vehicle using upcoming traffic signal information [46]. They report approximately 47% fuel consumption savings when this technique is utilized. Besides tailoring the vehicle speed profiles (i.e., acceleration and braking) in line with traffic condition, several recent studies explored potential fuel consumption savings by means of traffic control schemes in Intelligent Transport Systems. Barth and Boriboonsomsin found that CO2 emissions drop by up to 20% with a 8 decrease in traffic volume, especially traffic congestions, using traffic control techniques [47]. Other studies have echoed similar conclusions [48]–[51]. The third category of factors involves the roadway. Road topography and its physical characteristics such as the surface roughness affect fuel economy [52]. Yang et al. developed numerical simulations to study the effects of the road’s slope on EVs’ electricity consumption [52]. Their findings suggest that EVs’ electricity consumption increases with the uphill’s tilt angle and decreases with the downhill’s tilt angle. The fourth category relates to environmental factors including the temperature, the humidity, and the wind speed [53]. Aside from their impact on the engine’s performance, these environmental factors affect the fuel consumption indirectly through the vehicle attachments such as heating, ventilation, and air conditioning [54], [55], [56]. The fifth and last category corresponds to driver-related factors associated with fuel consumption [57]–[61]. Driver-related factors refer to driving style, which can vary the energy consumption by up to 30 − 40% [62]–[66]. Conscious avoidance of intense acceleration and deceleration maneuvers by driver can also save fuel consumptions [58], [67]. The dataset available for this thesis encompasses the vehicle’s speed and acceleration, a binary variable that captures whether the vehicle was driving on a weekday or a weekend, along with the hour of the day for each observation (the second category), and the grade of the road (the third category). Since the input data in this thesis has been collected over one month, it may not have been long enough to capture the environmental factors such as the temperature variation across the seasons (the fourth category). For future work, if available, it is advisable to incorporate other power consumption factors such as the efficiency of vehicle components (the first category). 2.2 Power consumption modeling Fuel consumption models can be classified by their granularity (resolution level) or by the modeling framework. Regarding their granularity, either macroscopic, mesoscopic or microscopic estimation approaches can be adopted to match the model’s intended use [68]. Microscopic models, such as the fuel consumption model developed by Rakha and Kamalanathsharma [69] estimate instantaneous rates 9 of change for model parameters using instantaneous driving cycle data. Macroscopic models, on the other hand, use aggregated data and are more suitable for fleet-wide long-term planning. Mesoscopic models use inputs that combine microscopic and macroscopic models, and tend to model daily vehicle operation. The results from this model are often described as fuel per distance travelled, for instance, liters/km or kWh/km. This approach has been used by a great number of authors in literature including [36], [37], [70]–[73]. Regarding their modeling framework, physical, or data-driven approaches can be utilized to capture the quantitative relationship between the power (or the fuel for ICE vehicles) and consumption impact factors (section 2.1). 2.2.1 Physical approaches Physical models, also called white-box, knowledge-driven, or simulation models, are mathematical representations of a system and describe physical and chemical processes using algebraic or differential equations. Physical fuel consumption models are established based on the vehicle’s powertrain system behavior and simulated dynamic vehicle operation [74]–[76]. Fuel consumption (and in the case of this thesis, power consumption) rates can be calculated from the traction required at the wheels (backward model) [43], [55], [77], [78] or alternatively, from the torque generated from the battery or the engine and transmitted forward to the wheels (forward model) [79]. Physical models use first-order principles, such as Newton's laws of motion and Maxwell's electromagnetism equations as the building blocks of the model. These models make a link between fuel or power consumption and powertrain parameters. Much effort has been made to develop physics based fuel consumption models for ICE vehicles [36], [80]–[82] as well as power consumption models for EV powertrains: [83]–[86]. Some methods derive fuel consumption primarily based on the distance traveled [87], [88]. However, due to the complexity of the vehicle dynamic system, the majority of the physical models require considerable vehicle parameters. This issue often makes the computer simulation of the physical models inefficient or infeasible [89]. Moreover, physical models have limitations in terms of scalability. Modifying parameters of the model to adapt a new vehicle’s configuration would induce increasingly greater complexities thus makes the computation process cumbersome and increases the modeling costs [75]. 10 2.2.2 Data-driven approaches Data-driven models also called black-box models characterize the system under study by capturing connections between the system state variables, e.g., concurrent input and output time series [90], [91]. Compared to physical models, data-driven models lack transparency and physics in their structure, often leading to wider confidence intervals when their accuracies are reported. However, feeding the model with real-world measurements can produce results that are closer to the system being modeled. Data-driven models can achieve significant predictive and explanatory adequacy without sound prior knowledge of the system behavior or even guiding hypotheses. They are specifically useful for relatively complex systems where building an accurate system model is prohibitively difficult. Concerning energy consumption modeling for EVs, data driven approaches consider the whole system as a black box and estimate the fuel consumption rates based on either driving cycles [66], [92]–[95] or real world measurements [90], [91], [96]–[103]. The later approach acquires data with temporal resolutions that are higher than those available from driving cycles. Additionally, these black box models can be combined with a priori physical knowledge. Such grey box models have been reported to provide improved fitting performance [104]. Data-driven modeling relies upon artificial intelligence which refers to the ability of machines to learn from data, based on training examples that include classification [90], pattern recognition, and predictive analysis [105]. The performance of data-driven models relies on measured data describing the system variables of interest and the desired output. Bearing the advantages of the data-driven approaches over the physical models in mind, and with the aid of the instantaneous dataset available for this study, this thesis takes a detailed data-driven approach to estimate instantaneous vehicular power consumption. 2.3 Literature on vehicle power consumption neural networks models 2.3.1 Literature review There currently exists a body of literature on models that exploit artificial neural networks to determine vehicular fuel demand under varying transportation modes, from aircraft [106], [107] to 11 gasoline [108], diesel [109], and bio-diesel [110] engines, conventional cars [111]–[113], heavy-duty vehicles [114], [115], hybrid vehicles [116], and plug-in hybrid vehicles [117]. Togun et al. proposed an ANN model based on a back-propagation learning algorithm that predicted torque and brake specific fuel consumption in a gasoline engine [108]. They performed experiments where they produced detailed engine characteristics such as spark advance, throttle position and engine speed and exploited these data as input layer components for training and testing their model. They held 18 samples out of a total of 81 for testing the model and used the rest for training their model of one hidden layer. The results of their model yielded 0.9997, 0.9971 for 𝑅𝑅2 correlation coefficients of the torque and brake specific fuel consumption respectively. Parlak et al. aimed at predicting engine-based specific fuel consumption and exhaust temperature of a diesel engine [109]. They utilized back propagation learning algorithm using the Neural Networks Toolbox in MATLAB. The model consists of 3 input nodes, followed by 7 hidden nodes in the hidden layer and finally 2 nodes at the output layer. Three input nodes represent three input parameters (engine speed, break mean effective pressure and injection timing) and the two output parameters represent the desired outcomes being specific fuel consumption and exhaust temperature. The network was trained using 60 values out of a total of 80 and was evaluated using the remaining untouched values. After training the network for about 900 cycles, their model generated a mean absolute relative error no larger than 2%. A three-layer feed forward ANN architecture was introduced by Cirak et al. [110] where the objective was to predict the torque of a diesel engine fueled with biodiesel and eurodiesel mixture fuels. The studied input variables were coolant and exhaust temperatures, engine speeds and biofuel mixtures, and the expected output was the engine torque. An experimental set of 60 values were available of which 30 were used for training the model and 11 values to test the trained network. More specifically, 5 samples were used for model validation and 6 samples were chosen randomly for testing the performance of the trained model. The model employed Levenberg-Marquardt optimization technique [118]. By iteration, the optimal number of neurons for hidden layers were found to be 15. The setup was performed in MATLAB. The mean squared error and the 𝑅𝑅2 correlation coefficient (defined in section 6.4) were calculated as 0.0007 and 0.98, respectively. Cheng et al. presented a big, data-based deep learning algorithm, capable of predicting vehicular speeds at both freeway and urban traffic conditions [110]. Four speed influencing 12 categories including driver behavior, route information (e.g., speed limit, route types, and route curvature), traffic flow and lastly weather factors are fed into a 5-layer adaptive Neuro-Fuzzy Inference system to be trained through supervised learning. The data comprises of 94 actual driving samples collected by a single test driver. 67% of the data (63 samples) was randomly selected and used for training the model and the rest (31 samples) was saved for validation and testing. The model was implemented in MATLAB and yielded an average RSME of 7.7311 which outperformed their calculated baseline models. A dataset comprising 3693 values was utilized by Kanarachos et al. for a surrogate method based on indirect monitoring using Recurrent Neural Networks (RNN) to estimate instantaneous fuel consumption for vehicles [112]. They utilized smartphone measurements of GPS position, speed, altitude, acceleration, and number of visible satellites. They then incorporated these speed influencing factors into two different RNN types and tuned their architectures by performing a parametric study. In order to avoid overfitting, they split the driving samples to portions of 70%, 15%, and 15% for training, validation, and testing, respectively. Their gradient-based algorithms were terminated after 1000 epochs which achieved better performance compared to the boosted tree and support vector regression methods. Predić et al. applied an instantaneous ANN based mathematical model to experimentally measured data recorded through on-board diagnostics equipment on passenger cars during the winter period in the city of Niš in Serbia [113]. The input variables they considered includes the speed of the vehicle, the city zone, the hour of day and the day of week. Their results suggest that the time of day, the ability to maintain a constant average speed in the given city zone and finally, the day of week are the most strongly related factors to their model’s output (the fuel consumption of the passenger cars). They also reported that driving on weekend in city of Niš would increase the fuel consumption to as high as 4%. The model evaluation based on the test data shows a RMSE and mean absolute percentage error of 1.42 and 13.3% respectively. The fuel consumption of heavy-duty vehicles using ANN techniques has been investigated extensively. Bhoraskar presented two approaches of modelling fuel consumption of long-haul heavy-duty trucks, particularly, semi-trailer trucks [114]. One of the approaches utilizes all the variables from their measurement data and the second one incorporates road and vehicle associated factors -which are almost half the number of available feature- as input parameters in the model only. They applied various machine learning techniques, including linear regression, support vector regression, 13 random forest, and neural networks on their real-world measured data from road tests which were sampled at the rate of 10 Hz. 75% of the data was used for training and the remaining 25% was saved for model validation and testing the performance of the model. Their analysis revealed that the neural network model had the greatest predictive power among all four methods. Statistical assessment of the models in terms of RMSE are 0.097, 0.13, 0.098, and 0.092 for linear regression, support vector regression, random forest, and the neural network models, respectively. Bhoraskar also noted that the engine torque was the most influencing factor affecting the fuel consumption rates. In another study by Almér on heavy-duty vehicles, various machine learning methods were employed and applied to a rich dataset to predict fuel consumption of heavy vehicles [115]. The data were collected from several sources including Scania’s Fleet Management system, GPS, and weather data as well as the vehicle configuration information and was aggregated into a single dataset afterwards. The original temporal data comprised a set of 985,600 values sampled at a mix of 1-minute and 10-minute rates. The dataset encompassed almost 40 features, acquired from 62 trucks. Almér used this dataset on five different methods: linear regression, random forests, support vector regression and shallow artificial neural networks. The data were split into 67%, 11% and 22% for training, validation, and testing purposes, respectively. The results suggest that firstly, excluding the linear regression model, the performances of the remaining three do not differ greatly. It was also found that data collection sampling time of 10 minutes shows similar performance compared to 1-minute collection sampling. Lastly, the road slope, vehicle speed and vehicle weight were reported as the three most influential factors determining fuel demand rates. ANN models for EV fuel economy have been used in a number of recent papers including the work of Galang [116]. A set of temporal driving measurements of over 40 minutes of on-road collected at 1 Hz was utilized to predict instantaneous hybrid vehicle fuel economy and emissions using neural network models implemented in a MATLAB Neural Time Series toolbox. The input attributes to the model included hybrid state (𝑜𝑜𝑜𝑜 𝑜𝑜𝑜𝑜 𝑜𝑜𝑜𝑜𝑜𝑜), vehicle speed (𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚ℎ𝑜𝑜𝑜𝑜𝑜𝑜) and acceleration, engine speed (revolution𝑚𝑚𝑚𝑚𝑚𝑚𝑜𝑜𝑚𝑚𝑚𝑚), two vehicle specifications including system high pressure and low pressure, throttle and break positions, and elevation of the road. The one hidden layer neural network model predicted fuel consumption rates within a mean absolute error of 0-5%, outperforming the numerical, physics-based Autonomy model and a custom HEV simulation model developed at Colorado State University. In this case, an increase in the length of the dataset used to train the model increased the accuracy of the results. Remarkably, Galang’s study shows that apart from the vehicle velocity and acceleration, 14 other input parameters did contribute to the enhancements of the predictions only to a small extend. In addition, it was concluded that tuning hyperparameters of the model architecture did not positively affect the model performance. In another work, Goebel et al. adapted five machine learning techniques to estimate the utility factor for PHEVs, which is defined as the ratio of miles travelled on electricity to total miles traveled [117]. His daily driving data were collected from 1768 individual Chevrolet Volt PHEVs with an average of 300 driving days per vehicle. Linear model, regression tree, random forest, support vector machine, and neural networks methods were applied to the data. The models were trained on 90% of the data and tested on 10% unseen. The final mean 𝑅𝑅2 coefficient of correlation for linear model, regression tree, random forest, support vector machine, and neural networks were calculated and reported as 0.817, 0.805, 0.801, 0.864, 0.868, and 0.861, respectively. Goebel compared the performance of these models with a simple simulation (a non-statistical method) and concluded that the machine learning techniques obtain more accurate and robust results at the cost of lower interpretability. They also concluded that, according to their linear regression model, the most predominant affecting factors in estimating the utility factor are the variance and the skewness of vehicle miles traveled distributions, and the frequency of long-distance driving. In this section, we have included relevant literature related to estimating power consumption using ANN modeling in the transportation sector. As expected, the previous research concerning BE-buses is limited and most of the literature is dedicated to modeling fossil-fuel vehicles. Even though the data from such studies are not derived from BE-buses, reviewing their modeling designs is still helpful when implementing similar models. 2.3.2 Research gap The previous research on fuel consumption modeling for road vehicles using machine learning techniques shows that ANNs are among the best methods to utilize for advance modeling purposes. However, prior works on power consumption ANN modeling lack in depth sensitivity analysis and parameter tuning [38], [109], [111], [112], [117]. Moreover, the majority of the models employ shallow neural networks where only one hidden layer was imbedded [108]–[110], [113]–[116]. This thesis does sensitivity analysis and parameter tuning with multiple hidden layers. 15 Many of the existing models had the access an extensive list of attributes or were even able to measure and collect the model input parameters. However, most of the time, such elaborate description of the fuel consumption factors is not available. For instance, Bhoraskar utilized dozens of detailed input parameters [114]. This makes the results hard to replicate and impractical to adopt for other scenarios of interest. In contrast, some studies did not incorporate tightly linked influential factors such as speed or acceleration of the vehicle to their models [39]. Some models used the same data for validation and testing the performance or left no data for testing at all [119]. This may lead to overfitting, which describes an undesirable behavior of the model where it is prone to memorizing idiosyncratic patterns in training samples instead of generalizing to unseen data [119]. Another common pitfall is the existence of inaccuracies in the raw data that are not detected nor treated. Input inaccuracies can lead to unexpected discrepancies and can decrease the model performance [120]. Table 1 compares several of the modeling features utilized in this thesis against other ANN modeling approaches cited throughout the paper. Table 1 Modeling components of this thesis along with previously developed vehicle power consumption neural networks models Data Model Test Case Analysis Electric Bus Data Database Size Data Filtering Anomaly Detection Deep or Shallow Network Hyperparameter Optimization Parametric Study Togun et al. [108] No 18 No No Shallow Partially No Parlak [109] No 80 No No Shallow No No Cirak et al. [110] No 60 No No Shallow Partially No Cheng et al. [111] No 94 No No Deep No No Kanarachos et al. [112] No 3693 No No Deep Partially Partially Predić et al. [113] No 250 Yes No Shallow Partially No Bhoraskar [114] No N.A. Yes No Deep Yes Yes Almér [115] No 985 600 Yes No Shallow Partially Yes 16 Galang [116] No 4200 No No Shallow Yes Yes Goebel et al. [117] No 1768 No No Shallow No Yes This thesis Yes 722 149 Yes Yes (SAX) Deep Yes (grid-search) Yes The data section in the table reiterates the dataset and the data treatment details of the reviewed studies. It describes the size of the dataset of the studies (Database Size) and whether the vehicle under study was an EV bus. The two columns under the model column show the modeling specifications of the studies. The first column shows whether the ANN models utilized several hidden layers or one only. The second column shows if the studies optimized the hyperparameter of their ANN models. Lastly, the test case analysis shows whether the studies evaluated the performance of their model against alternative model configurations and formulations. From the literature review, we conclude that there are four main gaps in the research related to electric buses power consumption modeling using deep neural networks: 1. The need of a better understanding of the optimal hyper-parameters in an ANN model, 2. The need to improve the data-treatment and preprocessing, by applying techniques such as SAX filtering, 3. The need to improve our understanding of the impact of the power consumption factors on power consumption estimation and, 4. The need to increase the understanding of how the power consumption factors improve the estimation of power consumption when using data-driven models. 17 Chapter 3 Dataset, data preprocessing and data processing This chapter describes the process of preparing the input data for the ANN model. The variables used in the model are selected in the feature engineering section. The following section presents details on micro-trips identification and the shape specification for the inputs to the model. 3.1 Dataset This analysis is based on a dataset of driving cycles for battery electric buses. The buses ran over 4 weeks. The embedded dataloggers instrumented on the buses recorded and logged instantaneous driving data during on-road operation. The devices are connected to the CAN bus with a J1939 connector. The collected dataset comprises of 722,149 samples at a varied sample-rate of 0.1-10 Hz. It encompasses temporal measurements of road topography (road grade) along with the speed profiles (bus speed and acceleration), date and time (day of the week, weekday or weekend) and the power consumption. There are samples for which the power attribute, being the model output, is negative. The reason is related to the regenerative braking implemented in some EVs to enable energy recovery [121]. 3.2 Data processing Since the frequency at which the data is measured and recorded fluctuates in time, the dataset is resampled to 1 Hz to reformat the data samples so that their time duration is equal2. Interpolation is used as the resampling method. Table 2 illustrates a few first rows of the dataset with second-by-second sampling distribution. Figure 2 shows a snapshot of the speed data. 2 In regression problems involving time series, a prediction is executed on the assumption that the sampling time (or the sample frequency) in the future is known, and consistent or equal to the one of the historical data used for the prediction. Resampling is used to ensure that a there is a reference sampling time in the dataset. 18 Table 2 First ten rows of the dataset with second-by-second sampling distribution. Bus Speed (𝑚𝑚𝑠𝑠) Bus Acceleration (𝑚𝑚𝑠𝑠2) Altitude (𝑚𝑚) Road Grade Day of week Hour of day Weekday vs Weekend Power (𝑘𝑘𝑘𝑘) 1.67 -1.69 12.023077 -0.792641 0 21 0 1.488 1.11 -0.264 11.984615 -1.319338 0 21 0 29.14 1.39 -0.035 11.946154 -1.984507 0 21 0 22.506 1.11 -0.399 11.907692 -1.58498 0 21 0 19.592 0.83 -0.217 11.869231 -1.984507 0 21 0 12.462 1.11 0.218 11.830769 -2.653143 0 21 0 27.218 1.39 0.717 11.792308 -1.984507 0 21 0 40.61 1.67 0.026 11.753846 -1.58498 0 21 0 50.654 2.22 -0.205 11.715385 -1.319338 0 21 0 56.296 2.78 -0.527 11.676923 -0.992551 0 21 0 48.608 3.06 -1.152 11.638462 -0.792641 0 21 0 48.608 Figure 2 A snapshot of the speed data 19 3.2.1 Feature engineering The reported literature suggests that power consumption models can benefit from incorporating acceleration along with the speed profile. The acceleration of the buses (Equation 1) is integrated to the original dataset which was obtained from the instantaneous speed. Acceleration𝑚𝑚 �𝑚𝑚𝑠𝑠2� = 𝑑𝑑𝑉𝑉𝑚𝑚 �𝑚𝑚𝑠𝑠 �𝑑𝑑𝑡𝑡𝑚𝑚(𝑠𝑠) Equation 1 Where, 𝑑𝑑𝑉𝑉𝑚𝑚 and 𝑑𝑑𝑡𝑡𝑚𝑚 are the change in vehicle speed and time between two consecutive instances (1 second), respectively. The grade of the road is also calculated and is attached to the original dataset. It is calculated using the instantaneous altitude and the distance which was obtained from instantaneous speed of the bus (Equation 2 and Equation 3): Distance𝑚𝑚(𝑚𝑚) = � 𝑑𝑑𝑉𝑉(𝑡𝑡)𝑚𝑚 �𝑚𝑚𝑠𝑠 �𝐵𝐵𝐴𝐴 𝑑𝑑𝑡𝑡𝑚𝑚(𝑠𝑠) Equation 2 Grade𝑚𝑚 = tan−1 𝑑𝑑𝐴𝐴𝑚𝑚(𝑚𝑚)𝑑𝑑𝐷𝐷𝑚𝑚(𝑚𝑚) Equation 3 3.2.2 Grade filtering Preliminary data investigation revealed erroneous altitude values. The road grade is calculated based on the altitude data, and therefore, the altitude errors have caused extreme, unrealistic gradients. Figure 3 exhibits a snapshot of the grade profile and highlights the flawed datapoints. Even small discrepancies in the process of acquisition and transmission of the raw altitude data may amplify the final gross error of the road grade. Therefore, following the research in [122]–[124] a Savitzky-Golay (SAVGOL) smoothing technique has been adopted to attenuate the noise on this signal. The Savitzky-Golay technique combines moving average and linear regression methods under the premise that the grade data is corrupted by only a small quantity of datapoints, hence the grade waveforms are preserved after the filtering process without great trend distortions. This technique 20 fits a polynomial function on equally spaced, adjacent datapoints (so-called windows) and assigns the value to central points of each of the successive windows. The optimal length of the window and the order of the polynomial depend on the size of the dataset and the noise power [125]. These values are tuned through a heuristic approach and the visualization of the results from various tests. Figure 4 depicts the Savitzky-Golay filtered grade profiles alongside the original signal and the corresponding tuned parameters. Figure 3 A snapshot of the grade profile, highlighting some the flawed datapoints. 21 Figure 4 The Savitzky-Golay filtered grade profiles alongside the original signal and the corresponding tuned parameters. 3.3 Data processing: micro-trips identification A trip denotes a terminus-to-terminus journey spanning from the beginning to the last bus stops in the bus schedule. A micro-trip (MT) on the other hand, is defined as a snippet of the speed–time profile between each pair of consecutive stops events. In other words, a MT begins at a zero speed, undergoes an acceleration phase followed by a cruise phase and finally, slows down by a deceleration phase and ends where the bus comes to the next adjacent halt. Figure 5 depicts a random MT speed profile from the dataset. 22 Figure 5 A MT speed profile. A widely used practice in working with recorded speed–time driving cycles is to juxtapose MTs [90], [126]. This technique retains the records that best represent the ups and downs in the data and filters out the prolonged standstill periods of the bus, as well as the layover intervals. In this study, juxtaposed MTs from speed–time profiles are obtained by the said fashion to form a continuous chain of driving events while the bus is moving. The dataset of this study was time indexed, and the indexation of each speed MT could be applied to all other attributes. Table 16 in the appendix shows an example of how MT selection from speed profiles can be expanded to the rest of the attributes. A trip may be seen as a pool of MTs with intermediate long halts (longer than two seconds). Therefore, to extract MTs, one can cut out the prolonged stops from the speed profile. Figure 6 illustrates the MTs and stops between them form a trip. 23 Figure 6 MTs and stops between them form a trip. 24 Chapter 4 SAX filtering Symbolic Aggregate approXimation filtering or SAX in short, includes the process of annotating a continuous sequential dataset (such as the timeseries data of this thesis) for interpretation and pattern recognition purposes. This study makes use of this robust filtering technique to detect uncommon patterns in MT speed profiles and proceeds the rest of analysis with most repeated patterns only. This process is detailed in the following sections. 4.1 Methodology SAX [129] transformation algorithm involves a discretization process of a normalized time-series with an arbitrary length of n and converting it into a Piecewise Aggregate Approximation (PAA) vector of length 𝜔𝜔, such that 𝜔𝜔 « n. Then, a symbol, usually a letter from the alphabet, is assigned to each of the PAA vector components. The symbol probabilities are equally spaced with respect to each other. To this end, 𝛼𝛼 areas under the gaussian distribution curve N (0,1) of the normalized time-series are sliced equally so that each area has a 1/𝛼𝛼 probability. The slices are marked as breakpoints 𝜷𝜷 = [𝛽𝛽1, 𝛽𝛽2, … , 𝛽𝛽𝛼𝛼−1]. The value in each time interval (there are 𝜔𝜔 time intervals) is compare to the breakpoints and a symbol corresponding to the closest breakpoint is assigned to the interval. The smallest breakpoint is mapped to the first letter, usually an ‘a’, whereas the second smallest breakpoint gets tagged as ‘b’ and so on. The letters are then concatenated to form a word. Figure 7 illustrates an example of how a MT can be represented by a word (bddb) using SAX. The shown MT is split into 4 segments (ω=4) with the alphabet size of 4 (A=4). As shown, the symbolic representation of the MT is bddb. 25 Figure 7 Representation of a MT by a word using SAX. 4.2 Motifs and discords Upon having the SAX words, they can be re-grouped based on the frequency of the occurrence. Most frequent and relatively infrequent words can be tagged as motifs and discords, respectively [130]. Motifs characterize typical dynamics of the system whereas discords capture anomalous behaviors, most likely arisen from certain failures or experimental errors (either random or environmental systematic errors) [130]. Discords are then filtered out for further investigations for potential anomaly detections (which is out of the scope of this thesis). 4.3 Implementation 4.3.1 Binning the MTs Miller et al. implemented the SAX method on an engineering building project with a constant 𝜔𝜔 and 𝐴𝐴 [131]. They developed an automated filter which analyzes daily building energy meter data, filtering out abnormal (infrequent) profiles. Since the sampling time and the time window of analysis were constant (daily basis), Miller et al. simply chose the subsequence period count as 4 (𝜔𝜔 = 4), aggregating the 24 hour of a day in every 6 hours. However, the dataset in this study is based on MT speed profiles which don’t necessarily have the same time window length (whereas it was always 24h in Miller et al. analysis). If the time-window is considered constant and large enough to contain large MTs (in the order of 200 seconds or more), 26 small MTs, with potentially representative driving patterns, would be oversimplified. On the other hand, if the time window is considered constant to include small MTs (in the order of 5 to 25 seconds), large MTs would have to be broken and some important patterns would be lost. Therefore, to avoid neglecting representative MTs based on different MT lengths in the feed, MTs are uniformly grouped into several batches or bins. The selection of the number of batches should be in accordance with the total MTs and the range of MTs. Bearing these two variables in this study in mind and in order to facilitate the understanding of the filtering process, the number of batches is set to be 10. Table 3 shows the distribution of the MTs along the 10 bins. The MTs at each batch are then extended to the longest MT in that same batch (Figure 8). In table 3, for each bin, the table shows the number of MTs within the bin (Absolute frequency, Percentage relative frequency), along with the duration of the shortest and the longest MT inside each bin (Shortest MT, Longest MT). Figure 8 Extending the length of the MTs at each batch to the longest MT in the same batch. Table 3 Illustration of the distribution of the MTs along the 10 bins. Properties of MTs in Bins Bin Number Absolute Frequency Percentage Relative Frequency Shortest MT Longest MT 1 477 9% 3 15 2 563 10% 15 22 3 585 11% 22 29 27 4 468 9% 29 34 5 530 10% 34 39 6 606 11% 39 47 7 521 10% 47 56 8 554 10% 56 71 9 571 11% 71 101 10 544 10% 101 521 4.3.2 Selection of the length (𝝎𝝎) and the alphabet count (𝑨𝑨) of patterns The first step to starting the SAX analysis after normalizing the model input (the MTs) is choosing 𝜔𝜔 and the alphabet size, 𝐴𝐴. The maximum speed of the extended MTs of each batch and the length of the extended MTs are different at each batch, and in all likelihood, constant 𝜔𝜔 or 𝐴𝐴 values void a fair and valid filtering. Thus, in this study, to yield the best filtering outcomes, 𝜔𝜔 or 𝐴𝐴 are selected independently for each batch (or bin), according to the maximum speed of the extended MTs in each batch and the length in time of the corresponding extended MTs. Moreover, shorter MTs usually have three phases in their speed–time profiles: acceleration, cruise, and deceleration phases (Figure 5). Longer MTs might undergo more phases in between the stop events; however, the number of phases is most likely not linearly proportional to the duration of the MTs. In order for 𝜔𝜔 to capture the ups and downs of MTs more accurately, the larger the number of phases a MT has, the larger the 𝜔𝜔 should be. Taking these into consideration, the 𝜔𝜔 or 𝐴𝐴 for each of the 10 bins were selected as follows: Table 4 The selected values for ω or A for each of the 10 bins. bin number 𝜔𝜔 and A 1 2 3 4 5 6 7 8 9 10 𝜔𝜔 4 5 5 5 5 5 5 5 5 5 A 3 3 4 4 4 4 4 4 5 6 These numbers (𝜔𝜔 and A) were selected to be small enough so that the number of outliers was not larger than 5% in each bin (following the assumption that 95% of MTs is a good representation of 28 the probability distribution of the MTs in the bin), but kept as high as possible to keep a granular representation. 4.3.3 Integration of bins and results The process of SAX filtering is performed on each batch with its respective values of 𝜔𝜔 and 𝐴𝐴. Motifs are pulled out from each of the batches with the threshold of 0.02 following the definition of Miller et al. in [131]. As described in Equation 4, the MTs of each bin are marked as motifs if their SAX representation exceeds the threshold times the total MTs of each bin. Motif𝑏𝑏𝑏𝑏𝑚𝑚𝑏𝑏ℎ𝑖𝑖 eligibility: number of MTs𝑆𝑆𝐴𝐴𝑆𝑆𝑖𝑖 > threshold × total number of MTs𝑏𝑏𝑏𝑏𝑚𝑚𝑏𝑏ℎ𝑖𝑖 Equation 4 Figure 9 and Figure 10 show a random motif and discord from the outcomes of the SAX analysis, respectively. Figure 9 A random outcome from the SAX analysis: a set of motifs from bin number 8. 29 Figure 10 A random outcome from the SAX analysis: a set of discords from bin number 8. Finally, the motifs from each bin are merged into a single dataset as inputs to the models (alternatively, discords could be marked as outliers and excluded from the model input dataset). 30 Chapter 5 Modeling approach: deep LSTM 5.1 ANN and DNN models The combined effect of massive databases with rapid means of performing calculations encourages the use of data-driven models such as ANN. ANNs are a series of algorithms that are inspired by and loosely modeled after the biological neural networks. ANNs endeavor to perform cognitive functions such as interpreting data and extracting non-trivial information from databases through iterative trial and error process -so called dataset training-, close to the way a human brain operates. ANN functionality can be envisioned as an effort to learn an approximate mapping function 𝑜𝑜 ∶ 𝑋𝑋 →𝑌𝑌 where the function 𝑜𝑜 produces sufficiently close outcomes (𝑌𝑌) to a certain reference from the input data (𝑋𝑋). ANNs have series of layers of nonlinear processing units, so-called nodes, or neurons, each of which connects to neighbor layers. The layers break down into three types: input, output and in between, the hidden layer. The information from the outside world flows into the model through the input layer. The data then, through the connection weights, flows into the hidden layer to be trained. The processed data arrives at the output layer on the other side of the network structure. At the hidden layer, the input neurons’ values are multiplied by connection weights. Then, all the weighted terms are aggregated in a summation together with a constant term called the bias. Each neuron has a value between zero (fully inactive) and one (fully active) where the greater the weighted sum of the neuron in a layer is, the stronger the signal which triggers the successive neuron will be. With the goal of introducing non-linearity to the network, and based on the summed weighted values, activation functions come into play to decide whether the next neuron should be activated or not. This process repeats until a satisfactory outcome based on the defined metrics is achieved. Figure 11 illustrates a neuron in a ANN architecture. 31 Figure 11 Schematic representation of a Neuron. ANNs have strong adaptability to modeling complex dynamical systems nonlinear and multi-dimensional dynamic systems with acceptable accuracies [132], [133] and are specifically promising for estimation problems. As summed in [134], deep neural networks (DNN) or deep learning in short is a sub-field of ANNs which can adopt multiple hidden layers to train the data, compared to early ANNs that could have a single hidden layer only (shallow learning). DNNs, thanks to their deep (multiple) layers (Figure 12), are flexible models that can process large datasets (several hundreds of thousands of observations) from complex systems and unravel the intricacies of their systems even further than regular ANNs can. Notably, employing successive shallow learning models may not enable them to emulate the effects of deep learning [135]. This is due to DNNs’ ability to learn all layers of representation jointly while ANNs cannot adapt to changes from subsequent layers without human intervention [135]. 32 Figure 12 Structure of a DNN model with 3 inputs, several hidden layers and one output. 5.2 Overfitting and early stopping In DNNs, as data flow through more hidden layers, the model learns more about elaborate features in the data, resulting in greater learning capacity. Nevertheless, networks with large numbers of hidden layers are prone to suffer from a phenomenon called over-fitting. Since deep neural networks are heavily parameterized, they fit very accurately to the training examples. However, it is commonplace for them to fail to fit to an unseen dataset (such as a dataset for testing the model or test dataset in short). In fact, such models tend to memorize the input-output relations in training dataset which disables their generalization capabilities and therefore, they perform poorly on the test dataset. Moreover, when training a model, a relatively small number of epochs (number of iterations the learning algorithm works forward and backward through the entire dataset) results in an underfit model, a model that does not fit a desire functional behavior. A compromise is to interrupt training and updating the weights past the point where the model performance on held-out validation set reaches a plateau or starts to degrade. This is done by Early Stopping regularization methods. Figure 13 shows a validation curve and the key stopping point at which no performance enhancement is made within certain maximum number of epochs. 33 Figure 13 Example of a train and test learning convergence curve, showcasing the early stopping principle. 5.3 RNNs and LSTM for time series In ordinary ANNs, the information travels in a single direction from the input to the hidden layer(s) and lastly the output in the network. For this reason, they are so-called "feedforward" neural networks. Recurrent neural networks (RNN) are distinct from feedforward networks in that they have internal states (memories) which allow their neurons to send feedback signals through feedback loops. By leveraging their memories, the RNNs can loop the persistent previous information back into the network and use it in the present task. A graphical illustration of this loop is depicted in a compact form in Figure 14. Figure 14 RNN loop structure for the tth input. 34 In the figure above, 𝑋𝑋𝑚𝑚 is the input sequence to the model at time “t”, yt is the output sequence, and ht is the internal state, being a representation of the past history. A simple formulation of the RNNs can be presented as: ℎ𝑚𝑚 = tanh(𝑎𝑎𝑋𝑋𝑚𝑚 + 𝑏𝑏 ℎ𝑚𝑚−1 + 𝑐𝑐) Equation 5 𝑦𝑦𝑚𝑚 = 𝑔𝑔(𝑑𝑑 ℎ𝑚𝑚 + 𝑒𝑒) Equation 6 Where 𝑎𝑎, 𝑏𝑏, 𝑐𝑐, 𝑑𝑑, and e are RNN parameters. RNNs have been proven robust for sequential and temporal data forecasting [136]; however, traditional RNNs are limited by vanishing or exploding gradient issues. This difficulty occurs in training ANNs where the sequential data is so long, and the context goes far back in time that it makes the updates to initial weights either shrink (vanishing gradient problem) or inflate (exploding gradient problem). As a result, the performance of the network degrades significantly. RNNs with Long short-term memory [137] (LSTM) units alleviate this problem by retaining the previous information from longer time periods. Hence, the LSTM’s so-called forget gates that regulate the flow of information can be better informed whether to retrieve the invaluable previous information or disregard irrelevant pieces. Figure 15 shows a graphical representation of an LSTM loop structure. 35 Figure 15 LSTM loop structure. In the graph, xt is the input sequence (at time ‘t’), ct-1 and ct are the memories from the previous LSTM outcome and the new updated memory, respectively. Likewise, the ht-1 and ht are the output from the previous LSTM event and the current output, respectively. The + and the × operations in the grey circles represent adding and scaling of the information, respectively. The rectangular units represent neural network layers. 𝝈𝝈 stands for a sigmoid activation layer while tanh is a hyperbolic tangent activation layer. These activation functions introduce non-linearity to the output LSTM cells, to capture broader behaviors, beyond a linear fit. Strong predictive capabilities of RNNs with LSTM architecture are reported in a broad range of studies [138], [139] including in fuel consumption estimation model developments [112]. More specifically, deep RNNs with LSTM units are recognized as one of the best fitted algorithms for time-series forecasts [135]. Taking these into consideration, a deep RNNs with LSTM architecture is opted for to estimate power consumptions of BE-buses with copious amount of time-series EB-bus and driving data as input variables. 5.4 Computational tools The following frameworks and libraries have been used throughout this project: 1. Numerical and scientific computations: NumPy [140] 2. Data analysis: pandas [141] 36 3. Visualizations: matplotlib [142], Seaborn [143] 4. Machine learning libraries: TensorFlow [144], Keras [145], scikit-learn [146] In particular in this study, Keras (version: 1.14.0) has been used to construct the deep RNNs with LSTM models. Keras is an open source, high-level deep learning application-programming-interface (API) and runs seamlessly as the frontend of deep learning engines such as TensorFlow [147], [148], Theano [149], [150], or Microsoft Cognitive Toolkit [151]. It is one of the most widely used platforms for practical applications of DNNs mainly due to the relative ease of developing custom architectures of DNNs through its modular framework and its capability for combining different architectures for building more complex models. 37 Chapter 6 Implementation of the system: ground-laying optimized DLSTM model This chapter describes the Implementations of the proposed DLSTM model. The model instantiation is explained and a hyper-parameter tuning procedure is laid out. Then, several input preprocessing steps that have to be taken before training the model including the timeseries to tensor conversion, the dataset split and finally, and the feature scaling are detailed. 6.1 Instantiating the model Designing an ANN architecture in Keras involves building, training, and evaluating a model. A model is an arbitrary assembly of layers, fine-grained and tailored to pertain to the specific problem of interest. In Keras, DNN models can be instantiated in two ways, by using either Sequential or functional API classes. Sequential API class, as the name implies, allows for stacking different types of neural network layers with different defined characteristics one after another. Functional API classes offer more flexibility for building more complex models such as models with shared layers or with multiple inputs or outputs [147]. For the purpose of this study, a Sequential API class has been opted and instantiated. Defining a complete ANN architecture involves specifying types and numbers of layers. Among common types of core layers used in ANN are Input, Dense, Recurrent layer. Each of the layers should be characterized with parameters including the input data size, number of neurons, and an activation function. The model is then to be compiled with a specific optimizer algorithm and a loss function of choice. These parameters, which are called hyper-parameters, govern the entire underlying learning process and heavily influence the model performance [152]. 6.2 Hyperparameter tuning Compromising between predictability and generalization power for DNN models through the selection of hyperparameters is not a trivial task. Thus, a systematic optimization process using Randomized Search [153] from scikit-learn library in Python has been adopted to achieve the best hyperparameter configuration for this estimation task. There is a dependency among different hyperparameters and the finest tuning of them for the model relies on an exhaustive grid-search over all parameters in each loop. However, such task requires a high dimensional space and training time thus it is not practical. To avoid such computational burdens, instead of assessing every possible hyperparameter combination, a list of recommended values for each hyper-parameter is selected. 38 Then, the final value for each hyper-parameter is selected among the list by running the model in test cases for each value and selecting the one that maximizes the metric of choice. The hyperparameter tuning test cases are listed in Table 5. Hyperparameters that outperformed others in each row are highlighted in the table (the metrics for the evaluation is the 𝑅𝑅2 coefficient, as detailed in section 6.4). Table 17 in the appendix summarizes brief definitions of some of the terms used in table 5. Table 5 Hyperparameter tuning test cases with selected hyperparameters. Activation linear, relu, tanh, sigmoid Batch size 64, 128, 256, 512, 1024, 2048, 4096 Number of hidden layers 1, 3, 5, 10, 15 Number of neurons 10, 50, 100, 1000 Optimizer SGD, Adam, RMSprop, Adagrad Patience 3, 5, 10, 20 6.3 Input preprocessing 6.3.1 Reshaping the timeseries data into 3-dimentioanl formatting One last step before training the model with the selected hyperparameters is preparing the multivariate time series dataset for the use of LSTM algorithm. This is done by reconstructing the shape of the dataset into a 3-dimentioanl structure wherein a mapping function from a series of vectorial predictor variable (X) to a scalar target value (y) is sought. The process consists of moving a rolling window of size 𝑘𝑘 backward in time sequence and converting the dataset into X and y pairs that are independent of other observations and identically distributed in time. The rolling window reflects the dependency of the prediction of power for the next step on the values X in the previous ‘W’ time steps. Table 6 shows a timeseries dataset of ten observations with three features (F1, F2, F3) and one target value (y). Table 7 and Table 8 illustrate how the transformation from timeseries to a tensor shape looks like for a window of size W=2 and W=3. 39 Table 6 Example of a timeseries with eight rows of data, three features, and one output 𝑋𝑋, 𝑦𝑦 𝑇𝑇𝑇𝑇𝑚𝑚𝑒𝑒 𝐗𝐗 𝒚𝒚 𝐹𝐹1 𝐹𝐹2 𝐹𝐹3 0 𝑋𝑋0,𝐹𝐹1 𝑋𝑋0,𝐹𝐹2 𝑋𝑋0,𝐹𝐹3 𝑦𝑦0 1 𝑋𝑋1,𝐹𝐹1 𝑋𝑋1,𝐹𝐹2 𝑋𝑋1,𝐹𝐹3 𝑦𝑦1 2 𝑋𝑋2,𝐹𝐹1 𝑋𝑋2,𝐹𝐹2 𝑋𝑋2,𝐹𝐹3 𝑦𝑦2 3 𝑋𝑋3,𝐹𝐹1 𝑋𝑋3,𝐹𝐹2 𝑋𝑋3,𝐹𝐹3 𝑦𝑦3 4 𝑋𝑋4,𝐹𝐹1 𝑋𝑋4,𝐹𝐹2 𝑋𝑋4,𝐹𝐹3 𝑦𝑦4 5 𝑋𝑋5,𝐹𝐹1 𝑋𝑋5,𝐹𝐹2 𝑋𝑋5,𝐹𝐹3 𝑦𝑦5 6 𝑋𝑋6,𝐹𝐹1 𝑋𝑋6,𝐹𝐹2 𝑋𝑋6,𝐹𝐹3 𝑦𝑦6 7 𝑋𝑋7,𝐹𝐹1 𝑋𝑋7,𝐹𝐹2 𝑋𝑋7,𝐹𝐹3 𝑦𝑦7 Table 7 Timeseries data in Table 6 reconstructed into a tensor structure with a rolling window size W=2. 𝑋𝑋, 𝑦𝑦 𝑇𝑇𝑇𝑇𝑚𝑚𝑒𝑒 𝑿𝑿 𝒚𝒚 1 [[𝑋𝑋0,𝐹𝐹1 , 𝑋𝑋0,𝐹𝐹2 , 𝑋𝑋0,𝐹𝐹3], [𝑋𝑋1,𝐹𝐹1 , 𝑋𝑋1,𝐹𝐹2 , 𝑋𝑋1,𝐹𝐹3]] 𝑦𝑦1 2 [[𝑋𝑋1,𝐹𝐹1 , 𝑋𝑋1,𝐹𝐹2 , 𝑋𝑋1,𝐹𝐹3], [𝑋𝑋2,𝐹𝐹1 , 𝑋𝑋2,𝐹𝐹2 , 𝑋𝑋2,𝐹𝐹3]] 𝑦𝑦2 3 [[𝑋𝑋2,𝐹𝐹1 , 𝑋𝑋2,𝐹𝐹2 , 𝑋𝑋2,𝐹𝐹3], [𝑋𝑋3,𝐹𝐹1 , 𝑋𝑋3,𝐹𝐹2 , 𝑋𝑋3,𝐹𝐹3]] 𝑦𝑦3 4 [[𝑋𝑋3,𝐹𝐹1 , 𝑋𝑋3,𝐹𝐹2 , 𝑋𝑋3,𝐹𝐹3], [𝑋𝑋4,𝐹𝐹1 , 𝑋𝑋4,𝐹𝐹2 , 𝑋𝑋4,𝐹𝐹3]] 𝑦𝑦4 5 [[𝑋𝑋4,𝐹𝐹1 , 𝑋𝑋4,𝐹𝐹2 , 𝑋𝑋4,𝐹𝐹3], [𝑋𝑋5,𝐹𝐹1 , 𝑋𝑋5,𝐹𝐹2 , 𝑋𝑋5,𝐹𝐹3]] 𝑦𝑦5 6 [[𝑋𝑋5,𝐹𝐹1 , 𝑋𝑋5,𝐹𝐹2 , 𝑋𝑋5,𝐹𝐹3], [𝑋𝑋6,𝐹𝐹1 , 𝑋𝑋6,𝐹𝐹2 , 𝑋𝑋6,𝐹𝐹3]] 𝑦𝑦6 7 [[𝑋𝑋6,𝐹𝐹1 , 𝑋𝑋6,𝐹𝐹2 , 𝑋𝑋6,𝐹𝐹3], [𝑋𝑋7,𝐹𝐹1 , 𝑋𝑋7,𝐹𝐹2 , 𝑋𝑋7,𝐹𝐹3]] 𝑦𝑦7 40 Table 8 Timeseries data in Table 6 reconstructed into a tensor structure with a rolling window size W=3. 6.3.2 Split input to train, test, validation Development of an ANN model depends on a trilogy of observations including training, validation, and test sets, each of which used in different stages of creating the model. The definition of these glossary terms according to Ripley [154] are as follows: “Training set: A set of examples used for learning, that is to fit the parameters of the classifier [for this thesis, a regressor].” “Validation set: A set of examples used to tune the parameters of a classifier [regressor], for example to choose the number of hidden units in a neural network.” “Test set: A set of examples used only to assess the performance of a fully specified classifier [regressor].” The validation process (which uses the validation set) is tied to the training process in an iterative loop in which information leaks implicitly from validation errors to tune the model. In other words, the model that is created based on the training data updates its parameters based on evaluations through the validation data. The primary objective for holding the validation set out is to provide an unbiased evaluation of the model fit on training dataset. Successively, to determine a score for the final tuned model, the model runs through the test set for comparison purposes. It is important to lock away the test set from training and validation phases 𝑋𝑋, 𝑦𝑦 𝑇𝑇𝑇𝑇𝑚𝑚𝑒𝑒 𝑿𝑿 𝒚𝒚 2 [[𝑋𝑋0,𝐹𝐹1 , 𝑋𝑋0,𝐹𝐹2 , 𝑋𝑋0,𝐹𝐹3], [𝑋𝑋1,𝐹𝐹1 , 𝑋𝑋1,𝐹𝐹2 , 𝑋𝑋1,𝐹𝐹3], [𝑋𝑋2,𝐹𝐹1 , 𝑋𝑋2,𝐹𝐹2 , 𝑋𝑋2,𝐹𝐹3]] 𝑦𝑦2 3 [[𝑋𝑋1,𝐹𝐹1 , 𝑋𝑋1,𝐹𝐹2 , 𝑋𝑋1,𝐹𝐹3], [𝑋𝑋2,𝐹𝐹1 , 𝑋𝑋2,𝐹𝐹2 , 𝑋𝑋2,𝐹𝐹3], [𝑋𝑋3,𝐹𝐹1 , 𝑋𝑋3,𝐹𝐹2 , 𝑋𝑋3,𝐹𝐹3]] 𝑦𝑦3 4 [[𝑋𝑋2,𝐹𝐹1 , 𝑋𝑋2,𝐹𝐹2 , 𝑋𝑋2,𝐹𝐹3], [𝑋𝑋3,𝐹𝐹1 , 𝑋𝑋3,𝐹𝐹2 , 𝑋𝑋3,𝐹𝐹3], [𝑋𝑋4,𝐹𝐹1 , 𝑋𝑋4,𝐹𝐹2 , 𝑋𝑋4,𝐹𝐹3]] 𝑦𝑦4 5 [[𝑋𝑋3,𝐹𝐹1 , 𝑋𝑋3,𝐹𝐹2 , 𝑋𝑋3,𝐹𝐹3], [𝑋𝑋4,𝐹𝐹1 , 𝑋𝑋4,𝐹𝐹2 , 𝑋𝑋4,𝐹𝐹3], [𝑋𝑋5,𝐹𝐹1 , 𝑋𝑋5,𝐹𝐹2 , 𝑋𝑋5,𝐹𝐹3]] 𝑦𝑦5 6 [[𝑋𝑋4,𝐹𝐹1 , 𝑋𝑋4,𝐹𝐹2 , 𝑋𝑋4,𝐹𝐹3], [𝑋𝑋5,𝐹𝐹1 , 𝑋𝑋5,𝐹𝐹2 , 𝑋𝑋5,𝐹𝐹3], [𝑋𝑋6,𝐹𝐹1 , 𝑋𝑋6,𝐹𝐹2 , 𝑋𝑋6,𝐹𝐹3]] 𝑦𝑦6 7 [[𝑋𝑋5,𝐹𝐹1 , 𝑋𝑋5,𝐹𝐹2 , 𝑋𝑋5,𝐹𝐹3], [𝑋𝑋6,𝐹𝐹1 , 𝑋𝑋6,𝐹𝐹2 , 𝑋𝑋6,𝐹𝐹3], [𝑋𝑋7,𝐹𝐹1 , 𝑋𝑋7,𝐹𝐹2 , 𝑋𝑋7,𝐹𝐹3]] 𝑦𝑦7 41 and avoid peeking (the leakage of knowledge from the test set into the model) for a true measure of generality and accuracy for the model [155]. For timeseries with no significant changes over the successive samples, it is common practice to extract the validation and the test subsets randomly, instead of proportionately dividing the dataset into three subsequent parts. The random sampling of records method is adopted for the dataset of this study with the second-by-second sampling distribution. Some research has been devoted to identifying the split fraction of the data for validation and testing for the most statistically significant results [156], [157]; however, this ratio remains a dependent of the specific dataset and the model of interest. It is common practice to partition the dataset into the training, the validation, and the test subset with 0.8:0.1:0.1 split ratio. However, having more test examples for the model will produce a more reliable estimation. For this reason and for simplicity’s sake, a split fraction of 0.5:0.25:0.25 is chosen for the training, validation, and testing, respectively. 6.3.3 Feature scaling The last step of preparing the numerical data for the input of a LSTM network is the scaling process. Features with different scales may create bias during training. For instance, a feature varying in the range (-0.1, 0,1) may not contribute as significantly throughout the training process as one in the range (-1000, 1000) [160]. Scaling techniques constrain the bounded range of each and every independent feature of the data to a smaller scale, thereby making relatively similar scales and a more digestible form of the feature for the training algorithm [160]. One common method for normalization is MinMaxScaler from scikit-learn library in Python. MinMaxScaler rescales each and every feature linearly such that all fall in the range [0,1] (or – [1 ,1] if there are negative values) while preserving the distribution shape of the original data. The technique is performed using the following formula [161]: 𝑋𝑋i, MinMaxScaled = 𝑋𝑋𝑚𝑚 − 𝑋𝑋𝑚𝑚𝑚𝑚𝑚𝑚𝑋𝑋𝑚𝑚𝑏𝑏𝑚𝑚 − 𝑋𝑋𝑚𝑚𝑚𝑚𝑚𝑚 Equation 7 42 where 𝑋𝑋𝑚𝑚 is a sample of one feature of the data and 𝑋𝑋𝑚𝑚𝑚𝑚𝑚𝑚 and 𝑋𝑋𝑚𝑚𝑏𝑏𝑚𝑚 are the minimum and the maximum values of the feature, respectively. 6.4 Figures of merit To evaluate the merits of proposed approaches, Root Mean Squared Error (denoted as RMSE) and the squared correlation coefficient (denoted as 𝑅𝑅2) also so-called absolute fraction of variance or the fitting coefficient are used. The two scores are mathematically defined as follows: 𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅 (𝒚𝒚𝒊𝒊, 𝒀𝒀𝒊𝒊) = �1𝑁𝑁 �(𝑦𝑦𝑚𝑚−𝑌𝑌𝑚𝑚)2𝑁𝑁𝑚𝑚=1 Equation 8 𝑅𝑅2 (𝒚𝒚𝒊𝒊, 𝒀𝒀𝒊𝒊) = (𝑁𝑁 ∑ 𝒚𝒚𝒊𝒊𝒀𝒀𝒊𝒊 −𝑁𝑁𝑚𝑚=1 ∑ 𝒚𝒚𝒊𝒊 ∑ 𝒀𝒀𝒊𝒊𝑁𝑁𝑚𝑚=1𝑁𝑁𝑚𝑚=1 )2��𝑁𝑁 ∑ (𝒀𝒀𝒊𝒊)2𝑁𝑁𝑚𝑚=1 − ∑ (𝒀𝒀𝒊𝒊)2𝑁𝑁𝑚𝑚=1 � − [𝑁𝑁 ∑ (𝒚𝒚𝒊𝒊)2𝑁𝑁𝑚𝑚=1 − ∑ (𝒚𝒚𝒊𝒊)2𝑁𝑁𝑚𝑚=1 ] = 1 − ∑ (𝒚𝒚𝒊𝒊 − 𝒀𝒀𝒊𝒊)2𝑁𝑁𝑚𝑚=1∑ (𝒚𝒚𝒊𝒊 − 𝒚𝒚�𝒊𝒊)2𝑁𝑁𝑚𝑚=1 Equation 9 where N is the total number of samples, 𝑦𝑦𝑚𝑚 , and 𝑦𝑦�𝑚𝑚 represent the observed and the average of the observed values and 𝑌𝑌𝑚𝑚 is the estimation of 𝑦𝑦𝑚𝑚 . RMSE is the standard deviation of the residuals. A smaller RMSE suggests that the model is more reliable for prediction. The 𝑅𝑅2 coefficient is the proportion of the variance of a dependent variable explained by the model predictions over the second sum of the errors or the total variance on a 0 – 100% scale. For the same dataset, a higher squared correlation coefficient generally indicates the model can establish a stronger mapping between the model output and the dependent variables. These two scoring indicators were selected to ensure that the model can be easily compared to other models, as either of these two are common model performance measures reported by other scholars. 43 Chapter 7 Results of the ground-laying optimized DLSTM model and other model configurations and formulations 7.1 Results of the ground-laying optimized DLSTM model Training and hyperparameter tuning of the DLSTM was performed on the processed data as specified in the previous chapter and the fine-tuned model is then instantiated. Figure 16 displays the loss function (see section 6.2) over time for the ground-laying optimized DLSTM model. The orange and the blue curves illustrate the loss function on the training and the test subsets of the dataset respectively (see section 6.3.2). Figure 16 Loss curves for the ground-laying optimized DLSTM model. As can be observed from the graph, the loss starts increasing after approximately 45 epochs (overfitting). Early Stopping regularization method was employed to interrupt the training process because further training produces loss of the generalization capabilities of the model, resulting in an inferior performance (see section 5.2). 44 Table 9 provides a brief quantitative outlook on the aggregated figures of merit of the ground-laying optimized DLSTM model. Table 10 shows global statistical measures including the aggregated average, the maximum, and the minimum of the actual and the predicted power values. Table 9 Key performance indicators of the ground-laying optimized DLSTM model. Model Key Performance Indicators RMSE [𝑘𝑘𝑘𝑘] Scaled RMSE 𝑅𝑅2 Ground-laying Optimized DLSTM 36.12435 0.07517 0.75566 Table 10 Quantitative summary of the Ground-laying optimized DLSTM Training and forecasting results: the maximum, the minimum, and the average of the actual and the predicted power values. Power Statistical Measures Actual Power [𝑘𝑘𝑘𝑘] Predicted Power [𝑘𝑘𝑘𝑘] Average 23.4 25.9 Min -224.2 -224.3 Max 245.3 230.5 The model has achieved an RMSE value of 0.07517 and an 𝑅𝑅2 value of 0.75566. In order to assess the goodness-of-fit of the model comprehensively, comparative evaluation of the model performance against other models with different model configurations and formulations is indispensable and is to be discussed in the next section. From a pairwise comparison of the statistical measures from Table 10, it is evident that the estimated power consumption corresponds well with the actual power values. Figure 17 allows for a more concrete comparisons between the prediction and the true values for a random two-minute time interval. The orange and the blue values represent the actual power and the predicted power respectively. 45 Figure 17 Actual versus the estimated power values for a random two-minute time interval. 7.2 Model configurations and parametric study over the DLSTM model A potent means to understanding the model is to benchmark its architecture and the employed methods and comparatively evaluate its performance with test cases [162]. In order to examine the efficacy of the ground-laying model presented in the previous chapter, a set of test cases are proposed and run in this chapter. These test cases will determine if any of the decisions that motivated the construction of the ground-laying model in the chapter 6 hold, thus will take measures to improve upon its performance. To this end, a set of four categories of different model configurations are defined, each of which include different test cases with different configurations of input parameters or modeling approaches. The test cases are compared against the ground-laying model posteriorly. To enable comparability, the model structure including the parameters and the methods employed in each case are kept constant across the test cases (with the exception of the test case variable under study). Finally, a model incorporating the specifications of the test cases with superior prediction scores than the ground-laying model (the optimal model) is created and implemented. The four different model configurations are introduced, and the results are discussed in the four following subsections. [kW] MTs 46 7.2.1 Basic model: least squares linear regression To examine the necessity of employing more sophisticated data-driven models than the traditional statistical models, a least square linear regression (LSLR) model is considered. Conventional statistical methods such as linear regression modeling are simple to implement yet capture the essentials and allow a direct interpretation. However, they can void these benefits since they cannot preserve and learn from the historical information in the time-series data and struggle to capture the non-linearity of the dynamics such as vehicles’ motions. Looking at the results from the LSLR model from the Table 11, a clear distinction between the ground-laying optimized DLSTM model and the LSLR model is evident. The RMSE and the 𝑅𝑅2 for this case are 0.10139 and 0.55512, respectively. Table 11 Key performance indicators of the least square linear regression model 7.2.2 Ground-laying model without MT extraction and SAX filtering To uncover the importance of the key data processing practices exploited in chapter 3 and chapter 4 including extraction and juxtaposition of MTs as well as SAX filtering, we dismiss each of the practices at a time in two different test cases (test cases number 2 and number 3 in Table 12). Looking at the results briefly, we find that both of the cases struggle to estimate the power consumption in comparison with the ground-laying model. Without employing the SAX filtering technique, the RMSE and the 𝑅𝑅2 are 0.07655 and 0.74881 respectively and without MT extraction technique, these figures drop even further (0.08355 and 0.44417 for RMSE and the 𝑅𝑅2 respectively). Test Category Test Number Test Case Key Performance Indicators 𝐑𝐑𝐑𝐑𝐑𝐑𝐑𝐑 (𝐬𝐬𝐬𝐬𝐬𝐬𝐬𝐬𝐬𝐬𝐬𝐬) 𝑹𝑹𝟐𝟐 Ground-laying Optimized DLSTM Model 0.07517 0.75566 Varying the modeling approach: the basic model 1 Least squares linear regression 0.10139 0.55512 47 Table 12 Key performance indicators of the models without MT extraction and SAX filtering 7.2.3 Ground-laying model with different data feature configurations A plausible means of escalating the ground-laying model would be by introducing new features to the input data or discarding some selected features that were already incorporated into the model. The inclusion of the new features into the model can offer further information associated with the model output. In such manner, the model can better take on the variations and the heterogeneities of the system and could potentially yield a superior predictive capability. New descriptive statistical features added to the model (each at a time to test cases 6, 7, 8) include the length of each trip, the average speed of each trip, the maximum speed of each trip. Two test cases incorporated two new features simultaneously (test cases 9 and 10 in Table 13). Among these new features introduced to the model, only the length of the trips (with the RMSE and the 𝑅𝑅2 values of 0.05916 and 0.80823 respectively for the corresponding model), in test 6, had a positive contribution to the model’s predictive performance. Other four test cases mentioned above yielded higher RMSE and lower 𝑅𝑅2 values, likely caused by the information redundancy among other features of the model input. Meanwhile, discarding suspiciously erroneous features restrains the flow of flawed information into the model thus can enhance its prediction accuracy. In this study, the grade of the road, as pointed out in section 3.2, originally exhibited extreme, unrealistic gradients at some timeslots. These discrepancies are treated with Savitzky-Golay smoothing filter; however, even the consequent de-noised grade signals might have introduced unreliable and unrealistic values to the model. On this Test Category Test Number Test Case Key Performance Indicators 𝐑𝐑𝐑𝐑𝐑𝐑𝐑𝐑 (𝐬𝐬𝐬𝐬𝐬𝐬𝐬𝐬𝐬𝐬𝐬𝐬) 𝑹𝑹𝟐𝟐 Ground-laying Optimized DLSTM Model 0.07517 0.75566 Dismissing key data engineering techniques: without MT extraction and SAX filtering 2 No MT 0.08355 0.44417 3 No SAX 0.07655 0.74881 48 account, test cases 4 and 5 are implemented in which the altitude and the grade of the road (which is calculated from the altitude) are not incorporated into the input data to the model. As it can be observed through Table 13, disregarding the altitude and its associated feature, the grade, from the model input outperforms the ground-laying model (the RMSE and the 𝑅𝑅2 values for the test case excluding the grade are 0.07007 and 0.82064 and for the one without the grade and the altitude are 0.06434 and 0.90611, respectively.) Table 13 Key performance indicators of the models with different data feature configurations 7.2.4 Changing the rolling window size (𝑾𝑾) Finally, the rolling window size (𝑘𝑘) test is devised to allow for studying the impact of the temporal dynamic behaviors in the model input. 𝑘𝑘 was set to 1 for the ground-laying model (see section 6.3.1) which means that the estimation of the power depends on the values of the input set for the current and previous time step. A larger 𝑘𝑘 feeds earlier dependencies in the input sequences to process the current input(s). Test Category Test Number Test Case Key Performance Indicators 𝐑𝐑𝐑𝐑𝐑𝐑𝐑𝐑 (𝐬𝐬𝐬𝐬𝐬𝐬𝐬𝐬𝐬𝐬𝐬𝐬) 𝑹𝑹𝟐𝟐 Ground-laying Optimized DLSTM Model 0.07517 0.75566 Varying the selected data features: different data feature configurations 4 No Grade 0.07007 0.82064 5 No Grade and Altitude 0.06434 0.90611 6 With Length of Each Trip 0.05916 0.80823 7 With Average Speed of Each Trip 0.07987 0.72388 8 With Maximum Speed of Each Trip 0.08270 0.70424 9 With Average Speed and Length of Each Trip 0.08325 0.70026 10 With Average Speed and Maximum Speed of Each Trip 0.07596 0.68902 49 The tested rolling window sizes were bounded between two and five, in increments of one. In other words, the estimation depends on the previous samples of the input dataset, varying from 2 previous times steps to 5 previous time steps. Table 14 Key performance indicators of the models with different rolling window size (𝑾𝑾) 7.3 Model improvement For completeness, we brought the superior test cases together and built a model comprising them all. Table 15 reiterates the selected test cases used to create the optimal model as well as the evaluation scores of this model (without the grade and the altitude and including the length of the MTs and the region of the city). The optimal model test case, expectedly, achieved the most robust estimation results (RMSE and the 𝑅𝑅2 values of 0.05747 and 0.92502, respectively). Notably, the 𝑅𝑅2 of the optimal model is 22% higher than the 𝑅𝑅2 achieved from the ground-laying model. Table 15 The selected test cases used to create the optimal model and the evaluation scores. Test Category Test Number Test Case Key Performance Indicators 𝐑𝐑𝐑𝐑𝐑𝐑𝐑𝐑 (𝐬𝐬𝐬𝐬𝐬𝐬𝐬𝐬𝐬𝐬𝐬𝐬) 𝑹𝑹𝟐𝟐 Ground-laying Optimized DLSTM Model 0.07517 0.75566 Changing the rolling window size (𝑾𝑾) 12 W = 2 0.07829 0.73280 13 W = 3 0.08105 0.71611 14 W = 4 0.08479 0.66986 15 W = 5 0.09225 0.62882 Selected Test Case Key Performance Indicators 𝐑𝐑𝐑𝐑𝐑𝐑𝐑𝐑 (𝐬𝐬𝐬𝐬𝐬𝐬𝐬𝐬𝐬𝐬𝐬𝐬) 𝑹𝑹𝟐𝟐 Discarding Grade and Altitude 0.06434 0.90611 Incorporating Trip Length 0.05916 0.80823 Model Utilizing All Selected Test Case 𝐑𝐑𝐑𝐑𝐑𝐑𝐑𝐑 (𝐬𝐬𝐬𝐬𝐬𝐬𝐬𝐬𝐬𝐬𝐬𝐬) 𝑹𝑹𝟐𝟐 The Optimal Model 0.05747 0.92502 50 Chapter 8 Conclusions, recommendations, and future research 8.1 Conclusions As alluded at the outset, the electrification of mobility is a transformational, viable technology towards decarbonized transportation. Modeling power consumption of EVs can hasten larger-scale adoption of them by addressing a range of challenges EVs are confronting at present. Inspired by the implicit values placed on modeling power consumption of EVs, this thesis has sought out to establish and implement a framework to achieve the power consumption estimations of BE-buses. In doing so, the currently existing body of literature on fuel consumption influencing factors was reviewed. Moreover, ANN models that estimate power demand for different transportation modes were examined. A majority of the reviewed models have common pitfalls summarized below: - The data utilized by some of the models includes an excessive list of attributes, whereas such detailed list of input parameters is not always available in practice. In contrast, other studies lack tightly linked influential factors to power consumption. - The inaccuracies in the measured data are merely handled and treated in most of the models, which could lead to unexpected discrepancies sub-optimal model performance. - Previous models generally exploit shallow neural networks with only one hidden layer imbedded in the model, as opposed to multiple hidden layers (deep learning). - Some of the models use the same learned data for the performance testing of the model which will disable the model’s generalization ability. - A majority of the models lack in depth sensitivity and parameter tuning analysis. The proposed learning framework in this thesis has addressed all these shortcomings. It is worth mentioning that the proposed learning framework is not restricted to EVs and may be applied to all types of vehicles upon the availability of the required data. After treating the dataset and dismissing irregular patterns by SAX filtering, a LSTM networks-based algorithm for pointwise estimation of the power consumption of the BE-buses was defined and 51 implemented, to model the relationship between the dataset attributes (encompassing the bus speed (𝑚𝑚𝑚𝑚) and the bus acceleration (𝑚𝑚𝑚𝑚2), temporal measurements of the grade of the road, the day of the week) and the power consumed (𝑘𝑘𝑘𝑘). Then, an optimization process was adopted to achieve the best hyperparameter configuration for the model. A network with 10 layers, 50 neurons at each layer, a batch size of 512, an early stopping callback patience of 10 epochs, the Adagrad optimizer and finally, the sigmoid activation function exhibited superior performance among other recommended structures. The model achieved an RMSE value 0.07517 of and an 𝑅𝑅2 value of 0.75566. To examine the efficacy of this model and to determine if any of the decisions that motivated the construction of the baseline model in chapter 6 hold, a set of four different model configurations and formulations were defined in chapter 7, each including test cases with different input parameters or processing methods. The RMSE for the test case with LSLR model is far above the ground-laying model and the poorest among all other test cases. The predictability of power for the LSLR model shows the merit of opting for the ground-laying DLSTM model. The test case that dismissed the MTs extraction step produced one of the poorest results, conforming to our expectation that providing more tailored input to the model by cutting the prolonged stops and standstill periods boosts its performance. As noted in section 7.2.2, we showed that employing SAX filtering did help the model with estimating the power, but in this case study, only to a small extend. This might be due to the fact that bus schedules and routes do not vary much, and irregular trips will not occur often to be identified by SAX filtering. Because this might not necessarily be the case for other vehicle datasets and study cases, the slight improvement we observed in the model accuracy when SAX filtering was employed should not void its usefulness for identifying irregularities. The test case that dismissed noisy features including the altitude and the grade of the road achieved the most robust estimation results (section 7.2.3). The grade of the road is highly sensitive to noises. Although it was smoothed out (section 3.2.2), it appears that it is still introducing overwhelming error to the model. That said, another selected test case is the region of the city which characterizes 52 the zones. Therefore, although the other test case suggests that the model would be better off without the information from the grade of the road, grade might be present as a hidden feature in the added city zone variable. We expected to observe competitive performances for the test cases with a larger 𝜔𝜔 against the ground-laying model caused by potential long dependencies on historical data. However, as can be seen in Table 14, the ground-laying model achieved the best test scores across all the test cases in this category. Hence, 𝑘𝑘 = 1 is persisted as for the optimal model. This can be attributed to the information homogeneity across our massive dataset. We persisted with 𝑘𝑘 = 1. Among the test cases, one that incorporated the length of each MT as a new attribute to the model improved the ground-laying DLSTM model as well. For completeness, the ground-laying model was overlaid with two superior test cases. This optimal model outperformed all other test-cases, yielding to the RMSE and the 𝑅𝑅2 values of 0.05747 and 0.92502, respectively. 8.2 Limitations and recommendations As is the case with any other data-driven model, the established learning framework, the outcomes and the resultant implications are conclusive for the particular dataset with the described buses and the geographical location where the data were collected form. Hence, this does not necessarily restrict the use of the proposed framework to other circumstances and settings. We see several promising directions for future work. First off, employing a sheer quantity of the data, i.e., having several months or years of data to feed the model might increase the accuracy of the DNN model. The input data in this study has been collected over one months which may not be long enough to capture the temperature variation across the seasons. If available, incorporating temperature data collected from a wider time window might help boost the model accuracy. Temperature effects can have a significant impact on battery performance (especially in high latitudes). The number of passengers on board as a measure of the total mass of the vehicle, speed limit of the road and weather-related factors such as whether the day was rainy or sunny can also be incorporated as new attributes to the model. In addition, as specified at section 2.1, driver-related factors, ranging from conservative to aggressive driving can substantially affect the power 53 consumption of the vehicle. Thus, their acceleration and braking behavior can also be considered as another input attribute for the model to help further boost the performance of the models. Furthermore, statistical hypothesis testing (also known as confirmatory data analysis) such as the significance-based hypothesis testing can be carried out in the future research. The strength of evidence from the outcomes drawn can be evaluated based on the test cases analysis provided in chapter 7. Such analysis will determine if there is any statistically significant difference between any pair of the test cases, or the test cases and the ground-laying model at a specified significance level (to enable a quantitative comparison). Moreover, analysis of variance [163] can be established to assess potential differences across several test cases. 54 References [1] R. K. Pachauri et al., Climate change 2014: synthesis report. Contribution of Working Groups I, II and III to the fifth assessment report of the Intergovernmental Panel on Climate Change. Ipcc, 2014. [2] T. R. Karl, A. Arguez, B. Huang, and H. Jay, “Possible artifacts of data biases in the recent global surface warming hiatus,” no. September, 2015, doi: 10.1126/science.aaa5632. [3] J. Cook, D. Nuccitelli, S. A. Green, and M. Richardson, “Quantifying the consensus on anthropogenic global warming in the scientific literature,” 2013, doi: 10.1088/1748-9326/8/2/024024. [4] S. S. Sleman, “Global Warming Could Change the Spectrum of Viral Infections in Europe,” J. Infect. Dis. Ther., vol. 03, no. 06, pp. 2–6, 2015, doi: 10.4172/2332-0877.1000248. [5] M. de Almeida D’Agosto, “Chapter 6 - Air pollutant and greenhouse gas emissions (GHG),” in Transportation, Energy Use and Environmental Impacts, M. de Almeida D’Agosto, Ed. Elsevier, 2019, pp. 227–257. [6] A. J. Cohen et al., “The global burden of disease due to outdoor air pollution,” J. Toxicol. Environ. Heal. - Part A, vol. 68, no. 13–14, pp. 1301–1307, 2005, doi: 10.1080/15287390590936166. [7] T. C. Bond et al., “Bounding the role of black carbon in the climate system: A scientific assessment,” J. Geophys. Res. Atmos., vol. 118, no. 11, pp. 5380–5552, 2013, doi: 10.1002/jgrd.50171. [8] F. Dominici, M. Greenstone, and C. R. Sunstein, “Science and regulation. Particulate matter matters.,” Science, vol. 344, no. 6181, pp. 257–9, 2014, doi: 10.1126/science.1247348. [9] U. Nations, “Transforming our world: The 2030 agenda for sustainable development,” New York United Nations, Dep. Econ. Soc. Aff., 2015. [10] P. Agreement, “Paris agreement,” in Report of the Conference of the Parties to the United Nations Framework Convention on Climate Change (21st Session, 2015: Paris). Retrived December, 2015, vol. 4, p. 2017. [11] Canada Energy Regulator, “Canada Energy Regulator, Canada’s Historical GHG Emissions,” 2019. [Online]. Available: https://www.nrcan.gc.ca/science-data/data-analysis/energy-data-analysis/energy-and-greenhouse-gas-emissions-ghgs/20063. [12] European Union, “Worldwide Emission Standards and Related Regulations - Passenger Cars / Light and Medium Duty Vehicles,” Cont. Futur. Motion, no. May, p. 210 pages, 2019. [13] Government of Canada, “On-Road Vehicle and Engine Emission Regulations,” 2019. [Online]. Available: https://pollution-waste.canada.ca/environmental-protection-registry/regulations/view?Id=46. 55 [14] M. Rupp, S. Schulze, and I. Kuperjans, “Comparative life cycle analysis of conventional and hybrid heavy-duty trucks,” World Electr. Veh. J., vol. 9, no. 2, 2018, doi: 10.3390/wevj9020033. [15] Z. Wu, M. Wang, J. Zheng, X. Sun, M. Zhao, and X. Wang, “Life cycle greenhouse gas emission reduction potential of battery electric vehicle,” J. Clean. Prod., vol. 190, pp. 462–470, 2018, doi: 10.1016/j.jclepro.2018.04.036. [16] P. Jaramillo, C. Samaras, H. Wakeley, and K. Meisterling, “Greenhouse gas implications of using coal for transportation: Life cycle assessment of coal-to-liquids, plug-in hybrids, and hydrogen pathways,” Energy Policy, vol. 37, no. 7, pp. 2689–2695, 2009, doi: 10.1016/j.enpol.2009.03.001. [17] K. Meisterling, “Life Cycle Assessment of Greenhouse Gas Emissions from Plug-in Hybrid Vehicles : Implications for Policy,” vol. 42, no. 9, pp. 3170–3176, 2008. [18] EPRI, “Environmental Assessment of Plug-In Hybrid Electric Vehicles,” vol. 1, 2007. [19] R. Wang, Y. Wu, W. Ke, S. Zhang, B. Zhou, and J. Hao, “Can propulsion and fuel diversity for the bus fleet achieve the win-win strategy of energy conservation and environmental protection?,” Appl. Energy, vol. 147, pp. 92–103, 2015, doi: 10.1016/j.apenergy.2015.01.107. [20] M. Mahmoud, R. Garnett, M. Ferguson, and P. Kanaroglou, “Electric buses: A review of alternative powertrains,” Renew. Sustain. Energy Rev., vol. 62, pp. 673–684, 2016, doi: 10.1016/j.rser.2016.05.019. [21] Y. Xu, F. E. Gbologah, D. Y. Lee, H. Liu, M. O. Rodgers, and R. L. Guensler, “Assessment of alternative fuel and powertrain transit bus options using real-world operations data: Life-cycle fuel and emissions modeling,” Appl. Energy, vol. 154, pp. 143–159, 2015, doi: 10.1016/j.apenergy.2015.04.112. [22] C. Weiller, “Plug-in hybrid electric vehicle impacts on hourly electricity demand in the United States,” Energy Policy, vol. 39, no. 6, pp. 3766–3778, 2011, doi: 10.1016/j.enpol.2011.04.005. [23] J. A. García Sánchez, J. M. López Martínez, J. Lumbreras Martín, M. N. Flores Holgado, and H. Aguilar Morales, “Impact of Spanish electricity mix, over the period 2008–2030, on the Life Cycle energy consumption and GHG emissions of Electric, Hybrid Diesel-Electric, Fuel Cell Hybrid and Diesel Bus of the Madrid Transportation System,” Energy Convers. Manag., vol. 74, pp. 332–343, Oct. 2013, doi: 10.1016/j.enconman.2013.05.023. [24] M. Duvall, E. Knipping, M. Alexander, L. Tonachel, and C. Clark, “Environmental assessment of plug-in hybrid electric vehicles, Volume 1: Nationwide greenhouse gas emissions,” Electr. Power Res. Institute, Palo Alto, CA, vol. 1015325, 2007. [25] A. Poullikkas, “Sustainable options for electric vehicle technologies,” Renew. Sustain. Energy Rev., vol. 41, pp. 1277–1287, 2015, doi: 10.1016/j.rser.2014.09.016. [26] N. O. Nylund, K. Erkkilä, and T. Hartikka, Fuel consumption and exhaust emissions of urban buses: Performance of the new diesel technology, no. 2373. 2007. 56 [27] W. J. Requia, M. Mohamed, C. D. Higgins, A. Arain, and M. Ferguson, “How clean are electric vehicles? Evidence-based review of the effects of electric mobility on air pollutants, greenhouse gas emissions and human health,” Atmos. Environ., vol. 185, no. May, pp. 64–77, 2018, doi: 10.1016/j.atmosenv.2018.04.040. [28] “Global EV Outlook 2020,” Glob. EV Outlook 2020, 2020, doi: 10.1787/d394399e-en. [29] BloombergNEF, “Electric Vehicle Outlook 2020,” 2019. [Online]. Available: https://bnef.turtl.co/story/evo-2020. [30] Government of Canada, “Eligible vehicles under the iZEV Program,” 2019. [Online]. Available: https://www.tc.gc.ca/en/services/road/innovative-technologies/list-eligible-vehicles-under-izev-program.html. [31] British Columbia government, “New act ensures B.C. remains leader on clean energy vehicles | BC Gov News,” 2019. [Online]. Available: https://news.gov.bc.ca/releases/2019EMPR0018-001077. [32] M. Messagie, F.-S. Boureima, T. Coosemans, C. Macharis, and J. Mierlo, “A Range-Based Vehicle Life Cycle Assessment Incorporating Variability in the Environmental Assessment of Different Vehicle Technologies and Fuels,” Energies, vol. 7, no. 3, pp. 1467–1482, Mar. 2014, doi: 10.3390/en7031467. [33] M. Messagie, K. Lebeau, T. Coosemans, C. Macharis, and J. van Mierlo, “Environmental and Financial Evaluation of Passenger Vehicle Technologies in Belgium,” Sustainability, vol. 5, no. 12, pp. 5020–5033, Nov. 2013, doi: 10.3390/su5125020. [34] R. A. Rodríguez, E. A. Virguez, P. A. Rodríguez, and E. Behrentz, “Influence of driving patterns on vehicle emissions: A case study for Latin American cities,” Transp. Res. Part D Transp. Environ., vol. 43, pp. 192–206, 2016, doi: 10.1016/j.trd.2015.12.008. [35] E. Demir, T. Bektaş, and G. Laporte, “A comparative analysis of several vehicle emission models for road freight transportation,” Transp. Res. Part D Transp. Environ., vol. 16, no. 5, pp. 347–357, 2011, doi: 10.1016/j.trd.2011.01.011. [36] W. Li, G. Wu, Y. Zhang, and M. J. Barth, “A comparative study on data segregation for mesoscopic energy modeling,” Transp. Res. Part D Transp. Environ., vol. 50, pp. 70–82, 2017, doi: 10.1016/j.trd.2016.10.029. [37] X. Qi, G. Wu, K. Boriboonsomsin, and M. J. Barth, “Data-driven decomposition analysis and estimation of link-level electric vehicle energy consumption under real-world traffic conditions,” Transp. Res. Part D, vol. 64, no. April 2017, pp. 36–52, 2017, doi: 10.1016/j.trd.2017.08.008. [38] M. Ahn, Kyoungho and Rakha, Hesham and Trani, Antonio and Van Aerde, “Estimating Vehicle Fuel Consumption and Emissions Based on Instantaneous Speed and Acceleration Levels,” J. Transp. Eng., vol. 128, no. 2, pp. 182–190, 2002, doi: 10.1061/(ASCE)0733-947X(2002)128:2(182). [39] S. Zargarnezhad, R. Dashti, and R. Ahmadi, “Predicting vehicle fuel consumption in energy distribution 57 companies using ANNs,” Transp. Res. Part D Transp. Environ., vol. 74, no. August, pp. 174–188, 2019, doi: 10.1016/j.trd.2019.07.020. [40] J. L. Sullivan, G. M. Lewis, and G. A. Keoleian, “Effect of mass on multimodal fuel consumption in moving people and freight in the U.S.,” Transp. Res. Part D Transp. Environ., vol. 63, no. July, pp. 786–808, 2018, doi: 10.1016/j.trd.2018.06.019. [41] J. D. Du, W. J. Han, Y. H. Peng, and C. C. Gu, “Potential for reducing GHG emissions and energy consumption from implementing the aluminum intensive vehicle fleet in China,” Energy, vol. 35, no. 12, pp. 4671–4678, 2010, doi: 10.1016/j.energy.2010.09.037. [42] T. Holdstock, A. Sorniotti, M. Everitt, M. Fracchia, S. Bologna, and S. Bertolotto, “Energy consumption analysis of a novel four-speed dual motor drivetrain for electric vehicles,” in 2012 IEEE Vehicle Power and Propulsion Conference, 2012, pp. 295–300, doi: 10.1109/VPPC.2012.6422721. [43] E. Yao, Z. Yang, Y. Song, and T. Zuo, “Comparison of electric vehicle’s energy consumption factors for different road types,” Discret. Dyn. Nat. Soc., vol. 2013, 2013, doi: 10.1155/2013/328757. [44] T. Tielert, M. Killat, H. Hartenstein, R. Luz, S. Hausberger, and T. Benz, “The impact of traffic-light-to-vehicle communication on fuel consumption and emissions,” 2010 Internet Things, IoT 2010, pp. 1–8, 2010, doi: 10.1109/IOT.2010.5678454. [45] B. Asadi and A. Vahidi, “Predictive Cruise Control : Utilizing Upcoming Traffic Signal Information for,” IEEE Trans. Control Syst. Technol., vol. 19, no. 3, pp. 707–714, 2011, doi: 10.1109/TCST.2010.2047860. [46] B. Asadi and A. Vahidi, “Predictive cruise control: Utilizing upcoming traffic signal information for improving fuel economy and reducing trip time,” IEEE Trans. Control Syst. Technol., vol. 19, no. 3, pp. 707–714, 2011, doi: 10.1109/TCST.2010.2047860. [47] M. Barth and K. Boriboonsomsin, “Real-world carbon dioxide impacts of traffic congestion,” Transp. Res. Rec., no. 2058, pp. 163–171, 2008, doi: 10.3141/2058-20. [48] M. Mascia et al., “Impact of Traffic Management on Black Carbon Emissions: a Microsimulation Study,” Networks Spat. Econ., vol. 17, no. 1, pp. 269–291, Mar. 2017, doi: 10.1007/s11067-016-9326-x. [49] E. Demir, T. Bektaş, and G. Laporte, “The bi-objective Pollution-Routing Problem,” Eur. J. Oper. Res., vol. 232, no. 3, pp. 464–478, 2014, doi: 10.1016/j.ejor.2013.08.002. [50] S. Beevers et al., “Traffic management strategies for emissions reduction: recent experience in London,” Energy Emiss. Control Technol., no. April, p. 27, Apr. 2016, doi: 10.2147/EECT.S69858. [51] K. Boriboonsomsin, M. J. Barth, W. Zhu, and A. Vu, “Eco-routing navigation system based on multisource historical and real-time traffic information,” IEEE Trans. Intell. Transp. Syst., vol. 13, no. 4, pp. 1694–1704, 2012, doi: 10.1109/TITS.2012.2204051. 58 [52] S. C. Yang, M. Li, Y. Lin, and T. Q. Tang, “Electric vehicle’s electricity consumption on a road with different slope,” Phys. A Stat. Mech. its Appl., vol. 402, pp. 41–48, May 2014, doi: 10.1016/j.physa.2014.01.062. [53] Z. Yi and P. H. Bauer, “Sensitivity Analysis of Environmental Factors for Electric Vehicles Energy Consumption,” 2015 IEEE Veh. Power Propuls. Conf. VPPC 2015 - Proc., no. 1, pp. 1–6, 2015, doi: 10.1109/VPPC.2015.7353012. [54] H. A. Yavasoglu, Y. E. Tetik, and K. Gokce, “Implementation of machine learning based real time range estimation method without destination knowledge for BEVs,” Energy, vol. 172, pp. 1179–1186, 2019, doi: 10.1016/j.energy.2019.02.032. [55] K. R. Kambly and T. H. Bradley, “Estimating the HVAC energy consumption of plug-in electric vehicles,” J. Power Sources, vol. 259, pp. 117–124, 2014, doi: 10.1016/j.jpowsour.2014.02.033. [56] J. H. Bundgaard, B. B. T. Madsen, and C. P. Nørskov, “Using Spatial and Temporal Context for Predicting Energy Consumption of Electric Vehicles,” Aalborg Universitet, 2019. [57] N. Lin, C. Zong, M. Tomizuka, P. Song, Z. Zhang, and G. Li, “An Overview on Study of Identification of Driver Behavior Characteristics for Automotive Control,” Math. Probl. Eng., vol. 2014, no. March, pp. 1–15, 2014, doi: 10.1155/2014/569109. [58] C. Bingham, C. Walsh, and S. Carroll, “Impact of driving characteristics on electric vehicle energy consumption and range,” IET Intell. Transp. Syst., vol. 6, no. 1, pp. 29–35, 2012, doi: 10.1049/iet-its.2010.0137. [59] J. C. Ferreira, J. de Almeida, and A. R. da Silva, “The Impact of Driving Styles on Fuel Consumption: A Data-Warehouse-and-Data-Mining-Based Discovery Process,” IEEE Trans. Intell. Transp. Syst., vol. 16, no. 5, pp. 2653–2662, Oct. 2015, doi: 10.1109/TITS.2015.2414663. [60] T. R. Waters, “Adaptive driver modeling using machine learning algorithms for the energy optimal planning of velocity trajectories for electric vehicles and realizing simultaneous lane keeping and adaptive speed regulation on accessible mobile robot testbeds,” Georgia Institute of Technology, 2017. [61] H. Ma, H. Xie, D. Huang, and S. Xiong, “Effects of driving style on the fuel consumption of city buses under different road conditions and vehicle masses,” Transp. Res. Part D Transp. Environ., vol. 41, pp. 205–216, 2015, doi: 10.1016/j.trd.2015.10.003. [62] I. M. Berry, “The Effects of Driving Style and Vehicle Performance on the Real-World Fuel Consumption of U.S. Light-Duty Vehicles,” Massachusetts Institute of Technology, 2010. [63] J. Van Mierlo, G. Maggetto, E. Van de Burgwal, and R. Gense, “Driving style and traffic measures-influence on vehicle emissions and fuel consumption,” Proc. Inst. Mech. Eng. Part D J. Automob. Eng., vol. 218, no. 1, pp. 43–50, Jan. 2004, doi: 10.1243/095440704322829155. [64] E. Ericsson, “Independent driving pattern factors and their infuence on fuel-use and exhaust emission 59 factors Eva,” Transp. Res. Part D Transp. Environ., vol. 6, pp. 325--345, 2001. [65] A. Diaz Alvarez, F. Serradilla Garcia, J. E. Naranjo, J. J. Anaya, and F. Jimenez, “Modeling the Driving Behavior of Electric Vehicles Using Smartphones and Neural Networks,” IEEE Intell. Transp. Syst. Mag., vol. 6, no. 3, pp. 44–53, 2014, doi: 10.1109/MITS.2014.2322651. [66] G. Wager, M. P. Mchenry, J. Whale, and T. Bräunl, “Testing energy ef fi ciency and driving range of electric vehicles in relation to gear selection,” Renew. Energy, vol. 62, no. 2014, pp. 303–312, 2015, doi: 10.1016/j.renene.2013.07.029. [67] A. Braun and W. Rid, “The influence of driving patterns on energy consumption in electric car driving and the role of regenerative braking,” Transp. Res. Procedia, vol. 22, pp. 174–182, 2017, doi: 10.1016/j.trpro.2017.03.024. [68] H. Yue, “Mesoscopic Fuel Consumption and Emissions Modeling,” Virginia Polytech. Inst. State Univ., p. 139, 2008. [69] H. Rakha and R. K. Kamalanathsharma, “Eco-driving at signalized intersections using V2I communication,” IEEE Conf. Intell. Transp. Syst. Proceedings, ITSC, pp. 341–346, 2011, doi: 10.1109/ITSC.2011.6083084. [70] B. Ke, C. Chung, and Y. Chen, “Minimizing the costs of constructing an all plug-in electric bus transportation system : A case study in Penghu,” Appl. Energy, vol. 177, pp. 649–660, 2016, doi: 10.1016/j.apenergy.2016.05.152. [71] M. Xylia, S. Leduc, P. Patrizio, F. Kraxner, and S. Silveira, “Locating charging infrastructure for electric buses in Stockholm,” Transp. Res. Part C Emerg. Technol., vol. 78, pp. 183–200, 2017, doi: 10.1016/j.trc.2017.03.005. [72] Y. Wang, Y. Huang, J. Xu, and N. Barclay, “Optimal recharging scheduling for urban electric buses: A case study in Davis,” Transp. Res. Part E Logist. Transp. Rev., vol. 100, pp. 115–132, 2017, doi: 10.1016/j.tre.2017.01.001. [73] T. Paul and H. Yamada, “Operation and charging scheduling of electric buses in a city bus route network,” 2014 17th IEEE Int. Conf. Intell. Transp. Syst. ITSC 2014, vol. 1, pp. 2780–2786, 2014, doi: 10.1109/ITSC.2014.6958135. [74] T. Sandberg, “Heavy Truck Modeling for Fuel Consumption Simulations and Measurements,” Link¨oping University, 2001. [75] C. M. Silva, T. L. Farias, H. C. Frey, and N. M. Rouphail, “Evaluation of numerical models for simulation of real-world hot-stabilized fuel consumption and emissions of gasoline light-duty vehicles,” Transp. Res. Part D Transp. Environ., vol. 11, no. 5, pp. 377–385, 2006, doi: 10.1016/j.trd.2006.07.004. [76] C. Lorf, R. F. Martínez-Botas, D. A. Howey, L. Lytton, and B. Cussons, “Comparative analysis of the energy consumption and CO2 emissions of 40 electric, plug-in hybrid electric, hybrid electric and 60 internal combustion engine vehicles,” Transp. Res. Part D Transp. Environ., vol. 23, pp. 12–19, 2013, doi: 10.1016/j.trd.2013.03.004. [77] W. Dib, A. Chasse, P. Moulin, A. Sciarretta, and G. Corde, “Optimal energy management for an electric vehicle in eco-driving applications,” Control Eng. Pract., vol. 29, pp. 299–307, 2014, doi: 10.1016/j.conengprac.2014.01.005. [78] F. Badin et al., “Evaluation of EVs energy consumption influencing factors, driving conditions, auxiliaries use, driver’s aggressiveness,” in 2013 World Electric Vehicle Symposium and Exhibition (EVS27), 2013, no. November, pp. 1–12, doi: 10.1109/EVS.2013.6914723. [79] L. Horrein, A. Bouscayrol, P. Delarue, J. N. Verhille, and C. Mayet, Forward and backward simulations of a power propulsion system, vol. 8, no. PART 1. IFAC, 2012. [80] M. Muratori, M. J. Moran, E. Serra, and G. Rizzoni, “Highly-resolved modeling of personal transportation energy consumption in the United States,” Energy, vol. 58, pp. 168–177, 2013, doi: 10.1016/j.energy.2013.02.055. [81] Q. Zhao, Q. Chen, and L. Wang, “Real-time prediction of fuel consumption based on digital map API,” Appl. Sci., vol. 9, no. 7, 2019, doi: 10.3390/app9071369. [82] J. M. López-Martínez et al., “Modelling the fuel consumption and pollutant emissions of the urban bus fleet of the city of Madrid,” Transp. Res. Part D Transp. Environ., vol. 52, pp. 112–127, 2017, doi: 10.1016/j.trd.2017.02.016. [83] C. Fiori and V. Marzano, “Modelling energy consumption of electric freight vehicles in urban pickup/delivery operations: analysis and estimation on a real-world dataset,” Transp. Res. Part D Transp. Environ., vol. 65, no. October, pp. 658–673, 2018, doi: 10.1016/j.trd.2018.09.020. [84] K. N. Genikomsakis and G. Mitrentsis, “A computationally efficient simulation model for estimating energy consumption of electric vehicles in the context of route planning applications,” Transp. Res. Part D Transp. Environ., vol. 50, pp. 98–118, 2017, doi: 10.1016/j.trd.2016.10.014. [85] X. Xu, H. M. A. Aziz, and R. Guensler, “A modal-based approach for estimating electric vehicle energy consumption in transportation networks,” Transp. Res. Part D Transp. Environ., vol. 75, no. February, pp. 249–264, 2019, doi: 10.1016/j.trd.2019.09.001. [86] X. Qi, G. Wu, K. Boriboonsomsin, M. J. Barth, and J. Gonder, “Data-driven reinforcement learning-based real-time energy management system for plug-in hybrid electric vehicles,” Transp. Res. Rec., vol. 2572, pp. 1–8, 2016, doi: 10.3141/2572-01. [87] M. Keskin and B. Çatay, “Partial recharge strategies for the electric vehicle routing problem with time windows,” Transp. Res. Part C Emerg. Technol., vol. 65, pp. 111–127, 2016, doi: 10.1016/j.trc.2016.01.013. [88] S. Erdoğan and E. Miller-Hooks, “A Green Vehicle Routing Problem,” Transp. Res. Part E Logist. Transp. 61 Rev., vol. 48, no. 1, pp. 100–114, Jan. 2012, doi: 10.1016/j.tre.2011.08.001. [89] M. Zhou, H. Jin, and W. Wang, “A review of vehicle fuel consumption models to evaluate eco-driving and eco-routing,” Transp. Res. Part D Transp. Environ., vol. 49, no. 5, pp. 203–218, 2016, doi: 10.1016/j.trd.2016.09.008. [90] R. Shankar and J. Marco, “Method for estimating the energy consumption of electric vehicles and plug-in hybrid electric vehicles under real-world driving conditions,” IET Intell. Transp. Syst., vol. 7, no. 1, pp. 138–150, 2013, doi: 10.1049/iet-its.2012.0114. [91] M. Neaimeh, G. A. Hill, Y. Hübner, and P. T. Blythe, “Routing systems to extend the driving range of electric vehicles,” IET Intell. Transp. Syst., vol. 7, no. 3, pp. 327–336, 2013, doi: 10.1049/iet-its.2013.0122. [92] N. Nylund and K. Erkkilä, “Evaluation of duty cycles for heavy-duty urban vehicles,” Final Rep. IEA AMF Annex XXIX, 2007. [93] P. Shen, Z. Zhao, J. Li, and X. Zhan, “Development of a typical driving cycle for an intra-city hybrid electric bus with a fixed route,” Transp. Res. Part D Transp. Environ., vol. 59, no. February, pp. 346–360, 2018, doi: 10.1016/j.trd.2018.01.032. [94] G. De Filippo, V. Marano, and R. Sioshansi, “Simulation of an electric transportation system at The Ohio State University,” Appl. Energy, vol. 113, pp. 1686–1691, 2014, doi: 10.1016/j.apenergy.2013.09.011. [95] O. Vilppo and J. Markkula, “Feasibility of electric buses in public transport,” World Electr. Veh. J., vol. 7, no. 3, pp. 357–365, 2015, doi: 10.3390/wevj7030357. [96] P. Sinhuber, W. Rohlfs, and D. U. Sauer, “Study on power and energy demand for sizing the energy storage systems for electrified local public transport buses,” 2012 IEEE Veh. Power Propuls. Conf. VPPC 2012, no. February 2016, pp. 315–320, 2012, doi: 10.1109/VPPC.2012.6422680. [97] B. Zhou et al., “Real-world performance of battery electric buses and their life-cycle benefits with respect to energy consumption and carbon dioxide emissions,” Energy, vol. 96, no. 2016, pp. 603–613, 2016, doi: 10.1016/j.energy.2015.12.041. [98] Z. Gao et al., “Battery capacity and recharging needs for electric buses in city transit service,” Energy, vol. 122, pp. 588–600, 2017, doi: 10.1016/j.energy.2017.01.101. [99] R. Prohaska, K. Kelly, and L. Eudy, “Fast charge battery electric transit bus in-use fleet evaluation,” 2016 IEEE Transp. Electrif. Conf. Expo, ITEC 2016, no. May, 2016, doi: 10.1109/ITEC.2016.7520220. [100] M. Rogge, E. van der Hurk, A. Larsen, and D. U. Sauer, “Electric bus fleet size and mix problem with optimization of charging infrastructure,” Appl. Energy, vol. 211, no. February 2017, pp. 282–295, 2018, doi: 10.1016/j.apenergy.2017.11.051. [101] M. Rogge, S. Wollny, and D. U. Sauer, “Fast charging battery buses for the electrification of urban public 62 transport-A feasibility study focusing on charging infrastructure and energy storage requirements,” Energies, vol. 8, no. 5, pp. 4587–4606, 2015, doi: 10.3390/en8054587. [102] T. Ly, D. Goehlich, and L. Heide, “Assessment of the Interaction of Charging System and Battery Technology for the Use in Urban Battery Electric Bus Systems,” 2016 IEEE Veh. Power Propuls. Conf. VPPC 2016 - Proc., 2016, doi: 10.1109/VPPC.2016.7791621. [103] U. Choi, “Commercial Operation of Ultra Low Floor Electric Bus for Seoul City Route,” 2012 IEEE Veh. Power Propuls. Conf., pp. 1128–1133, 2012, doi: 10.1109/VPPC.2012.6422619. [104] K. C. Tan and Y. Li, “Grey-box model identification via evolutionary computing,” Control Eng. Pract., vol. 10, no. 7, pp. 673–684, 2002, doi: 10.1016/S0967-0661(02)00031-X. [105] H. Taghavifar, H. Taghavifar, A. Mardani, and A. Mohebbi, “Modeling the impact of in-cylinder combustion parameters of di engines on soot and NOx emissions at rated EGR levels using ANN approach,” Energy Convers. Manag., vol. 87, pp. 1–9, 2014, doi: 10.1016/j.enconman.2014.07.005. [106] A. A. Trani, F. C. Wing-Ho, G. Schilling, H. Baik, and A. Seshadri, “A neural network model to estimate aircraft fuel consumption,” Collect. Tech. Pap. - AIAA 4th Aviat. Technol. Integr. Oper. Forum, ATIO, vol. 2, no. September, pp. 669–692, 2004, doi: 10.2514/6.2004-6401. [107] G. D. Schilling, “Modeling aircraft fuel consumption with a neural network,” Virginia Tech, 1997. [108] N. Kara Togun and S. Baysec, “Prediction of torque and specific fuel consumption of a gasoline engine by using artificial neural networks,” Appl. Energy, vol. 87, no. 1, pp. 349–355, 2010, doi: 10.1016/j.apenergy.2009.08.016. [109] A. Parlak, Y. Islamoglu, H. Yasar, and A. Egrisogut, “Application of artificial neural network to predict specific fuel consumption and exhaust temperature for a Diesel engine,” Appl. Therm. Eng., vol. 26, no. 8–9, pp. 824–828, 2006, doi: 10.1016/j.applthermaleng.2005.10.006. [110] B. Cirak and S. Demirtas, “An Application of Artificial Neural Network for Predicting Engine Torque in a Biodiesel Engine,” Am. J. Energy Res., vol. 2, no. 4, pp. 74–80, 2014, doi: 10.12691/ajer-2-4-1. [111] Z. Cheng and M. Chow, “A Big Data Based Deep Learning Approach for Vehicle Speed Prediction,” 2017 IEEE 26th Int. Symp. Ind. Electron., pp. 389–394, 2017, doi: 10.1109/ISIE.2017.8001278. [112] S. Kanarachos, J. Mathew, and M. E. Fitzpatrick, “Instantaneous vehicle fuel consumption estimation using smartphones and recurrent neural networks,” Expert Syst. Appl., vol. 120, pp. 436–447, 2019, doi: 10.1016/j.eswa.2018.12.006. [113] B. Predić, M. Madić, M. Roganović, M. Kovačević, and D. Stojanović, “Prediction of Passenger Car Fuel Consumption Using Artificial Neural Network : a Case Study in the City of Niš,” FACTA Univ. Ser. Autom. Control Robot., vol. 15, no. 2, pp. 105–116, 2016. [114] A. Bhoraskar, “Prediction of Fuel Consumption of Long Haul Heavy Duty Trucks Using Machine 63 Learning and Comparison of the Performance of Various Learning Techniques,” Delft University of Technology, 2019. [115] H. Almér, “Machine learning and statistical analysis in fuel consumption prediction for heavy vehicles,” KTH ROYAL INSTITUTE OF TECHNOLOGY SCHOOL, 2015. [116] A. Galang, “Predicting hybrid vehicle fuel economy and emissions with neural network models trained with real world data,” Colorado State University, 2017. [117] D. Goebel and P. Plötz, “Machine learning estimates of plug-in hybrid electric vehicle utility factors,” Transp. Res. Part D Transp. Environ., vol. 72, no. April, pp. 36–46, 2019, doi: 10.1016/j.trd.2019.04.008. [118] S. Roweis, “Levenberg-Marquardt Optimization,” Lect. Notes. Univ. Toronto, 1996. [119] Z. Cheng, M. Y. Chow, D. Jung, and J. Jeon, “A big data based deep learning approach for vehicle speed prediction,” IEEE Int. Symp. Ind. Electron., pp. 389–394, 2017, doi: 10.1109/ISIE.2017.8001278. [120] S. Wickramanayake and D. H. M. N. Bandara, “Fuel consumption prediction of fleet vehicles using Machine Learning: A comparative study,” 2nd Int. Moratuwa Eng. Res. Conf. MERCon 2016, pp. 90–95, 2016, doi: 10.1109/MERCon.2016.7480121. [121] L. Rambaldi, E. Bocci, and F. Orecchini, “Preliminary experimental evaluation of a four wheel motors, batteries plus ultracapacitors and series hybrid powertrain,” Appl. Energy, vol. 88, no. 2, pp. 442–448, 2011, doi: 10.1016/j.apenergy.2010.08.008. [122] R. W. Schafer, “What is a savitzky-golay filter?,” IEEE Signal Process. Mag., vol. 28, no. 4, pp. 111–117, 2011, doi: 10.1109/MSP.2011.941097. [123] E. Wood et al., “Appending high-resolution elevation data to GPS speed traces for vehicle energy modeling and simulation appending high-resolution elevation data to GPS speed traces for vehicle energy modeling and simulation,” Tech. report, NREL/TP-5400-61109, no. June, 2014. [124] A. Savitzky and M. J. E. Golay, “Smoothing and Differentiation of Data by Simplified Least Squares Procedures,” Anal. Chem., vol. 36, no. 8, pp. 1627–1639, 1964, doi: 10.1021/ac60214a047. [125] M. Sadeghi and F. Behnia, “Optimum window length of Savitzky-Golay filters with arbitrary order,” arXiv Signal Process., 2018. [126] X. Wu et al., “Real-world emissions and fuel consumption of diesel buses and trucks in Macao: From on-road measurement to policy implications,” Atmos. Environ., vol. 120, pp. 393–403, 2015, doi: 10.1016/j.atmosenv.2015.09.015. [127] and W. R. Maćkiewicz, Andrzej, “Principal components analysis (PCA),” Comput. Geosci., vol. 19, no. 3, pp. 303–342, 1993. 64 [128] L. Yiding Hua, ID and Jiang, Haobin and Tian, Huan and Xu, Xing and Chen, “A Comparative Study of Clustering Analysis Method for Driver’ s Steering Intention Classification and Identification under Different Typical Conditions,” RoadVehicles Surroundings Superv. On-Board Sensors Commun., p. 63, 2017, doi: 10.3390/app7101014. [129] E. Keogh, J. Lin, and A. Fu, “HOT SAX Finding the Most Unusual Time Series Subsequence: Algorithms and Applications,” Icdm, p. 8, 2005, doi: 10.1109/ICDM.2005.79. [130] J. Lin, E. Keogh, L. Wei, and S. Lonardi, “Experiencing SAX: A novel symbolic representation of time series,” Data Min. Knowl. Discov., vol. 15, no. 2, pp. 107–144, 2007, doi: 10.1007/s10618-007-0064-z. [131] C. Miller, Z. Nagy, and A. Schlueter, “Automated daily pattern filtering of measured building performance data,” Autom. Constr., vol. 49, no. PA, pp. 1–17, 2015, doi: 10.1016/j.autcon.2014.09.004. [132] V. Chiarazzo, L. Caggiani, M. Marinelli, and M. Ottomanelli, “A neural network based model for real estate price estimation considering environmental quality of property location,” Transp. Res. Procedia, vol. 3, no. July, pp. 810–817, 2014, doi: 10.1016/j.trpro.2014.10.067. [133] E. Siami-Irdemoosa and S. R. Dindarloo, “Prediction of fuel consumption of mining dump trucks: A neural networks approach,” Appl. Energy, vol. 151, pp. 77–84, 2015, doi: 10.1016/j.apenergy.2015.04.064. [134] J. Schmidhuber, “Deep Learning in neural networks: An overview,” Neural Networks, vol. 61, pp. 85–117, 2015, doi: 10.1016/j.neunet.2014.09.003. [135] F. Chollet, Deep Learning with Python. Manning Publications Co., 2018. [136] P.-A. Fawaz, Hassan Ismail and Forestier, Germain and Weber, Jonathan and Idoumghar, Lhassane and Muller, “Deep learning for time series classification: a review,” Data Min. Knowl. Discov., vol. 33, no. 4, pp. 917–963, 2019. [137] S. Hochreiter and J. Schmidhuber, “Long Short-Term Memory,” Neural Comput., vol. 9, no. 8, pp. 1735–1780, 1997, doi: 10.1162/neco.1997.9.8.1735. [138] N. C. Petersen, F. Rodrigues, and F. C. Pereira, “Multi-output bus travel time prediction with convolutional LSTM neural network,” Expert Syst. Appl., vol. 120, pp. 426–435, 2019, doi: 10.1016/j.eswa.2018.11.028. [139] F. M. Bianchi, E. Maiorino, M. C. Kampffmeyer, and A. Rizzi, Recurrent neural networks for short-term load forecasting: an overview and comparative analysis. Springer, 2017. [140] “NumPy Documentation.” [Online]. Available: https://numpy.org/doc/. [141] “pandas Documentation.” [Online]. Available: https://pandas.pydata.org/docs/. [142] “matplotlib Documentation.” [Online]. Available: https://matplotlib.org/3.2.1/contents.html. 65 [143] “seaborn Documentation.” [Online]. Available: https://seaborn.pydata.org/. [144] “TensorFlow Documentation.” [Online]. Available: https://www.tensorflow.org/guide. [145] F. Chollet, Deep Learning mit Python und Keras: Das Praxis-Handbuch vom Entwickler der Keras-Bibliothek. MITP-Verlags GmbH & Co. KG, 2018. [146] “scikit-learn Documentation.” [Online]. Available: https://scikit-learn.org/stable/. [147] TensorFlow, “https://www.tensorflow.org/,” 2019. . [148] S. S. Girija, “TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems,” Softw. available from tensorflow.org, vol. 39, no. 9, 2015. [149] “theano Documentation.” LISA lab, University of Montreal, 2018. [150] J. Bergstra et al., “Theano: A CPU and GPU math compiler in Python,” Proc. Python Sci. Comput. Conf., no. Scipy, pp. 1–7, 2010. [151] “Microsoft Cognitive Toolkit Documentation,” 2017. [Online]. Available: https://docs.microsoft.com/en-us/cognitive-toolkit/. [152] R. Pascanu, D. Tour, T. Mikolov, and D. Tour, “On the difficulty of training recurrent neural networks,” no. 2, 2013. [153] A. Géron, Hands-On Machine Learning with Scikit-Learn and TensorFlow. O’Reilly Media. [154] B. D. Ripley, Pattern Recognition and Neural Networks. Cambridge university press, 2007. [155] P. Russell, Stuart J and Norvig, Artificial Intelligence: A Modern Approach, Third Edit. Pearson Education London, 2010. [156] I. Guyon, “A scaling law for the validation-set training-set size ratio,” AT&T Bell Lab., vol. 1, no. 11, 1997. [157] I. Guyon, J. Makhoul, R. Schwartz, and V. Vapnik, “What Size Test Set Gives Good Error Rate Estimates?,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 20, no. 1, pp. 52–64, 1998. [158] V. Pareto, Manuale di economia politica., vol. 13. Società editrice libraria, 1919. [159] M. Chen, Y-S and Chong, P Pete and Tong, “Mathematical and Computer Modelling of the Pareto Principle,” Elsevier, vol. 19, no. 9, pp. 61–80, 1994. [160] C. M. Bishop, “Neural networks for pattern recognition,” Oxford Univ. Press, 1995. [161] Scikit-learn, “MinMaxScaler.” [Online]. Available: https://scikit-66 learn.org/stable/modules/generated/sklearn.preprocessing.MinMaxScaler.html. [162] Y. B. R Pascanu, T Mikolov, “On the difficulty of training recurrent neural networks,” Phylogenetic Divers. Appl. Challenges Biodivers. Sci., no. 2, pp. 41–71, 2013, doi: 10.1007/978-3-319-93145-6_3. [163] R. Miller, “Beyond ANOVA: Basics of Applied Statistics,” CRC Press, 1997. [164] “Activation,” Keras Documentation. [Online]. Available: https://faroit.com/keras-docs/1.2.0/activations/. [165] “What does ‘sample’, ‘batch’, ‘epoch’ mean?,” Keras Documentation. [Online]. Available: https://keras.io/getting_started/faq/. [166] “Usage of loss functions,” Keras Documentation. [Online]. Available: https://keras.io/api/losses/. [167] S. Shalev-Shwartz, Shai and Ben-David, Understanding machine learning: From theory to algorithms. Cambridge university press, 2014. [168] C. Res, C. J. Willmott, and K. Matsuura, “Advantages of the mean absolute error (MAE) over the root mean square error (RMSE) in assessing average model performance,” Clim. Res., vol. 30, pp. 79–82, 2005. [169] P. J. Huber, “Robust Estimation of a Location Parameter,” Springer Ser. Stat. B. Ser., 1964. [170] “Usage of optimizers,” Keras Documentation. [Online]. Available: https://faroit.com/keras-docs/0.2.0/optimizers/. [171] S. J. W. Suvrit Sra, Sebastian Nowozin, Optimization for Machine Learning. Cambridge, Mass. : MIT Press, 2012. [172] D. P. Kingma and J. L. Ba, “Adam: A Method for Stochastic Optimization,” arXiv Prepr. arXiv1412.6980, pp. 1–15, 2014. [173] J. Duchi, “Adaptive Subgradient Methods,” vol. 12, pp. 2121–2159, 2011, doi: 10.5555/1953048.2021068. 67 Appendix Table 16 How MT selection from speed profiles can be expanded to the rest of the attributes. Index Bus Speed (𝑚𝑚𝑚𝑚) Bus Acceleration (𝑚𝑚2𝑠𝑠) Altitude (𝑚𝑚) Road Grade Day of week Hour of day Weekday Power (𝑘𝑘𝑘𝑘) vs Weekend 615240 0 -0.009 21.3 5.81s5619 3 3 0 1.781 615241 0.14 -0.024 21.5 5.721027 3 3 0 5.894 ⋮ 615257 11.67 2.583 23.51818 3.059948 3 3 0 87.917 ⋮ 615273 0.83 -1.566 26.44 -1.2852 3 3 0 -9.471 615274 0 -0.832 26.4 -1.6246 3 3 0 1.716 ⋮ 615281 0 0.002 26.05455 -1.17294 3 3 0 1.962 615282 0.19 0.002 25.98182 -1.29625 3 3 0 2.023 ⋮ 615295 10.56 1.091 24.1 -2.21763 3 3 0 22.668 ⋮ 615316 1.11 -1.6 19.04545 -3.6172 3 3 0 -12.031 615317 0 -1.193 18.73636 -3.67809 3 3 0 1.845 ⋮ 615325 0 -0.08233 16.98 -4.94339 3 3 0 6.212 615326 0.28 -0.125 16.85 -4.978 3 3 0 5.965 68 ⋮ 615334 2.22 -0.09 15.775 -4.51973 3 3 0 -3.198 ⋮ 615343 0.28 -0.24 14.5 -4.2236 3 3 0 2.03 615344 0 -0.099 14.41818 -4.12769 3 3 0 2.152 ⋮ 615362 0 0.003 13.6 -1.7552 3 3 0 2.091 615363 0.28 0.73 13.6 -1.44572 3 3 0 12.669 ⋮ 615385 11.11 1.99 10.45 -0.58163 3 3 0 44.688 ⋮ 615397 0.56 -1.565 10.70909 0.611719 3 3 0 -1.599 615398 0 -0.581 10.75455 0.720938 3 3 0 2.03 Table 17 Hyperparameter jargon terms definitions. Activation [164] Activation function or activation is a mathematical function 𝑜𝑜(𝑥𝑥) that generates the output: 𝑜𝑜(𝑤𝑤𝑥𝑥 + 𝑏𝑏) for each neuron in the network where 𝑤𝑤𝑥𝑥 is the weighted sum of the inputs by the weights (𝑤𝑤) and the bias (𝑏𝑏). Types linear 𝑙𝑙𝑇𝑇𝑜𝑜𝑒𝑒𝑎𝑎𝑜𝑜 𝑎𝑎𝑐𝑐𝑡𝑡𝑇𝑇𝑎𝑎𝑎𝑎𝑡𝑡𝑇𝑇𝑜𝑜𝑜𝑜 𝑜𝑜𝑓𝑓𝑜𝑜𝑐𝑐𝑡𝑡𝑇𝑇𝑜𝑜𝑜𝑜 𝑜𝑜(𝑥𝑥) returns: 𝑥𝑥. relu relu or Rectified Linear Unit activation function 𝑜𝑜(𝑥𝑥) 69 returns: max (𝑥𝑥, 0). tanh tanh or Hyperbolic tangent activation function 𝑜𝑜(𝑥𝑥) returns: tanh(𝑥𝑥) = (𝑚𝑚𝑥𝑥− 𝑚𝑚−𝑥𝑥)(𝑚𝑚𝑥𝑥+ 𝑚𝑚−𝑥𝑥). sigmoid sigmoid activation function 𝑜𝑜(𝑥𝑥) returns: 1(1+ 𝑚𝑚−𝑥𝑥). Batch size [165] Number of sample partitions within a given epoch which are processed independently and in parallel. Sample [165] One element of a dataset. Epoch [165] One cycle of forward and backward pass of the learning algorithm over the full training dataset; used to separate training into distinct phases. Loss [166] Loss function or 𝑙𝑙() is defined to penalize the errors in prediction. It measures the discrepancy between the ANN outputs 𝑜𝑜 and the ground truth 𝑜𝑜. In the equations below, N is the total number of samples, 𝑜𝑜𝑚𝑚 represent the observed values and 𝑜𝑜𝑚𝑚 is the estimated 𝑜𝑜𝑚𝑚. Types mean_squared_error [167] (MSE, L2 loss) 𝑅𝑅𝑅𝑅𝑅𝑅(𝑜𝑜, 𝑜𝑜) = 1𝑁𝑁�(𝑜𝑜 − 𝑜𝑜)2𝑁𝑁𝑚𝑚=1 70 mean_absolute_error [168] (MAE, L1 loss) 𝑅𝑅𝐴𝐴𝑅𝑅(𝑜𝑜, 𝑜𝑜) = 1𝑁𝑁 ��𝑜𝑜 − 𝑜𝑜�2𝑁𝑁𝑚𝑚=1 huber_loss [169] 𝐻𝐻𝑓𝑓𝑏𝑏𝑒𝑒𝑜𝑜�𝑜𝑜, 𝑜𝑜� = �12 (𝑜𝑜 − 𝑜𝑜)2 𝑜𝑜𝑜𝑜𝑜𝑜 �𝑜𝑜 − 𝑜𝑜� ≤ 𝛿𝛿𝛿𝛿�𝑜𝑜 − 𝑜𝑜� −12 𝛿𝛿2 𝑜𝑜𝑡𝑡ℎ𝑒𝑒𝑜𝑜𝑤𝑤𝑇𝑇𝑠𝑠𝑒𝑒. Optimizer Optimizer is an algorithm which molds the ANN model by updating the weights (𝑤𝑤) and the biases (𝑏𝑏) in response to the outputs of the loss function 𝑙𝑙() in order to minimize it [170]. Types SGD [171] (Stochastic Gradient Descent) SGD partitions the entire dataset randomly and iteratively and makes adjustments to the weights of one subset of the partitioned dataset and proceeds towards an approximate direction of the steepest descent. Adam [172] (Adaptive Moment Estimation) Adam is an extension of SGD where it adaptively tunes the weights for each parameter in the optimization algorithm to accelerate the convergence rate. RMSprop [171] (root mean square propagation) The difference between RMSprop and SGD is how the gradient is calculated. RMSprop incorporates an exponentially decaying average of squared gradients which enables a faster convergence to the answer by restricting and damping oscillations in the vertical directions. Adagrad [173] (adaptive gradient) Adagrad adapts the updates to the slope of the loss function, meaning performing larger updates for infrequent parameters and smaller updates for frequent 71 parameters. By adapting the size of the updates to the importance of the parameters, Adagrad is a suitable optimizer for sparse data.
Thesis/Dissertation
eng
Mechanical Engineering
Vancouver : University of British Columbia Library
Attribution-NonCommercial-NoDerivatives 4.0 International
Graduate
Data-driven power estimation of electric buses using deep LSTM neural networks and symbolic aggregate approximation filtering
Text