Applied Science, Faculty of
Engineering, School of (Okanagan)
DSpace
UBCO
Aly, Mohamed Tarek
Master of Applied Science - MASc
University of British Columbia
There exists a growing demand for efficient gas sensing devices in applications involving indoor and outdoor air monitoring, food quality assessment, law enforcement, medical diagnoses and others. Advances in the field of replicating mammalian olfaction are still at infancy as compared to technologies developed to function close to the human senses of vision, hearing, and touch. Microfluidic olfaction detectors (MFODs) are an emerging class of detectors that are low-cost, compact, and highly selective, having a great potential for a wide range of gas monitoring applications. MFODs are, however, susceptible to fluctuations in environmental conditions, and suffer from prolonged sampling times, which hinder their introduction to outdoor applications. This thesis aims at overcoming these two obstacles by developing robust pattern recognition systems using machine learning techniques. To take into account the effect of temperature on MFOD, we perform a machine learning comparison of MFOD performance at three temperature ranges (5, 15, and 25°C) using a limited dataset (as it is challenging to collect substantial amount of data in many applications). By transforming the raw responses using dimensionality reduction techniques and inputting temperature as features, an efficient classifier was able to learn from the small datasets and reached a 95.5% classification accuracy for methane vs. methane-ethane mixure. A unique pattern recognition system was then designed to reduce the recognition time of a MFOD. This system involved: (i) the ensemble of four heterogeneous base machine learning models, (ii) a Long-Short Term Memory Neural Network (LSTM) to construct a portion of the transient response, and (iii) features from an additional fast response sensor. The system was applied to different transient response times to further identify the minimal transient time needed for the proposed system. This effort resulted in facilitating gas discrimination and quantification with less than 20% of the typical MFOD response time using methane and ethane as a case study. By pairing the pattern recognition systems with the MFOD, the performance is shown to improve significantly while keeping the production cost low. The proposed models significantly add to the capability of MFODs in detecting gases for outdoor applications.
MICROFLUIDIC OLFACTION DETECTOR FOR OUTDOOR APPLICATIONS: A PATTERN RECOGNITION APPROACH by Mohamed Tarek Aly B.Sc., United Arab Emirates University, UAEU 2017 A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF APPLIED SCIENCE in THE COLLEGE OF GRADUATE STUDIES (Mechanical Engineering) THE UNIVERSITY OF BRITISH COLUMBIA (Okanagan) January 2021 © Mohamed Tarek Aly, 2021 ii The following individuals certify that they have read, and recommend to College of Graduate Studies for acceptance, the dissertation entitled: Microfluidic Olfaction Detector for Outdoor Applications: A Pattern Recognition Approach submitted by Mohamed Tarek Aly in partial fulfillment of the requirements for the degree of Master of Applied Science in Mechanical Engineering Examining Committee: Dr. Mina Hoorfar, School of Engineering Supervisor Dr. Homayoun Najjaran, School of Engineering Co- Supervisor Dr. Abbas Milani, School of Engineering Supervisory Committee Member Dr. Anas Chaaban, School of Engineering Supervisory Committee Member Dr. Rehan Sadiq, School of Engineering University Examiner iii Abstract There exists a growing demand for efficient gas sensing devices in applications involving indoor and outdoor air monitoring, food quality assessment, law enforcement, medical diagnoses and others. Advances in the field of replicating mammalian olfaction are still at infancy as compared to technologies developed to function close to the human senses of vision, hearing, and touch. Microfluidic olfaction detectors (MFODs) are an emerging class of detectors that are low-cost, compact, and highly selective, having a great potential for a wide range of gas monitoring applications. MFODs are, however, susceptible to fluctuations in environmental conditions, and suffer from prolonged sampling times, which hinder their introduction to outdoor applications. This thesis aims at overcoming these two obstacles by developing robust pattern recognition systems using machine learning techniques. To take into account the effect of temperature on MFOD, we perform a machine learning comparison of MFOD performance at three temperature ranges (5, 15, and 25C) using a limited dataset (as it is challenging to collect substantial amount of data in many applications). By transforming the raw responses using dimensionality reduction techniques and inputting temperature as features, an efficient classifier was able to learn from the small datasets and reached a 95.5% classification accuracy for methane vs. methane-ethane mixure. A unique pattern recognition system was then designed to reduce the recognition time of a MFOD. This system involved: (i) the ensemble of four heterogeneous base machine learning models, (ii) a Long-Short Term Memory Neural Network (LSTM) to construct a portion of the transient response, and (iii) features from an additional fast response sensor. The system was applied to different transient response times to further identify the minimal transient time needed for the proposed system. This effort resulted in facilitating gas discrimination and quantification with less than 20% of the typical MFOD response time using methane and ethane as a case study. iv By pairing the pattern recognition systems with the MFOD, the performance is shown to improve significantly while keeping the production cost low. The proposed models significantly add to the capability of MFODs in detecting gases for outdoor applications. v Lay Summary In 1914, Alexander Graham Bell stated, “If you are ambitious to find a new science, measure a smell”. Sensing of gases can be looked at as 'smelling'. A substantial amount of research has been carried out on mammalian’s senses like vision, hearing, and touch, and instruments equivalently functioning close to these senses were manufactured. Conversely, the sense of smell, or olfaction, has not been heavily dealt with as its peers, and there exists no device that has been able to perform effectively close to the mammalian’s olfactory system. Microfluidic olfaction detectors (MFOD) are an emerging technology that have shown potential in many gas sensing applications. MFODs are, however, susceptible to fluctuations in environmental conditions, and suffer from prolonged sampling times, which limit their outdoor usage. By relying on artificial intelligence, the performance of MFODs recognition systems was improved at no extra cost or modifications of the hardware of these detectors. vi Preface The research presented in this thesis is the original work performed by the author. This thesis was supervised by Dr. Mina Hoorfar at the Advanced Thermo-Fluidic Laboratory (ATFL) in the School of Engineering, University of British Columbia. Part of this thesis will be published in different journals. The detail of the publications and the author’s contribution in them are explained below: [1] Mohamed Tarek Aly, Nishat Tasnim, Homayoun Najjaran, Mina Hoorfar, “Pattern recognition system for rapid detection of gases using microfluidic olfaction detector.” To be submitted to Sensors and Actuator B: Chemical Contribution: Conceptualization and Investigation, Experiments, Original manuscript. [2] Mohamed Tarek Aly, Nishat Tasnim, Matthew Barriault, Homayoun Najjaran, Mina Hoorfar, “Training classifiers on limited datasets obtained from microfluidic olfaction detectors operating under different temperatures.” To be submitted to Sensors and Actuator B: Chemical Contribution: Conceptualization and Investigation, Experiments, Original manuscript. vii Table of Contents Abstract ................................................................................................................................... iii Lay Summary ............................................................................................................................ v Preface ...................................................................................................................................... vi Table of Contents ....................................................................................................................vii List of Tables ............................................................................................................................ xi List of Figures..........................................................................................................................xii List of Abbreviations .............................................................................................................. xiv Acknowledgements .................................................................................................................. xv Dedication ............................................................................................................................... xvi Chapter 1: Introduction and Background ............................................................................... 1 1.1 Odors and the olfactory system ....................................................................................1 1.2 Volatile organic compounds and hydrocarbons ............................................................3 1.3 Gas sensing technologies .............................................................................................4 1.3.1 Metal Oxide Semiconductor (MOS/MOX) ...............................................................5 1.3.2 Conductive Polymers (CP) .......................................................................................6 1.3.3 Non-dispersive optical sensors .................................................................................7 1.3.4 Catalytic Beads ........................................................................................................9 1.3.5 Photoionization Detector (PID) ................................................................................9 1.3.6 Natural receptors .................................................................................................... 10 1.4 Gas sensing devices ................................................................................................... 11 1.4.1 Gas Chromatography Mass Spectrometer (GCMS) ................................................ 12 viii 1.4.2 Electronic noses (e-noses) ...................................................................................... 14 1.4.3 Microfluidic olfaction detectors ............................................................................. 15 1.5 Knowledge gap .......................................................................................................... 17 1.6 Objectives .................................................................................................................. 17 1.7 Organization of thesis ................................................................................................ 18 Chapter 2: Background on techniques used in pattern recognition...................................... 19 2.1 Dimensionality reduction techniques ......................................................................... 19 2.1.1 PCA ....................................................................................................................... 19 2.1.2 LDA ...................................................................................................................... 20 2.1.3 Kernel PCA ........................................................................................................... 20 2.1.4 MDS ...................................................................................................................... 21 2.1.5 DWT ..................................................................................................................... 21 2.2 Machine learning models ........................................................................................... 22 2.2.1 Neural networks ..................................................................................................... 22 2.2.2 Support vector machines ........................................................................................ 24 2.2.3 k-Nearest neighbors ............................................................................................... 25 2.2.4 Random forests and decision trees ......................................................................... 26 2.3 Related work regarding pattern recognition systems................................................... 27 Chapter 3: Training classifiers on limited datasets from microfluidic olfaction detector operating under different temperatures ................................................................................. 30 3.1 Chapter overview ....................................................................................................... 30 3.2 Introduction ............................................................................................................... 31 3.3 Apparatus .................................................................................................................. 34 ix 3.4 Methods ..................................................................................................................... 36 3.4.1 Data pre-processing ............................................................................................... 36 3.4.2 Data transformation ............................................................................................... 37 3.4.3 Machine learning training ...................................................................................... 39 3.4.4 Models performance evaluation ............................................................................. 43 3.5 Results and discussion ............................................................................................... 43 3.6 Field test validation.................................................................................................... 47 3.7 Conclusions ............................................................................................................... 48 Chapter 4: Pattern recognition system for rapid detection of gases using microfluidic olfaction detector ..................................................................................................................... 49 4.1 Chapter overview ....................................................................................................... 49 4.2 Introduction and background ..................................................................................... 50 4.3 Experimental setup .................................................................................................... 52 4.4 Data analysis ............................................................................................................. 57 4.4.1 Base models ........................................................................................................... 58 4.4.2 Ensemble learning.................................................................................................. 59 4.4.3 Signal response prediction ..................................................................................... 59 4.4.4 Regression ............................................................................................................. 60 4.5 Results and discussion ............................................................................................... 61 4.5.1 Comparing base models ......................................................................................... 61 4.5.2 Fusing CB .............................................................................................................. 62 4.5.3 Using ensembles of models .................................................................................... 63 4.5.4 Response prediction ............................................................................................... 64 x 4.5.5 Regression ............................................................................................................. 66 4.5.6 Detaching MOS from microchannel ....................................................................... 67 4.6 Conclusions ............................................................................................................... 68 Chapter 5: Summary, contribution’s and future work ......................................................... 69 5.1 Summary ................................................................................................................... 69 5.2 Contributions ............................................................................................................. 70 5.3 Limitation and future work ........................................................................................ 70 Bibliography ............................................................................................................................ 71 xi List of Tables Table 3.1: ANN hyper parameters searched ............................................................................... 40 Table 3.2: SVM hyper parameters searched ............................................................................... 41 Table 3.3: k-NN hyper parameters searched .............................................................................. 42 Table 3.4: RF hyper parameters searched .................................................................................. 42 Table 3.5: Training and testing accuracies for the different combinations of models and DR technique................................................................................................................................... 44 Table 3.6: Field test results ........................................................................................................ 48 Table 4.1: MFOD accuracies for different response times with base models .............................. 62 Table 4.2: MFOD + CB accuracies for different response times ................................................ 63 Table 4.3: MFOD + CB ensemble accuracies for different signal times ..................................... 64 Table 4.4: Accuracies for original and predicted responses ........................................................ 66 xii List of Figures Figure 1.1: Human olfaction system ............................................................................................2 Figure 1.2: Schematic of the parts that composes an NDIR..........................................................8 Figure 1.3: Schematic of GC components .................................................................................. 12 Figure 1.4: Schematic of MS ..................................................................................................... 13 Figure 1.5: Microfluidic olfaction detector ................................................................................ 16 Figure 2.1: A typical structure of a dense neural network .......................................................... 23 Figure 2.2: Support vector machine hyperplane ......................................................................... 25 Figure 2.3: A typical gas sensor (or e-nose) patter recognition system ....................................... 27 Figure 2.4: MOS response ......................................................................................................... 28 Figure 2.5: MFOD response through exposure and recovery stages ........................................... 29 Figure 3.1: The automated setup. ............................................................................................... 35 Figure 3.2: The SM-4-8200 Thermotron Environmental Chamber use ....................................... 35 Figure 3.3: Voltage responses of a MFOD ................................................................................. 36 Figure 3.5: The first two components for different DR techniques ............................................. 45 Figure 3.6: Box plots of accuracies by (a) different models and (b) DR techniques .................... 46 Figure 4.1: The components of the entire gas injection system. ................................................. 53 Figure 4.2: Exposure and recovery valve views ......................................................................... 54 Figure 4.3: Exposure and recovery valve ................................................................................... 55 Figure 4.4: (a) Reponses of MFOD with different exposure times, (b) CB responses ................. 56 Figure 4.5: Pattern recognition system used ............................................................................... 58 Figure 4.6: MFOD responses for different concentrations .......................................................... 60 Figure 4.7: Real and predicted responses for 10 s exposure data ................................................ 65 xiii Figure 4.8: MFOD response before and after detaching MOS from microchannel ..................... 68 xiv List of Abbreviations MFOD Microfluidic Olfaction Detector MOS Metal Oxide Semiconductor sensor CB Catalytic Bead sensor SVM Support Vector Machine ANN Artificial Neural Network k-NN K-Nearest Neighbor RF Random Forest PCA Principal Component Analysis kPCA Kernel Principal Component Analysis LDA Linear Discriminant Analysis MDS Multidimensional scaling DWT Discrete Wavelet Transform SVR Support Vector Machine Regressor LSTM Long-Short Term Memory Neural Network MLP Multi-Layer Perceptron MAE Mean Absolute Error MAPE Mean Absolute Percentage Error ppm part per million 3D Three-dimensional xv Acknowledgements I would like to express my deepest appreciation to my supervisor, Dr. Mina Hoorfar, for her support, patience, and encouragement during my masters, and for making this thesis possible. I extend my gratitude to my co-supervisor Dr. Homayoun Najjaran, for his valuable advices, and I further extend it to my committee members for their advisory. Special thanks to our devoted lab manager, Nishat Tasnim for her dedication and support. Lastly, a good thank you to all my lab mates, friends, and people I have known during this journey. xvi Dedication To my parents, Eng. Ghada and Dr. Tarek, whom I will always be deeply indebted to, for their wise counsel and sympathetic ear, and to my beloved sisters S & Y for giving me strength during the hard times 1 Chapter 1: Introduction and Background 1.1 Odors and the olfactory system A considerable amount of literature has been published on the properties of light, sound, and touch or pressure. The same applies to the human’s senses associated with these qualities, being the visual, auditory, and somatosensory systems. Scientists and researchers have successfully developed devices that perform in similar ways to human’s senses [1]. These devices are able to record the information on sight, sound or interaction through touching, and easily analyze and transfer them to other people. Yet, the sense of smell, or olfaction, has not been heavily dealt with as its peers. To date, no device has been able to perform effectively as the human’s olfactory system. It was not until the 1990s that the olfactory receptors were discovered, and by that the human olfactory system really understood [2]. In the light of this discovery, Richard Axel and Linda Buck, were both awarded the Nobel Prize in Physiology and Medicine 2004 [3]. One of the definitions of odors by Meriam-Webster’s dictionary is “a sensation resulting from adequate stimulation of the olfactory organ”. Other words used include: scent, smell, fragrance, and aroma [4]. The scientific community uses the term odor to refer to smells in general, although it usually refers to an unpleasant smell in the United States of America. All smells encountered in nature are nothing but volatile chemical compounds stimulating receptors in the nose. Simply, these receptors react to the stimulation by sending signals to the brain, and that’s how creatures perceive an odor [5]. A human’s olfactory system (Figure 1.1) consists of nostrils, olfactory epithelium tissue (includes all olfactory receptor neurons), olfactory bulb, olfactory tract, and the smell sensory cortex [6]. The process of detecting a smell starts with sniffing, which the literature emphasizes as being unrelated to the sense of smell [7] [8]. The temperature of the air rushing through the nostrils is first brought to the body’s temperature by 2 passing through the blood capillaries on the inner layer of the nose. The odorant’s molecules then bind to the olfactory receptors. This triggers the electrical signals from the receptor, through the tract, and to the sensory cortex in the brain [1]. A human nose includes about 400 different olfactory receptors having different affinities to different molecules. Every distinct odor would trigger different combinations of receptors, and each would be unique to that odor. 10,000 is the number of odors an average human is able to recognize [9], [10] As mentioned earlier, the sense of olfaction has not yet been heavily studied like other human senses. The other senses were also well standardized (such as the RGB standards of colors) which eased how they are transferred and analyzed. Contrarily to the RGB standard in colors, where only three types of receptors exist in the human eye, the standardization of the 10,000 odors coming from the 400 olfactory receptors is beyond complicated. Until now, scientists have not set a universal standard for odors [11]. There have been several attempts to standardize odors based on chemical structure or perception [12]. These attempts have been successful in identifying some, however, odors are perceived differently between people depending on the individual’s personal memories and experience [13]. This lack of standards made odors hard to represent or reproduce. Figure 1.1: Human olfaction system (Reproduced from [180] with permission from Elsevier) 3 In 1914, Alexander Graham Bell stated, “If you are ambitious to find a new science, measure a smell”. Despite the recent developments in olfaction technology, Bell’s statement is still relevant today. The unexplored areas in the young field leave a great room for innovations and new discoveries [14]. 1.2 Volatile organic compounds and hydrocarbons Volatile organic compounds (VOCs) and hydrocarbons are some of the most common gases around us, and most gas sensing devices are targeted towards sensing these two classes of gases. There is nothing easier to find in any city, town, or a small village than VOCs, which are compounds found in trillions of products used by nearly all the world’s population. It is defined by the US Code of Federal Regulations as “any compound of carbon, excluding carbon monoxide, carbon dioxide, carbonic acid, metallic carbides or carbonates, and ammonium carbonate, which participates in atmospheric photochemical reactions” [15]. It is, however, the high emissions of the VOCs that are regarded harmful to the environment. The Government of Canada defines Volatile Organic Compound (VOC) emissions as "a significant contributing factor in the creation of air pollution in urban areas". The government's official website also indicates that VOCs in Canada are emitted from paints, automotive refinishing products, personal care products, the printing industry, and from the application of asphalt. The government had taken five actions in the ten years’ period (from 2003 – 2013) towards decreasing VOC emissions. These included initiatives, and strict laws on industries to limit the use of chemicals leading to VOCs emissions [16]. In order to enforce these laws, the emissions had to be monitored. VOCs are not only related to air pollution and oil industries. Rotten food and beverages can also be a sources of certain VOCs [17], [18]. Surprisingly, recent research had shown that the presence of a VOC, usually in minute 4 concentrations, in the human breath can signalize the presence of a disease (both biological or physiological) like diabetes, cancer, or even schizophrenia [19]–[22]. Hydrocarbon gases, containing only hydrogen and carbon, are regarded as the simplest organic compounds [23]. Monitoring of hydrocarbons also plays an important role in the oil and gas industries. Methane, ethane, and ethylene are the major constituents of natural gas. Thus, detecting the presence of hydrocarbons around the pipeline networks indicates a leakage, and it becomes necessary to locate the leakage early and estimate its size to avoid catastrophes. The possibilities of such leakages can be 1 to 4%, which is not insignificant given the overall size of this industry [24]. A study estimated that a major pipeline rupture happens every 2.3 years in the Gulf of Mexico Region based on data in the ten years interval of 1996 and 2006 [25]. Hydrocarbons emissions are also a major contributor in the greenhouse effect by forming ground-level ozone in the atmosphere. These emissions can be in billions of cubic meters per year. Additionally, the inhalation of hydrocarbons gases may cause serious health risks on humans [26]. Taken together, these facts manifest solutions to monitoring of hydrocarbons and VOCs as essential in the fields of industry, health care, environment, and others. 1.3 Gas sensing technologies There exist plenty of technologies to sense gases, and many of the emerging technologies are also being utilized for gas sensing applications. Many gas sensing technologies are based on chemical, electrical, electrochemical, optical, calorimetric, acoustic, and other types of variations that occur as a result of the interactions of the sensing material to a certain gas in the ambient air [27]. In this section, only the technologies widely used for detecting VOCs and hydrocarbon gases are listed. 5 Sensors based on different technologies mainly differ in their sensitivities, response time and selectivity. Other factors arise such as the energy consumption, lifetime, and cost. These words are defined as: • Sensitivity- the minimum volume concentration of a gas to be detected. • Selectivity- the capability to recognize a specific gas amongst a mixture of different gases. • Response time- the time period taken by the sensor to generate a warning signal when exposed to a certain concentration of the gas [27]. • Drift- the gradual change over time in the physical or chemical properties of the sensor causing a shift in its sensing capability [28]. 1.3.1 Metal Oxide Semiconductor (MOS/MOX) Bielanski et al. [29] were the first to recognize semiconducting oxides as sensitive material. MOS were then introduced as gas sensors by Taguchi in 1971 [30]. Since then, MOS sensors became the most widely used gas sensors due to its low cost of fabrication, sensitivity, and ability to detect wide variety of gases. The sensing mechanism for MOS happens on the surface, where gas molecules are oxidized or reduced (depending on the gas). In clean air, oxygen molecules adsorb to the surface attracting electrons, leaving less free electrons on the MOS chip. When a reducing gas is introduced, it reacts with the adsorbed oxygen and the oxide itself, releasing the free electrons. Trapping and releasing of the electrons at the surface of MOS is what causes the change in resistivity and conductivity (based on whether the chip is of n-type or p-type). A MOS needs high temperature (usually hundreds of degrees Celsius) to operate. Therefore, a micro heater is fitted on the sensor to supply the needed heat. When connected to a voltage source, the MOS can act as a variable resistor. Its resistance changes depending on the gas sensed, and the temperature of the micro heater [31]. 6 MOS sensors can be classified into transition metal oxides (iron, nickel and chromium oxides), pre-transition-metal oxides (magnesium and aluminum oxides), and post-transition-metal oxides (includes zinc and tin oxides). The pre- and post-transition-metal oxides are usually referred to as non-transition-metal oxides. MOS sensors have high sensitivity towards many VOCs and hydrocarbons, and this sensitivity can further be increased by integrating more than one metal oxide material into one sensor [32]. MOS can also have relatively acceptable response times, and have a acceptable lifetime, although they might suffer from drift [28]. Nevertheless, the high-power consumption of micro heaters is a major concern for their use in portable devices. As discussed above, MOS sensors have high sensitivity, but their ability to distinguish between gases is low, in other words, they have poor selectivity. Their responses are also influenced by environmental factors. Several studies have revealed that MOS problems can in fact be overcome. Elmi et al. [33] created a low power consumption micro heater. In another attempt to lower power consumption of the micro heater, MOS surface was exposed to UV light, and it worked at room temperature [34]. The application of different intensities of UV light likewise allowed the MOS sensors to be tuned to detect certain gases. This was highly unexpected, and the reason for this poor selectivity was investigated in [35]. 1.3.2 Conductive Polymers (CP) Conductive Polymers in general are a relatively emerging technologies that are quickly finding application in different fields. They were discovered by Alan J. Heeger et al. [36] who were rewarded the Nobel Prize in Chemistry for the year 2000. CPs’ conductivity changes when exposed to different gases. CPs tend to be very inexpensive compared to other sensors, as they are only treated plastics. This is the major quality that fascinates researchers about them. Their low 7 cost and small size allowed researchers to build devices with thousands of CP sensors implemented [37]. CPs can also be easily merged with PCBs and integrated circuits [38]. As CPs sensors have come up only lately, they are not commercially available. Therefore, in order to use them, they need to be fabricated in-house. There is little known about the changes of the conductivity of the conductive polymers, and hence its behavior towards any gas is hard to expect. CPs are affected by the preparation methods, and the behavior of the same polymers might differ depending on the fabrication methods and conditions. CPs are also very sensitive to humidity. These factors variate the CP’s resistance values and complicates the expectation of the feedback from CPs to gases. Another major disadvantage of CPs is that the sensitivity of pure polymers is extremely weak, therefore, they are usually mixed with other organic or inorganic materials [39]. This process is called hybridization. CPs can be hybridized with MOS, which can yield good results in terms of enhancing sensitivity, and other characteristics as well [40]. One of work in this area [] showed the hybrids’ sensitivity improvement by twenty-fold, with a linear response between resistance and concentration. The lowest concentration of VOCs sensed through this method has been reported as 10 ppm [41]. However, hybridization was still unable to solve the problem of humidity. Most of the MOS-CP hybrids were tested on compounds involving nitrogen (like ammonia), and these attained a sensitivity of at least 1 ppm. Other hybrids aimed at gases other were able to sense in ppb level [42]. 1.3.3 Non-dispersive optical sensors Optical gas sensors, functioning based on optical absorption at specific wavelengths, include non-dispersive infra-red (NDIR), spectrophotometry, tunable diode laser spectroscopy (TDLS) and photoacoustic spectroscopy (PAS). The most common optical sensors are non-8 dispersive sensors because of their low cost and simple technique. They are composed of a micro bulb, light filters, a reflective light path (usually coated with gold), and a detector (Figure 1.2). The non-dispersive optical sensors are usually used in HVAC applications. Because they are optimized for CO2 detection, they are used in nearly all industries or fields associated with CO2 measurements, however they can be designed to detect other gases. Figure 1.2: Schematic of the parts that composes an ND IR (Reproduced from [43] with permission from IOP Science) There exist commercial non-dispersive infrared sensors that can detect VOCs [44]. The working principle is simple, the micro bulb emits light through the reflective light path, the light passes through filters, permitting only light with certain wavelengths related to the target gas and to the detector. At the same time, the gas flows from the inlet to the outlet in continuous stream. Light from micro bulb passes through the gas, if the targeted gas is present, it absorbs certain wavelengths and the detector measures that abruption [43]. Another enclosed parallel channel (often filled with NO2) acts as a reference. The signals from two detectors at the gas channel and reference channel are compared to account for any changes in temperature. The distance between the micro bulb and detector is critical, therefore both are placed in one cell when designed, together with the filters and gas inlet, pathway, and outlet. Absorption of the light by other gases is also inevitable, which causes significant errors in measurements. This also causes weak selectivity [45]. 9 1.3.4 Catalytic Beads Catalytic Beads or, as it is generally known, pellistors are sensors used to detect gases having high thermal conductivity, in other words, combustible gases. The term “pellistor” is derived from two words, the first part comes from “pellet”, and the second from “resistor” [27]. When a combustible gas reaches a pellistor, the pellistor burns. A temperature variation is created as a result of the combustion of the gas, and consequently, the change in the sensor is detected. Pellistors fall in two categories: catalytic bead sensors and thermal conductivity sensors, where the catalytic type is the most common, and is often referred to as pellistor. In the catalytic bead sensors, the combustion gas burns producing heat, which changes the resistance. The heat produced is proportional to the gas concentration. The thermal conductivity sensors work on a very similar concept as that of catalytic beads, but they measure the amount of heat loss [46]. The pellistors’ response time is usually few seconds, which is required in air monitoring indoors. The main disadvantages of pellistors are the poising and the weak selectivity. With time, compounds and impurities tend to compose over the beads forming a layer that affects its sensing sensitivity. The poisoning is often irreversible. Also, as the pellistors are sensitive to all combustible gases, it would be really ambitious to try to identify the gas based on its combustion, although the combustion strength depends on the concentration [27]. 1.3.5 Photoionization Detector (PID) PIDs are mostly used to detect VOCs, not hydrocarbons. Similar to other optical sensors, a light source is present, which is in this case a UV light source. The UV light ionizes the gas into positively and negatively charged ions. The ions are directed by means of electric field towards polarized electrodes, causing a current flow which is later turned into a voltage signal [47]. 10 PIDs can acquire extremely low sensitivity ranges reaching ppb levels. They are the sensors with the fastest responses, reaching sometimes less than 3 seconds [48]. However, the limitation of the UV lamp ionization energy greatly limits their applications. Not all gases can also be detected by PIDs. The ability of PIDs to ionize a gas depends on the ionization energy of the lamp. Most of the PIDs work with lamps having 10 or 10.6 eV. PIDs can ionize any gas having an ionization energy lower than 10.6 eV, but not higher [49]. Unfortunately, the existing commercial PIDs cannot detect the most common hydrocarbons in the natural gas industries: methane and ethane. The ionization energy of methane is between 12.6 to 13.6 eV, while that of ethane is between 11 to 12.1 eV [50]. 1.3.6 Natural receptors A biosensor is an analytical device used for the detection of an analyte. Biosensors combine a biologically derived material (such as antibodies, nucleic acids, enzymes, or aptamers) with a physicochemical detector [51]. Biosensors show great flexibility in terms of the analytes detected or type of signal output. It has the ability to detect biological or chemical molecules, and output a signal in different forms as electrical, optical, or mass changes [52], [53]. The biological elements can be of three types: enzyme-based, antibody-based, and aptamer-based. Enzyme-based biosensors typically stimulate redox reactions. The change caused by enzymes can be monitored and hence used to sense the surrounding chemical or biological matter. Two advantages appear for using enzymes as the biological element in biosensors: specific enzymes would have very high selectivity towards an analyte. The enzymes would react to the analyte by heating, or producing ions and/or light, which will allow for the change to be sensed in plenty of ways [54]. On the other hand, enzyme-based biosensors are less reliable as they can lose their activity easily. They are also more suitable for small sized analytes. 11 Antibody-based biosensors are used in the detection systems with the fastest responses [55], [56]. The major advantage to the antibody-based biosensors is that the sample of analytes does not need to be purified. Antibodies are costly and are not optimized to detect small-sized analytes. Aptamers are a single-stranded nucleic acid [57], [58]. What distinguishes aptamers over other biosensors are the long-term stability, low-cost and rapid synthesis [59]. Aptamers are more prone to toxicity, besides, their binding to analytes is rather weak. 1.4 Gas sensing devices Not all sensor technologies are found commercially; many are still in the research phase and need great improvements before they can be introduced to the market. Nevertheless, using the well-established sensing technologies (discussed before), many devices were built to harvest the sensing capability of a sensor. The most sophisticated devices are those that were able to overcome the problem of low selectivity. Solutions to poor selectivity in the literature can be categorized into two main categories: (i) separating the gases before they reach the sensor, and (ii) building an array of different sensors with different affinities. Separation is conventionally performed by many ways including chromatography, using membranes or filters, and lately by microfluidics. The most notable instrument that utilizes the gas separation principle is the gas chromatography mass spectrometer (GCMS). Electronic-nose (e-nose) systems, on the other hand, fall under the other category, and are comprised of sensor arrays. When a gas is introduced to the array, a specific response of the sensors collectively is triggered, which produce distinct signals referred to as ‘fingerprint’ responses. 12 This section presents three devices in more details, namely gas chromatography mass spectrometer (GCMS), electronic noses, and sensors paired with microfluidics. 1.4.1 Gas Chromatography Mass Spectrometer (GCMS) GCMS is a method combining two long known techniques of gas chromatography (GC) and mass spectrometry (MS). GC is a typical lab instrument that relies on the principle of chromatography to separate gases (and volatile compounds) in the analytical chemistry field. GC separates gases in the same manner as other chromatographs. A GC has a stationary phase and a mobile phase. The stationary phase is the column (usually several meters long) coiled inside the instrument, and the mobile phase is the inert gas that carries the sample through the column. The method of GC allows for superior separation performance [60]. In the gas chromatography process (Figure 1.3), a gas sample mixture is injected into the injection port, the sample is split before it reaches the column, and only a small amount of material enters the column. Figure 1.3: Schematic of GC components (Reproduced from [61] with permission from Elsevier) An inert gas carries the sample through the column [62], [63]. The column is usually coated with an oily non-volatile compound. Each component in the gas travels at different times (retention time), and hence speed. The difference in speeds depends on the polarity and molecular 13 weight, which affect how each component particles interact and gets slowed down by the column’s coating; hence, the components of the gas get separated. At the exit of the column, the separated gases hit a flame ionization detector in clusters of the different gases. This allows the detector to detect the quantities of each of the gas mixture components. MS (Figure 1.4) is another system that can identify different compounds of a gas mixture. It separates the components based on their mass and ionizes their molecules and measures the mass-to-charge ratios of each component, and hence identifies them. The MS composes of an ionization source, a mass spectrometer (where the components gets separated), and a detection unit. This method allows for high selectivity and is normally paired with a GC [64]. Figure 1.4: Schematic of MS (Reproduced from [65] with permission from Elsevier) The GCMS allows for great accuracy, and are used for forensic substance identification [66] for their outstanding sensitivity and selectivity [64]. As a result of their big size and conditions needed for their effective functioning, their usage is limited to indoors. The samples go through several extraction processes before they can be introduced to GCMS. The instrument should also be calibrated often which only adds more cost besides its expensive price. All these processes hinder its use as a real-time detector in a field. 14 Several attempts to improve GCMS device, while maintaining its high sensitivity and selectivity, have yielded great results. Zampoli et al. [67] developed a stand-alone miniaturized GC. A MOS sensor was used instead of the typical flame ionization detector. Unlike other GC devices that requires the supply of clean air from cylinders, this device filtered the environmental air. Using the MOS sensors allowed for more inexpensive calibration to fix the sensor’s drift, however, it should be performed weekly. Thus far, just like other miniaturized GCMSs, the miniature designed was still impractical to be used outdoors as its MOS sensor is calibrated to work on filtered air, and the samples of aromatics should be manually injected by a “syringe” to the column. Again the fabrication cost is high, although the research involved using a MOS (which is less expensive) instead of the identifier in other GCs [68]. 1.4.2 Electronic noses (e-noses) The e-noses try to mimic the human’s olfaction system by incorporating sensor arrays. Each sensor in this array has a different affinity towards different gases, and therefore produces a distinct responses, called “smell prints”, for each odor [69], [70]. Most of the arrays are created from MOS sensors, while others are created from conductive polymers [71]–[76].The “smell prints” are then identified using pattern recognition systems. E-noses have been considerably successful compared to single sensors in applications that require high selectivity. Nevertheless, e-noses are built using the existing sensor technologies, therefore suffer their limitations as well. Pattern recognition systems might also be challenging to build, especially when many sensors are involved. Other challenges unique to the e-noses are the inability to deal with multidimensional drifts of the sensor array, and the exceedingly frequent number of recalibrations [77]. Moreover, as each sensor in the array should be specific, e-noses 15 can be very expensive devices to fabricate and maintain [78]. These reasons typically limit the number of sensors to 3 to 32 sensors in the array [71]–[76]. Despite that, Beccherelli et al. [79] carried out an ambitious project to build an e-nose made up of 65,536 sensors. These sensors encompass only tens of different types of conducive polymer sensors, therefore there was a high degree of redundancy between the 65,536 sensors. The detector was successfully built, but not tested with any gases. Besides e-noses, bioelectronics noses exist and work on the same principle as that of the e-noses. When biological elements are used as the sensing material, the devices are given the name bioelectronic noses. The biosensors were a great discovery, and the number of applications they are being used for is increasing. The most widely applied olfactory-receptor-carrying cells are:, Escherichia coli, Saccharomyces cerevisiae and human embryonic kidney (HEK) [80]. However, their fabrication is very hard and complex, especially to extract the biological element and maintain it. They have a long way before they can be easily fabricated in labs, and perhaps, be available commercially. 1.4.3 Microfluidic olfaction detectors It was not until recently when microfluidics was used to enhance the selectivity of other sensors, predominantly MOS [81]–[83]. Microfluidic olfaction detectors (MFOD) working principle is based on the diffusion of different gases at different rates (i.e., their adsorption/desorption rates) through a coated microchannel to a MOS embedded at the channel's distal end, producing a unique response for each target gas (Figure 1.5). The channels are fabricated using a 3D printer, which maintains its low cost. These distinct responses can then be classified using pattern recognition systems to identify gases. 16 Figure 1.5: Microfluidic olfaction detector (Reproduced from [84] with permission from Elsevier) Paknahad et al. [28] tested different coatings to the inner layer of the microchannel and its dimensions. Microchannels were coated with one material, two, and three on top of each other. The different materials’ coatings affect the speed with which different gases diffuse through the channel and reach the sensor. The optimum coating was a three-layer coating of Chromium, Gold and then Parylene C on top of them. The responses were collected by exposing the sensor to the gas for 40 seconds, and then isolating the sensor from that medium to recover. The recovery time using the optimum coating was 150 seconds which is a significant progress over most of the fabricated sensors having their recovery time as 600 s. Other studies followed, proving the effectiveness of the detector by means of simulations [85], and building gases testing stations [86]. These studies provided a small size gas detector, but that alone does not make it portable. The responses were collected under controlled temperatures and relative humidity. The changes in temperature and humidity were not studied. Furthermore, most of the pattern recognition systems 17 involving the integration of microfluidics with MOS, used conventional ways to extract features from graphs and datasets. 1.5 Knowledge gap Developments of MFOD has been concerned with developing the microchannel and optimizing it for the maximum discrimination between gases. MFOD’s microchannel’s coating and dimensions were empirically optimized, and the results were further backed up by a simulation. MFOD was also tested on many different gases, and several sensing platforms have been built with them. As for the recognition systems used with their responses, recognition systems were used with the MFOD responses, in which the best recognition system was built using machine leaning models. The base machine learning models were compared when trained with several extracted features versus the raw responses, in which the raw responses showed to be better at training the models. MFODs still suffer from several problems limiting their capabilities in different applications. Drift due to fluctuations in environmental conditions like temperature have not been studied yet, although it has been demonstrated before that a membrane was able to neutralize the effect of humidity. 1.6 Objectives The main objective of this study is to facilitate the introduction of the MFODs to outdoor applications through overcoming two main obstacles: • The drift in the response due to the change in temperature of ambient air • The long sampling acquisition times Both limitations were overcome by the development of pattern recognition systems using data collected from an automated setup. 18 1.7 Organization of thesis This thesis is composed of five themed chapters and is structured following the integrated-article format. The five chapters are two chapters for introduction and background chapter, two body chapters, and a conclusion chapter. The body chapters are organized as separate articles with each having its own chapter overview, introduction, materials and methods section, results and discussion section, and conclusions. The first two chapters provides the required background and an overview of the existing literature review about olfaction in general, gas sensing technologies, devices, and components of the pattern recognition systems used. Chapter three discusses the development of a classifier for the microfluidic olfaction detector responses at colder temperatures. Challenges arose during collection of large datasets at colder temperatures (than the room temperature), and as a method to overcome that, several dimensionality reductions were implemented and paired with machine learning models. Chapter four explores the extent to which the sampling frequency of a single microfluidic olfaction detector can be increased. For this purpose, several modifications were applied on the typical microfluidic olfaction detector data sampling system, most importantly the design of a robust pattern recognition system, and it showed an improved performance with capturing lower response times than those previously used. The pattern recognition system was tried with other datasets to validate its possible expansion to other microfluidic olfaction detector applications. The fifth and final chapter summarizes the findings of the work, states the contributions, and outlines what future work. 19 Chapter 2: Background on techniques used in pattern recognition 2.1 Dimensionality reduction techniques Typically, there exist a significant number of correlated, irrelevant, and noisy features in samples’ features space [87]. Dimensionality reduction (DR) techniques, exploiting the aforementioned fact, represent the entire feature set in a smaller set of new features, removing the noise and redundancy. Thus, they input a set of features with the most discriminative information into the machine learning models, and hence, improve their accuracy. DR techniques fall under two main categories: feature extraction and feature selection [88]. Feature selection refers to dropping the least important of the original features, reducing the number of features. On the other hand, feature extraction refers to transforming the original features (by combining some of them) into a new set of fewer features, by recognizing the important patterns in the data [89], [90]. Feature selection techniques have applications mainly in text classification [91] and gene selection [92], whereas feature extraction is usually used in image classification [93] and pattern recognition application [94]. Consequently, all techniques used in this chapter are feature extraction techniques. Different feature extraction techniques can transform the data linearly or non-linearly and can be supervised or unsupervised (data labels included in the transformation or not). 2.1.1 PCA PCA is one of the most classical linear DR method, and has been used with electronic nose data in the past [95]. PCA works by identifying the transformed features with the highest variances, which implies that they are the features with the most information, and hence, the highest effect on the output. This is performed by calculating the eigenvectors and the eigenvalues. Eigenvectors 20 and eigenvalues help illustrate the patterns in the data. The feature with the highest variance is considered to be the first principal component, and the second highest is the second principal component, and so on. This ordering of principal components places those insignificant features at the bottom of the list, which allows PCA to eliminate them, and hence decrease dimensionality [96]. From the principal components, the original features can be extracted again, which is important in transfer applications, especially images [97]. 2.1.2 LDA LDA [98], in the case of binary classification problem, creates an axis that maximizes projections between the two class means, while minimizing the variation. In other words, relying on the data labels, the new axis maximizes class separability, while examples from the same class are put closely together. LDA performs this by first calculating the difference between the means of the two classes, and the variance within each class. Using Fisher's criterion, the separating line that maximizes the difference between the means and minimizes the in-class variation is constructed. LDA can also be used with multi-classification problems, and in this case it is called multiple discriminant analysis [99], [100]. 2.1.3 Kernel PCA The main limitation of linear PCA is its inability to grasp the non-linear trends and relationships in the data. A solution to this problem is to introduce nonlinear PCA. Some of the used nonlinear PCAs include the five-layer feedforward associative network, and the kernel PCA [101]. The kernel PCA first transforms the features into a high-dimensional space by applying a kernel function, then typical PCA methods are performed [102]. 21 2.1.4 MDS There are a number of similarities between MDS and PCA, yet they completely differ mathematically [103]. Instead of finding the correlations between the data as in PCA, MDS transforms the distances between the samples based on their features, then continues with the typical PCA steps. In fact, if the Euclidean distance is computed, MDS and PCA methods eventually yield the same components. There are two types of MDS, "metric", where the distances are proportional to the dissimilarities, and "non-metric”, where distances are in ordinal scale [104]. In supervised MDS, the labels are associated with the samples [103]. 2.1.5 DWT Fourier and wavelet transforms are also some of the commonly used e-nose signal processing transforms [105]. DWT have been used as a feature extraction tool for this purpose [106], [107], and it has shown to be more efficient than Fourier transform. The problem of using Fourier transform is the loss of the location of frequencies in time. Hence, a signal of increasing frequency in time and one of decreasing frequency, look the same after applying Fourier transform. Wavelet transform solves his problem by preserving the localization of different frequencies in time [108]. The original signal is decomposed into two responses after applying wavelet transform. The first response is a result of passing the signal through a low pass filter, which only conserves low frequency components, and is called approximation. While the second response is a result of passing the signal through a high pass filter, and is called detail. Both responses’ dimensions are only half of the original dimension as they are down sampled by two after they pass through the filters. Approximation coefficients can further be decomposed, and in this case the transform is called multi-level decomposition. More levels of decomposition mean less dimensions. In DWT, as opposed to continuous wavelet transform (CWT), wavelet coefficients are calculated at discrete 22 values of scale, where different frequencies are extracted. Different types of wavelets exist, the most common being Haar wavelet, and Daubechies wavelet [109]. 2.2 Machine learning models Recently, a considerable literature has been produced around the theme of using machine learning to classify gases and predict their concentrations [110]–[113]. Specifically, artificial neural network (ANNs), support vector machines (SVMs), and k-nearest neighbors (k-NNs) are the most widely, and maturely used models. Nonetheless, most of the literature has been based on assessments of signals from multiple sensors, and the signals would be mostly extracted features from the transient exposure state and the steady state that follows. Below, the common machine learning models used in gas detector’s pattern recognition systems are introduced. 2.2.1 Neural networks An artificial neural network (ANN) is a computational model inspired by biological neural networks in the human brain. ANNs employ a similar working principle to that of the real brains, with nodes as neurons, connections, and weights instead of dendrites and synapses, and activation functions instead of signals. ANNs are typically organized in layers, the first of which is called the input layer, the last is the output layer, and any layer in between is a hidden layer (as see in Figure 2.1). 23 Figure 2.1: A typical structure of a dense neural network. The left most vertical layer contains the input features, and the right most contains the outputs. The middle layers are the hidden layers. Hidden features in every node are a combination of the previous layer’s weighted activations. Neurons in the input layer take the values of the input features. Each neuron in the hidden and output layers take the sum of values of the previous neurons multiplied by weights and then transformed through what is called an “activation” function. This is known as feedforward propagation. After feed forwarding through the network with random weights, the output is compared with the targets through a cost function. This cost is carried backward to the previous layers, through an algorithm known as back propagation. Back propagation allows the network to update the weights, and hence learn. Many typical algorithms have shown to suffer from limitations when solving problems in high dimensions. This is where neural networks come to the fore: they can be used to form complex non-linear hypothesis, and to detect patterns that are rather complicated. Neural networks can solve classification or regression problems, supervised and unsupervised [114]. 24 2.2.2 Support vector machines Support vector machines (SVM) is a supervised learning classification algorithm that can be adapted to either multiclass classification or to regression. SVM algorithms were first coined by the mid-90s [115], and since then, they have become an extremely popular classification algorithm that have shown to excel over other algorithms in plenty of applications. For linearly separable data, SVMs tend to find the optimal hyperplane for separation (Figure 2.2). That is the hyperplane with the largest margin between the two classes. The margin between the hyperplane and data points is maximized by finding the smallest projection of the data points on the hyperplane. When defining the hyperplane, SVMs also rely on the data points closer to the boundaries, called support vectors. Other training points are not taken into consideration, and the solution becomes a linear combination of only the support vectors. This causes the complexity of an SVM model to remain unchanged by the number of the training data input features. Therefore, SVMs have shown to be robust in the cases where the number of training points are low compared to the number of features, and hence robust against overfitting [116]. For nonlinearly separable data, the kernel trick is used to map the training data onto a high dimensional space where the data can be linearly separable. The hyperplane is constructed in the new space, and new points are mapped to this new space by the kernel function. SVMs are binary classifiers by nature, however they can be adapted to either multiclass classification or to regression. An N different one-versus-rest classifiers approach is used for N-class problem [117]. For regression, SVM algorithm acts the same, however, the margin around the function is constructed to accommodate the function with respect to the error. 25 Figure 2.2: Support vector machine hyperplane (maximizing margins), (Reproduced from [118] with permission from Elsevier) 2.2.3 k-Nearest neighbors k-NNs are one of the simplest classification algorithms, and was described as a simple ‘lazy learning’ method [119]. k-NNs algorithms have no weights to learn, therefore, some data scientists do not regard them as a learning model. k-NNs rely on the fact that each class/cluster would have similar values of features. The new test point is classified based on the classes of the neighboring data points [120]. It is regarded as an instance-based learning method, as it solely relies on its memory of the data points it is trained on to determine the class of the new test point. The aforementioned reason causes k-NNs to be susceptible to noise and irrelevant features from training data. The unsupervised version of the k-NNs algorithm is called k-Means, in which they are used as a clustering algorithm. The main challenge in using k-means is in finding the optimal number of clusters, which is usually time consuming with no principled way to follow yet. Using k-NNs in a supervised learning problem, where labels are known, eases its implementation and makes it 26 attractive to use, especially because k-NNs are highly suitable for multiclass problems, even if its performance is likely to be poor. k-NNs can also be used in regression. 2.2.4 Random forests and decision trees Random Forests are another simple, yet most powerful machine learning classification algorithms. They consist of an ensemble of decision trees. Decision trees start with a root node, and the algorithm keeps splitting data, at nodes called internal nodes as long as the training data in the node consists of more than a class. The partitioning continues until it ends up with purely classified data, in which these ends are called leaves [121]. The decisions to split the data at the internal nodes is conducted with respect to a feature chosen, and a choice of a learned threshold assigned to the chosen feature. Different splitting criteria exist [122], and in general, they try to determine these thresholds and features to split on in an attempt to minimize the entropy, and hence maximize information gain. Entropy is defined as a way to measure the uncertainty of a class in a subset of points. A more desirable split maximizes the entropy loss between an internal node and a child node, and hence maximizes information gain, which is defined as reduction in the uncertainty following a split. One of the special traits that makes decision trees stand out from other machine learning algorithms is that it is not seen as a black box. Indeed, decision trees are highly interpretable, meaning that the reasoning behind a class label choice can be easily represented. Decision trees are also very good at ignoring noise. However, because decision trees use greedy search methods, it does not always attain the optimal solution. Decision trees can also be used for regression. Once the subset in which the new test point falls into is known, the average of the training examples in the subset is calculated and used as the prediction. 27 Random forests are an adaptation of decision trees that tries to create more sturdy models from decision trees. Although decision trees can effectively ignore noise, this is only true given proper pruning of the trees: a problem that can easily be solved when using Random forests [119]. Random forests consist of a bunch of K independently grown decision trees. K can be different subsets of training sets, and subsets of the features as well. These trees are trained to the full depth. When a new test point is introduced, it goes through all the trees, and the final prediction becomes the majority vote of the individual trees' predictions. 2.3 Related work regarding pattern recognition systems Most of the pattern recognition systems built for gas detectors follow the process shown in Figure 2.3. The systems involving machine learning techniques are usually built for e-noses, which turn consist of multiple sensors. Figure 2.3: A typical gas sensor (or e-nose) patter recognition system (Reproduced from [123] with permission from IEEE) Most of the previous studies solely used the steady state value of each sensor in their analysis, which is the saturation value of the sensor in the gas medium. Each test in this case (with an e-nose multiple sensors) produces a feature array equivalent in length to the number of sensors (one steady-state value from each sensor). Fewer studies, however, have utilized transient 28 responses through extracting features from them, which are usually the transient responses before the steady-state is reached (Figure 2.4). Figure 2.4: MOS response through exposing to the gas until the steady-state condition is reached (Reproduced from [124] with permission from Elsevier) Gaudioso et al. [125] used an array of five MOS sensors to classify methanol, ethanol, acetone, and benzene. They achieved accuracy of 94.74% with an SVM. They used the steady-state values of the MOSs, which usually take minutes to reach. Eduard et al. [124] extracted features from the transient response during exposure, and joined them with the steady-state responses from four MOSs to quantify ethanol, toluene, and o-xylene (exposed to concentrations of 25, 50 and 100 ppm for around 100 seconds). Using an ANN method, the accuracy was 100%, compared to only 66% accuracy if the transient features were not included in the analysis. Zhang et al. [112] used three decorated-graphene based MOS’s with three sensors. The time period of 600 seconds for the total of the exposure and recovery stages were recorded. Using a neural network, they predicted the concentration of a mixture gas of ammonia and formaldehyde but did not carry out any classification. The range tested against were 100 to 500 ppm, and the error was less than 1 ppm. Beccherelli et al. [126] used a 16,384 sensors’ device to test different concentrations of ethanol and 2-butanone in the range of around 350 to 20,000 ppm. The signal times were 120 seconds. Principal component analysis (PCA) was used to predict concentrations 29 of both gases separately, and the root mean square error of prediction was 524.5 ppm for ethanol, and 675.2 ppm for 2-butanone. The errors were acceptable on the high end of the concentration range, but not with low concentrations. The classification capability of the sensors was illustrated on plots showing that PCA transformation was able to form two clusters for the two gases, however, the performance was not quantitatively evaluated. All in all, the most suitable approach is dependent on the sensor type, response acquisition protocol, and application. The analysis of MFODs involved using responses that are fully transient (Figure 2.5), in which the recovery stage is started before the sensor even reaches saturation. This is to capture the special shapes produced by the diffusion and adsorption/desorption of the gas particles. The recognition systems used so far with MFODs have involved the following two approaches: (i) visually inspecting the location of 2 or 3 manually extracted features on a plot [86], [127]–[129], or (ii) using a single basic machine learning model for gases [130]. These two approaches allowed MFODs to successfully classify different gases, quantify them, or both. Figure 2.5: MFOD response (the curved blue line) through exposure and recovery stages (Reproduced from [31] with permission from Elsevier) 30 Chapter 3: Training classifiers on limited datasets from microfluidic olfaction detector operating under different temperatures 3.1 Chapter overview Despite MFODs potential, their responses across a wide range of temperatures have not yet been fully characterized. In this chapter, we present a machine learning based comparison of the performance of MFODs at three temperature ranges (5, 15, and 25C). In addition, we train machine learning models using a limited dataset, as it is challenging to collect substantial amount of data in many applications. As a proof of concept, methane and simulated natural gas of ethane-methane mixtures (1-3% ethane) were used as the target gases. Five dimensionality reduction methods were used to extract features from a small dataset for the training of four different machine learning models, for which the classification accuracies were evaluated. The overall results show that MFODs experience a marginal loss in their discrimination capabilities at lower temperatures. Dimensionality reduction techniques accounted for an average of 10.8% increase in the classification accuracy, and the highest classification accuracies for 5, 15, and 25C were found as 95.4%, 97.5%, and 99.5%, respectively. The classification accuracy of a single classifier trained on all temperature sets was 95.5%. The proposed classifier significantly adds to the capability of MFODs in detecting gases from limited data sets obtained in different environmental conditions. Future studies can expand this classifier to different humidity conditions, although the effect of humidity can be neutralized using certain membranes as demonstrated in a study before. 31 3.2 Introduction Fluctuations in environmental conditions, such as temperature, can decrease the sensitivity and selectivity of gas sensors and alter their expected responses. This presents a major challenge for outdoor gas detection systems, requiring calibration against a wide range of temperatures [131]. Microfluidic olfaction detectors (MFODs) are a relatively new technology that have shown potential as a low cost, miniaturized, selective detectors, however, the drift of their response due to temperature changes has not been fully characterized [126]–[128]. A few published studies have reported drifts in the MOS response (towards target gases) associated with changes in temperature, however, our understanding of this temperature effect is limited [133]. In such detectors, the combination of a microchannel with a MOS sensor can further influence the response signal, making the temperature dependent effect more challenging to predict. In essence, the selective characteristics of the microchannel is based on the rate of diffusion of gases along the channel as well as their adsorption and desorption on the channel walls. Both of these phenomena are affected by temperature, and thereby the detector’s response. Generating a simulation or mathematical model of the temperature-dependent drift effect on the response of MFODs is complex, primarily due the lack of a mechanistic understanding of the effect of temperature on the molecular interactions along the channel and MOS interface. Compared to traditional modeling approaches, machine learning based methods provide a viable alternative for predicting sensor responses, especially during the fluctuation of different parameters which is referred to as the drift. There are two main approaches that are currently adopted in the literature for preprocessing the drift: (1) using a reference-based technique, where older known responses are used as references, and the variation of the new signals from the reference is regularly used for compensation [77], [134]–[136]; (2) combining several system 32 identification and self-organization algorithms and techniques to counteract the drift effects [124], [137]–[140]. The general problem with most these approaches is their tendency to focus on compensating for any drift regardless of its source (i.e. aging, poisoning, temperature, humidity, etc.). Even the methods used specifically for the ambient environmental temperature drift have dealt with the baseline and steady-state responses of the MOS rather than the transient responses, which are required for MFODs. Another approach involves inducing changes to the hardware of the sensor through temperature modulation. Periodic modulation of the sensor’s heater voltage can produce responses for which the features are less reliant on the changes caused by the drift [141]–[143]. As the microchannel is another influencing factor on the response, this approach becomes ineffective with MFODs. The added components and power usage might also hinder the portability of the detector. In addition to the problem of compensating for the drift in the transient response of MFODs, the machine learning algorithms usually need large datasets to generate reliable predictions. Failing to provide sufficient data for the models creates the problem of data scarcity. Data scarcity is a common problem arising from the lack of sufficient quality and quantity of data for the training of machine learning models, which can cause them to drastically fail in classification [144], [145]. This results in a common problem known as “curse of dimensionality”, describing why many machine learning algorithms fail with high-dimensional data [146]. The problem lies in the ratio of features to the number of available data points. The relationship between the data points, needed to successfully generalize the model, grows exponentially as the number of features increases. And as a result of increasing the number of features, the ratio of the sample (i.e., the training dataset) to the total population (i.e., the real possible combinations of features) becomes infinitesimally small. The problem of data scarcity can be resolved by relying 33 on dimensionality reduction (DR) techniques, eliminating the need of wasting hardware resources to acquire more data. DR techniques represent the entire feature set in a smaller set of new features, removing the noise and redundancy [87]. Besides, DR saves up data storage space, ameliorates computational time, and can make data visualization conceivable if it decreases features to 3 dimensions or less. In this chapter, we demonstrate, for the first time, the performance of a MFOD at various temperatures to classify gases based on a limited dataset. The results are based on a case study where we collect small datasets for methane and simulated natural gas of ethane-methane mixtures (1-3% ethane) at 5, 15, and 25C. The accuracies of datasets at each temperature was evaluated first, and then a single classifier was trained on all datasets. Because the source of drift can be quantified (by measuring the temperature), incorporating it with the models can facilitate their learning of the drift direction. This saves the hustle of tweaking any of the previous methods [12]–[23] used for the drift compensation of MOS to compensate for the drift in the microchannel. Therefore, the approach in this study is to feed the models with features extracted by each DR technique separately alongside the temperature of each sample (as a feature). We compared the results from five DR techniques with four different machine learning models. The overall results show the highest classification accuracy of 95.5%, achieved by principal component analysis (PCA) data transformation, paired with a support vector machine (SVM). The proposed classifier, showing its robustness with MFODs, can be extended to other gases, and to compensate for different environmental conditions (such as humidity although it was illustrated before that the use of certain membranes with MFODs neutralized the effect of humidity [147]). 34 3.3 Apparatus The sensor used is a TGS 2611 from Figaro, fitted in the terminal end of a microchannel. The microchannel dimension and coating layers have been previously optimized to achieve the highest selectivity [84]. The automated setup used to collect data was previously described in [130]. The components of the automated setup are shown in Figure 3.1(a). The methane and ethane cylinders, connected to the gas chamber by mass flow controllers, were used to provide binary mixtures of 98% and 97% methane and 2 and 3% ethane, for a concentration range of 200 – 2000 ppm. The MFOD (Figure 3.1(b)) was placed in a valve setting on top of the gas chamber lid to control its exposure and recovery to the gas (Figure 3.1(c)). The setup was placed in an environmental chamber (Thermotron SM-4-8200, Figure 3.2) and set at different temperatures (5, 15 and 25C), and around 50 tests were collected at each temperature setpoint. For each test, the sensor response was recorded for a total of 200 s: 10 seconds for the baseline, 40 seconds during the detector exposure to the gas, and 150 seconds for recovery. Figure 3.3 shows the typical transient voltage responses of the MFOD at the same gas for different temperatures (Figure 3.3(a)), different concentrations of a gas at the same temperature (Figure 3.3(b)), and different gases at the same concentration and temperature (Figure 3.3(c)). 35 (a) (b) (c) Figure 3.2: The SM-4-8200 Thermotron Environmental Chamber use Figure 3.1: The automated setup. (a) The schematic of the setup components, (b) a sectional view of the MFOD showing the microchannel and MOS, (c) the valve used to control exposure and recovery of the MFOD (Reproduced from [130] with permission from Elsevier). 36 3.4 Methods A typical gas sensing pattern recognition system was followed in this study (Figure 3.4). After collecting the data experimentally, several anomalies are omitted by visually inspecting the data collected. The data was then preprocessed to remove noise, then transformed through five different DR techniques. The transformed data, together with the raw preprocessed responses are all used to train the four models. 3.4.1 Data pre-processing The controller’s sampling rate was 10 voltage readings/second. The first step was to apply a moving average filter. This filter is simple to implement, yet considered the leading method in removing any random noise, discontinuities or spikes [148]. The moving average filter acts like a sliding window that averages the values in the window to produce a final less noisy output signal. Setup collecting data Data preprocessing: Down sampling and moving average Dimensionality reduction: PCA, LDA, kPCA, MSD DWT Machine learning models: SVM, ANN, k-NN, RF Validation: loss function on validation sets Evaluating models on testing sets Figure 3.3: Voltage responses of a MFOD towards (a) 1100 ppm of natural gas at 5, 15, and 25 °C, (b) 200 to 500 ppm of natural gas at 5 ºC, (c)1000 ppm of methane and natural gas at 5 ºC Figure 3.4: Pattern recognition system used to build classifier 37 A symmetric averaging was used with the gas responses, meaning that the value of the point was averaged with points before and the same point after. The equation used is: 𝑦𝑖 =1𝑀∑ 𝑥𝑖+𝑗𝑗=(𝑀−1)2⁄𝑗=−(𝑀−1)2⁄ (3.1) where 𝑥𝑖+𝑗 and 𝑦𝑖 are the input and output values, respectively, and 𝑀 is the total number of points taken, which is set as 11 to average 5 points before and 5 after each value. Then, the data was down sampled by ten (1 voltage reading/second resulting in a 200-feature array for each test) and standardized to bring the mean of every feature to 0, and its variance to 1 (i.e., a normal distribution)[95][133]. Standardization improves the data set by compensating the slight variations between samples [95]. Besides, machine learning models like SVMs and k-NN require the data to be normalized before they are trained. The data was standardized using the following equation: 𝑧𝑖 =𝑥𝑖− ?̅?𝑠 (3.2) where ?̅? is the mean of a feature, and 𝑠 is the feature’s standard deviation. In this study, standardization was applied to the data before transformation by DR techniques, and on the transformed features before training machine learning models. 3.4.2 Data transformation The five DR techniques chosen in this thesis comprehensively represent the transformations methods in the literature. The five are namely principal component analysis (PCA), linear discriminant analysis (LDA), multidimensional (MDS), kernel PCA (kPCA), and discrete wavelet transform (DWT). PCA and LDA are linear methods, whereas MDS and kernel PCA are nonlinear. Likewise, PCA, Kernel PCA, and DWT are unsupervised learning methods, while LDA and MDS are supervised. These techniques were excessively used in the literature and 38 are mostly not entitled to a specific application. PCA, LDA, and DWT were widely used with electronic nose data, however the data they were used with were not the transient response, rather with features manually extracted from the response curves [95]. One of the highly used methods in e-noses literature in specific is to manually extract piecemeal features from the original curves, such as maximum values, slopes, area under the curve, time constants, etc. [124], [149], [150]. The choice of the features highly depends on the domain knowledge. It can sometimes lead to a better representation of the signals and a reduced dimensionality, especially when one test response spans for 10 minutes or more, in which raw curves become very hard to train. Conversely, in some cases, this feature extraction method fails to retain all the useful information and eliminates much of it and can actually decrease the performance of classifications models. Choosing features that will embody the discriminative features between curves can be quite hectic and is seldom worth the trouble when there are indications that the raw data can be learned, and that the transformation techniques can actually produce highest-valued features. Different number of components from each DR technique were tried with the machine learning models. For PCA, kPCA, and MDS, models were run with all components in the range of 2 to 20. Although explained variance for PCA was already 99% for the first component, more components were investigated to check if higher accuracies can be obtained. For kPCA, measuring the variance is challenging with nonlinear kernels, however, it is expected to be close to that of linear PCA, so the same range was explored. An exploratory run was carried out first to choose the kernel, and the ‘cosine’ kernel was eventually chosen. For MDS, firstly the same 2 to 20 range was explored, then other values in the factor of ten were tried to explore the higher range of 30 to 100. However, it did not yield much different accuracies. For the case of DWT, three ‘Daubechies 39 (‘db2’ [151], db4’[143], and ‘db6’ [152]) were tried each with 3 different levels of decomposition, totaling to 9 different wavelet transformations. Only the approximation coefficients are used, and they further underwent two more levels of decomposition. One of the highly-used methods in e-noses literature is to manually extract piecemeal features (such as maximum values, slopes, area under the curve, and time constants) from the original curves [124], [149], [150]. A preliminary case study was carried out for 8 to 10 hand designed features with the current dataset; however, they did show any sign of improvements in the accuracy, as this approach failed to retain all the useful information. Thus, the evaluation was not carried out on the full dataset with all models, and is not shown in the results and discussion section. 3.4.3 Machine learning training Four machine learning models were compared in this study, namely artificial neural networks (ANN), support vector machines (SVM), k-nearest neighbour (k-NN), and random forest (RF). These models were trained on the datasets acquired at each temperature separately, so the classification accuracy at each temperature was obtained. Then a single classifier was trained on the full set with temperatures included in the dataset as features. Cross validation was used to split training and validation sets, and a similar method was used to split testing data. In cross validation, the model is trained and tested on different generated sets from the original dataset, which reduces variability and increases the confidence in the accuracy values, especially for small datasets [153]. There is a bias-variance trade off with choosing the number of folds (different sets), however, the values of 5 and 10 are the most commonly used as they have shown to perform well empirically [154]. In this study, the full dataset was divided into 85 % training and 15 % testing sets, generated 20 times with the same random 40 seed for all the four models. Another 10 % of the training set was further divided for validation, and the remaining was kept for training. Using the validation sets, the hyperparameters of each model were tuned by grid searching which was conducted based on the hyperparameters performance (loss) on the data. Table 3.1 through Table 3.4 show the different hyper-parameters explored with the models. As the number of input features to the model changes remarkably between runs (the raw data has 200 features, and DR components range from 2 to 20 hyperparameters, which were set to be variable depending on the features. The f symbol in Table 3.1 stands for the number of features, and similarly with the max_features in RFs in Table 3.4. Table 3.1: ANN hyper parameters searched Hyper-parameter Grid searched values neurons f, 0.25f, 0.5f, 0.75f, 1.5f hidden layers 1, 2 activation relu, sigmoid, softmax Learning rate 0.001, 0.01, 0.1 Regularizer (l2) 0, 0.01, 0.1 For ANNs, the number of neurons required to efficiently train a model depends on the number of input features and output neurons, and usually not more than 2 hidden layers are needed for pattern recognition of graphs like the gas sensor responses [155]. Where there is only one output neuron, an efficient optimal number of neurons is usually a fraction of the input dimension. This widely-used approach in training neural networks [155] is to guide the grid searching approach followed in this research. Arbitrary fractions of the input dimension are exhaustively tried with one and two hidden layers. Different activation functions were tried in the neurons as shown in Table 3.1, except for the output node which was fixed to ‘sigmoid’. Another technique employed to bypass overfitting was early stopping (ES) [156]. When the model starts to overfit, 41 the loss of the validation set starts to increase while the training loss decreases simultaneously. It is at this point that the training is stopped promptly and the last recorded weights are saved. The neural networks are trained until overfitting is detected or to a maximum of 5000 epochs. Table 3.2: SVM hyper parameters searched Hyper-parameter Grid searched values C 100, 10, 5, 1 𝜸 0.00001, 0.001, 0.01, 1, 10, 100, 1000 kernel linear, rbf, sigmoid Several different hyper parameters are available to tune an SVM, the ones explored are in Table 3.2. There are plenty of kernel functions to choose from. The C symbol is the inverse of the regularization parameter (can be seen as 1/𝜆). It determines the level of tolerance the model has towards misclassified data, and the number of points to be considered as support vectors. The higher the value of C, the lower the tolerance is towards misclassified points, and the more prone the models are to noise, which results in more complex boundaries, and the more likely to overfit, and vice versa. When using RBF kernels, the parameter 𝛾 controls the influence of a training points chosen as support vectors on the hyeperparameter (𝛾 exists only for RBF kernels and is actually the inverse of the radius of influence of support vectors). Larger values for 𝛾 causes more complex hyperplanes [157]. 42 Table 3.3: k-NN hyper parameters searched Hyper-parameter Grid searched values n_neighbors 1,3,5,7,9,10,11,13,15,21,31,41,51 metric euclidean, manhattan weights uniform, distance Table 3.3 shows the different hyper-parameters explored for k-NN. The n-neighbors determine the number of neighboring points looked at to classify the test point. The metric is the measure of the distance from the test point to the neighboring points. The weights determine how to look at neighboring points. The uniform weights provide equal weights to all points, while the distance weights assign heavier weights to the points closer to the test point. Table 3.4: RF hyper parameters searched Hyper-parameter Grid searched values max_depth 5, 10, 15, 20, 50 max_features 0.25f, 0.5f, f min_samples_leaf 5,3,10 min_samples_split 4,8,12 n_estimator 10, 20, 50, 100 Regarding RF hyperparameters, the max_depth in Table 3.4 is for the maximum depth of a tree. The max_features determine how many features to look at when making each split. The min_samples_leaf is the minimum samples to be at a leaf node. A split will only happen if these minimum number of samples would be present in both new branches. The min_samples_split is the minimum number of samples to make a split. The n_estimator determines the number of trees 43 in the random forest. As more trees are added, the performance of RF usually improves, and the generalization error decreases. 3.4.4 Models performance evaluation With ANNs, ‘binary crossentropy’ loss function was used. The ‘hinge’ loss function was used with SVMs. k-NNs and RFs have slightly different performance measurements: k-NNs depend on the distance metric, whereas RFs depend on the quality of the split. After the models were finalized, the accuracies were calculated for the training and testing sets. The accuracy was calculated as the number of correctly classified points divided by the total number of points. The machine learning models were run with all the different datasets produced from the DR techniques, besides the raw data. PCA, kPCA, and MDS produced 19 sets each, DWT generated 9, and LDA and raw data created 1 set. Thus, the analysis ended up with 68 different datasets. Having 4 different models and applying 10-fold cross validation, a total of 2720 models were run. This does not count for the number of epochs for the hyperparameter tuning using grid searching. 3.5 Results and discussion The training and testing accuracies obtained by the different combinations of DR techniques and machine learning models at the three different temperatures and the full dataset are listed in Table 3.15. Only the accuracies yielded from the highest-valued hyperparameters and DR components were shown in the table. The use of the raw data to train the models resulted in poor accuracies. However, this is not entirely true for SVMs, as they scored acceptable accuracies (>90%) for the 15C and 25C sets. The underlying reason lies in the fact that SVM’s only base their choice of hyperplanes on a limited number of data points called “support vectors”, making SVMs always robust against high dimensional sets. 44 DR techniques significantly improved the accuracies of the models. The first two components of PCA, kPCA, MDS and LDA of the 15C training set were plotted in Figure 3.5 . It clearly shows that most of the DRs induced some sort of separation in the data, which made it easier for the models to classify. PCA (Figure 3.5(a)) showed good separation, and although more than 99% of the explained variance was in the first principal component, some samples are clearly going to be wrongly classified in a two-dimensional space. kPCA (Figure 3.5(b)) formed two clusters (right and left) which were not aligning with the two class labels. More dimensions (components) were needed for PCA and kPCA to improve their separation. Although MDS data had some sort of distinction in class centers with only 2 components (Figure 3.5(c)), using more components did not yield any improvements in accuracies although MDS was able to separate training data well with some models (see Table 3.5). LDA (Figure 3.5(d)) training sets were successfully separated as two classes (which can be seen in Table 3.5 as most of the LDA training sets scored 100% accuracy). However, the testing sets did not separate well. Taking the approximation coefficients from the DWT reduces the signal length by half in every level, SVM ANN k-NN RF Raw PCA LDA kPCA MDS DWT Raw PCA LDA kPCA MDS DWT Raw PCA LDA kPCA MDS DWT Raw PCA LDA kPCA MDS DWT 5C Train 80.0 100 99.0 85.1 84.8 91.9 74.8 97.6 99.1 94.9 88.1 95.4 74.5 100 100 95.8 100 84.0 82.3 98.4 99.1 97.4 79.4 85.7 Test 84.5 92.7 76.4 92.7 78.2 89.1 79.1 93.6 78.2 94.5 77.9 90.9 77.3 95.4 78.2 93.6 76.4 85.5 78.2 94.5 76.4 91.8 79.1 80.9 15C Train 92.1 99.4 100 99.6 98.3 100 91.4 96.9 100 91.8 95.7 97.0 100 100 100 100 95.2 100 93.2 98.3 100 96.5 93.7 90.1 Test 93.8 95.0 95.0 95.0 72.5 96.3 72.5 96.3 95.0 92.5 72.5 96.3 72.5 97.5 95.0 96.3 75.0 90.0 88.8 98.8 95.0 95.0 70.0 88.6 25C Train 98.5 99.1 100 99.9 75.8 100 75.8 99.8 100 99.5 77.2 99.7 100 100 100 94.3 75.8 75.8 75.8 99.9 100 97.0 79.9 75.8 Test 93.3 99.2 88.3 99.2 67.5 97.5 67.5 99.2 88.3 98.3 67.5 95.8 75.8 96.6 88.3 89.2 68.3 67.5 67.5 91.7 88.3 91.7 68.3 67.5 Full set Train 80.7 100.0 99.5 99.7 83.1 97.4 78.2 98.8 99.6 95.9 78.6 84.5 74.6 100 99.6 100 74.2 75.8 84.7 97.7 99.7 97.5 78.6 83.8 Test 80.6 95.5 72.7 91.8 77.4 83.7 82.3 92.9 72.7 91.0 75.9 83.0 76.8 91.0 72.9 89.2 75.5 75.9 78.6 86.9 72.5 90.5 75.1 80.2 Table 3.5: Training and testing accuracies for the different combinations of models and DR technique (high accuracies are highlighted) 45 therefore plotting them does not provide much information. All ‘db’ wavelet (‘db2’, db4’, and ‘db6’) were almost similar in accuracies when using the first level. Despite the second and third levels not yielding the best accuracies, they yielded close accuracies, yet decreasing the components to 52 and 27, respectively. Figure 3.5: The first two components for different DR techniques for the 15ºC training set for methane (cross symbol) and natural gas (dot symbol) results: (a) PCA, (b) kPCA, (c) MDS, and (d) LDA To better assess the effects of the DR techniques and models on the data, and their interplay, box plots of the test sets accuracies (for each model and DR) were plotted in Figure 3.6(a) and (b) for each model and each of the DR techniques, respectively. All models scored accuracies in the same range, which means that they all failed to learn on certain transformed datasets. The best 46 accuracies on average were scored by SVMs, then ANNs, which could have reach higher accuracies given more data. Regarding the DR techniques, the highest accuracies were obtained using PCA, followed by kPCA. Both DR techniques did not have a high standard deviation, meaning that they retained high averages with all models. Interestingly, the lowest accuracies were not achieved by the raw data, but with MDS (although MDS has a similar working principle to PCA). The main reason behind this significant difference between PCA and MDS performance is that PCA captures the variances between the data points, hence maximizing separation for the models (which is more relevant with this dataset). On the other hand, MDS captures the similarities in distance between the points (which might not have been relevant given the data collected at different concentrations). Another clear benefit of using the DR techniques was the reduced computational time: the model training time on PCA compared to that of the raw data reduced by 75.6%. Figure 3.6: Box plots of accuracies by (a) different models and (b) DR techniques The cells with the highest accuracies for each temperature set were highlighted in Table 3.5. The best accuracies were as follows: 95.4% for 5C, 98.8% for 15C, and 99.2% for 25C. The trend in increasing accuracies with the temperature indicate that the sensor is more accurate on tests collected at temperatures closer to the room temperature. The observed decrease in the 47 accuracy of the models at the lower temperatures can be attributed to the lower signal amplitude, indicating that the sensitivity of the detector decreases at low temperatures, and hence it becomes harder to classify low concentration points. Three combinations of the models and the DR techniques reached the highest score of 99.2%, which further supports the observation that the sensor response at 25C was easier to learn. Also, the one classifier built on the entire dataset (refer to as Full set in Table 3.5) scored 95.5% on the testing set. This was the average score from cross validation (i.e., from 20 different testing sets). Out of the misclassified points, 54.5% were samples collected at 5C, 27.5% at 15C, and 18.2% at 25C. 3.6 Field test validation The model with the highest accuracy (PCA and SVM) was installed to a portable device that was used to trace natural gas leakage and discriminate it from background methane emissions. The portable device was used by Hetek Solutions Inc. to detect natural gas at 4 locations in two different days, and was simultaneously compared to the Hetek probe (a separate probe showing the total concentration of hydrocarbons) used to measure total hydrocarbons in air. To build the regression model, more data was collected at room temperature (unlike what presented this chapter), and the model was a support vector regressor. Table 3.6 shows the 6 measurements obtained at 4 different leaking locations. The device achieved 100% classification accuracy. Because the leakage locations were continuously emitting gas, the predicted concentration would depend on the exposure time, and how close the gas was sampled from the leakage spot. Therefore, there was no definite way to completely check if the concentrations are 100% accurate. However, two points validated the results: (1) the larger leaks produce higher concentrations, and the concentration was proportional to the reading from the probe, and (2) the ethane percentage compared to methane was as expected, which is <2%. 48 Table 3.6: Field test results Location Predicted gas True gas Predicted Concentration (ppm) Methane Ethane 1 Natural gas Natural gas 1380 19 2 Natural gas Natural gas 1032 12.5 3 Natural gas Natural gas 2360 41 4 Natural gas Natural gas 1305 18.5 Natural gas Natural gas 1229 17.6 Natural gas Natural gas 1198 17.2 3.7 Conclusions In this chapter, we determined the response of a microfluidic olfaction detector (MFODs) against changes in temperature as a step towards their integration into portable gas sensing devices for outdoor applications. The detector was tested with limited samples of methane and methane-ethane mixtures at 5, 15, and 25C. The combinations of five DR techniques and four machine learning models were tested, and their performance compared. The DR techniques significantly improved the learning of the models, and the best method was PCA, followed by kPCA. SVM was the best model on average. Particularly, PCA transformed data trained with SVM achieved a 95.5% classification accuracy, where 54.5% of the misclassified samples were from the samples collected at 5, 27.5% were at 15, and 18.2% were at 25C. The results indicate that MFOD can be calibrated for different temperatures with a marginal loss in the accuracy as temperature decreases. Future studies can investigate drift compensation due to other factors. Also, for smaller datasets, transfer learning can be another technique to improve artificial neural networks (ANN) performance. Similar training methods can be applied to other target gases to develop gas monitoring solutions in the domains of health, safety, and environment. 49 Chapter 4: Pattern recognition system for rapid detection of gases using microfluidic olfaction detector 4.1 Chapter overview Microfluidic olfaction detectors (MFOD) was able to improve the selectivity of MOS by the microchannel addition, however, this further prolonged the recovery time due to the delay in the gas movement as the result of diffusion. The most intuitive approach to obtain a device with higher sampling rates is to use multiple sensors/detectors to sample sequentially. However, large number of detectors can significantly increase the weight, power draw, and calibration and drift efforts. This chapter aims at reducing the sampling time (by more than half) of a MFOD to be able to use a smaller number of detectors to achieve the same frequency. Such reduction is achieved here by developing a pattern recognition system involving: (i) the ensemble of four heterogeneous base machine learning models, (ii) a Long-Short Term Memory Neural Network (LSTM) to construct a portion of the transient response, (iii) features from an additional fast response sensor, (iv) and detachment of MOS from the microchannel for accelerated recovery. A measurement campaign to collect different transient response times (for the same gases and concentrations) is carried out to find the minimal transient time needed for the proposed recognition system. Using data collected from methane and ethane (as a case study), the results of classification and regression demonstrate the need for only 17.5% of the previously used response time without any loss in accuracy. This reduces the total testing cycle of a MFOD by more than 60%, which further adds to the ability of the selective MFODs by enabling faster detection of gases. 50 4.2 Introduction and background Gas monitors for measuring toxic and combustible gases for public and environmental safety applications require gas sensors with fast responses and high sampling rates [158]–[161]. Some of the gas surveillance applications are typically carried out by installing detectors at fixed monitoring stations that continuously intake samples [162], or on mobile devices such as unmanned vehicles [163], [164]. These applications predominantly use metal oxide semiconductor (MOS) gas sensors [126]–[128], which are cost-effective and highly sensitive towards a wide variety of gases; however, MOS sensors suffer from low selectivity and prolonged signal acquisition times. Microfluidic olfaction detectors (MFOD) exploit the merits of MOS while offering an improved selectivity, yet slows down gas movement, and hence further extends sampling times, especially recovery times. A generic approach for obtaining higher sampling frequencies is to design devices with multiple sensors/detectors sampling sequentially [165]. However, the number of detectors to be used in the device depends on each detector’s individual testing cycle time. The current response capturing time of the MFOD is 200 seconds, and an additional 150 seconds of idle time is needed by the MOS and microchannel to completely recover after completing the data capture process. Given the existing MFOD testing cycle time of 350 seconds, to obtain, for instance, a frequency of 1 sample per 10 seconds, 35 sensors are needed. Any decrease in the sensors testing cycle time implies a decrease in the number of detectors, and hence, a decrease in power draw from MOS heaters, weight, cost, and the laborious maintenance job (related to calibration and drift). Efforts for the reduction of MOS’s sampling time are frequently concerned with the development of novel sensing elements (with faster response times) by coating the MOS chip with nanoparticles [166]–[168], or doping them [169]–[171]. A key issue is repeatability as many 51 variations are involved in the preparation and fabrication processes of these modifications. An alternative approach would be to enhance the recognition systems to be able to detect the analyte in a shorter response time. Optimizing the software, rather than the hardware, is an attractive solution that ensures that the modifications are repeatable and easily reproducible on all detector units. In this chapter, machine learning methods are implemented to reduce the signal time of an MFOD without any physical changes to the structure of the sensing element or coating. Classification and regression of the methane and ethane (as a case study) with different gas exposure and recovery times are investigated. A pattern recognition system involving four base models, their ensembles, and a Long-Short Term Memory Neural Network (LSTM) is designed to minimize the possible time needed for gas identification. Ensemble learning is a process that builds a robust predictor by combining multiple models to provide a strong prediction. The models combined can be homogeneous or heterogeneous depending on the algorithm(s) used for training. In essence, it is the former if the same algorithm is used but trained on different training datasets or with different hyperparameters. It is heterogeneous if different algorithms (e.g., MLPs and SVMs) are used, which is the case in this study. Long short-term memory (LSTM) networks, on the other hand, are a type of neural networks considered as the leading method in sequence prediction, especially time series data. LSTMs have been widely used in application with sequences like speech recognition [172], [173], and forecasting stock market exchange rates [174], [175]. MFOD responses to different hydrocarbon are similar, and usually follow the same trends in exposure and recovery. Hence, based on a portion of the response and older data, it is possible for a model to learn how to reconstruct the remaining part before it is even captured. In this chapter, the response of a catalytic bead (CB) sensor (chosen due to its sensitivity to combustible gases 52 [28][26]) is also added to that of the MFOD to gain more features in a shorter time. Lastly, the idle recovery time was reduced by detaching the MOS from the microchannel, and then attaching it back for the next test. The results show the practicality of the proposed methods by reducing the time by more than 60% while keeping it cost-effective, portable, and repeatable. 4.3 Experimental setup An automated setup was designed to collect the required datasets for the training of robust machine learning models (Figure 4.1). The setup includes a gas injection system, valve controlling the exposure and recovery mechanism, and an electrical system to control and synchronize all the electrical components. The MOS sensor (a Figaro TGS 2611 with the sensitivity range of 500 - 5,000 ppm) is placed at the terminal end of a 3D-printed microchannel (30 mm in length, 3 mm in width, and 500 μm in height). The optimal dimensions, coatings, and fabrication process were previously reported in [85][31]. The CB sensor is Figaro TGS 6810 with the sensitivity range of 500 to 50,000 ppm. The tubes connecting the cylinders (methane and ethane from Praxair with purity of 99.9%) to the aluminum gas chamber (150 x 150 x 150 mm) passed through a mass flow controllers (MFCs; G-series from MKS Instruments, Inc., controlled by a LabVIEW program) and pneumatic solenoid valves which facilitate the control of the flow rate and time, respectively. To stabilize MFCs flow, a tubing bypass is implemented after the MFC and the exhaust. A printed circuit board (PCB) is designed to accommodate the different component of the control circuits, and a raspberry pi is used for control. An air compressor is connected to the chamber to flush it with ambient air between the tests. 53 Figure 4.1: The components of the entire gas injection system. Showing the cylinders and compressor, tubing, solenoid valves, the gas chamber, and tubes connecting them Four views of the valve designed to control the MFOD and CB are shown in Figure 4.2. The valve consists of 11 parts, 2 actuators, and the 2 fitted sensors. All parts were 3D printed, and assembled by screws with o-rings for sealing. The sub figures show (a) the front view of the valve during exposure, (b) the front view during recovery and detaching, (c) the side view during exposure, and (d) the side view during recovery and detaching. During exposure, an actuator rotates the slider to allow air from inside the chamber to reach the sensors (Figure 4.2 (a),(c)). After exposure, the actuator rotates the slider back to lock the air inside the gas chamber and allow clean air inside the sensors to recover. The linear actuator was placed to pull the sensors upward, detaching the MOS from the microchannel. The MOS sensor is then detached after completing the capture time (Figure 4.2 (b), (d)). Figure 4.3 shows two views of the printed valves. Compressor MFCs Solenoid valves mountExposure and recovery valve Gas Chamber 54 Figure 4.2: (a) Front view of the valve during exposure, (b) front view during recovery and detaching, (c) side view during exposure, and (d) side view during recovery and detaching MOS CB Microchannel Slider Linear actuator Rotational actuator (b) (a) (c) (d) 55 A standard signal response time used to predict gases using MFOD is 200 s: 10 seconds for the baseline, 40 seconds of exposure to the gas, and 150 seconds of recovery. In this study, the response acquisition started directly with the beginning of the exposure stage. Diffusion of gases is slow in the microchannel, therefore, the baseline value is retained in the response for the first 5 seconds of exposure. For the recovery time, 150 seconds of the MFOD response are recorded for all tests. Responses of CB’s were recorded until 10 seconds after exposure ended. Figure 4.4(a) shows the MFOD response (in Volt) at the three exposure times for the same gas and concentration, while Figure 4.4(b) shows an example of the CB response for two concentrations (in mVolt). Figure 4.3: Exposure and recovery valve 56 Figure 4.4: (a) Reponses of MFOD with different exposure times, (b) CB responses Automated setup running protocol: 1) The exhaust and compressor valves open. The chamber is first purged by the air from the compressor. Purging happened for 2 minutes. 2) After the chamber is clean, solenoids controlling methane and ethane are opened for a set time to allow the right concentration of both gases to be injected into the chamber. Gases are then left in the chamber for 140 seconds to diffuse in the chamber and form a homogenous mixture. 3) Data capture starts, and after 10 seconds of capturing the baseline, the rotational servo actuator exposes the two sensors to the gas for the set exposure time. Then, the actuator returns them to the recovery position afterwards. 4) Sensors responses are still recorded for 150 seconds while sensors are left to recover. 5) After recovery, the linear actuator pulls both sensors so that the MOS is detached from the channel, and it retains this position for some time. The linear actuator then pushes the sensors back to the normal position. -0.10.10.30.50.70.91.11.31.50 50 100 150 200Voltage (V)Signal time (s)40 s20 s10 s-1-0.500.511.520 10 20 30 40 50Voltage (mV)Signal time (s)4400 ppm4500 ppm(b) (a) 57 6) The chamber is purged and the same cycle is repeated for the next test. 4.4 Data analysis Different combinations of the exposure and recovery times were inputted into the machine learning models. Evaluation of response times started by evaluating the full response length with 150 seconds of recovery, then taking shorter response lengths by incrementing steps of 10 seconds. As an example, for the 40-seconds exposure data, the first 190 seconds of the response were used to train the model and accuracy of the classifiers were noted, then models were trained with 180 seconds and accuracy noted (and so on in steps of 10 s) until 40 s (the beginning of the recovery time). The same procedure was conducted with 20- and 10-s exposure data, starting evaluation with 170 and 160 s of the response and ending with 20 s and 10 s, respectively. The steps of running the analysis were as follow: first, different response times of only MFOD for all datasets were evaluated. Then, the same procedure was repeated, but with the CB response fused. The MFOD and CB data performance were then evaluated with ensembles of four base models. Eventually, an LSTM was used to reproduce a portion of the recovery stage without the need to record it experimentally, and the ensemble models were used to evaluate the new datasets. Although techniques like principal component analysis (PCA) and kernel principal component analysis (kPCA) have proven to be superior with similar smaller datasets (as in Chapter 3:), no significant improvement in accuracy was observed by studying the effect of different transformation techniques explored at different stages of the data analysis. Accordingly, no transformation technique was used in the analysis. 58 The pattern recognition system used in this study is shown in Figure 4.5. In the following subsections, we explain how each of these methods were applied, namely: (i) base models, (ii) ensembles, (iii) LSTM, and (iv) regression models. Figure 4.5: Pattern recognition system used 4.4.1 Base models Grid searching was the method followed to obtain the optimal hyperparameters for all models. Cross validation of 10 folds was applied to split into training and testing sets. A portion of training set (10%) was taken for validation to choose best hyperparameters. Base models used are: (i) Support Vector Machine (SVM) for which different 𝐶 parameters and kernels were tried, and for ‘RBF’ kernel, 𝛾 was also varied; (ii) Multi-Layer Perceptron (MLP) in which two layers were used. Several activation functions were explored for the hidden layers; however, the output layer’s activation function was set to sigmoid. A maximum of 4000 runs were carried out, and early stopping was used to terminate further training if it does not yield improvement in accuracy; (iii) k-Nearest Neighbors (k-NN) where the number of neighbors, metric to decide, and the weight of each neighbor were varied; and finally (iv) Random Forests (RF) for which the number of tress, maximum depth of a tree, maximum features taken into consideration for a split, minimum samples on a leaf were varied. 59 4.4.2 Ensemble learning In this study, we fused the four base models’ predictions at the decision level using three methods: hard voting [176], soft voting [177], and stacking [178]. In hard voting, each model outputs a class prediction, and the decision is made by majority voting (as we used four models, two votes for each class was considered methane). In soft voting, the probability predictions are considered, and the final class probability determines the prediction. In stacking, a meta-model was used to determine the relation between each base model’s prediction, which was k-NN in this study. Because the model’s performance on the test sets varied with concentration, the amplitudes of the samples were used as inputs to the meta-model besides the probability prediction of the four models. The amplitudes are expected to give the meta-model an indication of the sample’s concentration. 4.4.3 Signal response prediction LSTMs were used to reconstruct a part of the recovery response of the MFOD based on a shorter captured response. The training dataset of the MFOD response was sliced into windows with certain length. The subsequent time stamp outside the window was desired the target output which was used to train the LSTM. Grid search was used to obtain the LSTM with the minimal mean squared error between the predicted and original responses. The chosen LSTM consisted of two hidden layers with four and three times as many nodes as the input layer in the first and second layers, respectively. It used activation function of ‘relu’ in all layers, and the optimization algorithm was Adams (with the learn rate of 0.005 and decay of 1e-2) [179]. The learned model predicts only one-time step; therefore, a predicted time step is used again as an input to predict the next time step. The best performance was obtained with a training 60 window of 7 seconds, and a predicted signal of 20 seconds. The error of the predicted signal to the real one propagates as more points are predicted. Although the width of the training window and hyperparameters was based on the lowest mean squared error, the length of the predicted response was chosen based on the classification accuracy. 4.4.4 Regression Within a class of hydrocarbons, higher concentrations yield larger amplitudes which increase with the increase in the concentration. Figure 4.6(a) shows 6 MFOD responses to methane in the range of 1300 to 2700 ppm, (b) shows the plot of some amplitude against the concentration in the range of 500 to 3000 ppm, in which it shows that they vary somehow linearly. Therefore, regressors can be built for each class separately by training them on the amplitudes of that class and having the concentration as targets. Using the classification result and the amplitude, the relevant regressor is then chosen and the concentration is predicted. In this work, two regressors were built, one for methane and one for ethane. Figure 4.6: (a) Six MFOD responses based on 20 second exposure time, (b) the maximum amplitude of the response as a function of concentrations 61 4.5 Results and discussion To compare the effectiveness of each component of the pattern recognition system, the classification accuracies are obtained after (i) applying each base model, (ii) adding the CB response, (iii) implementing ensembles of different base models, and (iv), using the LSTM prediction. These methods were applied on different transient times of the MFOD responses. The final goal is to identify the minimum response time (in identifying the target gas) with no loss in accuracy. 4.5.1 Comparing base models Table 4.1 shows the accuracies over the response time for the three different exposure times using the four models. The majority of the highest accuracies at each signal time is obtained using SVMs on the 40 second exposure tests (see along the horizontal arrow). The maximum accuracy was 100%, using the responses collected at 40-seconds exposure and a total response time of 190 seconds with SVM (boxed in the table), which indicates that this combination provides the maximum amount of information. The trends of the classification accuracies show that the recovery stage present the major portion of information needed for classification. The 40 seconds exposure dataset improves accuracy when longer response is fed into the models (see along the vertical arrows); the same applies for 20-second exposure dataset. For 10-second exposure dataset, on the other hand, after a certain time point, the recovery response does not add any more information, and the extra signal time might cause the models to overfit. 62 Table 4.1: MFOD accuracies for different response times with base models Response Time (s) 10 s 20 s 40 s SVM k-NN RF MLP SVM k-NN RF MLP SVM k-NN RF MLP 10 83.6 61.4 58.6 82.1 - - - - - - - - 20 75.7 72.1 74.3 81.1 89.2 91.7 88.3 88.3 - - - - 30 84.3 80.0 75.0 80.7 86.7 90.8 82.5 87.5 - - - - 40 91.4 73.6 75.0 81.6 91.7 92.5 89.2 87.8 94.6 88.5 81.5 90.0 50 91.4 81.4 77.1 83.6 91.7 90.8 88.3 87.9 92.3 88.5 88.5 90.8 60 94.3 82.1 73.6 84.9 90.0 90.8 90.0 88.2 92.3 92.3 84.6 91.2 70 92.9 82.1 75.7 85.0 90.8 91.7 88.3 88.3 93.1 89.2 86.9 92.1 80 92.9 74.3 72.9 86.1 90.8 91.7 90.0 88.3 94.6 91.5 87.7 93.1 90 92.9 80.7 74.3 86.8 90.8 91.7 89.2 88.4 96.2 91.5 86.2 93.7 100 92.9 82.1 72.9 87.6 93.3 91.7 90.8 88.5 96.2 92.3 88.5 94.2 110 94.3 80.0 75.0 88.2 92.5 91.7 87.5 88.8 96.9 92.3 88.5 94.6 120 94.3 78.6 72.1 88.7 94.2 92.5 91.7 89.0 97.7 92.3 86.2 95.0 130 94.3 79.3 75.7 89.2 92.5 94.2 92.5 89.1 97.7 93.1 90.0 95.3 140 94.3 82.9 75.7 89.2 91.7 94.2 85.8 89.3 97.7 92.3 86.9 95.6 150 79.3 84.3 80.0 89.1 95.8 94.2 91.7 89.3 97.7 92.3 86.9 95.9 160 85.0 82.1 77.9 89.4 95.0 94.2 89.2 89.4 98.5 92.3 89.2 96.1 170 - - - - 92.5 95.0 85.0 89.5 97.7 93.1 90.0 96.3 180 - - - - - - - - 98.5 93.1 85.4 96.5 190 - - - - - - - - 100.0 93.1 88.5 96.8 4.5.2 Fusing CB After fusing the CB response, the accuracies significantly increase (see Table 4.2): an accuracy of 100% was obtained with a total of 120 seconds from the 40 second exposure tests, saving 80 seconds worth of data capture to reach the same accuracy as the data without a CB sensor. Classification accuracy of 99.2% was reached with shorter signal times, which is acceptable. Also 20-second exposure dataset accuracies surpassed those of the 40 second exposure tests with signal times lower than 100 s. 63 Table 4.2: MFOD + CB accuracies for different response times 4.5.3 Using ensembles of models Using ensembles further improve the prediction accuracies, as it can be seen in Table 4.3, where stacking presents the best ensemble method. With only 50 seconds of data capture, 100% classification accuracy was reached in the 10-second exposure tests. Again, for the case of the 10 second exposure tests, the accuracies at longer times. Response time (s) 10 s 20 s 40 s SVM k-NN RF MLP SVM k-NN RF MLP SVM k-NN RF MLP 10 74.3 84.3 85.7 63.6 - - - - - - - - 20 82.9 82.1 84.3 75.7 90.0 92.5 88.3 94.2 - - - - 30 91.4 85.0 82.9 80.0 92.5 94.2 86.7 94.2 - - - - 40 95.7 86.4 83.6 90.7 90.8 93.3 90.8 93.3 92.3 90.0 90.0 91.5 50 95.0 87.9 81.4 96.4 99.2 93.3 86.7 93.3 90.8 90.0 90.0 92.3 60 97.1 91.4 82.1 95.7 99.2 97.5 86.7 92.5 91.5 95.4 90.8 93.8 70 98.6 94.3 82.1 95.7 91.7 96.7 84.2 95.0 93.1 97.7 90.0 93.8 80 97.9 92.9 78.6 94.3 98.3 96.7 90.0 92.5 96.2 97.7 84.6 95.4 90 97.1 94.3 79.3 95.0 99.2 88.3 86.7 94.2 97.7 97.7 86.2 96.9 100 96.4 93.6 78.6 94.3 99.2 88.3 88.3 94.2 98.5 96.2 93.8 97.7 110 95.7 87.1 82.9 95.0 98.3 95.0 88.3 95.0 99.2 96.2 89.2 98.5 120 95.0 94.3 82.9 95.7 98.3 95.0 90.8 95.0 100.0 96.2 87.7 99.2 130 95.0 94.3 82.1 96.4 98.3 95.8 87.5 95.8 100.0 96.2 90.0 100.0 140 95.7 92.1 80.7 96.4 98.3 95.0 95.0 95.8 100.0 96.2 90.0 100.0 150 95.7 92.1 82.9 96.4 98.3 95.0 89.2 95.8 100.0 95.4 87.7 100.0 160 95.7 92.9 82.9 96.4 98.3 94.2 87.5 96.7 100.0 93.8 91.5 100.0 170 - - - - 99.2 90.8 91.7 95.0 100.0 94.6 87.7 100.0 180 - - - - - - - - 100.0 95.4 89.2 100.0 190 - - - - - - - - 100.0 96.9 86.2 100.0 64 Table 4.3: MFOD + CB ensemble accuracies for different signal times Response time (s) 10 s 20 s 40 s Soft Hard Stacking Soft Hard Stacking Soft Hard Stacking 10 85.7 85.0 89.3 - - - - - - 20 85.7 84.3 89.3 95.0 95.0 94.4 - - - 30 93.6 90.0 92.9 94.2 95.0 88.9 - - - 40 97.1 95.7 96.4 94.2 94.2 94.4 94.6 94.6 93.8 50 97.1 96.4 100.0 97.5 97.5 100.0 93.1 93.8 93.8 60 97.1 96.4 100.0 100.0 100.0 94.4 96.2 95.4 95.0 70 97.1 97.1 100.0 96.7 94.2 94.4 96.9 96.2 95.0 80 97.9 97.1 100.0 100.0 100.0 100.0 96.9 96.9 95.0 90 97.1 97.1 96.4 97.5 94.2 100.0 97.7 97.7 100.0 100 96.4 95.0 96.4 98.3 95.8 100.0 98.5 98.5 100.0 110 95.7 95.7 96.4 100.0 99.2 100.0 99.2 99.2 100.0 120 97.1 95.7 96.4 100.0 100.0 100.0 100.0 100.0 100.0 130 97.1 97.1 100.0 100.0 100.0 100.0 100.0 100.0 100.0 140 96.4 96.4 96.4 100.0 100.0 100.0 100.0 100.0 100.0 150 95.0 96.4 96.4 100.0 99.2 100.0 100.0 100.0 100.0 160 96.4 97.1 96.4 100.0 98.3 100.0 100.0 100.0 100.0 170 - - - 100.0 98.3 100.0 100.0 100.0 100.0 180 - - - - - - 100.0 100.0 100.0 190 - - - - - - 100.0 100.0 100.0 4.5.4 Response prediction The results in Table 4.4 show that the LSTM can be used to improve the classification on the 10-second exposure responses shorter than 50 seconds. Hence, the signal responses of the 10 seconds exposure set between 30 and 60 seconds were trained and predicted. The reason the 10 seconds exposure data was considered instead of the 20 seconds exposure data is that it provides a shorter recovery time. Figure 4.7 shows three original MFOD responses, and the corresponding three curves with the regions of the 30 to 54 seconds predicted. The average mean square error of most curves for different training windows was about 5 mV. However, the error seemed to reach 12 mV for low concentration data and as low as 2 mV for high concentration data. 65 Figure 4.7: Real and predicted responses for 10 s exposure data Table 4.4 shows the classification accuracies of the 10-second exposure set for the length of 30 to 55 seconds, evaluated by the stacking ensembles. The first column indicates the time the response capturing stopped. The table compares the accuracies between the original responses and that of the same responses but with added 20 seconds predicted by the trained LSTM. Response prediction remarkably improved classification accuracies, and 100% accuracy was reached with only 35 seconds of recorded response, which further saved 15 seconds of response capturing. 66 Table 4.4: Accuracies for original and predicted responses Response time Original curves Original curves + 20 s predicted 30 92.9 97.9 35 96.4 100 40 96.4 100 45 97.1 100 50 100.0 100 55 100.0 100 4.5.5 Regression Two regression models were built for ethane and methane. The average mean absolute error (MAE), which is the average of the absolute differences between true concentration and the predicted one, and the mean average percentage error (MAPE) which is the percentage of the error to the true concentration were compared for models trained on the full MFOD response length (35 seconds from the original + 20 seconds predicted in this case), and on only the amplitude value. The models compared were the support vector machine regressor (SVR), k-nearest neighbor, multi-layer perceptron, a random forest, and a k-NN stacking ensemble. The results are presented for methane and ethane in Table 4.5 and Table 4.6, respectively. The results show that by relying only on the amplitude to train and test the concentrations, lower errors were achieved. The stacking ensemble model yielded the lowest error for both gases, with an average MAE error of 64 ppm, and a MAPE of 3.5%. 67 Table 4.5: Regression results for methane Full response Amplitude MAE (ppm) MAPE (%) MAE (ppm) MAPE (%) SVR 551.7 33.9 117.3 5.9 k-NN 164.7 7.6 157.2 7.4 MLP 2679.0 100.0 233.5 11.7 RF 290.8 15.3 273.4 13.9 Stacking 153.3 7.8 73.3 4.4 Table 4.6: Regression results for ethane Full response Amplitude MAE (ppm) MAPE (%) MAE (ppm) MAPE (%) SVR 541.6 28.5 90.3 5.4 k-NN 184.9 8.5 178.6 8.3 MLP 2429.9 100.0 165.1 6.6 RF 280.5 14.7 348.3 15.8 Stacking 163.6 8.44 54.6 2.4 4.5.6 Detaching MOS from microchannel To examine the efficiency of detaching the MOS from the microchannel, the response of 10 seconds exposure of MFOD was recorded for 5 concentrations of 500, 1500, 2500, 3500, 4500, and 5000 ppm. Different concentrations require different recovery times, but overall, detaching has shown a decrease of 30 to 40% of the recovery time. Figure 4.8 shows 300 seconds of the response of MFOD for a 10-second exposure to 2000 ppm of ethane when the MOS remained in the microchannel during the whole recovery process, and when it is detached after 35 seconds of the response capture (which is the response needed to recognise the gas). The response after detaching looks unstable, and it eventually settles at a baseline higher than that obtained when the MOS is in the enclosure of the MFOD. Without detaching the MFOD reaches 1% of the baseline value at around 180 seconds, which is 170 seconds after exposure. On the other hand, with 68 detaching, it reached a stable value at around 120 seconds, which is 110 seconds of recovery, shortening the recovery time by 35.2%. Furthermore, after reaching a stable baseline with the detached MOS, it was attached again to the microchannel channel, and the baseline of the MOS returned back to that of the MFOD, which indicates that the microchannel has also recovered. Figure 4.8: MFOD response before and after detaching MOS from microchannel after 35 seconds of signal capture 4.6 Conclusions In this study, we developed a method to decrease the response time of a MFOD for the rapid detection of gases at high sampling frequency. The pattern recognition system was designed using several machine learning methods paired with hardware changes such as the addition of CB and a detachment recovery methodology. A case study was conducted using the developed recognition system with ethane and methane target gases for a concentration range of 500 to 5000 ppm. The results indicated a reduction in MFOD total sampling time by more than 60%, and more than 80% decrease in the identification time. The proposed methodology can be applied in the development of MFOD based devices for cost-effective and accurate outdoor gas surveillance devices. 69 Chapter 5: Summary, contribution’s and future work 5.1 Summary The performance of the MFOD was first characterized against three temperature ranges. An automated setup placed in an environmental chamber (controlling its ambient air temperature) was used to collect data. Small datasets were collected for methane and methane-ethane mixtures in the range of 200 to 2000 ppm. Five dimensionality reduction (DR) techniques were used to extract features from the raw responses to facilitate the classification task of machine learning models. The DR techniques improved the performance of machine learning models, and the classifier reached 95.5% accuracy in classification. The results indicate that MFOD performs better at temperatures closer to the room temperatures. A study was then carried out to decrease the sampling time of the MFOD, and hence increase its sampling frequency. Another automated setup was designed to carry out the required experiments. Shorter transient responses were first recorded and evaluated with the models to determine the shortest response time needed to efficiently discriminate and quantify methane and ethane. Then, the response of a catalytic bead sensor was fused with the MFOD response to improve accuracy. Using an ensemble of four heterogeneous base models, the accuracy was further improved. An LSTM was then used to predict a portion of the recovery response shapes to allow for a shorter signal capture. These modifications combined accounted for a successful discrimination and quantification of the gases using only 17.5% of the response. Finally, the detaching of the MOS from the microchannel was tested to investigate its improvement of the recovery time and it reduced the idle recovery time by around 35%, eventually resulting in a 63% total reduction in the MFOD sampling time. 70 5.2 Contributions Two main limitations of MFODs for outdoor usage were solved. The MFODs can be calibrated to work at different temperatures. Also, continuous gas monitoring devices can be built using MFODs. Multiple sensor devices can now be built with more than 60 % less in power usage and weight for the same sampling frequency, and are robust to environmental fluctuations. The main contribution was solving limitations without any need for hardware modification and all the difficulties associated like optimizations, simulations, experiment, etc. The proposed recognition system is applicable to sensing of other gases by the MFOD with only small changes. 5.3 Limitation and future work The following future work can be conducted: • to investigate drift compensation for ageing, and poisoning of the MFOD’s sensor and microchannel. • to improve pattern recognition system for quantifying gas concentrations in a mixture of more than two gases. • to install MFODs on a device designed for mobile robots. • to use membrane for flow-based applications. In essence, MFODs might not be effective when sampling gas is flowing, unless they are placed in an enclosure. Membranes embedded in channels can be another alternative to adding selectivity to MOS in the case of applications involving flow. 71 Bibliography [1] T. Son, M., Lee, J., Ko, H., & Park, “Bioelectronic Nose: An Emerging Tool for Odor Standardization,” Trends Biotechnol., pp. 301–307, 2017, doi: http://dx.doi.org/10.1016/j.tibtech.2016.12.007. [2] R. Buck, L., Axel, “A novel multigene family may encode odorant receptors: a molecular basis for odor recognition,” Cell, vol. 65, pp. 175–187, 1991, doi: https://doi.org/10.1016/0092-8674(91)90418-X. [3] T. N. Foundation, “Nobel Prize in Physiology and Medicine 2004,” 2004. https://www.nobelprize.org/nobel_prizes/medicine/laureates/2004/. [4] Merriam Webster, “odor,” 2018. https://www.merriam-webster.com/dictionary/odor. [5] R. Katz, B., Matsunami, H., Rinberg, D., Scott, K., Wachowiak, M., & Wilson, “Receptors, Circuits, and Behaviors: New Directions in Chemical Senses,” J. Neurosci., vol. 28, pp. 11802– 11805, 2008, doi: 10.1523/JNEUROSCI.3613-08.2008. [6] A. C. G. and J. E. Hall, Textbook of Medical Physiology, 16th ed. 2006. [7] J. Mainland, “The Sniff is Part of the Olfactory Percept,” Chem. Senses, vol. 31, pp. 181-196., 2006, doi: 10.1093/chemse/bjj012. [8] N. P. Sobel, “Sniffing and Smelling: Separate Subsystems in the Human Olfactory Cortex,” Nature, vol. 392, pp. 282–285, 1998, doi: 10.1038/32654. [9] E. A. Hallem, “Coding of odors by a receptor repertoire,” Cell2, vol. 125, pp. 143–160, 2006, doi: 10.1016/j.cell.2006.01.050. [10] B. Malnic, J. Hirono, T. Sato, and L. B. Buck, “Combinatorial receptor codes for odors,” Cell, vol. 96, no. 5, pp. 713–723, 1999, doi: 10.1016/S0092-8674(00)80581-4. [11] R. Gottfried, J., & Dolan, “The nose smells what the eye sees: crossmodal visual 72 facilitation of human olfactory perception,” Neuron, vol. 39, pp. 375–386, 2003, doi: https://doi.org/10.1016/S0896-6273(03)00392-1. [12] N. Khan, M., Luk, C., Flinker, A., Aggarwal, A., Lapid, H., Haddad, R., & Sobel, “Predicting odor pleasantness from odorant structure: pleasantness as a reflection of the physical world,” J. Neurosci., vol. 27, pp. 10015–10023, 2007, doi: https://doi.org/10.1523/JNEUROSCI. [13] N. Haddad, R., Khan, R., Takahashi, Y., Mori, K., Harel, D., & Sobel, “A metric for odorant comparison,” Nat. Methods, vol. 5, pp. 425–429, 2008. [14] P. M. Wise, M. J. Olsson, and W. S. Cain, “Quantification of odor quality,” Chem. Senses, vol. 25, no. 4, pp. 429–443, 2000, doi: 10.1093/chemse/25.4.429. [15] E. P. Agency, “Chapter I - ENVIRONMENTAL PROTECTION AGENCY,” in Code of Federal Regulations, 2012, pp. 138–143. [16] G. of Canada, “Environment and Climate Change Canada,” 2017. http://www.ec.gc.ca/cov-voc/default.asp?lang=En&n=C064ECF0-1%5C. [17] L. M. Zhua, “Integrated microfluidic gas sensor for detection of volatile organic compounds in water,” Sensors Actuators B Chem., vol. 121, pp. 679–698, 2007. [18] M. A. Bunge, “On-line monitoring of microbial volatile metabolites by proton transfer reaction-mass spectrometry,” Appl. Environ. Microbiol., vol. 74, pp. 2179–2186, 2008. [19] M. H. Mohamed Paknahad, Jannat SMohamed Paknahad, Jannat Singh Bachhal, Ali Ahmadi, Mina Hoorfaringh Bachhal, Ali Ahmadi, “Highly Selective Multi-Target 3D-Printed Microfluidic-Based Breath Analyzer,” in IEEE 29th International Conference on Micro Electro Mechanical Systems (MEMS), 2016, no. January, pp. 905–908. [20] L. C. A. Amorim and Z. de L. Cardeal, “Breath air analysis and its use as a biomarker in 73 biological monitoring of occupational and environmental exposure to chemical agents,” J. Chromatogr. B Anal. Technol. Biomed. Life Sci., vol. 853, no. 1–2, pp. 1–9, 2007, doi: 10.1016/j.jchromb.2007.03.023. [21] Y. Xie, R. Xing, Q. Li, L. Xu, and H. Song, “Three-dimensional ordered ZnO-CuO inverse opals toward low concentration acetone detection for exhaled breath sensing,” Sensors Actuators, B Chem., vol. 211, pp. 255–262, 2015, doi: 10.1016/j.snb.2015.01.086. [22] M. Phillips, “Prediction of lung cancer using volatile biomarkers in breath,” Cancer Biomark., vol. 3, no. 2, pp. 95–109, 2007. [23] J. G. Speight, “Chemical and Physical Properties of Hydrocarbons,” in Handbook of Industrial Hydrocarbon Processes, 2011, pp. 325–353. [24] D. A. Kirchgessner, R. A. Lott, R. M. Cowgill, M. R. Harrison, and T. M. Shires, “Estimate of methane emissions from the U.S. natural gas industry,” 1997. doi: 10.1016/S0045-6535(97)00236-1. [25] SLR, “Assessing Risk and Modeling a Sudden Gas Release Due to Gas Pipeline Ruptures,” 2009. [26] S. Kuppusamy et al., Total Petroleum Hydrocarbons in the Environment. 2020. [27] H. Liu, X., Cheng, S., Liu, H., Hu, S., Zhang, D., & Ning, “A Survey on Gas Sensing Technology,” Sensors, vol. 12, pp. 9635–9665, 2012, doi: 10.3390/s120709635. [28] M. Paknahad, “Development of highly selective single sensor microfluidic-based gas detector,” University of British Columbia, 2017. [29] J. D. and J. H. A. BIELANSKI, “ELECTRIC CONDUCTIVITY AND CATALYTIC ACTIVITY OF SEMICONDUCTING OXIDE CATALYSTS,” Nature, vol. 179, pp. 668–669, 1957, doi: https://doi-org.ezproxy.library.ubc.ca/10.1038/179668a0. 74 [30] N. Taguchi, “Gas Detective Device,” 1971. [31] & H. M. Paknahad, M., Bachhal J. C., Ahmadi A., “Characterization of channel coating and dimensions of microfluidic-based gas detectors,” Sensors Actuators B Chem., vol. 241, pp. 55–64, 2017. [32] X. L. Zhu, C.L., Chen, Y.J., Wang, R.X., Wang, L.J., Cao, M.S., & Shi, “Synthesis and Enhanced Ethanol Sensing Properties of α-Fe2O3/ZnO Heteronanostructures,” Sensors Actuators B, vol. 140, pp. 185–189, 2009, doi: https://doi.org/10.1016/j.snb.2009.04.011. [33] I. Elmi, S. Zampolli, E. Cozzani, F. Mancarella, and G. C. Cardinali, “Development of ultra-low-power consumption MOX sensors with ppb-level VOC detection capabilities for emerging applications,” Sensors Actuators, B Chem., vol. 135, no. 1, pp. 342–351, 2008, doi: 10.1016/j.snb.2008.09.002. [34] J. Saura, “Gas-sensing properties of SnO2 pyrolytic films subjected to ultraviolet radiation,” Sensors Actuators B, vol. 17, pp. 211–214, 1994, doi: https://doi.org/10.1016/0925-4005(93)00874-X. [35] G. Li, W., & Xu, “Unexpected Selectivity of UV Light Activated Metal-Oxide-Semiconductor Gas Sensors by Two Different Redox Processes,” J. Sensors, vol. 2016, pp. 1–6, 2016, doi: http://dx.doi.org/10.1155/2016/4306154. [36] T. N. Foundation, “The Nobel Prize in Chemistry 2000,” 2000. . [37] R. Bernabei, M., Persaud, K.C., Pantalei, S., Zampetti, E., & Beccherelli, “Large-scale chemical sensor array testing biological olfaction concepts,” IEEE Sens. J., vol. 12, pp. 3174–3183, 2012, doi: 10.1109/JSEN.2012.2207887. [38] K. T. Chiu, S.H., & Tang, “Towards a Chemiresistive Sensor-Integrated Electronic Nose: A Review,” Sensors, vol. 13, pp. 14214–14247, 2013, doi: 75 https://doi.org/10.3390/s131014214. [39] M. Janata, J., & Josowicz, “Conducting polymers in electronic chemical sensors,” Nat. Mater., vol. 2, pp. 19–24, 2003, doi: https://doi.org/10.1038/nmat768. [40] H. Park, S. J., Park, C. S., & Yoon, “Chemo-Electrical Gas Sensors Based on Conducting Polymer Hybrids,” Polymers (Basel)., vol. 9, p. 155, 2017, doi: https://doi.org/10.3390/polym9050155. [41] C. C. Quan, L., Sun, J., Bai, S., Luo, R., Li, D., Chen, A., & Liu, “A flexible sensor based on polyaniline hybrid using ZnO as template and sensing properties to triethylamine at room temperature,” Appl. Surf. Sci., vol. 399, pp. 583–591, 2017, doi: https://doi.org/10.1016/j.apsusc.2016.12.133. [42] J. Lee, J.S., Shin, D.H., Jun, J., & Jang, “Multidimensional polypyrrole/iron oxyhydroxide hybrid nanoparticles for chemical nerve gas agent sensing application,” ACS Nano, vol. 7, pp. 10139–10147, 2013, doi: 10.1021/nn404353w. [43] R. P. Hodgkinson, J., & Tatam, “Optical gas sensing: a review,” Meas. Sci. Technol., vol. 24, p. 012004, 2013, doi: 10.1088/0957-0233/24/1/012004. [44] “nenvitech,” 2018. http://www.nenvitech.com/wp-content/uploads/2018/06/DS3706-IRNET-P-32-methane-Rev-81.pdf. [45] H. N. Kannath, A., & Rutt, “Development of low cost instrumentation for non-invasive detection of Helicobacter Pylori,” in Advanced Biomedical and Clinical Diagnostic Systems, p. 2007, doi: 10.1117/12.700168. [46] F. Tardy, P., Coulon, J.R., Lucat, C., & Menil, “Dynamic thermal conductivity sensor for gas detection,” Sensors Actuators B, vol. 2004, no. 98, pp. 63–68, doi: https://doi.org/10.1016/j.snb.2003.09.019. 76 [47] J. Szulczyn ́ski, B., & Ge ̨bicki, “Currently Commercially Available Chemical Sensors Employed for Detection of Volatile Organic Compounds in Outdoor and Indoor Air,” Environment, vol. 4, pp. 1–15, 2017, doi: 10.3390/environments4010021. [48] “Alphasense,” 2016. http://www.alphasense.com/WEB1213/wp-content/uploads/2016/09/PID-AH2.pdf. [49] R. Bogue, “Detecting gases with light: a review of optical gas sensor technologies,” Sens. Rev., vol. 35, pp. 133–140, 2015, doi: https://doi.org/10.1108/SR-09-2014-696. [50] “National Institute of Standards and Technology,” 2017. https://webbook.nist.gov/cgi/cbook.cgi?Name=ethane&Units=SI&cIE=on. [51] & H. M. Luka, G., Ahmadi, A., Najjaran, H., Alocilja, E., DeRosa, M., Wolthers, K., Malki, A., Aziz, H., Althani, A., “Microfluidics Integrated Biosensors: A Leading Technology towards Lab-on-a-Chip and Sensing Applications,” Sensors, vol. 15, pp. 30011–30031, 2015, doi: https://doi.org/10.3390/s151229783. [52] C. A. Ruan, C., Zeng, K., Varghese, O.K., & Grimes, “A magnetoelastic bioaffinity-based sensor for avidin,” Biosens. Bioelectron., vol. 19, pp. 1695–1701, 2004, doi: 10.1016/j.bios.2004.01.005. [53] H. Long, F., Zhu, A., & Shi, “Recent advances in optical biosensors for environmental monitoring and early warning,” Sensors, vol. 13, pp. 13928–13948, 2013, doi: 10.3390/s131013928. [54] A. P. F. Subrahmanyam, S., Piletsky, S.A., & Turner, “Application of Natural Receptors in Sensors and Assays,” Anal. Chem., vol. 74, pp. 3942–3951, 2002, doi: 10.1021/ac025673+. [55] S. Saerens, D., Huang, L., Bonroy, K., & Muyldermans, “Antibody Fragments as Probe in 77 Biosensor Development,” Sensors, vol. 8, pp. 4669–4686, 2008, doi: 10.3390/s8084669. [56] R. Yalow, “Assay of plasma insulin in human subjects by immunological methods,” Nature1, vol. 185, pp. 1648–1649, 1959, doi: https://doi.org/10.1038/1841648b0. [57] B. A. Nimjee, S.M., Rusconi, C.P., & Sullenger, “Aptamers: An emerging class of therapeutics,” Annu. Rev. Med., vol. 56, pp. 555–583, 2005, doi: 10.1146/annurev.med.56.062904.144915. [58] J. Ellington, A., & Szostak, “In vitro selection of RNA molecules that bind specific ligands,” Nature, vol. 346, pp. 818–822, 1990, doi: 10.1038/346818a0. [59] D. Ciobanu, M., & Cliffel, “Glucose and Lactate Biosensors for Scanning Electrochemical Microscopy Imaging of Single Live Cells,” Anal. Chem., vol. 80, pp. 2717–2727, 2008, doi: 10.1021/ac7021184. [60] K. H. Kim, “Performance characterization of the GC/PFPD for H2S, CH3SH, CH3SCH3, and CH3SSCH3 in air,” Atmos. Environ., vol. 39, pp. 2235–2242, 2005. [61] J. A. D. and R. N. Eric Stauffer, “Gas Chromatography and Gas Chromatography–Mass Spectrometry,” in Chromatographic Theory, 2008, p. 293. [62] S. M. Van Ruth, “Methods for gas chromatography-olfactometry: A review,” Biomol. Eng., vol. 17, no. 4–5, pp. 121–128, 2001, doi: 10.1016/S1389-0344(01)00070-3. [63] J. Dorman, F. L., Overton, E. B., Whiting, J. J., Cochran, J. W., & Gardea-Torresdey, “Gas Chromatography,” Anal. Chem., vol. 80, pp. 4487–4497, 2008, doi: 10.1021/ac020210p. [64] Z. P. and F. G. K. O. D. Sparkman, Gas Chromatography and Mass Spectrometry: A Practical Guide, vol. 2. 2010. [65] S. N. Ahmed, “Radiation spectroscopy,” in Physics and Engineering of Radiation 78 Detection, 2015, pp. 689–730. [66] S. J. Cadd, L. Mota, D. Werkman, M. Islam, M. Zuidberg, and M. De Puit, “Extraction of fatty compounds from fingerprints for GCMS analysis,” Anal. Methods, vol. 7, no. 3, pp. 1123–1132, 2015, doi: 10.1039/c4ay02434f. [67] S. Zampolli et al., “Real-time monitoring of sub-ppb concentrations of aromatic volatiles with a MEMS-enabled miniaturized gas-chromatograph,” Sensors Actuators, B Chem., vol. 141, no. 1, pp. 322–328, 2009, doi: 10.1016/j.snb.2009.06.021. [68] R. E. Karasek, W.F., & Clement, Basic Gas Chromatography–Mass Spectrometry: Principles and Techniques. 2012. [69] J. W. Gardner and P. N. Bartlett, “A Brief History of … Electronic Materials A Brief History of … Electronic Materials,” Sensors Actuators B Chem., vol. 18, no. 19, pp. 211–220, 1994. [70] K. Arshak, E. Moore, G. M. Lyons, J. Harris, and S. Clifford, “A review of gas sensors employed in electronic nose applications,” Sens. Rev., vol. 24, no. 2, pp. 181–198, 2004, doi: 10.1108/02602280410525977. [71] H. Singh et al., “Metal oxide SAW E-nose employing PCA and ANN for the identification of binary mixture of DMMP and methanol,” Sensors Actuators, B Chem., vol. 200, pp. 147–156, 2014, doi: 10.1016/j.snb.2014.04.065. [72] J. He, L. Xu, P. Wang, and Q. Wang, “A high precise E-nose for daily indoor air quality monitoring in living environment,” Integr. VLSI J., vol. 58, no. December 2016, pp. 286–294, 2017, doi: 10.1016/j.vlsi.2016.12.010. [73] and F. H. G. Praveen K. Sekhar, Eric L. Brosha, Rangachary Mukundan, “Monitoring and Homeland Security,” Electrochem. Soc. Interface, pp. 35–40, 2010. 79 [74] J. Gębicki, T. Dymerski, and J. Namies̈nik, “Monitoring of odour nuisance from landfill using electronic nose,” Chem. Eng. Trans., vol. 40, no. Special Issue, pp. 85–90, 2014, doi: 10.3303/CET1440015. [75] G. Wei, G. Li, J. Zhao, and A. He, “Development of a LeNet-5 gas identification CNN structure for electronic noses,” Sensors (Switzerland), vol. 19, no. 1, 2019, doi: 10.3390/s19010217. [76] B. Szulczyński et al., “Different ways to apply a measurement instrument of E-nose type to evaluate ambient air quality with respect to odour nuisance in a vicinity of municipal processing plants,” Sensors (Switzerland), vol. 17, no. 11, 2017, doi: 10.3390/s17112671. [77] M. Holmberg, F. Winquist, I. Lundström, F. Davide, C. Dinatale, and A. D’Amico, “Drift counteraction for an electronic nose,” Sensors Actuators, B Chem., vol. 36, no. 1–3, pp. 528–535, 1996, doi: 10.1016/S0925-4005(97)80124-4. [78] W. J. Harper, “The strengths and weaknesses of the electronic nose,” Headsp. Anal. Foods Flavors, vol. 59, pp. 59–71, 2001, doi: https://doi.org/10.1007/978-1-4615-1247-9_5, DOI. [79] R. Beccherelli, E. Zampetti, S. Pantalei, M. Bernabei, and K. C. Persaud, “Design of a very large chemical sensor system for mimicking biological olfaction,” Sensors Actuators, B Chem., vol. 146, no. 2, pp. 446–452, 2010, doi: 10.1016/j.snb.2009.11.031. [80] J. Vidic, “Bioelectronic Nose Based on Olfactory Receptors,” in Intelligent and Biosensors, 2010, p. 377. [81] F. Hossein-Babaei and A. Amini, “Recognition of complex odors with a single generic tin oxide gas sensor,” Sensors Actuators, B Chem., vol. 194, pp. 156–163, 2014, doi: 10.1016/j.snb.2013.12.061. 80 [82] M. Paknahad, V. Ghafarinia, and F. Hossein-Babaei, “A microfluidic gas analyzer for selective detection of biomarker gases,” 2012 IEEE Sensors Appl. Symp. SAS 2012 - Proc., pp. 10–14, 2012, doi: 10.1109/SAS.2012.6166309. [83] V. Ghafarinia, A. Amini, and M. Paknahad, “Gas identification by a single gas sensor equipped with microfluidic channels,” Sens. Lett., vol. 10, no. 3–4, pp. 845–849, 2012, doi: 10.1166/sl.2012.2590. [84] M. Paknahad, J. S. Bachhal, A. Ahmadi, and M. Hoorfar, “Characterization of channel coating and dimensions of microfluidic-based gas detectors,” Sensors Actuators, B Chem., vol. 241, pp. 55–64, 2017, doi: 10.1016/j.snb.2016.10.048. [85] M. Mohaghegh Montazeri, A. O’Brien, and M. Hoorfar, “Understanding microfluidic-based gas detectors: A numerical model to investigate fundamental sensor operation, influencing phenomena and optimum geometries,” Sensors Actuators, B Chem., vol. 300, no. August, 2019, doi: 10.1016/j.snb.2019.126904. [86] M. M. Montazeri, N. De Vries, A. D. Afantchao, A. O’Brien, P. Kadota, and M. Hoorfar, “Development of a Sensing Platform for Nuisance Sewer Gas Monitoring: Hydrogen Sulfide Detection in Aqueous Versus Gaseous Samples,” IEEE Sens. J., vol. 18, no. 19, pp. 7772–7778, 2018, doi: 10.1109/JSEN.2018.2866305. [87] M. Dash and H. Liu, “Consistency-based search in feature selection,” Artif. Intell., vol. 151, no. 1–2, pp. 155–176, 2003, doi: 10.1016/S0004-3702(03)00079-1. [88] W. Pindah, S. Nordin, A. Seman, and M. S. Mohamed Said, “Review of dimensionality reduction techniques using clustering algorithm in reconstruction of gene regulatory networks,” in I4CT 2015 - 2015 2nd International Conference on Computer, Communications, and Control Technology, Art Proceeding, 2015, no. I4ct, pp. 172–176, 81 doi: 10.1109/I4CT.2015.7219560. [89] S. Pölsterl, S. Conjeti, N. Navab, and A. Katouzian, “Survival analysis for high-dimensional, heterogeneous medical data: Exploring feature extraction as an alternative to feature selection,” Artif. Intell. Med., vol. 72, pp. 1–11, 2016, doi: 10.1016/j.artmed.2016.07.004. [90] N. Chumerin and M. M. Van Hulle, “Comparison of two feature extraction methods based on maximization of mutual information,” Proc. 2006 16th IEEE Signal Process. Soc. Work. Mach. Learn. Signal Process. MLSP 2006, pp. 343–348, 2006, doi: 10.1109/MLSP.2006.275572. [91] W. Liu, J. Xiao, and M. Hong, “Comparison on feature selection methods for text classification,” ACM Int. Conf. Proceeding Ser., pp. 82–86, 2020, doi: 10.1145/3380625.3380677. [92] T. Almutiri and F. Saeed, Review on feature selection methods for gene expression data classification, vol. 1073. Springer International Publishing, 2020. [93] M. Ye, C. Ji, H. Chen, L. Lei, H. Lu, and Y. Qian, “Residual deep PCA-based feature extraction for hyperspectral image classification,” Neural Comput. Appl., vol. 32, no. 18, pp. 14287–14300, 2020, doi: 10.1007/s00521-019-04503-3. [94] T. Tuncer, S. Dogan, and A. Subasi, “Surface EMG signal classification using ternary pattern and discrete wavelet transform based feature extraction for hand movement recognition,” Biomed. Signal Process. Control, vol. 58, p. 101872, 2020, doi: 10.1016/j.bspc.2020.101872. [95] R. Gutierrez-Osuna, “Pattern analysis for machine olfaction: A review,” IEEE Sens. J., vol. 2, no. 3, pp. 189–202, 2002, doi: 10.1109/JSEN.2002.800688. 82 [96] L. Smith, “A tutorial on Principal Components Analysis,” 1988. doi: 10.1080/03610928808829796. [97] S. C. Ng, “Principal component analysis to reduce dimension on digital image,” in Procedia Computer Science, 2017, vol. 111, pp. 113–119, doi: 10.1016/j.procs.2017.06.017. [98] N. Kumar, A. Makkar, N. Kumar, and A. Makkar, “Machine Learning Techniques,” in Practical Machine Learning for Data Analysis Using Python, 2020, pp. 67–85. [99] C. H. Park and H. Park, “A relationship between linear discriminant analysis and the generalized minimum squared error solution,” SIAM J. Matrix Anal. Appl., vol. 27, no. 2, pp. 474–492, 2006, doi: 10.1137/040607599. [100] J. Chen, J. Ye, and Q. Li, “Integrating global and local structures: A least squares framework for dimensionality reduction,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2007, pp. 1–8, doi: 10.1109/CVPR.2007.383040. [101] H. Yin, “Nonlinear dimensionality reduction and data visualization: A review,” Int. J. Autom. Comput., vol. 4, no. 3, pp. 294–303, 2007, doi: 10.1007/s11633-007-0294-y. [102] I. A. Grinevich, “Nespetsificheskaia aktivnaia immunoterapiia v kompleksnom lechenii zlokachestvennykh novoobrazovaniǐ.,” Vrac. delo Kiev, vol. 1319, no. 5, pp. 8–11, 1991. [103] A. Ghodsi, “Dimensionality Reduction A Short Tutorial,” 2006. [Online]. Available: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.88.3592&rep=rep1&type=pdf%5Cnhttp://stats.stackexchange.com/questions/7111/pca-for-images-arrays-with-high-dimensionality. [104] S. J. Wright, “Chapter 6 Algorithms,” in Primal-dual interior-point methods, pp. 107–125. 83 [105] J. Yan et al., “Electronic nose feature extraction methods: A review,” Sensors (Switzerland), vol. 15, no. 11, pp. 27804–27831, 2015, doi: 10.3390/s151127804. [106] E. Llobet et al., “Wavelet transform and fuzzy ARTMAP-based pattern recognition for fast gas identification using a micro-hotplate gas sensor,” Sensors Actuators, B Chem., vol. 83, no. 1–3, pp. 238–244, 2002, doi: 10.1016/S0925-4005(01)01047-4. [107] F. Tian et al., “Detection of wound pathogen by an intelligent electronic nose,” Sensors Mater., vol. 21, no. 3, pp. 155–166, 2009, doi: 10.18494/sam.2009.578. [108] Y.-L. Wu, D. Agrawal, and A. El Abbadi, “A comparison of DFT and DWT based similarity search in time-series databases,” pp. 488–495, 2000, doi: 10.1145/354756.354857. [109] K. pong Chan and A. W. chee Fu, “Efficient time series matching by wavelets,” in Proceedings - International Conference on Data Engineering, 1999, pp. 126–133, doi: 10.1109/icde.1999.754915. [110] T. Nakamoto, A. Iguchi, and T. Moriizumi, “Vapor supply method in odor sensing system and analysis of transient sensor responses,” Sensors Actuators, B Chem., vol. 71, no. 3, pp. 155–160, 2000, doi: 10.1016/S0925-4005(99)00186-0. [111] M. Gaudioso, W. Khalaf, and C. Pace, “On the Use of the SVM Approach in Analyzing an Electronic Nose,” 2008, pp. 42–46, doi: 10.1109/his.2007.16. [112] D. Zhang, J. Liu, C. Jiang, A. Liu, and B. Xia, “Quantitative detection of formaldehyde and ammonia gas via metal oxide-modified graphene-based sensor array combining with neural network model,” Sensors Actuators, B Chem., vol. 240, pp. 55–65, 2017, doi: 10.1016/j.snb.2016.08.085. [113] S. De Vito et al., “Gas concentration estimation in ternary mixtures with room 84 temperature operating sensor array using tapped delay architectures,” Sensors Actuators, B Chem., vol. 124, no. 2, pp. 309–316, 2007, doi: 10.1016/j.snb.2006.12.039. [114] S. S. Cross, R. F. Harrison, and R. L. Kennedy, Introduction to neural networks, 1st ed., vol. 346, no. 8982. UCL Press, 1995. [115] S. R. Sain and V. N. Vapnik, The Nature of Statistical Learning Theory, 1st ed., vol. 38, no. 4. Springer, 1996. [116] R. Konieczny and R. Idczak, “Mössbauer study of Fe-Re alloys prepared by mechanical alloying,” Hyperfine Interact., vol. 237, no. 1, pp. 1–8, 2016, doi: 10.1007/s10751-016-1232-6. [117] B. E. Boser, V. N. Vapnik, and I. M. Guyon, “Training Algorithm Margin for Optimal Classifiers,” Perception, pp. 144–152, 1992. [118] D. A. P. and D. M. Schnyer, “Support vector machine,” in Machine Learning, 2020, p. 102. [119] A. Singh, N. Thakur, and A. Sharma, “A review of supervised machine learning algorithms,” Proc. 10th INDIACom; 2016 3rd Int. Conf. Comput. Sustain. Glob. Dev. INDIACom 2016, pp. 1310–1315, 2016. [120] A. C. Lorena et al., “Comparing machine learning classifiers in potential distribution modelling,” Expert Syst. Appl., vol. 38, no. 5, pp. 5268–5275, 2011, doi: 10.1016/j.eswa.2010.10.031. [121] J. R. Quinlan, “Learning decision tree classifiers,” ACM Comput. Surv., vol. 28, no. 1, pp. 71–72, 1996, doi: 10.1145/234313.234346. [122] C. A. Survey, L. Rokach, and O. Maimon, “Top-Down Induction of Decision Trees,” vol. 35, no. 4, pp. 476–487, 2005. 85 [123] R. Gutierrez-Osuna and H. T. Nagle, “A method for evaluating data-preprocessing techniques for odor classification with an array of gas sensors,” IEEE Trans. Syst. Man, Cybern. Part B Cybern., vol. 29, no. 5, pp. 626–632, 1999, doi: 10.1109/3477.790446. [124] E. Llobet, J. Brezmes, X. Vilanova, J. E. Sueiras, and X. Correig, “Qualitative and quantitative analysis of volatile organic compounds using transient and steady-state responses of a thick-film tin oxide gas sensor array,” Sensors Actuators, B Chem., vol. 41, no. 1–3, pp. 13–21, 1997, doi: 10.1016/S0925-4005(97)80272-9. [125] M. Gaudioso, W. Khalaf, and C. Pace, “On the Use of the SVM Approach in Analyzing an Electronic Nose,” pp. 42–46, 2008, doi: 10.1109/his.2007.16. [126] M. Bernabei, K. C. Persaud, S. Pantalei, E. Zampetti, and R. Beccherelli, “Large-scale chemical sensor array testing biological olfaction concepts,” IEEE Sens. J., vol. 12, no. 11, pp. 3174–3183, 2012, doi: 10.1109/JSEN.2012.2207887. [127] P. Mehrabi et al., “Fabrication of SnO2 composite nanofiber-based gas sensor using the electrospinning method for tetrahydrocannabinol (THC) detection,” Micromachines, vol. 11, no. 2, 2020, doi: 10.3390/mi11020190. [128] S. Janfaza et al., “A Nanostructured Microfluidic Artificial Olfaction for Organic Vapors Recognition,” Sci. Rep., vol. 9, no. 1, pp. 1–8, 2019, doi: 10.1038/s41598-019-55672-z. [129] M. Paknahad, C. Mcintosh, and M. Hoorfar, “Selective detection of volatile organic compounds in microfluidic gas detectors based on ‘like dissolves like,’” Sci. Rep., vol. 9, no. 1, pp. 1–11, 2019, doi: 10.1038/s41598-018-36615-6. [130] M. Barriault, I. Alexander, N. Tasnim, A. O’Brien, H. Najjaran, and M. Hoorfar, “Classification and Regression of Binary Hydrocarbon Mixtures using Single Metal Oxide Semiconductor Sensor With Application to Natural Gas Detection,” Sensors Actuators, B 86 Chem., vol. 326, no. October 2020, p. 129012, 2021, doi: 10.1016/j.snb.2020.129012. [131] B. K. C. G. Korotcenkova, “Instability of metal oxide-based conductometric gas sensors and approaches to stability improvement (short survey),” Sensors Actuators, B Chem., vol. 156, no. 2, pp. 527–538, 2011, doi: 10.1016/j.snb.2011.02.024. [132] F. Hossein-Babaei and V. Ghafarinia, “Gas analysis by monitoring molecular diffusion in a microfluidic channel,” Anal. Chem., vol. 82, no. 19, pp. 8349–8355, 2010, doi: 10.1021/ac101767r. [133] J. Watson, “The tin oxide gas sensor and its applications,” Sensors and Actuators, vol. 5, no. 1, pp. 29–42, 1984, doi: 10.1016/0250-6874(84)87004-3. [134] J. E. Haugen, O. Tomic, and K. Kvaal, “A calibration method for handling the temporal drift of solid state gas-sensors,” Anal. Chim. Acta, vol. 407, no. 1–2, pp. 23–39, 2000, doi: 10.1016/S0003-2670(99)00784-9. [135] S. Holmin, C. Krantz-Rülcker, I. Lundström, and F. Winquist, “Drift correction of electronic tongue responses,” Meas. Sci. Technol., vol. 12, no. 8, pp. 1348–1354, 2001, doi: 10.1088/0957-0233/12/8/350. [136] T. Artursson, T. Eklöv, I. Lundström, P. Mårtensson, M. Sjöström, and M. Holmberg, “Drift correction for gas sensors using multivariate methods,” J. Chemom., vol. 14, no. 5–6, pp. 711–723, 2000, doi: 10.1002/1099-128X(200009/12)14:5/6<711::AID-CEM607>3.0.CO;2-4. [137] C. Di Natale, E. Martinelli, and A. D’Amico, “Counteraction of environmental disturbances of electronic nose data by independent component analysis,” Sensors Actuators, B Chem., vol. 82, no. 2–3, pp. 158–165, 2002, doi: 10.1016/S0925-4005(01)01001-2. 87 [138] D. Hui, L. Jun-Hua, and S. Zhong-Ru, “Drift reduction of gas sensor by wavelet and principal component analysis,” Sensors Actuators, B Chem., vol. 96, no. 1–2, pp. 354–363, 2003, doi: 10.1016/S0925-4005(03)00569-0. [139] M. Zuppa, C. Distante, P. Siciliano, and K. C. Persaud, “Drift counteraction with multiple self-organising maps for an electronic nose,” Sensors Actuators, B Chem., vol. 98, no. 2–3, pp. 305–317, 2004, doi: 10.1016/j.snb.2003.10.029. [140] M. Holmberg, F. A. M. Davide, C. Di Natale, A. D’Amico, F. Winquist, and I. Lundström, “Drift counteraction in odour recognition applications: Lifelong calibration method,” Sensors Actuators, B Chem., vol. 42, no. 3 B, pp. 185–194, 1997, doi: 10.1016/S0925-4005(97)80335-8. [141] R. Aigner, F. Auerbach, P. Huber, R. Müller, and G. Scheller, “Sinusoidal temperature modulation of the Si-Planar-Pellistor,” Sensors Actuators B. Chem., vol. 18, no. 1–3, pp. 143–147, 1994, doi: 10.1016/0925-4005(94)87073-X. [142] M. Roth, R. Hartinger, R. Faul, and H. E. Endres, “Drift reduction of organic coated gas-sensors by temperature modulation,” Sensors Actuators, B Chem., vol. 36, no. 1–3, pp. 358–362, 1996, doi: 10.1016/S0925-4005(97)80096-2. [143] R. Ionescu and E. Llobet, “Wavelet transform-based fast feature extraction from temperature modulated semiconductor gas sensors,” Sensors Actuators, B Chem., vol. 81, no. 2–3, pp. 289–295, 2002, doi: 10.1016/S0925-4005(01)00968-6. [144] J. Van Hulse, “Data Quality in Data Mining and Machine Learning,” Florida Atlantic University, 2007. [145] J. Li, J. Zhao, and K. Lu, “Joint feature selection and structure preservation for domain adaptation,” in IJCAI International Joint Conference on Artificial Intelligence, 2016, vol. 88 2016-Janua, pp. 1697–1703. [146] R. E. Bellman, Adaptive Control Processes. London: Princeton University Press London, 2016. [147] M. Paknahad, J. S. Bachhal, and M. Hoorfar, “Diffusion-based humidity control membrane for microfluidic-based gas detectors,” Anal. Chim. Acta, vol. 1021, pp. 103–112, 2018, doi: 10.1016/j.aca.2018.03.021. [148] S. W. Smith, The scientist and engineer’s guide to digital signal processing. 1997. [149] H. Rahimzadeh, M. Sadeghi, M. Ghasemi-Varnamkhasti, S. A. Mireei, and M. Tohidi, “On the feasibility of metal oxide gas sensor based electronic nose software modification to characterize rice ageing during storage,” J. Food Eng., vol. 245, no. June 2018, pp. 1–10, 2019, doi: 10.1016/j.jfoodeng.2018.10.001. [150] S. Balasubramanian, S. Panigrahi, C. M. Logue, H. Gu, and M. Marchello, “Neural networks-integrated metal oxide-based artificial olfactory system for meat spoilage identification,” J. Food Eng., vol. 91, no. 1, pp. 91–98, 2009, doi: 10.1016/j.jfoodeng.2008.08.008. [151] R. Kaur, R. Kumar, A. Gulati, C. Ghanshyam, P. Kapur, and A. P. Bhondekar, “Enhancing electronic nose performance: A novel feature selection approach using dynamic social impact theory and moving window time slicing for classification of Kangra orthodox black tea (Camellia sinensis (L.) O. Kuntze),” Sensors Actuators, B Chem., vol. 166–167, pp. 309–319, 2012, doi: 10.1016/j.snb.2012.02.067. [152] J. Yan et al., “Feature extraction from sensor data for detection of wound pathogen based on electronic nose,” Sensors Mater., vol. 24, no. 2, pp. 57–73, 2012, doi: 10.18494/sam.2012.734. 89 [153] H. A. Martens and P. Dardenne, “Validation and verification of regression in small data sets,” Chemom. Intell. Lab. Syst., vol. 44, no. 1–2, pp. 99–121, 1998, doi: 10.1016/S0169-7439(98)00167-1. [154] M. Kuhn and K. Johnson, Applied predictive modeling. 2013. [155] G. Bin Huang, “Learning capability and storage capacity of two-hidden-layer feedforward networks,” IEEE Trans. Neural Networks, vol. 14, no. 2, pp. 274–281, 2003, doi: 10.1109/TNN.2003.809401. [156] L. Prechelt, “Early Stopping - But When?,” in Neural Networks: Tricks of the Trade, Berlin: Springer, Berlin, Heidelberg, 1998, pp. 55–69. [157] R. G. Brereton and G. R. Lloyd, “Support Vector Machines for classification and regression,” Analyst, vol. 135, no. 2, pp. 230–267, 2010, doi: 10.1039/b918972f. [158] W. Wang et al., “Development of a Pd/Cu nanowires coated SAW hydrogen gas sensor with fast response and recovery,” Sensors Actuators, B Chem., vol. 287, no. September 2018, pp. 157–164, 2019, doi: 10.1016/j.snb.2019.02.047. [159] F. I. M. Ali, S. T. Mahmoud, F. Awwad, Y. E. Greish, and A. F. S. Abu-Hani, “Low power consumption and fast response H2S gas sensor based on a chitosan-CuO hybrid nanocomposite thin film,” Carbohydr. Polym., vol. 236, no. February, p. 116064, 2020, doi: 10.1016/j.carbpol.2020.116064. [160] L. Li, J. Tan, M. Dun, and X. Huang, “Porous ZnFe2O4 nanorods with net-worked nanostructure for highly sensor response and fast response acetone gas sensor,” Sensors Actuators, B Chem., vol. 248, pp. 85–91, 2017, doi: 10.1016/j.snb.2017.03.119. [161] H. B. Na et al., “A fast response/recovery ppb-level H2S gas sensor based on porous CuO/ZnO heterostructural tubule via confined effect of absorbent cotton,” Sensors 90 Actuators, B Chem., vol. 297, no. July, p. 126816, 2019, doi: 10.1016/j.snb.2019.126816. [162] T. G. Van Kessel et al., “Methane Leak Detection and Localization Using Wireless Sensor Networks for Remote Oil and Gas Operations,” Proc. IEEE Sensors, vol. 2018-Octob, pp. 2018–2021, 2018, doi: 10.1109/ICSENS.2018.8589585. [163] M. Rossi, D. Brunelli, A. Adami, L. Lorenzelli, F. Menna, and F. Remondino, “Gas-drone: Portable gas sensing system on UAVs for gas leakage localization,” Proc. IEEE Sensors, vol. 2014-Decem, no. December, pp. 1431–1434, 2014, doi: 10.1109/ICSENS.2014.6985282. [164] T. Kersnovski, F. Gonzalez, and K. Morton, “A UAV system for autonomous target detection and gas sensing,” IEEE Aerosp. Conf. Proc., pp. 1–12, 2017, doi: 10.1109/AERO.2017.7943675. [165] J. Gonzalez-Jimenez, J. G. Monroy, and J. L. Blanco, “The multi-chamber electronic nose-an improved olfaction sensor for mobile robotics,” Sensors, vol. 11, no. 6, pp. 6145–6164, 2011, doi: 10.3390/s110606145. [166] Y. Lin et al., “Preparation of Pd nanoparticle-decorated hollow SnO2 nanofibers and their enhanced formaldehyde sensing properties,” J. Alloys Compd., vol. 651, pp. 690–698, 2015, doi: 10.1016/j.jallcom.2015.08.174. [167] Z. Zhang et al., “H2 response characteristics for sol–gel-derived WO3-SnO2 dual-layer thin films,” Ceram. Int., vol. 43, no. 9, pp. 6693–6699, 2017, doi: 10.1016/j.ceramint.2017.02.065. [168] E. M. El-Maghraby, A. Qurashi, and T. Yamazaki, “Synthesis of SnO2 nanowires their structural and H2 gas sensing properties,” Ceram. Int., vol. 39, no. 7, pp. 8475–8480, 2013, doi: 10.1016/j.ceramint.2013.01.112. 91 [169] C. Feng, F. Teng, Y. Xu, Y. Zhang, T. Fan, and T. Lin, “Au-nanoparticle-decorated SnO2 nanorod sensor with enhanced xylene-sensing performance,” Int. J. Appl. Ceram. Technol., vol. 15, no. 3, pp. 742–750, 2018, doi: 10.1111/ijac.12823. [170] L. Xiao, S. Shu, and S. Liu, “A facile synthesis of Pd-doped SnO2 hollow microcubes with enhanced sensing performance,” Sensors Actuators, B Chem., vol. 221, pp. 120–126, 2015, doi: 10.1016/j.snb.2015.06.099. [171] C. H. Chang, K. W. Lin, H. H. Lu, R. C. Liu, and W. C. Liu, “Hydrogen sensing performance of a Pd/HfO2/GaOx/GaN based metal-oxide-semiconductor type Schottky diode,” Int. J. Hydrogen Energy, vol. 43, no. 42, pp. 19816–19824, 2018, doi: 10.1016/j.ijhydene.2018.08.213. [172] M. Markitantov and O. Verkholyak, Automatic recognition of speaker age and gender based on deep neural networks, vol. 11658 LNAI. 2019. [173] J. Zhao, X. Mao, and L. Chen, “Speech emotion recognition using deep 1D & 2D CNN LSTM networks,” Biomed. Signal Process. Control, vol. 47, pp. 312–323, 2019, doi: 10.1016/j.bspc.2018.08.035. [174] H. S. Sim, H. I. Kim, and J. J. Ahn, “Is Deep Learning for Image Recognition Applicable to Stock Market Prediction?,” Complexity, vol. 2019, 2019, doi: 10.1155/2019/4324878. [175] Y. Mäkinen, J. Kanniainen, M. Gabbouj, and A. Iosifidis, “Forecasting jump arrivals in stock prices: new attention-based network architecture using limit order book data,” Quant. Financ., vol. 19, no. 12, pp. 2033–2050, 2019, doi: 10.1080/14697688.2019.1634277. [176] R. Atallah and A. Al-Mousa, “Heart Disease Detection Using Machine Learning Majority Voting Ensemble Method,” 2019 2nd Int. Conf. New Trends Comput. Sci. ICTCS 2019 - 92 Proc., 2019, doi: 10.1109/ICTCS.2019.8923053. [177] S. Karlos, G. Kostopoulos, and S. Kotsiantis, “A soft-voting ensemble based co-training scheme using static selection for binary classification problems,” Algorithms, vol. 13, no. 1, 2020, doi: 10.3390/a13010026. [178] S. Agarwal and C. R. Chowdary, “A-Stacking and A-Bagging: Adaptive versions of ensemble learning algorithms for spoof fingerprint detection,” Expert Syst. Appl., vol. 146, 2020, doi: 10.1016/j.eswa.2019.113160. [179] D. P. Kingma and J. L. Ba, “Adam: A method for stochastic optimization,” 3rd Int. Conf. Learn. Represent. ICLR 2015 - Conf. Track Proc., pp. 1–15, 2015. [180] J. M. Casey Trimmer, “The Olfactory System,” in Conn’s Translational Neuroscience, 2017, p. 377.
Thesis/Dissertation
eng
Mechanical Engineering
Vancouver : University of British Columbia Library
Attribution-NonCommercial-NoDerivatives 4.0 International
Graduate
Microfluidic olfaction detector for outdoor applications : a pattern recognition approach
Text