Applied Science, Faculty of
Engineering, School of (Okanagan)
DSpace
UBCO
Yu, Jingshuo
Master of Applied Science - MASc
University of British Columbia
The urgent need for advanced fire detection systems stems from the increased intensity in fire events that cause massive property losses and irreversible damage. To overcome the limitations of traditional fire detection systems such as smoke detectors, computer vision (CV) algorithms have been adopted to improve fire detection accuracy. A CV-based fire detection system takes images from fire monitoring camera as input and output fire-related information. To identify the installation location of fire monitoring cameras, this research performed compartment-level fire risk analysis based on fire risk analysis method for engineering (FRAME). Relevant fire risk factors were quantified using a building information modeling (BIM) model, and their relative importance was evaluated through the analytic hierarchy process (AHP). The resulting compartment-specific fire risk values provide data-driven guidance for deploying cameras in high-risk areas of a building. The case study result of a residential building aligns with historical fire incident statistics, which validates the proposed framework. In terms of fire detection model, when compared to single-modal model, multi-modal model has gained attention because it leverages the richer information presented in both RGB and thermal images. However, prevalent multi-modal fire detection methods significantly increase model complexity by requiring two separate streams in the backbone to process RGB and thermal images independently. To address this issue, this thesis proposes a 4-channel single-stream fire detection method based on YOLOv5, which concatenates RGB and thermal images to form the required 4-channel input. Comparison experiments with dual-stream YOLOv5 models using add fusion and transformer fusion demonstrate that the four-channel single-stream model reduces the model complexity while improving detection accuracy. To further enhance the accuracy of multi-modal fire detection and reduce model complexity, this study redesigned the YOLOv5's C3 module by integrating the Convolutional Block Attention Module (CBAM) to form the C3CBAM module and introduced the SCYLLA-IoU (SIoU) loss function. By comparing its performance with that of state-of-the-art models in multi-modal object detection such as YOLOv5-based dual-stream model, this study showcases that the proposed approach improves fire detection in diverse conditions presented in the selected dataset.
A RISK-INFORMED AND LIGHTWEIGHT SYSTEM FOR MULTI-MODAL FIRE DETECTION by Jingshuo Yu B.Eng., Wuhan University of Science and Technology, 2022 A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF APPLIED SCIENCE in THE COLLEGE OF GRADUATE STUDIES (Civil Engineering) THE UNIVERSITY OF BRITISH COLUMBIA (Okanagan) October 2025 © Jingshuo Yu, 2025 ii The following individuals certify that they have read, and recommend to the College of Graduate Studies for acceptance, a thesis entitled: A risk-informed and lightweight system for multi-modal fire detection submitted by Jingshuo Yu in partial fulfilment of the requirements for the degree of Master of Applied Science Examining Committee: Dr. Qian Chen, School of Engineering Supervisor Dr. Zheng Liu, School of Engineering Supervisory Committee Member Dr. Suliman Gargoum, School of Engineering Supervisory Committee Member Dr. Xiaoping Shi, Faculty of Computer Science, Mathematics, Physics and Statistics University Examiner iii Abstract The urgent need for advanced fire detection systems stems from the increased intensity in fire events that cause massive property losses and irreversible damage. To overcome the limitations of traditional fire detection systems such as smoke detectors, computer vision (CV) algorithms have been adopted to improve fire detection accuracy. A CV-based fire detection system takes images from fire monitoring camera as input and output fire-related information. To identify the installation location of fire monitoring cameras, this research performed compartment-level fire risk analysis based on fire risk analysis method for engineering (FRAME). Relevant fire risk factors were quantified using a building information modeling (BIM) model, and their relative importance was evaluated through the analytic hierarchy process (AHP). The resulting compartment-specific fire risk values provide data-driven guidance for deploying cameras in high-risk areas of a building. The case study result of a residential building aligns with historical fire incident statistics, which validates the proposed framework. In terms of fire detection model, when compared to single-modal model, multi-modal model has gained attention because it leverages the richer information presented in both RGB and thermal images. However, prevalent multi-modal fire detection methods significantly increase model complexity by requiring two separate streams in the backbone to process RGB and thermal images independently. To address this issue, this thesis proposes a 4-channel single-stream fire detection method based on YOLOv5, which concatenates RGB and thermal images to form the required 4-channel input. Comparison experiments with dual-stream YOLOv5 models using add fusion and transformer fusion demonstrate that the four-channel single-stream model reduces the model complexity while improving detection accuracy. To further enhance the accuracy of multi-modal fire detection and reduce model complexity, this study redesigned the YOLOv5's C3 module by iv integrating the Convolutional Block Attention Module (CBAM) to form the C3CBAM module and introduced the SCYLLA-IoU (SIoU) loss function. By comparing its performance with that of state-of-the-art models in multi-modal object detection such as YOLOv5-based dual-stream model, this study showcases that the proposed approach improves fire detection in diverse conditions presented in the selected dataset. v Lay Summary The goal of my research work is to develop an advanced CV-based fire detection system for building environment. To achieve this, I developed a building fire risk analysis framework based on FRAME and AHP method to determine the optimal locations for installing fire monitoring cameras. Then for the fire detection model part, unlike conventional multi-modal models that process RGB and thermal images separately, my model combines the two types of images into a four-channel image to feed into the model, which is further strengthened by the integration of C3CBAM and SIoU loss function. My work not only ensures that fire monitoring cameras can be installed in high fire-risk locations to provide input for the detection model, but also improves the accuracy and efficiency of multi-modal fire detection. vi Preface This thesis presents my independent research conducted under the supervision of Dr. Qian Chen. Generative AI was used to check the grammar and improve the readability of the document. The work related to Contributions 2 and 3 has been published in this paper: J. Yu and Q. Chen, âA lightweight four-channel multi-modal model to improve computational performance of automated fire detection,â J. Saf. Sci. Resil., p. 100254, Sept. 2025, doi: 10.1016/j.jnlssr.2025.100254. vii Table of Contents Abstract ......................................................................................................................................... iii Lay Summary .................................................................................................................................v Preface ........................................................................................................................................... vi Table of Contents ........................................................................................................................ vii List of Figures ............................................................................................................................... ix List of Tables ..................................................................................................................................x List of Abbreviations ................................................................................................................... xi Acknowledgements .................................................................................................................... xiii Dedication ................................................................................................................................... xiv Chapter 1: Introduction ................................................................................................................1 Chapter 2: Literature review ........................................................................................................5 2.1 Building fire risk assessment .......................................................................................... 5 2.2 Single-modal fire detection ............................................................................................. 6 2.3 Multi-modal (RGB-T) object detection .......................................................................... 7 2.4 Paired RGB-T fire dataset ............................................................................................. 10 Chapter 3: Methodology..............................................................................................................12 3.1 Fire risk assessment ...................................................................................................... 12 3.2 Fire detection model design .......................................................................................... 19 Chapter 4: Results........................................................................................................................27 4.1 Fire risk evaluation result.............................................................................................. 27 4.2 Performance of fire detection model............................................................................. 28 4.2.1 Evaluation metrics ................................................................................................ 29 viii 4.2.2 Performance comparison of different YOLO versions ......................................... 30 4.2.3 Performance comparison of selected models ........................................................ 31 4.3 Practical implications .................................................................................................... 34 4.4 Limitations .................................................................................................................... 35 Chapter 5: Conclusions and future work ..................................................................................38 5.1 Conclusions ................................................................................................................... 38 5.2 Future work ................................................................................................................... 39 Bibliography .................................................................................................................................42 Appendices ....................................................................................................................................47 Appendix A ............................................................................................................................... 47 A.1 Smooth L1 Loss ........................................................................................................ 47 A.2 Intersection over Union (IoU) ................................................................................... 47 A.3 Generalized IoU (GIoU) ........................................................................................... 47 A.4 Distance IoU (DIoU)................................................................................................. 48 A.5 Complete IoU (CIoU) ............................................................................................... 48 Appendix B ............................................................................................................................... 49 ix List of Figures Figure 1.1 Research framework ...................................................................................................... 4 Figure 2.1 Architecture of YOLOv5 model (adapted from [29] original version) ......................... 7 Figure 3.1 The value of level number ð¬ ....................................................................................... 15 Figure 3.2 BIM model of evaluated residential building .............................................................. 18 Figure 3.3 Floor plan of evaluated residential building ................................................................ 19 Figure 3.4 Creation of four-channel image input.......................................................................... 21 Figure 3.5 Architecture of CBAM ................................................................................................ 21 Figure 3.6 YOLOv5âs backbone after modification ..................................................................... 24 Figure 3.7 Scheme for calculation of angle cost and distance cost .............................................. 24 Figure 4.1 Overall fire risk of each compartment ......................................................................... 28 Figure 4.2 Visual results of compared models on the selected RGB-T wildfire dataset .............. 33 Figure 4.3 AP50 and AP50:95 evolutions in the training epochs ................................................. 34 Figure 4.4 Structure fires at industrial properties by leading causes: 2017-2021 [60] ................. 36 Figure 5.1 A complete data pipeline of our system ...................................................................... 41 x List of Tables Table 1.1 Comparison of wildfire and indoor fire .......................................................................... 2 Table 3.1 Fixed fire load density Qi values .................................................................................. 13 Table 3.2 Mobile fire load density Qm values .............................................................................. 13 Table 3.3 Average value of m by item type .................................................................................. 14 Table 3.4 Recommended values for destruction temperature T ................................................... 14 Table 4.1 Weights of fire risk factors determined by AHP .......................................................... 27 Table 4.2 Compartment number-name mapping table ................................................................. 28 Table 4.3 Accuracy comparison of 4-channel models based on various YOLO versions ........... 30 Table 4.4 Efficiency comparison of 4-channel models based on various YOLO versions .......... 31 Table 4.5 Accuracy comparison of our model and other models ................................................. 32 Table 4.6 Efficiency comparison of our model and other models ................................................ 33 xi List of Abbreviations AHP Analytic Hierarchy Process AP Average Precision BIM Building Information Modeling CBAM Convolutional Block Attention Module CFT Cross-Modality Fusion Transformer CIoU Complete Intersection over Union CNN Convolutional Neural Network COCO Common Objects in Context CV Computer Vision DIoU Distance Intersection over Union DL Deep Learning FN False Negative FP False Positive FPS Frames Per Second FRAME Fire Risk Assessment Method for Engineering GFLOPs Giga Floating-Point Operations GIoU Generalized Intersection over Union IoU Intersection over Union mAP mean Average Precision RGB-D RGB-Depth RGB-T RGB-Thermal xii SCA Spatial and Channel-wise Attention SE Squeeze-and-Excitation SIoU SCYLLA Intersection over Union SiLU Sigmoid Linear Unit SOTA State-of-the-art TP True Positive UAV Unmanned Aerial Vehicle YOLO You Only Look Once xiii Acknowledgements I would like to express my gratitude to everyone who has helped me throughout my graduate study, especially my supervisor, Dr. Qian Chen, who provided continuous support for completion of my research work. The constant encouragement from my parents has also been indispensable. I am grateful for the opportunity to work in the Construction Integration and Digitalization Lab, where I was surrounded by helpful lab mates and where my contributions were valued. Special thanks go to the facility management team at The University of British Columbia, Okanagan Campus, whose long-term collaboration on the Campus as a Living Lab project has been crucial to my research. xiv Dedication May everyone be treasured. 1 Chapter 1: Introduction In the US, the annual number of residential building fires is over 350k, which cost around 8.5 billion dollar loss and cause 2700 deaths yearly [1]. The wildfires even contribute to more than 3 million hectares loss in forest area every year [2]. The recent California wildfires caused an estimated $52â$57 billion in damages, destroying over 1,000 properties, displacing 80,000 residents, and leaving 170,000 people without power. At least five lives were lost, with insured losses potentially reaching $10 billion [3]. While smoke and flame detectors have been widely installed, their functionalities are significantly restricted due to the reason that they cannot detect detailed information of fires like the location and may contribute a lot to false fire alarms given that detected smoke may not be caused by fire incidents [4]. Other fire detection methods include public reports, which relies on the supervision of the public and still accounts for the detection of 40% of wildfires in British Columbia, Canada [5]. The problem with public reports is that fire incidents can only be located by coincidence, which is not always in time. Overall, people are expecting more reliable and automated fire detection systems. The emergence of deep learning (DL) [6] has revolutionized CV applications such as fire detection with improved accuracy. As the first step to deploy CV-based fire detection system, efforts on determining where to install the fire monitoring camera to get input for the models remain inadequate. We presented the comparison of wildfire and indoor fire in Table 1.1. For fire detection purpose, DL models can handle the detection of both wildfires and indoor fires as they share similar visual features. DL models also resolve false alarm and delayed response problems faced by the traditional detection methods of wildfires and indoor fires. However, in terms of fire risk analysis for camera installation, the influencing factors are different: wildfire risk is determined by terrain, vegetation, and weather conditions [7], [8], [9], whereas indoor fire risk is 2 primarily determined by architectural layout and building codes (e.g., material selection, fire protection system) [10]. This feature makes BIM particularly suitable for evaluating indoor fire risk, as it provides rich information of buildingâs layout and detailed design. In terms of fire risk evaluation framework, fire risk assessment method for engineering (FRAME) [11] has proven to be promising for quantifying building fire risk [12], which is crucial for determining the installation location of fire monitoring camera. In this thesis, we focus on the indoor fire risk evaluation, we believe that combining BIM-extracted information and FRAME-based framework allows for reliable building fire risk evaluation that aligns with historical fire statistics. Aspect Wildfire Indoor fire Common detection method Public report, land user report, patrols Smoke detector, heat detector Detection method limitation Delayed response False alarm, delayed response Risk determinant Terrain, vegetation, weather condition Architectural layout, building codes Risk analysis tool Geographic information system (GIS) BIM Table 1.1 Comparison of wildfire and indoor fire In terms of fire detection models, single-modal models using RGB images have been widely adopted due to the affordability of RGB cameras and the accessibility of RGB fire image datasets. These applications, leveraging models such as you only look once (YOLO) [13] and ensemble learning [14] approaches, have proven adoptable but face limitations in accuracy due to the limited information provided by RGB images only [15]. To address these challenges, thermal imaging that is capable of capturing heat emissions [16] has been introduced as a 3 complementary modality. However, integrating RGB and thermal data often involves dual-stream architectures and fusion operators that increase model complexity and computational demand. Building on the challenges of traditional RGB-T methods, four-channel single-stream object detection, where RGB and thermal images are concatenated into a single input, offers a promising lightweight solution. As an approach that eliminates additional parameters needed for dual-stream processing while retaining multi-modal advantages, although the four-channel technique has been explored in fields like RGB-Depth (RGB-D) detection [11,12], its application in fire detection remains inadequate. Furthermore, studies incorporating advanced modules such as the CBAM [19] and loss functions like SIoU [20] have demonstrated potential for enhancing detection accuracy and model efficiency. We believe that, as the end-to-end nature of the single-stream model leverages the inherent feature learning capability of native CNN architecture, single-stream model can outperform dual-stream model that uses additional fusion operators in fire detection. And the integration of CBAM and SIoU can further improve the accuracy of fire detection model. This thesis aims to bridge the gap in fire detection by proposing a fire risk evaluation framework to guide the installation of fire monitoring cameras in buildings and developing a lightweight and high-performance multi-modal fire detection model. As displayed in Figure 1.1, this thesis has three contributions: â¢ We proposed a fire risk evaluation framework based on FRAME and expert interview. The framework contains five fire risk factors and their values for each compartment were extracted from the BIM model. Then the AHP method was applied to determine the weights of fire risk factors and calculate the overall fire risk of each compartment, which 4 can be used to identify the installation location of fire monitoring cameras. A case study was conducted on a residential building to validate the framework. â¢ We designed a lightweight four-channel single-stream fire detection model that effectively combines RGB and thermal information without requiring separate processing streams. This design reduces the model complexity while improving detection accuracy, making it more efficient for real-time fire detection applications. â¢ To further enhance detection accuracy, we integrated CBAM with the C3 module of YOLOv5 (forming the C3CBAM module) and replaced YOLOv5âs default CIoU loss with more comprehensive SIoU loss, which improves feature representation while decreasing the computational load. Figure 1.1 Research framework Channel AttentionSpatial AttentionC A Angel cost istance costShape costIoU costSIoU loss uide camera installationImprove modelperformance 2 4 channel multi modal fire detection model 1 Fire risk assessment 3 Further enhancement of model odel blocksR camera hermal cameraOutput I modelFire load factorSpread factor enting factor Fire risk factors Fire risk tract A 5 Chapter 2: Literature review 2.1 Building fire risk assessment Although numerous models have been developed for fire detection, limited attention has been given to determining optimal locations for installing fire monitoring cameras. Evaluating fire riskâwhich is crucial for selecting appropriate camera installation sitesâhas been explored in various studies. Moghim et al. [7] and Jiang et al. [8] evaluated regional wildfire risk with historical fire data by training machine learning and deep leaning models respectively. The used wildfire risk factors include vegetation, weather, slope, etc. However, indoor fire risk evaluation involves different fire risk factors. rzeziÅska and ryant [21] introduced a semi-quantitative method for evaluating building fire strategies using a structured scoring system based on factors like ignition sources and fire system. However, the method does not explicitly calculate fire risk values. Instead, it relies on relative scoring, which limits its usefulness for applications requiring detailed risk prioritization. Chen et al. [22] proposed a Bayesian Network model to assess fire risk in historical buildings, combining expert knowledge, data, logic, and variable weights. It includes both static and dynamic risk factors and is applied to a case study. However, it does not derive the fire risk of each compartment within the building. Based on FRAME, Wang et al. [12] evaluated the fire risk of compartments within a post office with its BIM model. While FRAME provides a solid foundation, its use in our context has limitations. Our goal is to guide the optimal installation of fire monitoring cameras, which requires prioritizing factors that influence early fire detection and suppressionânamely, the likelihood of ignition, potential for fire spread, and the presence of fire control systems. Some of FRA âs considerations, such as evacuation logistics or firefighter access, are not relevant to our application. Additionally, it does not account for human activities that elevate fire risk (e.g., 6 indoor smoking). To address this gap, we proposed a focused fire risk evaluation framework tailored to support the selection of installation location for fire monitoring cameras by combining various fire risk factors and the AHP method to assign their weights. 2.2 Single-modal fire detection With the rise of DL [6] in CV tasks like object detection [23], CV-based fire detection has increasingly adopted DL techniques for more precise recognition of fires. Benefited from the affordable price of RGB cameras and the availability of RGB fire images, many researches have been done on RGB based fire detection. In [13], a fire detection dataset was composed with online images from search engines and publicly available datasets, then it was utilized to train a DL model based on YOLOv5 [24], whose architecture is shown in Figure 2.1. Similarly, Xu et al. [14] collected fire images from public datasets like BowFire [25] and trained an ensemble learning model combining YOLOv5 [24] and EfficientDet [26]. However, there are evidences showing that single-modal fire detection with RGB images have several limitations in real-world applications [15], [27]. Firstly, RGB cameras have relatively high requirement on illumination, which restricts their uses in low-visibility conditions like nighttime and cloudy weather. Secondly, reflection of flames on other objects (e.g., windows) can cause false positives in low-light environments. Thirdly, for RGB cameras mounted on the unmanned aerial vehicle (UAV), capturing small scale fires can be challenging, bad visibility of fire targets in the images can harm the accuracy of the following detection task. There are also works on thermal image-based single-modal fire detection. For example, Li et al. [28] conducted detection on a thermal fire dataset collected by combustion experiments with combustible materials. However, poor contextual information contained in thermals images could lead to false recognition caused by other heat-emitting objects [15]. A feasible solution to address the limitations of single-modal 7 fire detection is combining RGB and thermal data to leverage the complementary strengths of both modalities. RGB images provide rich contextual information, while thermal images ensure robust thermal signatures, improving detection precision and reducing false positives. Figure 2.1 Architecture of YOLOv5 model (adapted from [29] original version) 2.3 Multi-modal (RGB-T) object detection In general, multi-modal (RGB-T) object detection takes two distinct backbone streams for the feature extraction of RGB and thermal images respectively. Various works have been done to design fusion operators for RGB and thermal feature maps in the DL network architecture. According to the stage the fusion happens in the network, there are 4 types of fusion techniques: early fusion, middle fusion, late fusion, and deep fusion [30], [31]. As displayed by Figure 2.2, while early, middle, and late fusion strategies perform fusion operations at a single stage only, deep fusion conducts multiple fusion operations and passes the results of each fusion into subsequent feature extraction processes. El Ahmar et al. [31] proposed the enhanced sigmoid gating module to increase the accuracy of RGB-T based object detection. They applied four fusion operators: element-wise addition, element-wise multiplication, bilaterally-guided fusion [32], and multi-modal factorized bilinear pooling [33] on the ResNet [34] and ConvNeXt 8 [35] based networks and compared their early fusion performance on a city scene dataset containing multi-modal images of cars and pedestrians. Fang et al. [36] presented a fusion operator based on transformer [37] and applied it in a YOLOv5 based structure utilizing deep fusion strategy. Testing results on FLIR [38], LLVIP [39], and VEDAI [40] datasets show that RGB-T based object detection outperforms both single-modal detection, and the introduction of transformer fusion operator brings higher mean average precision (mAP) than addition fusion. Figure 2.2 Four types of fusion strategies (adapted from [30] original version) The problem with most existing RGB-T networks is the heavy model size and increased inference time brought by the introduction of fusion operators and additional stream for learning thermal feature in the backbone. The former doubles the parameter number in the backbone, the later would further worsen this especially when the fusion operator itself is relatively complex or the fusion strategy is deep fusion. For example, compared to original single-stream YOLOv5s, the dual-stream version increases the parameter number by 60%. In [36], when incorporated with 9 transformer-based fusion operator, the parameter number of dual-stream model increases by more than six times relative to that of the original version. To make fire detection model lightweight, Pincott et al. [4] selected the lightweight MobileNetV2 [41] as the baseline of their model. Efforts have also been made on modifying the original module structure to reduce model complexity [13], [28]. And considering grayscale thermal images with the same value in every channel are the widely used form of thermal images in RGB-T datasets, specifically reserving a stream for the duplicated channels of thermal images could be a waste of computing power. Four-channel object detection is an extension of standard object detection where, instead of just using the traditional three RGB channels, an additional fourth channel to store non-RGB information is incorporated. In this way, the stream to handle non-RGB input gets removed as only one stream is required to receive the four-channel input, thus the four-channel single-stream model is lightweight compared with the dual-stream model. The four-channel design has gained attention in RGB-D object detection [17], [18], which leverages the extra spatial information provided by depth data in the fourth channel to improve the accuracy and robustness of the detection systems. For example, Liu et al. [17] compared the performance of single-stream (four-channel input) and dual-stream models, the results show that the former outperforms the latter on several open-source RGB-D datasets. In RGB-T object detection field, Marques et al. [27] used a four-channel network to detect people and vehicle in the forest, but the performance comparison of four-channel and dual-stream models was not investigated. Currently there is no work applying the four-channel method on fire detection. To address the heavyweight drawback, we designed a lightweight multi-modal fire detection model from three aspects: First, we selected YOLOv5 as our baseline, which is more lightweight than other object detection networks like vision transformer (ViT) [42] and real-time 10 detection transformer (RT-DETR) [43]. Second, we deployed the four-channel design to make a single-stream multi-modal model, which is more lightweight than the commonly used dual-stream multi-modal model. Third, we replaced the C3 module at the rear of model backbone with C3CBAM, which simplifies the module structure and makes our model more lightweight than the original model with C3 module. 2.4 Paired RGB-T fire dataset In the few publicly available RGB-T based fire detection research, Toulouse et al. [44] provided a dataset containing paired RGB-T images of fires, but the dataset is relatively small and mainly contains large-scale fires, and the benefit brought by thermal images are not explored by experiments. Chen et al. [45] collected the FLAME2 dataset with paired RGB and thermal images and trained a CNN-based model proposed in [46]. However, the images are highly repetitive, and only image-level annotations (Fire/No Fire) are offered, which cannot help to precisely identify the location of fires in the images. In their following work, FLAME3 [47] also provides only image-level annotations. Rui et al. [15] collected a multi-modal dataset and designed a dual-stream model based on ResNet [34], their model achieved an IoU of 0.872 and an F1-score of 0.931 on segmentation task under day and night environment. The dataset contains images of fires in various sizes under different illumination, and these images are captured from two perspectives: the ground camera (HIK DS-2CD1T45-LA RGB-T integrated gimbal camera), and the UAV DJI M300 equipped with Zenmuse H20T camera. The variations of fire sizes, illumination, and camera angles enable the dataset to better represent real-world scenarios and have less biases, for which we select it to train and test the model in our work. The detailed information of these open source paired RGB-T fire datasets is listed in Table 2.1. 11 Dataset Perspective Time Annotation Image# Corsican [44] Ground Day/Night Pixel-level 635 FLAME2 [45] UAV Day Image-level 53541 FLAME3 [47] UAV Day Image-level 738 RGB-T wildfire [15] Ground/UAV Day/Night Pixel-level 1367 Table 2.1 Open source paired RGB-T fire datasets 12 Chapter 3: Methodology 3.1 Fire risk assessment We conducted an in-depth interview with the team manager of the facility management team at the University of British Columbia, Okanagan Campus, and identified relevant fire risk factors for our application by referencing the FRAME framework and expert experience. The expert provided fire system factor and human activity to complement fire load factor, spread factor, and venting factor from the FRAME. During the interview, we also developed the aggregated pairwise comparison matrix required for the AHP analysis. The expert was asked to compare each pair of criteria using a standardized scale from 1 to 9, where 1 means equal importance and 9 means extreme importance of one over the other. Then a case study was conducted on a local residential building. The fire load factor ð quantifies the amount of combustible material per unit area in a building. It is calculated using the fixed fire load density ðð and the mobile fire load density ðð, whose common values are presented in Table 3.1 and Table 3.2 respectively. These values represent the combustible content inherent to the structure and the content related to building activity, respectively. The fire load factor ð is given by Equation 3-1: ð =23â log(ðð + ðð) â 0.55 (3 â 1) The formula's logarithmic operation is employed to mirror the standard temperature-time curves used in fire science [48] and make the final risk calculation less sensitive to errors when estimating extremely high fire loads. Building Type Description ðð (MJ/m²) A Completely non-flammable 0 13 B Not easy to burn 100 C1 Wood structure with non-combustible coating 300 C2 Fire-resistant structure with combustible floor 300 D Only structural materials non-flammable 1000 E Combustible structural materials 1500 Table 3.1 Fixed fire load density ð¸ð values Scenario Description ðð (MJ/m²) a1 Low fire hazard occupancies 200 a2 Office 400 a3 Dwellings 500 a4 School 200 a5 Hospital 250 a6 Hotel 250 Table 3.2 Mobile fire load density ð¸ð values Spread factor ð indicates how easy a fire can spread through a building and the influence of its spread. In this context, the spread factor ð is modeled as the product of three sub-factors: ð = 5 â ðð â ð â ð (3 â 2) Fire spread factor ðð is influenced by the average dimension ð (volume-to-surface ratio), the destruction temperature ð, at which materials ignite or lose structural integrity: ðð = 1.3 âð1000 â (0.1 â log ð)(3 â 3) Item Type ð 14 Stacked items 1 Small industrial products 0.1 Film-type industrial products 0.01 Grains, marbles 0.001 Table 3.3 Average value of ð by item type Material ð (°C) Flammable liquid 20 Human beings, plastics, electronics 100 Textiles, wood, paper, food 200 Average content of residential buildings 250 Machinery, household appliances 300 Metals 400 Non-combustible construction materials 500 Table 3.4 Recommended values for destruction temperature ð» The area factor ð reflects how fire spreads within a given compartment, depending on its shape. A long, narrow space has different fire behavior than a compact square one. ð is calculated as: ð =ð + 5 â âð2 â ð3200(3 â 4) Where ð is the longest distance between opposite sides, ð is the equivalent width (total area divided by ð). 15 The level factor ð indicates the vertical influence of the fire, namely upward by smoke and heat, and downward by heat, water damage and collapse. As displayed in Equation 3-5, it is determined by the level number ð¸: ð = (|ð¸| + 3|ð¸| + 2)0.7|ð¸|(3 â 5) Figure 3.1 The value of level number ð¬ As displayed in Figure 3.1, the main access level is designated as level 0. Levels located above this are numbered as positive integers (e.g., 1, 2, 3), while levels below are represented by negative integers e.g., â1, â2, â3 . In cases where partial or intermediate floors e ist between standard levelsâcommonly referred to as incomplete floorsâtheir floor area contributions can be expressed using decimal values. For example, an incomplete floor that provides an additional 40% of a full floor area above level 1 may be represented as level 1.4. The venting factor ð£ quantifies the capability of a building to release smoke and heat generated during a fire. During combustion, a large volume of hot smoke accumulates near the ceiling, forming a smoke layer. If not adequately vented, the buildup of smoke and heat can lead to flashover, where the entire compartment becomes engulfed in flames, making fire control virtually impossible. To evaluate how well smoke can escape a compartment compared to the 16 amount of smoke generated, the venting factor is calculated using the empirical formula in Equation 3-6: ð£ = 0.84 + 0.1 â log ðð â âð â ââ (3 â 6) ðð is the mobile fire load density mentioned above. â is the height from floor to ceiling, which affects how much smoke can accumulate before it impedes visibility or breathing. Venting ratio ð is the ratio of total smoke venting area to the floor area of the compartment. It accounts for all aerodynamic openings such as windows and roof vents. The higher value it is, the more effective the building is at venting smoke. The fire system factor ð represents the availability of fire protection systems (i.e., heat/smoke detector, sprinkler) in a building to detect and control a fire during its early development stages. These systems play a role in limiting fire growth, reducing smoke generation, and enhancing occupant safety and property protection. The value of the fire system factor is binary: it is set to 0.9 if fire system is installed, and 1 if there is no such system. The human activity factor ð» quantifies the impact of occupant behavior and high-risk activities on the likelihood of fire ignition within a building compartment. Based on statistical data from the National Fire Information Database of Canada [49], the three most common causes of residential fires are cooking (29%), smoking or open flame (27%), and heating (16%). These activities collectively account for 72% of fire incidents and are therefore used to model human-related fire risk. To construct a balanced and scalable index, the human activity factor is composed of a base value of 0.5ârepresenting general background riskâand an additional 0.5 assigned proportionally to the presence of the three human activities. The contributions of the activities 17 are normalized to sum to 0.5, maintaining consistency with the total residential fire statistics. The human activity factor is defined by Equation 3-7: ð» = 0.5 + â ð¤ð3ð=1â ð´ð (3 â 7) In this expression, ð´ð is a binary indicator equal to 1 if the ith activity is present in the evaluated compartment and 0 if it is absent. The weights ð¤ð represent the normalized contribution of each activity, derived from its proportion in residential fire causes. Specifically, ð´1, ð´2, and ð´3 correspond to cooking, smoking/open flame, and heating, respectively. The corresponding weights ð¤1, ð¤2, and ð¤3 are 0.201, 0.188, and 0.111 respectively, which together sum to 0.5. As a result, the value of ð» ranges from 0.5 (when none of the three activities is present) to 1.0 (when all three are present), allowing the model to reflect variable levels of human-induced fire risk. To quantify the relative importance of the fire risk factors mentioned above, the pairwise comparison matrix collected from expert review for AHP was then normalized, and the priority vector (weights) was computed by normalizing the sum of each row in the normalized comparison matrix. Consistency of the matrix was validated using the Consistency Ratio, which compares the consistency index of the matrix with a randomly generated matrix. A CR less than 0.1 indicates acceptable consistency in the expert judgments. Each compartment within the studied building was evaluated based on the five identified factors. The overall fire risk score for each compartment was calculated using a weighted sum model in Equation 3-8: 18 Fire Risk = â ð¤ððð=1â ð ð (3 â 8) Where ð¤ð is the AHP-derived weight for fire risk factor ð, ð ð is the normalized score of fire risk factor ð for a compartment, and ð is the total number of fire risk factors. Then we conducted the case study on a local residential building in Kelowna. As described in Chapter 1, indoor fire risk is determined by architectural layout and building code. The selected building shares similar layout and the same building code with the on-campus residential buildings, which makes our expertise experience highly relevant. The BIM model and floor plan of the selected building are displayed in Figure 3.2 and Figure 3.3. Figure 3.2 BIM model of evaluated residential building 19 Figure 3.3 Floor plan of evaluated residential building 3.2 Fire detection model design In this study, we choose YOLOv5 as the baseline model as it remains widely adopted in fire detection studies [13], [14], [27], [50]. Besides that, the Cross-Modality Fusion Transformer (CFT) [36] based on YOLOv5 continues to achieve top rankings in multi-modal object detection benchmarks [51], which makes it a convincing object for comparison purpose. While there are newer versions of YOLO models, they often come with greater model size. The detailed 20 comparisons of YOLOv5, YOLOv8 [52], YOLOv10 [53], and YOLO11 [54] are presented in Chapter 4. All source codes of baselines are from Ultralytics [54]. The following modifications are made on the YOLOv5 to improve its performance of the fire detection: â¢ Adjusted the depth of the convolutional kernels in the first Conv module to enable it to accept four-channel inputs. This modification avoids the significant increase in parameters typically caused by fusion operators and an additional stream in traditional RGB-T fire detection. â¢ Integrated the CBAM module with the C3 module at the backbone's tail to form the C3CBAM module, aiming to further reduce model size while improving the accuracy of fire detection. â¢ Replaced the default CIoU loss function of YOLOv5 with SIoU loss function. This modification boosts modelâs regression speed and detection accuracy by introducing the angle cost to optimize the regression path. As the RGB and thermal images have the same resolution, to provide four-channel input for the model, we used OpenCV to directly concatenate the thermal channel with the RGB image formulate a four-channel input image for the model. In this way, both color and thermal information are carried by single image as displayed in Figure 3.4. 21 Figure 3.4 Creation of four-channel image input Figure 3.5 Architecture of CBAM Then we combined the four-channel YOLOv5 with CBAM [55], which learns both channel and spatial attention weights and refines input feature with the learned weights. As displayed by Figure 3.5, in CBAM, the input feature map ð¹ â âð¶Ãð»Ãð will go through channel attention module ðð â âð¶Ã1Ã1 and spatial attention module ðð â â1Ãð»Ãð in sequence: 22 ð¹â² = ðð(ð¹)â¨ð¹ (3 â 9) ð¹â²â² = ðð (ð¹â²)â¨ð¹â² (3 â 10) In Equation 3-9 and Equation 3-10, â¨ stands for the element-wise multiplication of feature map and learned attention values, ð¹â² and ð¹â²â² are the output feature maps after element-wise multiplication with channel and spatial attention values. Channel attention module and spatial attention module can be represented with Equation 3-11 and Equation 3-12: ðð(ð¹) = ð (ðð¿ð(ð´ð£ððððð(ð¹)) + ðð¿ð(ððð¥ðððð(ð¹)))= ð (ð1 (ð0(ð¹ðð£ðð )) + ð1(ð0(ð¹ððð¥ð )))(3 â 11) ðð (ð¹â²) = ð(ð¶ððð£ð([ð´ð£ððððð(ð¹â²); ððð¥ðððð(ð¹â²)]))= ð (ð¶ððð£ð([ð¹ðð£ðð â² ; ð¹ððð¥ð â² ]))(3 â 12) In the channel attention module, the input feature map ð¹ undergoes spatial level average pooling and max pooling. The outputs (ð¹ðð£ðð and ð¹ððð¥ð ) of these pooling operations are passed through a shared multilayer perceptron (MLP) (ð0 â âð¶ðÃð¶ and ð1 â âð¶Ãð¶ð) and then summed. The summed result is processed by a sigmoid function to obtain the channel attention values. These values are then multiplied elementwise with the input feature map to produce the input feature map ð¹â² for the spatial attention module. In the spatial attention module, the input feature map ð¹â² first undergoes channel-wise max pooling and average pooling, and the outputs (ð¹ðð£ðð â² ððð ð¹ððð¥ð â² ) of these operations are stacked. The stacked feature map is then processed through a convolution operation with a ð Ã ð kernel (ð¶ððð£ð) followed by a sigmoid function to obtain the spatial attention values. These values are multiplied elementwise with along the channel dimension, resulting in the final output ð¹â²â² of the spatial attention module. 23 In this study, we constructed the C3CBAM module and placed it at the rear of backbone. The difference of C3 module and refined C3CBAM module are shown in Figure 3.6. The basic structure of the C3 module in YOLOv5s involves splitting the input feature map into two streams: one stream is passed through a CBS module and a BottleNeck module, while the other stream bypasses only a CBS module. After processing, the outputs of both streams are concatenated along the channel dimension, and a final CBS module is applied to fuse the combined features. The C3CBAM module inherits the dual-stream structure of the C3 module. In this design, the first stream passes through two CBS modules followed by a CBAM module, while the second stream remains unprocessed and is directly added to the first streamâs output to form the moduleâs output. As shown in Figure 2.1, the CBS module consists of a convolutional layer, a batch normalization layer, and a SiLU activation function. Its trainable parameters are contributed by the convolutional layer. The BottleNeck module is composed of two CBS modules, so its trainable parameters are also contributed by the convolutional layer. As displayed by Figure 3.5, CBAM is divided into two parts: channel attention and spatial attention. The spatial attention mechanism requires only a two-channel convolutional kernel to process the feature map formed by concatenating the results of channel-wise max pooling and average pooling. Therefore, the trainable parameters in CBAM are mainly from the MLP used in channel attention, which can also be viewed as a convolutional layer of size 1Ã1. However, since the scaling ratio of the MLP is larger than that of typical convolution operations, it requires fewer convolutional kernels. Thus, under the same number of input channels, CBAM offers greater parameter efficiency compared to CBS, and similarly, C3CBAM is more efficient than C3. 24 In YOLOv5's original C3 module, the input feature map is processed in two streams: the first stream passes through one CBS module and n BottleNeck modules, while the second stream passes through one CBS module. The outputs of these two streams are concatenated and fed into another CBS module to produce the final output. In the refined C3CBAM module, the input feature map passes through two CBS layers and the CBAM module (including channel attention and spatial attention). The resulting feature map is then added to the original input to produce the final output. The modified backbone of YOLOv5 is represented in Figure 3.6. Figure 3.6 YOLOv5âs backbone after modification Figure 3.7 Scheme for calculation of angle cost and distance cost Then we replaced CIoU loss function in YOLOv5 with SIoU loss function. There have been several generations of bounding box loss functions [56], [57], [58], [59], we summarized them in the order they were proposed in Appendix A and listed the formulas of SIoU in this C S C S C3 C S C3 C S C3 C S C3C A S F640 640 4C ottle eckC3 C SC SC SC A C3C A C S C S25 section. SIoU includes 4 cost functions: angle cost, distance cost, shape cost, and IoU cost. Compared to CIoU applied in YOLOv5, SIoU innovatively introduces angle cost, which is defined by Equation 3-13 â Equation 3-17: Î = 1 â 2 â sin2 (arcsin(ð¥) âð4) (3 â 13) ð¥ = {ðâð= sin(ð¼) , ð¼ â¤ð4ðð¤ð= sin(ð½) , ðð¡âððð¤ðð ð(3 â 14) ð = â(ððð¥ðð¡ â ððð¥)2+ (ððð¦ðð¡ â ððð¦)2(3 â 15) ðâ = max (ððð¦ðð¡, ððð¦) â min (ððð¦ðð¡, ððð¦) (3 â 16) ðð¤ = max(ððð¥ðð¡, ððð¥) â min(ððð¥ðð¡, ððð¥) (3 â 17) The ð¥ determines the angle to be minimized. When ð¼ is less than ð 4â , ð¼ is minimized; otherwise, ð½ is minimized. ððð¥ðð¡, ððð¥, ððð¦ðð¡ and ððð¦ are the x, y coordinate of the ground truth and predicted bounding box respectively, ð¶â and ð¶ð¤ represent the height and width of the box formed with the center points of the ground truth box and the predicted box as its vertices. The second cost function in SIoU Loss is distance cost considering angle cost introduced above, it is defined by Equation 3-18 â Equation 3-21: Î = â (1 â ðâð¾ðð¡)ðð¡=ð¥,ð¦(3 â 18) ðð¥ = (ððð¥ðð¡ â ððð¥ðð¤â²)2(3 â 19) ðð¦ = (ððð¦ðð¡ â ððð¦ðââ² )2(3 â 20) 26 ð¾ = 2 â Î (3 â 21) The ðð¤â² and ðââ² are the width and height of the smallest enclosing box. The third cost function is shape cost handling aspect ratio deviation, which can be represented by Equation 3-22 â Equation 3-24: Ω = â (1 â ðâðð¡)ðð¡=ð¤,â(3 â 22) ðð¤ =|ð¤ â ð¤ðð¡|max(ð¤, ð¤ðð¡)(3 â 23) ðâ =|â â âðð¡|max(â, âðð¡)(3 â 24) Where parameter ð determines the amount of attention paid to shape cost and is set as 4 in our experiment. Finally, SIoU loss function considering IoU loss is defined by Equation 3-25: âðð¼ðð = 1 â ð¼ðð +Î + Ω2(3 â 25) 27 Chapter 4: Results 4.1 Fire risk evaluation result The pairwise comparison matrix developed through consultation with the facility team yielded the following factor weights: Fire risk factor Fire load Spread Venting Fire system Human activity Weight 0.0468 0.1251 0.1008 0.4800 0.2474 Table 4.1 Weights of fire risk factors determined by AHP The consistency ratio of the matrix was 0.045, which is below the acceptable threshold of 0.1, indicating a reliable level of consistency in expert evaluations. Based on the derived weights and the condition of each compartment, we listed fire risk values in Figure 4.1 and compartment number-name mapping in Table 4.2. The detailed information of each compartment is listed in Appendix B. Despite being equipped with a fire protection system, Compartment 106 (Dining) still exhibits the highest fire risk. After deriving compartment-level fire risk values, we can use these results to guide camera installation. Specifically, compartments with higher risk valuesâsuch as Dining (including kitchen) and Bedroomsâare identified as prioritized installation locations. This result aligns with historical data on residential fires [60], [61], demonstrating the effectiveness of the proposed framework. 28 Figure 4.1 Overall fire risk of each compartment Number 101 102 103 104 105 Name Garage Mudroom Washroom Stairway Den Number 106 201 202 203 204 Name Dining Bedroom1 Ensuite WIC Loft Number 205 206 207 208 209 Name Laundry Storage Bathroom Bedroom2 Bedroom3 Table 4.2 Compartment number-name mapping table 4.2 Performance of fire detection model The proposed model was tested using the RGB-T paired wildfire open dataset presented in [15], which consists of 1367 pairs of fire images covering various lighting conditions, flame sizes, and shooting angles. The same dataset was tested in the SOTA models proposed in [36] leveraging transformer-based fusion operator. 29 4.2.1 Evaluation metrics Our accuracy metrics include AP50, AP50:95, and F1 Score. As there is only one class in our task, we adopted commonly used object detection average precision metrics AP50 and AP50:95 proposed in Common Objects in Context (COCO) [62] instead of the mAP used for multi-class datasets. AP is calculated with Equation 4-1: ð´ð = â« ð(ð)ðð10= â« ðððððð ððð ð(ð ððððð)10= â«ðððð + ð¹ðð (ðððð + ð¹ð)10(4 â 1) IoU equals the overlap area divided by the union area of predicted bounding box and ground truth bounding box. The case where a predicted bo âs relative position to the ground truth box meets the IoU threshold is considered a true positive (TP); otherwise, it is considered a false positive (FP). If a ground truth box is not detected by any predicted box meeting the IoU threshold, it is regarded as a false negative (FN). The result calculated using the above equations at an IoU threshold of 0.5 is referred to as AP50, in other words, when IoU ï³ 0.5, the prediction is valid under AP50 criterion. AP50:95 is the mean of multiple AP values computed at IoU thresholds ranging from 0.5 to 0.95, with an interval of 0.05. The F1 Score is the harmonic mean of precision and recall, offering a balanced measure of detection quality. The efficiency metrics provide insights into the computational and deployment characteristics of the models. GFLOPs (Giga Floating-Point Operations) quantifies the total number of computations required for inference; lower GFLOPs indicate greater computational efficiency. FPS (Frames Per Second) indicates how many frames the model can process per second, directly reflecting inference speed. The parameter number refers to the number of 30 trainable weights in the model; smaller models with fewer parameters tend to be faster and more suitable for edge deployment. Finally, memory usage measures the amount of memory required to load and run the model, where lower values are beneficial for resource-constrained environments. All models were trained with a batch size of 32, patience of 100, and 200 epochs, while other parameters were set to the default values in the source code. The training GPU was RTX 4070. 1095 image pairs were used for training, the remaining 272 image pairs were used for testing. 4.2.2 Performance comparison of different YOLO versions The accuracy and efficiency comparison of 4-channel models based on various YOLO versions are presented in Table 4.3 and Table 4.4 respectively. Overall, 4-channel YOLOv5s provides an excellent trade-off between accuracy and efficiency. It demonstrates comparable accuracy to newer YOLO variants while maintaining much lower computational cost, making it highly suitable for real-time and resource-constrained applications, thus we selected the lightweight YOLOv5s as the basis for our experiments. Model Data AP50 AP50:95 F1 Score 4-chan YOLOv5s RGB+T 86.1 46.5 86.02 4-chan YOLOv8s RGB+T 86.6 46.3 82.83 4-chan YOLOv10s RGB+T 85.4 46.0 82.15 4-chan YOLO11s RGB+T 87.9 45.0 83.44 Table 4.3 Accuracy comparison of 4-channel models based on various YOLO versions Model GFLOPs FPS (i7-13700K) Parameter # (Million) Memory (MB) 4-chan YOLOv5s 16.2 9.87 7.02 14.12 31 4-chan YOLOv8s 28.7 9.56 11.14 22.01 4-chan YOLOv10s 24.8 8.81 8.07 16.16 4-chan YOLO11s 21.6 9.10 9.43 18.75 Table 4.4 Efficiency comparison of 4-channel models based on various YOLO versions 4.2.3 Performance comparison of selected models We implemented ablation experiments on YOLOv5, which include single-modal fire detection with RGB and thermal images respectively, and multi-modal detection with 4-channel model, CBAM, and SIoU. We also compared the performance of our model with YOLOv5-based dual-stream model using add fusion and transformer-based fusion proposed in [36], which is a SOTA (state-of-the-art) model in multi-modal object detection [51]. In both add fusion and transformer-based fusion, RGB and thermal images are processed through identical backbones created by duplication, and fusion is performed at multiple stages. In the add fusion approach, the output of the fusion operator is directly fed into the modelâs head for detection. In contrast, in transformer-based fusion, the output of the fusion operator is passed to the next stage for continued feature learning, as shown in the deep fusion strategy illustrated in Figure 2.2. For detailed implementation, refer to [36]. The accuracy metrics of selected models are listed in Table 4.5. Compared with original single-modal model and dual-stream multi-modal model with fusion operators, 4-channel multi-modal YOLOv5 achieves higher accuracy. It also has the same scale of parameter numbers with the original single-modal model because modifying the input from 4 to 3 channels only affects the first convolutional layer, which is a small part of the total parameters in the model. The deeper layers, which hold most of the parameters, remain unchanged. As explained in Chapter 3, replacing C3 with C3CBAM reduced the parameter number. In our case, the C3 module contains 32 1,182,720 parameters, while the C3CBAM module contains only 863,074 parameters. Overall, our 4-channel YOLOv5 combined with CBAM and SIoU achieves the highest AP while introducing the fewest parameters. Visual results of compared models are presented in Figure 4.2, our model design increases the detection precision of multi-scale fires under various illumination. The evolutions of AP50 and AP50:95 values in the training process are shown in Figure 4.3, the accuracy of our model dominates in most stages. Table 4.6 compares the efficiency of selected models, highlighting that our model achieves the best balance with the lowest GFLOPs, fewest parameters, and smallest memory usage. It is worth noting that, aside from the heavyweight YOLOv5s+CFT model, FPS differences across most models are minor due to the processor's ability to load full models and process dual streams simultaneously. However, on resource-constrained devices, larger models may fail to be loaded due to limited memory and experience slower inference. Model Data AP50 AP50:95 F1 Score YOLOv5s RGB 76.1 37.6 81.37 YOLOv5s Thermal 77.6 38.2 77.10 YOLOv5s+add RGB+T 83.9 40.7 84.92 YOLOv5s+CFT RGB+T 82.8 42.1 85.85 4-chan YOLOv5s RGB+T 86.1 46.5 86.02 4-chan YOLOv5s+SIoU RGB+T 87.2 48.6 85.65 4-chan YOLOv5s+CBAM RGB+T 87.4 47.6 87.00 Ours RGB+T 89.3 49.5 88.49 Table 4.5 Accuracy comparison of our model and other models 33 Model Data GFLOPs FPS (i7-13700K) Parameter # (Million) Memory (MB) YOLOv5s RGB 15.9 9.15 7.02 14.12 YOLOv5s Thermal 15.9 9.80 7.02 14.12 YOLOv5s+add RGB+T 27.4 10.08 11.28 22.39 YOLOv5s+CFT RGB+T 40.7 6.99 44.52 87.49 4-chan YOLOv5s RGB+T 16.2 9.87 7.02 14.12 4-chan YOLOv5s+SIoU RGB+T 16.2 9.67 7.02 14.12 4-chan YOLOv5s+CBAM RGB+T 15.9 9.64 6.70 13.50 Ours RGB+T 15.9 9.81 6.70 13.50 Table 4.6 Efficiency comparison of our model and other models Figure 4.2 Visual results of compared models on the selected RGB-T wildfire dataset. (a) YOLOv5 on RGB images. (b) YOLOv5 on thermal images. (c) YOLOv5 + add fusion. (d) YOLOv5 + transformer-based fusion. (e) Our model: 4-channel YOLOv5 + CBAM + SIoU. Due to limited space, you may need to zoom in to see the details while reading 34 Figure 4.3 AP50 and AP50:95 evolutions in the training epochs 4.3 Practical implications In our case study, the fire risk evaluation results closely align with historical fire incident data, supporting the validity of the proposed framework. The framework is applicable to various built environments, including residential and campus buildings. While the current implementation leverages the BIM model of the target building to extract the necessary structural and usage information, it can also operate with other detailed building representations. For example, if architectural drawings containing the required information are available, they can serve as an alternative input source for the risk evaluation process, ensuring flexibility in practical applications. The proposed fire detection model demonstrates high detection accuracy across diverse settings in the selected dataset, enabling it to deliver reliable multi-modal fire detection tasks. As multispectral cameras have been applied for fire detection systems in both industry [63] and 35 academia [16], our multi-modal model can be embedded as the algorithm for these systems, which retains both the robustness of multi-modal detection and the lightweight edge of single-modal detection. And it is applicable to the detection of both wildfires and indoor fires as they share similar visual and thermal characteristics. Additionally, the lightweight single stream design significantly decreases the model size, making it well-suited for deployment in resource-constrained environments like edge devices, enabling real-time fire monitoring in both urban and outdoor environments. 4.4 Limitations We validated our fire risk evaluation framework on a local residential building. However, the fire risk factors considered in our framework may not be sufficient for other types of buildings. For example, as presented in Figure 4.4, the industrial building fire incidents can be caused by machine malfunction and electrical failure [64], which are not considered in our framework. In this case, special attention should be given to the area for placing machines that are prone to malfunction or where numerous circuits are installed. 36 Figure 4.4 Structure fires at industrial properties by leading causes: 2017-2021 [64] We proposed our fire risk evaluation framework by interviewing the team manager of the campus facility team, although the framework has been validated by case study, opinions from more experts can be included to prevent potential bias caused by only referring to a single expert. And local expert familiar with campus buildings might overlook risks in other settings, such as industrial buildings. For these buildings, when applying the fire risk evaluation framework, suggestions from corresponding experts should also be considered. The fire risk values in our case study are fixed, however, these values could be dynamic in real-world applications. For example, the changes of stored material type can influence the spread factor in our framework. Thus, the fire risks should be reevaluated regularly to mitigate the impact of dynamic fire risk change. Modular and flexible layout of camera network can also be considered to make the relocation easier. 37 We focus on fire detection model design and fire risk analysis in this thesis. However, the fire detection pipeline is not fully automated, and the detection result is not displayed in a user-friendly way, which may limit the practicability of our system in real-world application. In our system, the images fed to detection model are captured manually with camera, which is not suitable for 24-hr monitoring. Besides that, the output results of detection model are directly displayed in the terminal, which is not user-friendly and cannot ensure secure access. To make the system more practical, automated data streaming and user interface are still required. Continuous video monitoring also raises privacy concerns, particularly under some regional regulations like the EU GDPR [65]. When applying the fire detection system, locally stored data should be removed regularly, involved humans should also be informed of the potential privacy issue. Additionally, compared to traditional fire detection system (e.g., smoke detector), higher cost of thermal camera limits the extensive application of our multi-modal fire detection system. As the dataset selected for our work includes relatively high-resolution images, using low-resolution images as the system input could harm the detection accuracy and the capability of detecting small fire targets [66]. 38 Chapter 5: Conclusions and future work 5.1 Conclusions This study proposed a multi-modal fire detection system covering fire risk evaluation and lightweight model design. Firstly, this study addressed the critical but often overlooked problem of determining where to install fire monitoring cameras. By leveraging the FRAME methodology and integrating it with AHP, a fire risk assessment framework was developed to evaluate compartment-level risk based on multiple factors, including fire load, spread potential, venting capacity, fire protection systems, and human activity. The relative importance of these factors was determined through expert consultation, and fire risk scores were calculated using a BIM model of a residential building. This enabled the identification of high-risk zones where camera installation would be most effective. Secondly, to address the current challenges in computational complexity of existing multi-modal fire detection models, this study introduced a lightweight four-channel single-stream model that integrates RGB and thermal images into a single input. Specifically, this study modified the YOLOv5 backbone to accept four-channel inputs and incorporated the CBAM module and the SIoU loss function in order to achieve improved accuracy and reduced computational complexity. By comparing the performance of the proposed model with state-of-the-art models in multi-modal object detection such as YOLOv5-based dual-stream model, the detection experimental results demonstrate that the four-channel model outperforms traditional dual-stream YOLOv5 architectures with additive and transformer-based fusion methods. The proposed model not only enhances detection performance but also significantly reduces the model size, making it more efficient for real-world applications. The inclusion of the CBAM module further improves feature representation by emphasizing relevant spatial and channel-wise 39 features, while the SIoU loss function enhances localization precision by addressing shortcomings of existing bounding box regression losses. In multi-modal fire detection, increasing model size does not always lead to improved accuracy; in fact, it can introduce significant computational overhead and may even prevent deployment on resource-limited devices. Instead, carefully designing efficient fusion strategiesâsuch as the 4-channel approach used in our workâcan enhance accuracy while simultaneously reducing model size. Our proposed architecture is modular and can be easily extended to more modern detection frameworks beyond YOLOv5. The four-channel fusion strategy, the CBAM integration, and the use of SIoU loss are all plug-and-play components and can be integrated into newer architectures with minimal modification, allowing future models to benefit from our lightweight yet effective multi-modal fire detection approach. 5.2 Future work Potential future works of our fire detection system are listed in this section. â¢ Although the proposed fire risk evaluation framework was validated on a residential building, its current design mainly reflects factors relevant to residential and campus environments. This may limit its applicability to other building types. Future work should therefore expand the library of risk indicators and engage a broader range of experts across different domains and regions. This would help reduce bias and ensure that the framework is adaptable to diverse building environments. â¢ To develop a more practical fire detection system, several works about the development of automated fire detection system can be referenced. Marques et al. [27] developed a system to monitor fire-related activities in forests. In their system, the drone imageâ¯captureâ¯module streams captured R and thermal images to a 40 transmissionâ¯module, which feeds a cloudâ¯server containing three modulesâprocessing, analysis, and resultsâ¯presentation. he processing module align RGB and thermal images that have different resolution. After align image pairs pass through analysis module, the results are then distributed to various end-user devices through the results presentation module. Similarly, Yang et al. [16] proposed a fire detection system composed of four layers: device layer, data collection layer, fire detection layer, and use interface layer. Data collection layers collected images from device and feed them to fire detection layer. The real-time detection results are then push to the user interface, which also displays historical data for users who are granted access to the system. Based on these works, a more complete version of our system as displayed in Figure 5.1 can be developed in the future. Compared with the framework in Figure 1.1, a data streaming server is applied to capture images from cameras and feed them as the input of the fire detection model, which saves the effort of manual image collection. At the rear of the pipeline, the output results will be displayed in an interface to be securely accessed by authorized users. â¢ To void the privacy issue brought by continuous video surveillance, techniques to conceal humans in the captured can be considered. For example, Jain et al. [67] presents a privacy-preserving fire detection system for indoor surveillance. To enable safe fire surveillance in private spaces, this paper proposes a system that ensures occupant privacy. It uses a modified near-infrared camera to capture images that effectively conceal human presence and activities. Future implementations integrated with such approach will allow continuous fire monitoring without compromising occupant privacy. 41 Figure 5.1 A complete data pipeline of our system â¢ In case of false alarm triggered by human cooking with open flame, rules can be set to reduce prediction confidence when fire is detected with humanâs track nearby. Prediction results from other resources like smoke detector can also be fused to provide more reliable detection. I modelFire load factorSpread factor enting factor Fire risk factors Fire risk tract A Channel AttentionSpatial AttentionC A Angel cost istance costShape costIoU costSIoU loss uide camera installationImprove modelperformance 2 4 channel multi modal fire detection model 1 Fire risk assessment 3 Further enhancement of model odel blocksR camera hermal cameraOutputCaptureCaptureFeedFeed isplayStreamingserverInterface42 Bibliography [1] âResidential fire estimate summaries (2013-2022 ,â U.S. Fire Administration. Accessed: Jan. 14, 2025. [Online]. Available: https://www.usfa.fema.gov/statistics/residential-fires/ [2] J. acCarthy, J. Richter, S. yukavina, . Weisse, and . arris, â he Latest ata Confirms: Forest Fires Are etting Worse,â Aug. 2024, Accessed: Jan. 14, 2025. [Online]. Available: https://www.wri.org/insights/global-trends-forest-fires [3] âCalifornia Wildfiresâ amage stimated Above $50 ,â Ainvest. Accessed: Jan. 14, 2025. [Online]. Available: https://www.ainvest.com/news/california-wildfires-damage-estimated-above-50b-2501101004997d407d9a80e5 [4] J. incott, . W. ien, S. Wei, and J. K. Calautit, âIndoor fire detection utilizing computer vision-based strategies,â Journal of Building Engineering, vol. 61, p. 105154, Dec. 2022, doi: 10.1016/j.jobe.2022.105154. [5] . W. Service, â ow wildfire is detected - rovince of ritish Columbia.â Accessed: Feb. 11, 2024. [Online]. Available: https://www2.gov.bc.ca/gov/content/safety/wildfire-status/wildfire-response/how-wildfire-is-detected [6] Y. LeCun, Y. engio, and . inton, â eep learning,â Nature, vol. 521, no. 7553, pp. 436â444, May 2015, doi: 10.1038/nature14539. [7] S. oghim and . ehrabi, âWildfire assessment using machine learning algorithms in different regions,â fire ecol, vol. 20, no. 1, p. 104, Dec. 2024, doi: 10.1186/s42408-024-00335-2. [8] W. Jiang et al., âWildfire risk assessment using deep learning in uangdong rovince, China,â International Journal of Applied Earth Observation and Geoinformation, vol. 128, p. 103750, Apr. 2024, doi: 10.1016/j.jag.2024.103750. [9] . A. Finney, â he challenge of quantitative risk analysis for wildland fire,â Forest Ecology and Management, vol. 211, no. 1â2, pp. 97â108, June 2005, doi: 10.1016/j.foreco.2005.02.010. [10] B. Meacham, B. Poole, J. Echeverria, and R. Cheng, Fire Safety Challenges of Green Buildings. Springer Science & Business Media, 2013. [11] . Smet, âFRA 2008 echnical reference guide,â 2008. [12] L. Wang, W. Li, W. Feng, and R. Yang, âFire risk assessment for building operation and maintenance based on I technology,â Building and Environment, vol. 205, p. 108188, Nov. 2021, doi: 10.1016/j.buildenv.2021.108188. [13] C. Lv, H. Zhou, Y. Chen, D. Fan, and F. i, âA lightweight fire detection algorithm for small targets based on YOLOv5s,â Sci Rep, vol. 14, no. 1, p. 14104, June 2024, doi: 10.1038/s41598-024-64934-4. [14] R. Xu, . Lin, K. Lu, L. Cao, and Y. Liu, âA Forest Fire etection System ased on nsemble Learning,â Forests, vol. 12, no. 2, p. 217, Feb. 2021, doi: 10.3390/f12020217. [15] X. Rui, Z. Li, X. Zhang, Z. Li, and W. Song, âA R -Thermal based adaptive modality learning network for dayânight wildfire identification,â International Journal of Applied Earth Observation and Geoinformation, vol. 125, p. 103554, Dec. 2023, doi: 10.1016/j.jag.2023.103554. [16] X. Yang, Y. Li, and Q. Chen, âAutomated image-based fire detection and alarm system using edge computing and cloud-based platform,â Internet of Things, vol. 28, p. 101402, Dec. 2024, doi: 10.1016/j.iot.2024.101402. 43 [17] Z. Liu, S. Shi, Q. uan, W. Zhang, and . Zhao, âSalient object detection for R -D image by single stream recurrent convolution neural network,â Neurocomputing, vol. 363, pp. 46â57, Oct. 2019, doi: 10.1016/j.neucom.2019.07.012. [18] X. Zhao, L. Zhang, Y. ang, . Lu, and L. Zhang, âA Single Stream etwork for Robust and Real-Time RGB-D Salient Object etection,â in European conference on computer vision (ECCV), in Lecture Notes in Computer Science. Glasgow, United Kingdom: Springer, 2020, pp. 646â662. doi: 10.1007/978-3-030-58542-6_39. [19] S. Woo, J. Park, J.-Y. Lee, and I. S. Kweon, âC A : Convolutional Block Attention odule,â in European conference on computer vision (ECCV), in Lecture Notes in Computer Science. Munich, Germany: Springer, 2018, pp. 3â19. doi: 10.1007/978-3-030-01234-2_1. [20] Z. evorgyan, âSIoU Loss: ore owerful Learning for ounding o Regression,â May 2022, arXiv: 2205.12740. doi: 10.48550/arXiv.2205.12740. [21] . rzeziÅska and . ryant, âRisk Inde ethodâA Tool for Building Fire Safety Assessments,â Applied Sciences, vol. 11, no. 8, p. 3566, Apr. 2021, doi: 10.3390/app11083566. [22] J. Chen, L. ing, J. Ji, and J. Zhu, âA Combined ethod to uild ayesian etwork for Fire Risk Assessment of istorical uildings,â Fire Technol, vol. 59, no. 6, pp. 3525â3563, Nov. 2023, doi: 10.1007/s10694-023-01475-8. [23] A. Voulodimos, . oulamis, A. oulamis, and . rotopapadakis, â eep Learning for Computer ision: A rief Review,â Computational Intelligence and Neuroscience, vol. 2018, pp. 1â13, 2018, doi: 10.1155/2018/7068349. [24] G. Jocher, YOLOv5 by Ultralytics. (May 2020). Python. Accessed: Nov. 06, 2024. [Online]. Available: https://github.com/ultralytics/yolov5 [25] â owFire ataset.â Accessed: ov. 06, 2024. [Online]. Available: https://bitbucket.org/gbdi/bowfire-dataset/downloads/ [26] . an, R. ang, and Q. . Le, â fficient et: Scalable and fficient Object etection,â in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA: IEEE, June 2020, pp. 10778â10787. doi: 10.1109/CVPR42600.2020.01079. [27] T. Marques, S. Carreira, R. Miragaia, J. Ramos, and A. ereira, âApplying deep learning to real-time UAV-based forest monitoring: Leveraging multi-sensor imagery for improved results,â Expert Systems with Applications, vol. 245, p. 123107, July 2024, doi: 10.1016/j.eswa.2023.123107. [28] D. Li, T. Yang, Z. Jin, W. Si-qi, and L. Quan-yi, âYOLOv8-EMSC: A lightweight fire recognition algorithm for large spaces,â Journal of Safety Science and Resilience, vol. 5, no. 4, pp. 422â431, Dec. 2024, doi: 10.1016/j.jnlssr.2024.06.003. [29] Ultralytics, âArchitecture Summary.â Accessed: Jan. 22, 2025. [Online]. Available: https://docs.ultralytics.com/yolov5/tutorials/architecture_description [30] S. Alaba, A. urbuz, and J. all, âA Comprehensive Survey of eep Learning Multisensor Fusion-based 3D Object Detection for Autonomous Driving: Methods, Challenges, Open Issues, and Future irections,â Aug. 23, 2022. doi: 10.36227/techrxiv.20443107.v2. [31] W. El Ahmar et al., â nhanced hermal-R Fusion for Robust Object etection,â in 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops 44 (CVPRW), Vancouver, BC, Canada: IEEE, June 2023, pp. 365â374. doi: 10.1109/CVPRW59228.2023.00042. [32] . . asandi, . Liu, Y. assoud, and R. Laganière, âSensor Fusion Operators for Multimodal 2D Object etection,â in International Symposium on Visual Computing, San Diego, CA, USA: Springer, 2022, pp. 184â195. doi: 10.1007/978-3-031-20713-6_14. [33] Y. assoud, âSensor Fusion for 3 Object etection for Autonomous ehicles,â University of Ottawa, 2021. [Online]. Available: http://dx.doi.org/10.20381/ruor-27029 [34] K. e, X. Zhang, S. Ren, and J. Sun, â eep Residual Learning for Image Recognition,â in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA: IEEE, June 2016, pp. 770â778. doi: 10.1109/CVPR.2016.90. [35] Z. Liu, H. Mao, C.-Y. Wu, C. Feichtenhofer, . arrell, and S. Xie, âA Conv et for the 2020s,â in 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA, 2022, pp. 11976â11986. [36] F. Qingyun, . apeng, and W. Zhaokui, âCross-Modality Fusion Transformer for ultispectral Object etection,â Oct. 04, 2022, arXiv: 2111.00273. doi: 10.48550/arXiv.2111.00273. [37] A. Vaswani et al., âAttention is All you eed,â in Advances in Neural Information Processing Systems 30, Long Beach, CA, USA, 2017. [38] âFree FLIR hermal ataset for Algorithm raining.â Accessed: ov. 08, 2024. [Online]. Available: https://www.flir.ca/oem/adas/adas-dataset-form/ [39] X. Jia, C. Zhu, . Li, W. ang, and W. Zhou, âLL I : A isible-infrared Paired Dataset for Low-light ision,â in 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), Montreal, BC, Canada: IEEE, Oct. 2021, pp. 3489â3497. doi: 10.1109/ICCVW54120.2021.00389. [40] S. Razakarivony and F. Jurie, â ehicle detection in aerial imageryâ¯: A small target detection benchmark,â Journal of Visual Communication and Image Representation, vol. 34, pp. 187â203, Jan. 2016, doi: 10.1016/j.jvcir.2015.11.002. [41] M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen, â obile et 2: Inverted Residuals and Linear ottlenecks,â in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT: IEEE, June 2018, pp. 4510â4520. doi: 10.1109/CVPR.2018.00474. [42] A. Dosovitskiy et al., âAn Image is Worth 16 16 Words: ransformers for Image Recognition at Scale,â June 03, 2021, arXiv: arXiv:2010.11929. doi: 10.48550/arXiv.2010.11929. [43] Y. Zhao et al., â Rs Beat YOLOs on Real-time Object etection,â in 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA: IEEE, June 2024, pp. 16965â16974. doi: 10.1109/CVPR52733.2024.01605. [44] T. Toulouse, L. Rossi, A. Campana, T. Celik, and . A. Akhloufi, âComputer vision for wildfire research: An evolving image dataset for processing and analysis,â Fire Safety Journal, vol. 92, pp. 188â194, Sept. 2017, doi: 10.1016/j.firesaf.2017.06.012. [45] X. Chen et al., âWildland Fire etection and Monitoring Using a Drone-Collected R /IR Image ataset,â IEEE Access, vol. 10, pp. 121301â121317, 2022, doi: 10.1109/ACCESS.2022.3222805. 45 [46] A. Shamsoshoara, F. Afghah, A. Razi, L. Zheng, . Z. Fulé, and . lasch, âAerial imagery pile burn detection using deep learning: he FLA dataset,â Computer Networks, vol. 193, p. 108001, July 2021, doi: 10.1016/j.comnet.2021.108001. [47] B. Hopkins et al., âFLA 3 ataset: Unleashing the ower of Radiometric hermal UA Imagery for Wildfire anagement,â ec. 03, 2024, arXiv: arXiv:2412.02831. doi: 10.48550/arXiv.2412.02831. [48] Fire-resistance tests - Elements of building construction, ISO 834, 1975. [49] âStatistics Canada Report.â Accessed: June 16, 2025. [Online]. Available: http://nfidcanada.ca/statistics-canada-report/ [50] Z. Xue, . Lin, and F. Wang, âA Small arget Forest Fire etection odel ased on YOLOv5 Improvement,â Forests, vol. 13, no. 8, p. 1332, Aug. 2022, doi: 10.3390/f13081332. [51] â apers with Code - LLVIP Benchmark (Pedestrian Detection .â Accessed: ov. 28, 2024. [Online]. Available: https://paperswithcode.com/sota/pedestrian-detection-on-llvip [52] R. arghese and S. ., âYOLOv8: A ovel Object etection Algorithm with nhanced erformance and Robustness,â in 2024 International Conference on Advances in Data Engineering and Intelligent Computing Systems (ADICS), Chennai, India: IEEE, Apr. 2024, pp. 1â6. doi: 10.1109/ADICS58448.2024.10533619. [53] A. Wang et al., âYOLOv10: Real-Time End-to- nd Object etection,â in Advances in Neural Information Processing Systems (NeurIPS), Vancouver, BC, Canada, 2024, pp. 107984â108011. [54] G. Jocher, J. Qiu, and A. Chaurasia, Ultralytics YOLO. (Jan. 2024). Python. Accessed: June 05, 2025. [Online]. Available: https://github.com/ultralytics/ultralytics [55] S. Woo, J. Park, J.-Y. Lee, and I. S. Kweon, âC A : Convolutional lock Attention odule,â in Computer Vision â ECCV 2018, vol. 11211, V. Ferrari, M. Hebert, C. Sminchisescu, and Y. Weiss, Eds., in Lecture Notes in Computer Science, vol. 11211. , Cham: Springer International Publishing, 2018, pp. 3â19. doi: 10.1007/978-3-030-01234-2_1. [56] R. irshick, âFast R-C ,â in IEEE International Conference on Computer Vision (ICCV), Santiago, Chile: IEEE, 2015, pp. 1440â1448. [57] H. Rezatofighi, N. Tsoi, J. wak, A. Sadeghian, I. Reid, and S. Savarese, â eneralized Intersection Over Union: A etric and a Loss for ounding o Regression,â in 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA: IEEE, June 2019, pp. 658â666. doi: 10.1109/CVPR.2019.00075. [58] Z. Zheng, . Wang, W. Liu, J. Li, R. Ye, and . Ren, â istance-IoU Loss: Faster and etter Learning for ounding o Regression,â AAAI, vol. 34, no. 07, pp. 12993â13000, Apr. 2020, doi: 10.1609/aaai.v34i07.6999. [59] Z. Zheng et al., â nhancing eometric Factors in Model Learning and Inference for Object etection and Instance Segmentation,â IEEE Trans. Cybern., vol. 52, no. 8, pp. 8574â8586, Aug. 2021, doi: 10.1109/TCYB.2021.3095305. [60] â150 ome Fire Statistics,â Insuranceopedia. Accessed: July 08, 2025. [Online]. Available: https://www.insuranceopedia.com/homeowners-insurance/home-fire-statistics [61] . Spearpoint and C. opkin, âA Study of the ime of ay and Room of Fire Origin for welling Fires,â Fire Technol, vol. 56, no. 4, pp. 1465â1485, July 2020, doi: 10.1007/s10694-019-00934-5. 46 [62] âCOCO - Common Objects in Conte t.â Accessed: ov. 28, 2024. [Online]. Available: https://cocodataset.org/#home [63] â eat ro i-Spectrum Thermal Cameras Deliver Advanced Fire Detection for Waste Recycling Facilities,â ikvision. Accessed: Aug. 17, 2025. [Online]. Available: http://www.hikvision.com/ca-en/newsroom/blog/heatpro-bi-spectrum-thermal-cameras-deliver-advanced-fire-detect/ [64] âFire in Industrial or anufacturing roperties | F A Research.â Accessed: July 25, 2025. [Online]. Available: https://www.nfpa.org/education-and-research/research/nfpa-research/fire-statistical-reports/fires-in-us-industrial-or-manufacturing-properties [65] â ersonal ata,â eneral ata rotection Regulation R . Accessed: July 25, 2025. [Online]. Available: https://gdpr-info.eu/issues/personal-data/ [66] V. Thambawita, I. Strümke, S. A. Hicks, P. Halvorsen, S. Parasa, and M. A. Riegler, âImpact of Image Resolution on eep Learning erformance in ndoscopy Image Classification: An perimental Study Using a Large ataset of ndoscopic Images,â Diagnostics, vol. 11, no. 12, p. 2183, Nov. 2021, doi: 10.3390/diagnostics11122183. [67] A. Jain and A. Srivastava, â rivacy-Preserving Efficient Fire Detection System for Indoor Surveillance,â IEEE Transactions on Industrial Informatics, vol. 18, no. 5, pp. 3043â3054, Sept. 2021, doi: 10.1109/TII.2021.3110576. 47 Appendices Appendix A This appendix summarizes the key bounding box regression loss functions discussed in Chapter 2, along with the definitions of all mathematical symbols used. A.1 Smooth L1 Loss âð ðððð¡âð¿1(ð¥, ð¦) = {0.5(ð¥ â ð¦)2, ðð|ð¥ â ð¦| < 1|ð¥ â ð¦| â 0.5, ðð¡âððð¤ðð ð(ð´ â 1) â¢ ð¥: Vertexâs coordinates of ground truth bounding bo â¢ ð¦: Vertexâs coordinates of predicted bounding bo A.2 Intersection over Union (IoU) ð¼ðð =ð´ â© ðµð´ âª ðµ(ð´ â 2) âð¼ðð = âððð¼ðð (ð´ â 3) ðâð¼ðððð¼ðð= â1ð¼ðð(ð´ â 4) â¢ ð´: Predicted bounding box â¢ ðµ: Ground truth bounding box A.3 Generalized IoU (GIoU) ðºð¼ðð = ð¼ðð â|ð¶ â (ð´ âª ðµ)||ð¶|(ð´ â 5) âðºð¼ðð = 1 â ðºð¼ðð = 1 â ð¼ðð +|ð¶ â (ð´ âª ðµ)||ð¶|(ð´ â 6) 48 â¢ ð¶: Minimum enclosing rectangle that can encompass both the predicted and ground truth boxes A.4 Distance IoU (DIoU) ð·ð¼ðð = ð¼ðð âð2ð2(ð´ â 7) âð·ð¼ðð = 1 â ð·ð¼ðð = 1 â ð¼ðð +ð2ð2(ð´ â 8) â¢ ð: Euclidean distance between the center points of the predicted box and the ground truth box â¢ ð: Diagonal length of the smallest enclosing box that contains both predicted and ground truth boxes â¢ ð2ð2 : Distance penalty A.5 Complete IoU (CIoU) ð¶ð¼ðð = ð·ð¼ðð â ï¡ð£ = ð¼ðð âð2ð2â ð¼ð£ (ð´ â 9) ð¼ =ð£(1 â ð¼ðð) + ð£(ð´ â 10) ð£ =4ð2(ðððð¡ððð¤ðð¡âðð¡â ðððð¡ððð¤â)2(ð´ â 11) âð¶ð¼ðð = 1 â ð¶ð¼ðð = 1 â ð¼ðð +ð2ð2+ ð¼ð£ (ð´ â 12) â¢ ð¼: the trade-off parameter between distance penalty and aspect ratio â¢ ð£ is the aspect ratio used to evaluate the consistency between the aspect ratio of predicted box and ground truth box â¢ ð¤ðð¡: Width of ground truth box â¢ âðð¡: Height of ground truth box â¢ ð¤: Width of predicted box â¢ â: Height of predicted box 49 Appendix B This appendix presents the detailed information of compartments in the evaluated residential building. Name # Area Length Width Fire load Spread Venting Fire system Human activity Fire risk Garage 101 21.15 3.68 5.75 1.385 0.651 1.110 0.9 0.3 0.7643 Mudroom 102 6.89 1.96 3.52 1.385 0.452 0.806 0.9 0.3 0.7088 Washroom 103 1.9 0.89 2.13 1.385 0.313 0.565 1 0.3 0.7151 Stairway 104 6.86 3.33 2.06 1.385 0.305 0.724 0.9 0.456 0.7207 Den 105 7.28 2.23 3.26 1.385 0.428 0.689 0.9 0.582 0.7637 Dining Area 106 59.76 7.47 8 1.385 0.835 0.851 0.9 1 0.9344 Bedroom1 201 17.62 4.18 4.22 1.385 0.633 1.110 0.9 0.582 0.8318 Ensuite 202 4.49 1.49 3.01 1.385 0.493 0.574 1 0.3 0.7386 WIC 203 3.87 1.49 2.6 1.385 0.443 0.677 0.9 0.3 0.6947 Loft 204 18.49 3.28 5.63 1.385 0.784 0.780 0.9 0.582 0.8175 Laundry 205 4.38 1.68 2.61 1.385 0.444 0.703 0.9 0.3 0.6974 Storage 206 1.86 1.16 1.61 1.385 0.312 1.110 0.9 0.3 0.7219 Bathroom 207 4.48 2.96 1.51 1.385 0.298 1.110 1 0.3 0.7681 50 Bedroom2 208 13.14 3.28 4.01 1.385 0.609 0.719 0.9 0.582 0.7894 Bedroom3 209 15.21 3.94 3.86 1.385 0.592 0.746 0.9 0.582 0.7900
Thesis/Dissertation
eng
Civil Engineering
Vancouver : University of British Columbia Library
Attribution-NonCommercial-NoDerivatives 4.0 International
Graduate
Chen, Qian
A risk-informed and lightweight system for multi-modal fire detection
Text