Applied Science, Faculty of
Engineering, School of (Okanagan)
DSpace
UBCO
Yun, Hongguang
Doctor of Philosophy - PhD
University of British Columbia
Composite materials are widely used in aerospace, wind energy, and shipbuilding, offering significant benefits such as superior strength-to-weight ratios. However, they are prone to damage from impacts, necessitating advanced methods for damage localization to ensure safety and reduce maintenance costs. Deep learning-based ultrasonic guided waves (UGW) methods are increasingly recognized for their effectiveness in locating damage in composites. This study aims to develop deep neural networks (DNN) models for damage localization, focusing on two main challenges: limited dataset sizes and the integration of physical knowledge. Three specific application scenarios were identified: independent and identically distributed ($i.i.d.$) damage localization, domain generalized damage localization, and out-of-distribution (OOD) damage localization. Firstly, to achieve accurate damage localization, incorporating transducer coordinates with UGW signals is essential. This study developed a multi-dimensional data fusion DNN model that integrates a convolutional neural network-based encoder for high-dimensional wave signals and a novel random Fourier projection head for transducer coordinates. The proposed method outperformed existing DNN-based methods, achieving an average absolute distance error of approximately 2 millimeters in damage localization on plate-like composite structures. Secondly, distribution discrepancies often occur between training and testing data, leading to reduced model performance. This study proposed a domain generalization method to mitigate this issue in a multi-class classification task. A feature distance maximization mechanism was introduced to improve modelâs resistance to such distribution discrepancies. The proposed method outperformed leading DNN-based models, achieving an average accuracy of around 90%, compared to less than 85% for other methods. Thirdly, this study proposes a physics-informed neural network model to address the challenging OOD damage localization. The damage localization process was divided into two stages: extracting the time-of-flight from UGW, and estimating the location. The proposed method outperformed existing DNN-based and traditional methods, achieving relative prediction errors of 0.08 meters, compared to over 0.1 meters for other models. Overall, this study introduces three methodologies that are designed for different levels of data availability and the embedding of physical knowledge. Each method contributes to improving the robustness and accuracy of damage localization, paving the way for effective and efficient damage assessment in composite materials.
Deep neural networks for damagelocalization in composite materialswith ultrasonic guided wavesbyHongguang YunB.Sc., Xiâan Jiaotong University, 2016M.Sc., Xiâan Jiaotong University, 2019A THESIS SUBMITTED IN PARTIAL FULFILLMENT OFTHE REQUIREMENTS FOR THE DEGREE OFDOCTOR OF PHILOSOPHYinThe College of Graduate Studies(Electrical Engineering)THE UNIVERSITY OF BRITISH COLUMBIA(Okanagan)May 2024© Hongguang Yun, 2024The following individuals certify that they have read, and recommendto the College of Graduate Studies for acceptance, a thesis/dissertation en-titled:Deep neural networks for damage localization in compositematerials with ultrasonic guided wavessubmitted by Hongguang Yun in partial fulfillment of the requirementsof the degree of Doctor of PhilosophyDr. Zheng Liu, School of EngineeringSupervisorDr. Abbas Milani, School of EngineeringSupervisory Committee MemberDr. Shashank Pant, National Research Council CanadaSupervisory Committee MemberDr. Hadi Mohammadi, School of EngineeringUniversity ExaminerDr. Yuu Ono, Carleton UniversityExternal ExamineriiAbstractComposite materials are widely used in aerospace, wind energy, andshipbuilding, offering significant benefits such as superior strength-to-weightratios. However, they are prone to damage from impacts, necessitating ad-vanced methods for damage localization to ensure safety and reduce mainte-nance costs. Deep learning-based ultrasonic guided waves (UGW) methodsare increasingly recognized for their effectiveness in locating damage in com-posites. This study aims to develop deep neural networks (DNN) modelsfor damage localization, focusing on two main challenges: limited datasetsizes and the integration of physical knowledge. Three specific applicationscenarios were identified: independent and identically distributed (xIxIsI)damage localization, domain generalized damage localization, and out-of-distribution (OOD) damage localization.Firstly, to achieve accurate damage localization, incorporating trans-ducer coordinates with UGW signals is essential. This study developed amulti-dimensional data fusion DNN model that integrates a convolutionalneural network-based encoder for high-dimensional wave signals and a novelrandom Fourier projection head for transducer coordinates. The proposedmethod outperformed existing DNN-based methods, achieving an averageabsolute distance error of approximately 2 millimeters in damage localiza-tion on plate-like composite structures.Secondly, distribution discrepancies often occur between training andtesting data, leading to reduced model performance. This study proposed adomain generalization method to mitigate this issue in a multi-class classifi-cation task. A feature distance maximization mechanism was introduced toimprove models resistance to such distribution discrepancies. The proposedmethod outperformed leading DNN-based models, achieving an average ac-curacy of around 90%, compared to less than 85% for other methods.Thirdly, this study proposes a physics-informed neural network model toaddress the challenging OOD damage localization. The damage localizationprocess was divided into two stages: extracting the time-of-flight from UGW,and estimating the location. The proposed method outperformed existingDNN-based and traditional methods, achieving relative prediction errorsiiiof 0.08 meters, compared to over 0.1 meters for other models. Overall,this study introduces three methodologies that are designed for differentlevels of data availability and the embedding of physical knowledge. Eachmethod contributes to improving the robustness and accuracy of damagelocalization, paving the way for effective and efficient damage assessment incomposite materials.ivLay SummaryComposite materials are increasingly employed across aerospace, windenergy, and shipbuilding sectors due to their superior strength-to-weightratios and corrosion resistance. Despite their advantages, these materials aresusceptible to damage from impacts and environmental stress, necessitatingaccurate damage localization to prevent operational risks. This study aimsto leverage deep neural networks for composite damage localization usingultrasonic guided waves.Specifically, this study highlights the balance between data dependencyand physical knowledge integration, examining three application scenar-ios: xIxIsI damage localization, domain-generalized damage localization, andOOD damage localization. These scenarios reflect varied data availabilityand physical knowledge needs. Accordingly, three DNN models were devel-oped to fuse multi-dimensional data, address data distribution discrepancies,and utilize physics-informed approaches to enhance damage localization ac-curacy. This approach demonstrates the potential of leveraging DNN meth-ods in improving damage assessment in composite materials.vPrefaceThis dissertation is based on the research work conducted in the Schoolof Engineering at the University of British Columbia, Okanagan Campus,under the supervision of Prof. Zheng Liu. I accomplished the literaturereview, methods development, experimental validation, and writing of thisdissertation. I also conducted the ultrasonic guided wave inspection at theAerospace Research Centre, National Research Council Canada, Ottawa,Montréal Road Campus, with the support of Dr. Shashank Pant and MarkGenest. Prof. Zheng Liu guided me during the finalization of the researchscope, plan, and development. The publications in which I primarily partic-ipated during the Ph.D. program are listed below. Bold fonts indicate theprimary author, while the â sign indicates the corresponding author.A version of chapter 2 has been published. Yun H, Rayhana R, PantS, Genest M, Liu Zâ. Nonlinear ultrasonic testing and data analytics fordamage characterization: A review. Measurement. 2021 Dec 1. I conductedall the literature investigation and manuscript writing.Chapter 3 is based on work conducted in National Research CouncilCanadaAerospace Research Centre by me, with support from Marc Genestand Dr. Shashank Pant. I was responsible for conducting impact dam-age inspections using ultrasonic guided waves, immersion ultrasonic testing,and destructive microscope inspection. A version of chapter 3 has beenpublished. Yun H, Pant S, Genest M, Li L, Liu Zâ. Impact damagecharacterization in composite plate using ultrasonic guided waves. In2023IEEE International Ultrasonics Symposium (IUS) 2023 Sep 3 (pp. 1-4).IEEE. I conducted all the method development, experimental validation,and manuscript writing.A version of chapter 4 has been published. Yun H, Feng K, Rayhana R,Pant S, Genest M, Liu Zâ. A multidimensional data fusion neural networkfor damage localization using ultrasonic guided wave. IEEE Transactions onInstrumentation and Measurement. 2023 Aug 1. I conducted all the methoddevelopment, experimental validation, and manuscript writing.A version of chapter 5 has been published. Yun H, Wang R, Rayhana R,Pant S, Genest M, Liu Zâ. WaveCLR: Contrastive learning of guided wavevirepresentations for composite damage identification. IEEE Transactions onInstrumentation and Measurement. 2024 Apr 8. I conducted all the methoddevelopment, experimental validation, and manuscript writing.The lists of publications are summarized as below.Journal Publications1. Yun H, Rayhana R, Pant S, Genest M, Liu Zâ. Nonlinear ultra-sonic testing and data analytics for damage characterization: A re-view. Measurement. 2021 Dec 1.2. Yun H, Feng K, Rayhana R, Pant S, Genest M, Liu Zâ. A multi-dimensional data fusion neural network for damage localization usingultrasonic guided wave. IEEE Transactions on Instrumentation andMeasurement. 2023 Aug 1.3. Yun H, Wang R, Rayhana R, Pant S, Genest M, Liu Zâ. Wave-CLR: Contrastive learning of guided wave representations for compos-ite damage identification. IEEE Transactions on Instrumentation andMeasurement. 2024 Apr 8.4. Yun H, Carlone M, Liu Zâ. Topic modeling of maintenance logsfor linac failure modes and trends identification. Journal of AppliedClinical Medical Physics. 2022 Jan;23(1):e13477.Conference Proceedings1. Yun H, Pant S, Genest M, Li L, Liu Zâ. Impact damage charac-terization in composite plate using ultrasonic guided waves. In2023IEEE International Ultrasonics Symposium (IUS) 2023 Sep 3 (pp. 1-4). IEEE.Other Co-authored Publications1. Feng K, Ji JC, Ni Q, Yun H, Zheng J, Liu Zâ. A novel vibration indi-cator to monitor gear natural fatigue pitting propagation. StructuralHealth Monitoring. 2023 Sep;22(5):3126-40.2. Rayhana R, Yun H, Liu Zâ, Kong X. Automated Defect-DetectionSystem for Water Pipelines Based on CCTV Inspection Videos of Au-tonomous Robotic Platforms. IEEE/ASME Transactions on Mecha-tronics. 2023 Sep 6.vii3. Wang R, Yun H, Rayhana R, Bin J, Zhang C, Herrera OE, LiuZâ, Mérida W. An adaptive federated learning system for communitybuilding energy load forecasting and anomaly prediction. Energy andBuildings. 2023 Sep 15;295:113215.4. Lei Z, Yun H, Tian F, Wen G, Liu Zâ. A Multi-scale ConvolutionalAutoencoder with Attention Mechanism for Fault Diagnosis of Ro-tating Machinery. InApplications of Generative AI 2024 Mar 6 (pp.601-617). Cham: Springer International Publishing.viiiTable of ContentsAbstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iiiLay Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vPreface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viTable of Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . ixList of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiiiList of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvList of Abbreviation . . . . . . . . . . . . . . . . . . . . . . . . . xxiAcknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . .xxiiiDedication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .xxivChapter 1: Introduction . . . . . . . . . . . . . . . . . . . . . . . 11.1 Background and Motivation . . . . . . . . . . . . . . . . . . . 11.2 Research Gaps and Objectives . . . . . . . . . . . . . . . . . . 51.3 Thesis Structure and Organization . . . . . . . . . . . . . . . 9Chapter 2: Literature Review . . . . . . . . . . . . . . . . . . . 132.1 Nondestructive Testing Techniques for Composite DamageLocalization . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132.1.1 Infrared Thermography . . . . . . . . . . . . . . . . . 132.1.2 Laser Shearography . . . . . . . . . . . . . . . . . . . 142.1.3 Acoustic Emission . . . . . . . . . . . . . . . . . . . . 152.1.4 Ultrasonic Testing . . . . . . . . . . . . . . . . . . . . 162.2 Ultrasonic Guided Wave for Damage Localization . . . . . . . 17ix2.2.1 Hand-crafted Feature Extraction for Damage Local-ization . . . . . . . . . . . . . . . . . . . . . . . . . . . 172.2.2 Probabilistic Feature Extraction for Damage Localiza-tion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192.3 Deep Neural Networks for Damage Localization . . . . . . . . 202.3.1 Standard Deep Neural Network Damage Localization . 212.3.2 Domain Generalized Damage Localization . . . . . . . 242.3.3 OOD Damage Localization . . . . . . . . . . . . . . . 252.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26Chapter 3: Composite Damage Inspection using UltrasonicGuided Waves . . . . . . . . . . . . . . . . . . . . . 273.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273.2 Composite Damage Inspection using Nonlinear UGW . . . . 283.2.1 Inspection Overview . . . . . . . . . . . . . . . . . . . 283.2.2 Detailed Inspection Procedure . . . . . . . . . . . . . 293.3 Public Composite Damage Inspection Dataset using UGW . . 333.4 Data Analysis of UGW Signals . . . . . . . . . . . . . . . . . 343.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44Chapter 4: Multi-dimensional Data Fusion for Composite Dam-age Localization . . . . . . . . . . . . . . . . . . . . 454.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 454.2 Standard Deep Neural Network for Damage Localization . . . 464.2.1 Problem Definition . . . . . . . . . . . . . . . . . . . . 464.2.2 Multi-dimensional Input Data from Ultrasonic Testing 464.3 Multi-dimensional Data Fusion Model for Damage Localization 474.3.1 Model Architecture . . . . . . . . . . . . . . . . . . . . 474.3.2 Kernel Regression . . . . . . . . . . . . . . . . . . . . 494.3.3 Random Fourier Feature Projection . . . . . . . . . . 514.4 Experimental Results and Discussion . . . . . . . . . . . . . . 544.4.1 Data Description . . . . . . . . . . . . . . . . . . . . . 544.4.2 Ablation Studies . . . . . . . . . . . . . . . . . . . . . 554.4.3 Comparative Studies with SOTA Methods . . . . . . . 604.4.4 Robustness Study Regarding the Data Availability . . 624.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65Chapter 5: Contrastive Learning for Domain Generalized Com-posite Damage Localization . . . . . . . . . . . . . 665.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66x5.2 Domain Generalization Problem in Damage Localization . . . 675.2.1 Problem Definition . . . . . . . . . . . . . . . . . . . . 675.2.2 Domain Generalization of Deep Neural Network Models 685.3 Contrastive Learning for Domain Generalization Damage Lo-calization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 685.3.1 Model Architecture . . . . . . . . . . . . . . . . . . . . 695.3.2 Mutual Inter-class Distance Maximization . . . . . . . 715.3.3 Intra-class Distance Maximization . . . . . . . . . . . 735.3.4 Training strategy . . . . . . . . . . . . . . . . . . . . . 755.4 Experimental Results and Discussion . . . . . . . . . . . . . . 765.4.1 Data Description . . . . . . . . . . . . . . . . . . . . . 765.4.2 Ablation Studies . . . . . . . . . . . . . . . . . . . . . 795.4.3 Comparative Studies with SOTA Methods . . . . . . . 825.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85Chapter 6: Physics-informed Neural Network for Out-of-distributionComposite Damage Localization . . . . . . . . . . . 876.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 876.2 Out-of-distribution Problem in Damage Localization . . . . . 886.2.1 Problem Definition . . . . . . . . . . . . . . . . . . . . 886.2.2 Revisiting Analytical Methods . . . . . . . . . . . . . 886.3 Physics-informed Neural Network for OOD Damage Local-ization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 896.3.1 Model Architecture . . . . . . . . . . . . . . . . . . . . 916.3.2 Distance-of-flight Extractor . . . . . . . . . . . . . . . 926.3.3 Location Solver . . . . . . . . . . . . . . . . . . . . . . 956.4 Experimental Results and Discussion . . . . . . . . . . . . . . 976.4.1 Data Description . . . . . . . . . . . . . . . . . . . . . 976.4.2 Ablation Studies . . . . . . . . . . . . . . . . . . . . . 976.4.3 Comparative Studies with SOTA Methods . . . . . . . 996.4.4 Comparative Studies with Analytical Methods . . . . 1016.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108Chapter 7: Conclusion . . . . . . . . . . . . . . . . . . . . . . . . 1097.1 Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . . 1097.2 Future Works . . . . . . . . . . . . . . . . . . . . . . . . . . . 112Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114Appendices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131xiAppendix A: Feature Space Visualization of Key Mechanisms . . . 131A.1 Visualization of RFF Projection Benefit . . . . . . . . 131A.2 Visualization of WaveCLR Mechanism . . . . . . . . . 133Appendix B: Destructive Inspection of Composite Plate . . . . . . 136xiiList of TablesTable 3.1 Mechanical properties of IM7/977-3 prepreg materialfor plate manufacture. . . . . . . . . . . . . . . . . . . 28Table 3.2 Coordinates of eight transducers installed on the CFRPplate. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36Table 3.3 Coordinates of nine predefined disk damage locationson the CFRP plate. . . . . . . . . . . . . . . . . . . . . 36Table 4.1 Detailed architecture of proposed method with ResNet18as encoder for LUGW dataset. . . . . . . . . . . . . . 56Table 4.2 Comparative results on the use of low-dimensional dataon two datasets. . . . . . . . . . . . . . . . . . . . . . . 60Table 4.3 Comparative results of various DNN-based methodsfor damage localization on NUGW dataset. . . . . . . 62Table 4.4 Comparative results of various DNN-based methodsfor damage localization on LUGW dataset. . . . . . . 63Table 4.5 Comparative results (MMD (mm)) of various DNN-based methods for damage localization under threedifferent data split settings on NUGW dataset. . . . . 64Table 4.6 Comparative results (MMD (mm)) of various meth-ods under three different data split settings on LUGWdataset. . . . . . . . . . . . . . . . . . . . . . . . . . . 64Table 5.1 Comparison of different deep learning paradigms fordata requirements. . . . . . . . . . . . . . . . . . . . . 69Table 5.2 Definition of the data group according to the locationof damages. . . . . . . . . . . . . . . . . . . . . . . . . 77Table 5.3 Definition of classification tasks according to the datagroups in the source and target domain. . . . . . . . . 78Table 5.4 Accuracy results (%) of ablation study on six modelsacross two datasets using transducer T1 as transmitter. 81xiiiTable 5.5 Comparative accuracy (%) results of various DNN-based methods on NUGW dataset using different trans-ducers as the transmitter. . . . . . . . . . . . . . . . . 84Table 5.6 Comparative F1 score (%) results of various DNN-based methods on NUGW dataset using different trans-ducers as the transmitter. . . . . . . . . . . . . . . . . 84Table 5.7 Comparative accuracy (%) results of various DNN-based methods on LUGW dataset using different trans-ducers as the transmitter. . . . . . . . . . . . . . . . . 85Table 5.8 Comparative F1 score (%) results of various DNN-based methods on LUGW dataset using different trans-ducers as the transmitter. . . . . . . . . . . . . . . . . 85Table 6.1 MND (m) results ablation studies on four models onNUGW dataset. . . . . . . . . . . . . . . . . . . . . . . 99Table 6.2 MND (m) results ablation studies on four models onLUGW dataset. . . . . . . . . . . . . . . . . . . . . . . 99Table 6.3 Comparative MMD (m) results with various DNN-based methods on NUGW dataset. . . . . . . . . . . . 101Table 6.4 Comparative MMD (m) results with various DNN-based methods on LUGW dataset. . . . . . . . . . . . 102xivList of FiguresFigure 1.1 Sectional view of delamination damage within a multi-layer composite plate after impact. © Hongguang Yun 2Figure 1.2 Illustration of relation of data availability and distri-bution difference between training and testing data.© Hongguang Yun . . . . . . . . . . . . . . . . . . . . 6Figure 1.3 Illustration of training and testing data split formatfollow xIxIsI principle. © Hongguang Yun . . . . . . . 8Figure 1.4 Illustration of training and testing data split formatfollow domain discrepancy principle. © HongguangYun . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8Figure 1.5 Illustration of training and testing data split formatfollow OOD distribution principle. © Hongguang Yun 8Figure 1.6 Flow diagram of the thesis structure. © HongguangYun . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10Figure 2.1 Architecture of a fully-connected neural network with5 layers. © Hongguang Yun . . . . . . . . . . . . . . 21Figure 2.2 Architecture of LeNet and illustration of convolutionoperation and pooling operation. © Hongguang Yun . 23Figure 2.3 Architecture of 1DCNN with basic convolutional, pool-ing, and fully-connected layer. © Hongguang Yun . . 24Figure 3.1 The procedure of the impact damage inspection usingnonlinear UGW technique. © Hongguang Yun . . . . 29Figure 3.2 The SMART Layers PZT transducer by Acellent Tech-nologies, Inc.. [1] . . . . . . . . . . . . . . . . . . . . . 29Figure 3.3 Flowchart of the installation process for PZT trans-ducers on the CFRP plate. © Hongguang Yun . . . . 30Figure 3.4 Illustration of CFRP plate with eight PZT transduc-ers installed. © Hongguang Yun . . . . . . . . . . . . 31xvFigure 3.5 Calculated phase velocity dispersion curve of the IMM/977-3 specimen. Synchronous pair of modes s0 and s1 ata synchronous phase velocity of around 6280 m/s. ©Hongguang Yun . . . . . . . . . . . . . . . . . . . . . 32Figure 3.6 Immersion ultrasonic C-Scan results of #1 CFRP plateafter 19J impact. © Hongguang Yun . . . . . . . . . 33Figure 3.7 Immersion ultrasonic C-Scan results of #1 CFRP plateafter 17J impact. © Hongguang Yun . . . . . . . . . 34Figure 3.8 Immersion ultrasonic C-Scan results of #1 CFRP plateafter 16.5J impact. © Hongguang Yun . . . . . . . . 35Figure 3.9 Metallic disks with varying size used as simulationdamages. © Hongguang Yun . . . . . . . . . . . . . . 35Figure 3.10 Configuration of 12 transducers and 28 different dam-age locations of LUGW dataset [2]. Used with per-mission from Sage. . . . . . . . . . . . . . . . . . . . . 37Figure 3.11 Pitch-catch configuration of the nonlinear UGW sig-nals: top panel displays the transmitted waveform;bottom panel shows the received waveform. © Hong-guang Yun . . . . . . . . . . . . . . . . . . . . . . . . 38Figure 3.12 Visualization of the nonlinear UGW signals from sevenreceivers collecting data from the same transmitter.© Hongguang Yun . . . . . . . . . . . . . . . . . . . . 39Figure 3.13 Comparative visualization of received nonlinear UGWsignal: before and after a band-pass filtering. © Hong-guang Yun . . . . . . . . . . . . . . . . . . . . . . . . 39Figure 3.14 Visualization of the nonlinear UGW signal in the fre-quency domain, showing both fundamental and sec-ond harmonic frequencies. © Hongguang Yun . . . . 40Figure 3.15 Time-frequency representation of the nonlinear UGWsignal, showing both fundamental and second har-monic wave packets. © Hongguang Yun . . . . . . . . 40Figure 3.16 Pitch-catch configuration of the linear UGW signals:top panel displays the transmitted waveform; bottompanel shows the received waveform. © Hongguang Yun 41Figure 3.17 Visualization of the linear UGW signals from six re-ceivers collecting data from the same transmitter. ©Hongguang Yun . . . . . . . . . . . . . . . . . . . . . 42Figure 3.18 Comparative visualization of received linear UGWsignal: before and after a band-pass filtering. © Hong-guang Yun . . . . . . . . . . . . . . . . . . . . . . . . 42xviFigure 3.19 Visualization of the linear UGW signal in the fre-quency domain, showing only the fundamental har-monic frequency. © Hongguang Yun . . . . . . . . . . 43Figure 3.20 Time-frequency representation of the linear UGW sig-nal, showing the fundamental harmonic wave packet.© Hongguang Yun . . . . . . . . . . . . . . . . . . . . 43Figure 4.1 Illustration of data structure and methods for UGW-based damage localization: (a) available low and highdimensional data, (b) conventional method with low-dimensional data used, (c) current DL-based methodwith high dimensional data used, and (d) proposedDL-based with multidimensional data incorporated.© Hongguang Yun . . . . . . . . . . . . . . . . . . . . 48Figure 4.2 The architecture of proposed multidimensional datafusion neural network framework. © Hongguang Yun 49Figure 4.3 The 1D convolutional block used in the encoder mod-ule. © Hongguang Yun . . . . . . . . . . . . . . . . . 50Figure 4.4 Illustration of CFRP plate with eight PZT transduc-ers installed. © Hongguang Yun . . . . . . . . . . . . 55Figure 4.5 Training loss of models with and without optimizingthe pi and bi. All four models were trained 150 epochswith pi initialized as 1 and bi initialized at eight dif-ferent scales of standard deviations Ï: (a) Ï = 0I1,(b) Ï = 1, (c) Ï = 2, (d) Ï = M, (e) Ï = 10, (f)Ï = 1N, (g) Ï = 32, (h) Ï = N4. © Hongguang Yun . . 58Figure 4.6 Training MND (mm) of models with and without op-timizing the pi and bi. All four models were trained150 epochs with pi initialized as 1 and bi initializedat eight different scales of standard deviations Ï: (a)Ï = 0I1, (b) Ï = 1, (c) Ï = 2, (d) Ï = M, (e) Ï = 10,(f) Ï = 1N, (g) Ï = 32, (h) Ï = N4. © Hongguang Yun 59Figure 4.7 Training and testing MND (mm) results of the pro-posed model with bi sampled from a Gaussian distri-bution at a different scale of standard deviation. ©Hongguang Yun . . . . . . . . . . . . . . . . . . . . . 60xviiFigure 5.1 Illustration of learned feature space of different deeplearning paradigms. (a) Standard deep learning: mod-els are trained for source domain data fitting with-out considering maximizing mutual feature distance.(b) Proposed domain generalization methods. Modelperformance in the target domain is improved by pullinginter-class and intra-class features away from eachother in the source domain as much as possible. ©Hongguang Yun . . . . . . . . . . . . . . . . . . . . . 70Figure 5.2 Illustration of the WaveCLR architecture with a dual-pathway in the training process. © Hongguang Yun . 72Figure 5.3 Illustration of traditional and proposed contrastivelearning structure and training mechanism: (a) tra-ditional contrastive learning structure, (b) learningmechanism of traditional contrastive learning, whichsimplifies multi-class classification to multiple binaryclassification (c) proposed WaveCLR structure, and(d) momentum update in proposedWaveCLR to max-imize mutual inter-class distance. © Hongguang Yun 74Figure 5.4 The composite plate being inspected with eight trans-ducers mounted and three disk damages. © Hong-guang Yun . . . . . . . . . . . . . . . . . . . . . . . . 77Figure 5.5 Configuration of 12 transducers and 28 different dam-age locations [2]. Used with permission from Sage. . . 78Figure 5.6 Illustration of ultrasonic guided waves of baseline anddamage signal from LUGW dataset with one trans-mitter and six receivers. © Hongguang Yun . . . . . 80Figure 5.7 Illustration of ultrasonic guided waves of baseline anddamage signal from NUGW dataset with one trans-mitter and seven receivers. © Hongguang Yun . . . . 80Figure 6.1 Illustration of the ellipse method for damage localiza-tion: transitioning from a single transmitter-receiverpath to multiple paths. © Hongguang Yun . . . . . . 90Figure 6.2 Architecture of the proposed physics-informed neuralnetworks for damage localization. © Hongguang Yun 92Figure 6.3 Illustration of the standard recurrent block in recur-rent neural networks. © Hongguang Yun . . . . . . . 92xviiiFigure 6.4 Illustration of forget, input, and output gates mecha-nism in the long short-term memory neural networks.© Hongguang Yun . . . . . . . . . . . . . . . . . . . . 94Figure 6.5 Architecture illustration of the feature extractor ofthe proposed physics-informed neural networks. ©Hongguang Yun . . . . . . . . . . . . . . . . . . . . . 95Figure 6.6 Architecture illustration of the location solver of theproposed physics-informed neural networks. © Hong-guang Yun . . . . . . . . . . . . . . . . . . . . . . . . 96Figure 6.7 Illustration of the training and testing data split ofLUGW dataset [2]. Reproduced with permission fromSage. . . . . . . . . . . . . . . . . . . . . . . . . . . . 98Figure 6.8 Illustration of the actual damage location and pre-dicted location using ellipse method on NUGW dataset.© Hongguang Yun . . . . . . . . . . . . . . . . . . . . 103Figure 6.9 Illustration of the actual damage location and pre-dicted location using proposed PINNmodel on NUGWdataset. © Hongguang Yun . . . . . . . . . . . . . . . 104Figure 6.10 Illustration of the actual damage location and pre-dicted location using ellipse method on LUGW dataset.© Hongguang Yun . . . . . . . . . . . . . . . . . . . . 106Figure 6.11 Illustration of the actual damage location and pre-dicted location using proposed PINNmodel on LUGWdataset. © Hongguang Yun . . . . . . . . . . . . . . . 107Figure 7.1 Illustration of the distribution of RFF projected co-ordinates using t-SNE. © Hongguang Yun . . . . . . 132Figure 7.2 t-SNE visualization of latent feature spaces of keycomponents in WaveCLR model. Features with dif-ferent labels are distinguished by different colours.(a) Shows the ResNet18 model as the baseline. (b)shows the model with standard contrastive learningbut without stratified sampling. (c) shows stratifiedsampling used on the top of the model (b). (d) themodel with momentum update shows the maximizedmutual inter-class distance. (e) and (f) shows the fea-ture space with the complete WaveCLR model withand without a two-stage training strategy. It is bestviewed in colour. © Hongguang Yun . . . . . . . . . . 135xixFigure 7.3 Illustration of the front and back view of the area in-troduced by an 19J impact of plate #1. © HongguangYun . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137Figure 7.4 Illustration of the cross-section view showing the depthdimension after cutting plate #1. © Hongguang Yun 138Figure 7.5 Detailed illustration of the internal damages: delam-ination and matrix cracks. © Hongguang Yun . . . . 139xxList of Abbreviation1DCNN 1D Convolutional neural networks. 492DCNN 2D Convolutional neural networks. 61AE Acoustic emission. 13BVID Barely-visible impact damage. 2CFRP Carbon fibre reinforced polymer. 11CNN Convolutional neural networks. 3CWT Continuous wavelet transform. 3DNN Deep neural networks. iiiDoF Distance-of-flight. 91DWT Discrete wavelet transform. 3EMD Empirical mode decomposition. 18FCNN Fully-connected neural networks. 3xIxIsI Independent and identically distributed. iiiIRT Infrared thermography. 13LSTM Long short-term memory. 93LUGW Composite damage inspection dataset usinglinear ultrasonic guided wave testing tech-nique. 33MMD Maximum mean discrepancy. 82MND Mean normalized distance. 54NDT Non-destructive testing. 1NTK Neural tangent kernel. 52NUGW Composite damage inspection dataset usingnonlinear ultrasonic guided wave testing tech-nique. 32OOD Out-of-distribution. iiiPINN Physics-informed neural networks. 9PZT Piezoelectric transducer. 29RAPID Reconstruction algorithm for probabilistic in-spection of defects. 3RFF Random Fourier feature. 48xxiRNN Recurrent neural network. 92SHG Second harmonic generation. 27SHM structural health monitoring. 1SOTA State-of-the-art. 60STFT Short-time Fourier transform. 3ToF Time-of-flight. 3t-SNE t-Distributed stochastic neighbour embed-ding. 131TVDI Time-varying damage index. 60UGW Ultrasonic guided wave. iiiUT Ultrasonic testing. 1WVD Wigner-Ville distribution. 18xxiiAcknowledgementsI am deeply grateful to everyone who supported me throughout my Ph.D.journey. Foremost, I extend my heartfelt thanks to my supervisor, Dr.Zheng Liu, for his unwavering guidance and support during this challengingjourney. His expertise, patience, and encouragement have been invaluableto my development as a researcher and have significantly contributed to thesuccess of this thesis.My gratitude also goes to Dr. Shashank Pant for his invaluable adviceand assistance throughout my research. I am equally thankful to MarcGenest, who significantly contributed to my experimental work and becamea dear friend during my time in Ottawa. Special thanks to Dr. Lucy Li fromthe National Research Council Canada for providing the composite platesfor my research and for imparting her extensive knowledge of compositematerials.I am also grateful to my supervisory committee member, Dr. AbbasMilani, my external examiner, Dr. Yuu Ono, and university examiner, Dr.Hadi Mohammadi, for their contributions to my defence.I want to acknowledge all my colleagues at the Intelligent Sensing, Diag-nostics, and Prognostics Research Lab (ISDPRL) for their invaluable sup-port.Special thanks are owed to my parents, who have supported me through-out my years of education.xxiiiDedicationThis dissertation is dedicated to my wife, for endless patience, under-standing, and love that motivates me to be the best version of myself.xxivChapter 1Introduction1.1 Background and MotivationPlate-like composite structures are widely used in aerospace [3â5], windturbines [6â8], and shipbuilding [9], etc. Specifically, composite materialsare increasingly used in aircraft structures due to their superior materialproperties over metallic structures, such as a higher strength-to-weight ratioand better corrosion resistance [10]. The Boeing 787 airplane is composedof more than 50% composite materials in its structural components [11].Likewise, composite materials for aircraft also constitute a major industrywithin Canadaâs economy, such as aircraft industry in Winnipeg, Canada[12]. As part of the Canadian aerospace industry, they contributed closeto 27 billion Canadian dollars to the Gross Domestic Product and providedover 212,000 jobs to the Canadian economy [12, 13].Notwithstanding their numerous advantages, composite materials ex-hibit certain vulnerabilities. Specifically, they are prone to damage causedby foreign object impacts, time-varying loads, and harsh operational envi-ronments. Such conditions can lead to internal damages that might remainminimally visible on the surface, as depicted in Figure. 1.1 [14, 15]. Suchinvisible damage increases the likelihood of requiring hangar or dispatchchecks, which are major drivers of maintenance costs in aircraft operations[16]. Furthermore, the continued development of unidentified internal dam-age can cause significant structural damage and pose additional safety haz-ards. Consequently, minimizing maintenance costs and ensuring the safetyof operations necessitate a significant focus on damage localization in thestructural health monitoring (SHM) of composite materials, which is theinitial and critical step in the following assessment process [17, 18].Non-destructive testing (NDT) is crucial for monitoring the conditionof composite structures without causing additional damage [19]. There areseveral NDT techniques, such as visual inspection, ultrasonic testing (UT),infrared thermography, laser shearography, etc. Ultrasonic waves can be di-vided into bulk waves and guided waves. Longitudinal and shear waves arethe two modes of bulk waves that are widely used in UT. The particle oscil-1Figure 1.1: Sectional view of delamination damage within a multi-layercomposite plate after impact. © Hongguang Yunlation direction of the longitudinal and shear wave is parallel and transverseto the wave propagation direction, respectively. Guided wave on the otherhand consists of three modes - Rayleigh, shear-horizontal, and Lamb waves.Rayleigh wave is also known as a surface wave for propagating near the freesurface and has been used for surface damage detection. Shear-horizontalwave is particularly used in damage characterization in adhesive bonds andlaminated composite plates [20â22]. Lamb wave, which is made up of a su-perposition of longitudinal and shear modes is the most widely used guidedwave. Although Lamb wave is multi-modal and dispersive, it provides anattractive solution for detecting damage in thin plates and shell-like struc-tures, as well as, for its ability to propagate over long distances covering alarger inspection area.In this study, the UGW carrying the Lamb waves is selected to realizethe damage localization for several reasons. First, unlike point-by-point in-spection methods such as infrared thermography, UGW can propagate longdistances over a plate-like structure while interrogating the entire thickness[23]. Secondly, UGW also allows for on-site application, minimizing down-time and further reducing costs [16]. Thirdly, UGW is particularly effectivefor detecting internal damage, making it ideal for locating barely-visible im-pact damage (BVID) in composites. Finally, when combined with nonlin-ear UT techniques, UGW can detect micro-damage, such as delamination,2enabling early damage localization to prevent major damage progression[24â27].Therefore, this study aims to locate damages in composite materialsusing UGW. As UGW propagates through the specimen, damage in com-posite materials leads to linear or nonlinear changes to UGW signals, such asamplitude reduction [28] and changes in the time-of-flight (ToF). Two tradi-tional methods, Reconstruction algorithm for probabilistic inspection of de-fects (RAPID) [29] and the ellipse method [30], have been developed basedon these features to locate damages. The RAPID method focuses on con-structing a mathematical model for damage localization based on amplitudechanges in UGW. The core of ellipse method is to extract damage-relatedwave packets using traditional signal processing techniques to obtain ToFs.Time-frequency transform methods such as the short-time Fourier trans-form (STFT) [31, 32], singular value decomposition [33], and continuouswavelet transform (CWT) [34] have been applied to extract ToF. Althoughsignificant progress has been made, conventional methods still face manychallenges in research and practice. First, they require expert knowledgein signal processing and an understanding of complex wave propagation incomposites. Then, for RAPID algorithm, building mathematical models of-ten involves assumptions about damage distribution, potentially introducingbias and affecting the robustness and generalization of the methods. Like-wise, for ellipse method, the multi-modal and dispersive nature of UGWcomplicates signal analysis, making it difficult to isolate damage-inducedwave packets from noise.Consequently, there is growing interest in applying deep learning tech-niques for analyzing wave signals through DNN without the need for signalprocessing steps [35]. Fully-connected neural networks (FCNN) are one ofthe earliest, simplest, and most effective deep learning structures and havebeen widely employed in damage identification with UGW. FCNN mod-els relies on using effective hand-crafted features to locate damages, suchas dominant eigenvalues of UGW matrix [36], time domain statistics [37],and ToFs [38]. Another deep learning architecture that has been extensivelystudied for damage identification uses convolutional neural networks (CNN).CNN-based models also requires signal processing techniques to provide in-put features, such as discrete wavelet transform (DWT), discrete Gabortransform, and CWT [39, 40]. Zhang et al. [41] proposed a novel time-varying feature, defined as the energy ratio of the current signal to the base-line signal, to enhance damage localization. The proposed feature extractedfrom multiple transmitter-receiver paths is input into a multi-headed CNNmodel. The model effectively identified damages on an aluminum plate3and demonstrated its significant superiority over traditional ellipse-baseddamage localization methods. In addition to FCNN and CNN, other deeplearning methods have also been explored. Wang et al. [42] introduced anovel graph-in-graph convolutional network to enhance damage detectionand localization using UGW signals. This was achieved by constructingspatial-temporal feature representations and a global graph to highlight dif-ferences among signals, thereby achieving superior accuracy and stability,even with limited and imbalanced datasets. Zhang et al. [43] developedan interpretable prototypical network for damage localization with embed-ding an attention module into the traditional CNN block to feature superiorgeneralization ability.While numerous studies have adopted deep learning methods for damagelocalization with UGW, most simply apply widely recognized DNN modelswithout addressing the unique aspects that differentiate them from con-ventional deep learning applications, such as image recognition and naturallanguage processing. However, there are relatively few DNN-based investi-gations that are specifically tailored to the unique characteristics of damagelocalization using UGW. This situation presents two major research chal-lenges in advancing DNN models for UGW-based damage localization.One significant challenge in damage localization is the relatively smalldataset sizes compared to those in established deep learning applicationssuch as image recognition. For instance, ImageNet [44] provides over 14million labelled images for training, while datasets for composite damagelocalization often suffer from limitations due to the high costs and time-consuming nature of data collection. This gap substantially affects the ap-plicability of deep learning, as conventional models presuppose that train-ing and testing data are identically and independently distributed (xIxIsI),a condition that is frequently unmet in damage localization scenarios. Thescarcity of data can lead to discrepancies in data distribution between train-ing and testing stages, potentially impairing the performance of DNN mod-els.Another challenge is the integration of physical knowledge into the de-sign of deep learning models. For example, traditional approaches in com-puter vision employ convolution operations, leveraging the shift-invarianceproperty for image recognition, which allows CNN to identify images evenwhen they do not precisely match their training examples. Likewise, dam-age localization involves different and more intricate physical constraints,constituting an ill-posed problem where minor variations in input can re-sult in substantial discrepancies in output. In such scenarios, it becomesimperative to infuse more physical knowledge into the model to steer its be-4haviour, ensuring accurate interpretation of changes and precise predictionof damage localization.To advance DNN-based damage localization using UGW, it is crucial toaddress the challenges associated with data availability and the embeddingof physical knowledge. Consequently, this thesis outlines three applicationscenarios that consider both data availability and the requirements for em-bedding physical knowledge. These scenarios offer a comprehensive view onleveraging DNN models for effective damage localization using UGW.1.2 Research Gaps and ObjectivesDNN models, inherently data-driven, are capable of learning predictivefunctions with minimal prior knowledge by being tailored directly to train-ing data. That is, when the xIxIsI requirement is satisfied, minimal physicalknowledge embedding is necessary. In essence, there exists a trade-off be-tween data requirements and the need for embedding physical knowledgein model construction. Specifically, the availability of data determines thediscrepancy in data distribution, thereby influencing the level of physicalknowledge required to achieve satisfactory performance in the testing phase.As illustrated in Figure. 1.2, as the availability of training data decreases,the difference between training and testing data correspondingly increases.Consequently, the need for embedding more physical knowledge into themodel becomes increasingly significant. Without incorporating additionalprior knowledge, there is a risk of overfitting to the training data, which canresult in performance drop on testing data. Embedding more physical pri-ors enables the model to acquire actual, relevant knowledge during training,enhancing its performance on testing data.To this end, this study aims to advance DNN-based damage localizationusing UGW by addressing two primary challenges: effective data utilizationand the integration of physical knowledge into DNN model construction.Taking into account the unique characteristics of DNN models and the de-mands of industrial applications, this research explores three applicationscenarios to thoroughly examine these challenges. These scenarios are dif-ferentiated by the disparity in data distribution between the training andtesting phases. Specifically, the research gaps identified within these threescenarios provide a structured overview of the current state of the field.1. Scenario 1xIxIsI damage localization. Under the assumption of (xIxIsI) betweentraining and testing data, it is presumed that both sets of data follow5Figure 1.2: Illustration of relation of data availability and distribution dif-ference between training and testing data. © Hongguang Yunthe same distribution. As illustrated in Figure. 1.3, considering dam-age occurred on multiple locations of a specimen, multiple measure-ments at each location constitute the complete dataset. This datasetis then randomly divided into training and testing sets. This approachto data distribution is commonly used in deep learning applications.Research gap. The xIxIsI assumption reduces the necessity of in-tegrating explicit physical knowledge, given the DNN modelâs strongcapacity for data fitting. Thus, the primary research gap concernshow to incorporate all input data within the context of damage lo-calization using UGW. Traditional DNN models are adept at process-ing high-dimensional data, such as time-series wave signals. However,for damage localization, it is essential not only to handle the high-dimensional UGW signals but also to integrate the low-dimensionaltransducer location data. Existing DNN-based studies have failed toincorporate transducer locations into the model effectively. Moreover,directly combining low-dimensional with high-dimensional data couldhinder the modelâs ability to converge. Therefore, developing a DNNmodel that can manage multi-dimensional input data for damage lo-6calization using UGW is imperative.2. Scenario 2Domain generalized damage localization. Discrepancies in datadistribution between training and testing data can arise in practicalapplications. These discrepancies may result from changes in inspec-tion configuration or damage conditions. For instance, as depicted inFigure. 1.4, the training dataset is composed of damages of size A lo-cated at multiple positions. In practice, the size of the damage couldvary to size B. This variance can lead to variations in the input data,even for the same location.Research gap. With limited data available for training, a DNNmodel is prone to overfitting during the training phase. Ignoring dis-crepancies in data distribution can lead to a drop in performance onthe testing data. Therefore, it is essential for the model to exhibitstrong resilience against such discrepancies.3. Scenario 3OOD damage localization. This scenario represents a more gener-alized application, frequently encountered in industrial settings, wherethe testing data is OOD in comparison to the training data. As il-lustrated in Figure. 1.5, the damage is located in the blue area fortraining, while during the testing phase, damage could occur in thered area. In essence, the testing data are completely unseen by theDNN model in comparison to the training data.Research gap. The OOD task presents a significant challenge toDNN models. As typical data-driven approaches, DNN models insuch scenarios cannot merely learn to fit the data; they must alsoinfer the inherent physical principles underlying the model. However,unlike large language models that acquire linguistic knowledge fromextensive datasets, the availability of data for training in the contextof damage localization using UGW is markedly limited.The overall objective is to construct DNN models to locate damagesunder regarding the specific application requirements. To attain this goal,training the model to excel with the training data is essential, but it is alsocrucial for the model to effectively handle discrepancies between the trainingand testing data. Furthermore, in addressing these three research gaps, thisstudy aims to offer a comprehensive solution to this challenge across threeapplication scenarios.7Figure 1.3: Illustration of training and testing data split format follow xIxIsIprinciple. © Hongguang YunFigure 1.4: Illustration of training and testing data split format follow do-main discrepancy principle. © Hongguang YunFigure 1.5: Illustration of training and testing data split format follow OODdistribution principle. © Hongguang Yun81. Objective 1xIxIsI damage localization. To accurately locate damage, this studyaims to develop a multi-dimensional data fusion DNN model capableof efficiently integrating high-dimensional UGW signal data with low-dimensional transducer location data.2. Objective 2Domain generalized damage localization. To mitigate the datadistribution discrepancies between training and testing data, this studyendeavors to create a domain-generalized DNN model utilizing con-trastive learning. This approach aims to diminish the effects of distri-bution discrepancies by enhancing the modelâs resilience to variationsin input data. The DNN model is expected to exhibit robustnessagainst such discrepancies, even when trained on limited data.3. Objective 3OOD damage localization. To compensate for the scarcity of train-ing data and to learn the latent physical principles governing damagelocalization, this study aims to construct physics-informed neural net-works (PINN). The proposed PINN model is designed to integrateunderlying physical laws directly into the learning process, therebyenhancing the modelâs ability to generalize from limited data by lever-aging physical knowledge.1.3 Thesis Structure and OrganizationThis thesis is comprised of seven chapters and one appendix, as shownin Figure. 1.6. Following the introduction is a literature review, and theexperiment design and procedure for composite damage inspection usingUGW are presented. Then, three chapters correspond to the three scenar-ios are described. The thesis concludes with a chapter on conclusions andfuture work. Additionally, an appendix chapter is included at the end. Thecontents of each chapter are summarized as follows:â Chapter 1 outlines the research background and motivation behindusing DNN-based approaches for damage localization in compositematerials with UGW. And it further specifies the research objectives,which include developing deep learning methods tailored to the uniquechallenges of damage localization using UGW.9Figure 1.6: Flow diagram of the thesis structure. © Hongguang Yun10â Chapter 2 conducts a literature review on NDT techniques for damagelocalization, then explores traditional UGW methods. It concludeswith a review of DNN approaches for damage localization, categorizedinto three scenarios based on data availability and physical knowledgeintegration.â Chapter 3 describes the composite damage inspection procedure usingUGW on a carbon fibre reinforced polymer (CFRP) plate. To examineUGW signal behaviour for composite damage localization, an impactdamage inspection experiment using UGW and the nonlinear UGWtechnique is conducted. Additionally, a public damage localizationinspection using the linear ultrasonic testing technique is introducedto demonstrate the generalization ability of the DNN methods. Thesetwo inspections provides the dataset for the validation of the DNNmodels for damage localization. In addition, UGW signal visualizationand analysis in time domain and frequency domain are conducted toprovide the evidence of using UGW to locate composite damages.â Chapter 4 presents a multi-dimensional data fusion neural network de-signed for damage localization. It introduces a novel approach usinga random Fourier projection head to integrate low-dimensional datainto the model, aiding in accelerating model convergence. Experi-mental results affirm that the proposed model enhances localizationaccuracy beyond that of models solely utilizing UGW signals.â Chapter 5 introduces a domain generalization neural network thatemploys contrastive learning to maximize the distance between inter-class and intra-class features, thereby enhancing the modelâs toleranceto data distribution discrepancies resulting from input variations. Theeffectiveness of the proposed model is validated against two types ofinput variations: varying damage sizes and slight location variations.â Chapter 6 proposes a PINN model for OOD damage localization. Thedevelopment of the PINN model is inspired by the analytical ellipselocalization method, integrating substantial physical constraints intothe model. This enables the model to accurately locate damage thatis completely unseen during the training phase.â Chapter 7 concludes the dissertation by summarizing the entire re-search, highlighting the contributions made and outlining future work.This chapter encapsulates the findings, underscoring the advancement11in damage localization using UGW through the development of novelDNN models and methodologies.â The Appendices provides feature space visualizations of the key mech-anisms proposed in Chapters 4 and 5. Additionally, it includes thedestructive inspection results of the plate after impact, showing thedetails of the internal cracks and delamination damage in the inspectedcomposite structure.12Chapter 2Literature Review 1This chapter presents a comprehensive review of the literature on com-posite damage localization, examining various NDT techniques. The reviewstarts with imaging-based methods, such as infrared thermography (IRT)and laser shearography, followed by mechanical vibration-based methods in-cluding acoustic emission (AE) and UT. Subsequently, the chapter explorestraditional approaches employing UGW for damage localization. The focusthen shifts to DNN-based methods for localizing composite damage usingUGW, discussed within the three application scenarios outlined in Chapter1.2.1 Nondestructive Testing Techniques forComposite Damage Localization2.1.1 Infrared ThermographyIRT is a technique based on thermal radiation, captured by an infraredcamera emitted from the specimenâs surface. Damage presence alters thematerialâs thermal behaviour, creating localized variations in infrared emis-sion detectable through thermography measurements. As an imaging-basedNDT technique, IRT provides a straightforward approach to damage lo-calization, essentially involving scanning the specimen. Consequently, theeffectiveness of using IRT hinges on the ability to distinguish damage-relatedemissions.IRT can be operated in either a passive or an active mode. Passive IRTdirectly measures the surface temperature for evaluation, while active IRTmeasures the surface temperature after applying some form of thermal ex-citation. Passive IRT is typically used for materials that are not thermallybalanced [45â47]. Active thermography is the most widely used technique1A version of chapter 2 has been published. Yun H, Rayhana R, Pant S, GenestM, Liu Z. Nonlinear ultrasonic testing and data analytics for damage characterization:A review. Measurement. 2021 Dec 1. I conducted all the literature investigation andmanuscript writing.13for NDE of composite aerospace parts [48, 49]. Xie et al. [50] explored thedetection of impact damage in sandwich structures using IRT after impactsby a hemispherical-nosed head at energies from 22 to 77 J, demonstratingthat IRT can effectively identify damage typesmatrix cracking, delamina-tion, and fibre breakagethrough distinct temperature distribution patterns.The research also examines how impact energy, fibre thickness, and stack-ing sequence influence damage, suggesting that IRT can detect subsurfacedefects, offering insights into the damage dynamics of these structures. Weiet al. [51] employ DNN and infrared thermography to detect and segmentimpact damage in thermally stressed, curved CFRP plates. By analyzingmid-wave and long-wave infrared sequences from pulsed thermography, theproposed method can identify localized damage with the segmentation abil-ity of the used DNN models. Efforts to develop a compact IRT system tomake it more feasible in application have also been made. Farmaki et al.[52] proposed a novel thermographic strategy that merges two active in-frared techniques for rapid analysis of damage in aircraft-grade composites,promising substantial maintenance cost and time reductions. This method,equipped with an ultra-compact module for autonomous NDT inspection,demonstrates high precision and efficiency in assessing aircraft fuselage andwing skin parts. Since IRT is a point-by-point technique, to enable fasterinspection, researchers are trying to combine it with other NDT techniques.Padiyar et al. [53] proposed combining the phased array technique withIRT to characterize BVID on aircraft structures. Balasubramaniam et al.[54] employ a global-local strategy combining the UGW technique with IRT,where UGW is utilized to locate the damage and IRT is used to quantifythe damage in composite structures.IRT offers the advantage of not requiring physical contact with the spec-imen, facilitating the inspection of parts with irregular surfaces where sensorinstallation is challenging. However, IRT as a point-by-point scanning tech-nique, it is inefficient for inspecting large structures. Additionally, IRTâseffectiveness is sensitive to environmental conditions and the configurationof the instrumentation, requiring significant expertise for field application.Lastly, IRT has difficulty in identifying deep internal damages due to theminimal heat fluctuations produced compared to surface damage [55].2.1.2 Laser ShearographyLaser Shearography is another imaging-based NDT technique, where en-abling by a laser-speckle shearing interferometry. A laser illuminates thesample, and its image is captured by a charge-coupled device camera through14a beam shearing element, creating laterally shifted images or shearograms.These shearograms, which are interferograms showing deformation changesunder different loads, are analyzed for variations in the interference patterns.These patterns reveal the rate of deformation change, helping to identify sur-face and subsurface defects based on the disturbances in the fringe patterns,thus enabling the classification of various defects.Similar to IRT, laser shearography is another point-by-point NDT tech-nique that requires scanning the specimen to identify damage. The essence ofusing laser shearography for damage localization lies in enhancing the sensi-tivity and stability of damage detection. Existing studies on laser shearogra-phy for damage identification focus on developing different loading systems,which are crucial for improving damage detection capabilities. Lopes et al.[56] introduce two speckle shearography systems for assessing damage inlaminated composite plates, employing stroboscopic and double pulse laserilluminations for temporal and spatial phase modulations, respectively. Anovel differentiation approach for phase maps before post-processing signif-icantly improves damage localization, especially with the first system, byemphasizing the fourth-order spatial derivative of mode shapes for accuratedamage detection.The advantages of shearography include its reduced susceptibility tonoise compared to other NDT methods and its economic benefits, such ashigh inspection rates measured in m2Lstr. It is particularly effective in in-specting honeycomb and foam composite structures [57]. However, a signifi-cant limitation is its difficulty in detecting defects other than delamination.Shearographyâs sensitivity to specimen movement can result in false damageidentification and a lack of resilience to uncertainties. Moreover, it gener-ally identifies damage only up to a depth of 23 mm [58, 59]. Like IRT, lasershearography is often combined with other NDT techniques. Zhang et al.[60] proposed to synchronize the stroboscopic laser with UGW to markedlyenhance defect imaging quality, providing a significant improvement overother imaging-based NDT techniques.2.1.3 Acoustic EmissionThe occurrence and development of damage, such as fibre breakage, ma-trix micro-cracking, and delamination, will generate mechanical vibrationsin the material, resulting in stress waves [61, 62]. Unlike other active NDTtechniques, AE testing is a widely used passive NDT technique that identi-fies damages by monitoring the release of stress waves from localized sourceswithin the specimen. With multiple sensors installed on the specimen, AE15can locate damage similarly to UT.The key of damage localization using AE lies in precisely identifyingthe ToF of AE waves induced by damage. AE has been proven effective inidentifying damage in plates with complex geometries [63] and honeycombsandwich structures [64, 65]. Eaton et al. [66] utilized Delta T mapping totrain a model empirically for ToF determination of AE signals. Sikdar etal. [67] improved ToF extraction using a particle swarm optimization algo-rithm, while Li et al. [68] enhanced ToF calculation accuracy within highoperational noise using empirical wavelet transform, Akaike information cri-terion, and genetic algorithm, validating their method by identifying cracksin steel-concrete composite beams. Additional signal processing techniqueslike wavelet packet transform [69] and CWT [70] have also been exploredto refine ToF extraction accuracy. Moreover, there have been endeavours toautomate damage localization using AE signals through deep learning meth-ods, including CNN [71, 72], deep transfer learning [73], and unsupervisedlearning [74], exhibiting the broad techniques enhancing AEâs application indamage detection.AE has various advantages of quick inspection rate and high sensitivity.However, the AE has limitation on identify small damage [62]. In addition,using AE requires great expertise to identify damage onsite [59].2.1.4 Ultrasonic TestingUltrasonic waves can be divided into bulk waves and guided waves. Lon-gitudinal and shear waves are the two modes of bulk waves that are widelyused in UT. The particle oscillation direction of the longitudinal and shearwave is parallel and transverse to the wave propagation direction, respec-tively. Guided wave on the other hand consists of three modes - Rayleigh,shear-horizontal, and Lamb waves. Rayleigh wave is also known as a surfacewave for propagating near the free surface and has been used for surfacedamage detection. Shear-horizontal wave is particularly used in damagecharacterization in adhesive bonds and laminated composite plates [20â22].Lamb wave, which is made up of a superposition of longitudinal and shearmodes is the most widely used guided wave. Although Lamb wave is multi-modal and dispersive, it provides an attractive solution for detecting damagein thin plates and shell-like structures, as well as, for its ability to propagateover long distances covering a larger inspection area. In this study, the UGWcarrying the Lamb waves is selected to realize the damage localization.The linear UT techniques utilize dissipation of wave energy due to atten-uation, reflection, and change of phase to characterize damages [75]. How-16ever, these linear features are insensitive to microscopic damages such asmicro-cracks and delamination [76]. This is because the physical mecha-nism of micro damage wave interaction is full of nonlinearity [77]. A majordifference between linear UT and nonlinear UT is that in the latter thepresence of damages would generate new frequency harmonics [78].2.2 Ultrasonic Guided Wave for DamageLocalization2.2.1 Hand-crafted Feature Extraction for DamageLocalizationTo accurately localize damage using UGW, it is essential to extract fea-tures from the wave signals that are indicative of damage. Over the pastdecade, two features have emerged as the most effective and extensivelystudied: the ToF and amplitude attenuation. The ToF refers to the propa-gation time of the wave packet from the transmitter, through the damaged,and then to the receiver. These wave packets can either be the scatteredversion of the original wave or waves newly induced by the damage, owingto the nonlinearity of the guided waves and the materials involved. Underthe assumption that a baseline signal (collected in the absence of damage)subtracts from the received wave signals, it is typically presumed that anyobserved signals are exclusively induced due to damage and physical bound-aries. In the method based on amplitude attenuation, it is assumed thatdamage absorbs energy from the guided waves, leading to a reduction in theamplitude of the received signals. By assessing the level of reduction, thelocation of the damage can be approximately determined.ToF-based Method The most straightforward and effective method basedon ToF is known as the ellipse method. The core equation describes thismethod is detailed in Equation 2.1. By extracting the ToF from the UGWand calculating the wave propagation velocity, the travel distance for atransmitter-receiver pair can be established. This travel distance sets thepotential damage location along an elliptical trajectory. The precise lo-cation of the damage can be identified by the intersection point of threeellipses, derived from at least three ToF measurements from three differenttransmitter-receiver pairs. Kehlenbach and Das [30] utilized this methodwith three transmitter-receiver pairs in a pitch-catch configuration to local-ize damages. Additionally, researchers employing pulse-echo configurations17have validated the effectiveness of this approach [79, 80]. Lu et al. [81]enhanced the solution process for Equation 2.1 by implementing a nonlinearleast squares Gauss-Newton optimization algorithm.ci =â(xs â xit)2 + (ys â yit)2eg(f)+â(xs â xir)2 + (ys â yir)2eg(f)(2.1)One of the major factor deciding the accuracy of the ToF-based methodfor damage localization is the accuracy of ToF extraction from the signal.Since the guided waves has the nature of dispersion and multi-modal, thefrequency and waveform of the wave packets will change during the prop-agation. It is usually required to use time-frequency transform techniquesto extract the first damage-induced wave packets from the complex wavesignals. Researchers have investigated the effectiveness of using STFT, Em-pirical Mode Decomposition (EMD), Wigner-Ville distribution (WVD), andCWT [82â84]. Niethammer et al. [82] investigated the effectiveness of fourtime-frequency transform techniques to analyze UGW signals. They foundthat the WVD is not capable of representing UGW signals in the time-frequency domain with a good resolution. Compared with Wavelet analysis,EMD does not require a base signal to be selected in advance. Ara etal.[85] combined EMD and DWT to extract non-redundant features. Al-though EMD has achieved good results in other fault diagnosis fields, it hasa problem of mode confusion, which limits its application in ultrasonic signalpre-processing. Cantero-Chinchilla et al. [86] stated that the CWT providesan overall better performance than EMD, WVD, and STFT on calculatingToF data of UGW signals.Many researchers aim to utilize additional signal processing methods toestimate the ToF. For instance, mode decomposition methods have been ap-plied to enhance the precision of extracting damage-induced ToF from com-plex wave packets [87]. Sun et al. [88] proposed a novel frequency-mixed,point-focusing guided wave electromagnetic acoustic transducer, accompa-nied by signal processing techniques including variational mode decomposi-tion and CWT. The proposed method enables the accurate determination ofdamage positions, plate thicknesses, and the classification of damage typeswith a high identification accuracy of approximately 95%, without the needfor sophisticated feature engineering.Amplitude Attenuation-based Method RAPID algorithm is a promi-nent method that relies on amplitude attenuation [29] to identify damage.18It employs correlation analysis to measure changes in guided wave signalsunder normal and damage-appeared conditions. Damage probability is es-timated by analyzing the severity of signal changes across different sensorpairs, building a mathematical model upon the attenuation and how dam-age affect signal paths. Thiene et al. improved the RAPID algorithm byintroducing an optimal sensor placement algorithm aimed at maximizingarea coverage within a sensor network, specifically for complex structureslike aircraft panels [89]. It incorporates physical and geometrical consider-ations of UGW propagation, including amplitude attenuation. Utilizing agenetic algorithm, the approach optimizes sensor placement for high damagelocalization probability, independent of damage characteristics, showcasingits adaptability to complex composite structures. Deng et al. [90] improvedthe RAPID algorithm by adjusting the RAPID methodâs scaling parameterand assigning weights to critical detection paths based on features extractedfrom UGW signals using machine learning models. The improved methodachieve better damage localization results compared to the original RAPIDalgorithm. However, the proposed method only validated on simulation sig-nals.2.2.2 Probabilistic Feature Extraction for DamageLocalizationThere is another avenue of study where researchers aim to use proba-bilistic methods to improve the estimation of the ToF. Bayesian approachesare capable of reducing uncertainties from measurements and time-varyingconditions, thus making it possible to accurately estimate the ToF. Uncer-tainties in the ToF extractor can arise from:1. Dispersion effects: Guided waves have dispersive and multi-modalin nature, which may cause mode conversion and overlap in the re-ceived wave signals. Mode conversion and overlap would affect thecalculation of ToF data.2. Heisenberg principle: When time-frequency transform is used toextract the wave packet scattered by damage, the Heisenberg principlewould affect the accuracy of the ToF data estimation.Yan [91] established a Bayesian model using the ToF data to updatethe probability density function of damage location and wave group veloc-ity. Numerical and experimental results showed that the proposed methodwas able to provide a more accurate estimation of the wave velocity and19damage location compared to the ellipse method. Ng [92] extended theestimation parameters to include damage length, depth, and location. Ahybrid particle swarm optimizer was used in the Bayesian model. The esti-mated damage length and depth errors are less than 2%. Fendzi et al. [93]further developed Yanâs [91] work to make it applicable in the anisotropicmaterials. He and Ng used the Bayesian approach to identify multiple de-lamination [94] and cracks [95] in beams and found that P( mode is moresensitive for identifying small crack (1 mm) than b( mode. To address theproblem of unappropriated selection of time-frequency transform methodsin calculating the ToF, Cantero-Chinchilla et al. [86] proposed to build aprobabilistic model to select the TF transform method automatically. Huoet al. [96] fuse the elliptical method and RAPID method based on theBayesian inference approach. The proposed method outperformed the tra-ditional elliptical method. However, the author did not provide comparisonresults of proposed methods with other Bayesian approaches. Zhao et al.[97] improved the uncertainty quantification by introducing Gibbs samplingto iteratively draw samples of damage related-parameters.Although significant progress has been made, traditional methods stillface many challenges in research and in practice. First, these methods re-quire expert knowledge of signal processing and complex wave propagationin composite structures. The multi-modal and dispersive nature of the UGWfurther complicates the received signals as it is difficult to extract a partic-ular wave packet induced by damage without interference from other noise.Building a mathematical model needs to manually assume the damage dis-tribution function, which may introduce prior bias and thus deteriorate therobustness and generalization ability.2.3 Deep Neural Networks for DamageLocalizationWith the rapid advancement of computing technology, deep learningmethods have become extensively applied in classification and pattern recog-nition fields. These models outshine at processing complex and multi-modaldata to find hidden indicators of damage. Unlike traditional approaches,most of deep learning models eliminate the need for delicate hand-craftedfeature extraction due to their powerful data handling capabilities. Recently,deep learning has attracted significant attention for damage localization us-ing UGW. This section reviews the most recent deep learning-based stud-ies across three application scenarios introduced previously: xIxIsI damage20Figure 2.1: Architecture of a fully-connected neural network with 5 layers.© Hongguang Yunlocalization, domain generalized damage localization., and OOD damagelocalization, highlighting their advancements and applications.2.3.1 Standard Deep Neural Network Damage LocalizationTraditional deep learning paradigms are based on the xIxIsI assumption,where training data (source domain) and testing data (target domain) arepresumed to follow the same distribution. The application of deep learningmodels to damage localization using UGW initially adhered to this assump-tion. Research in this field has progressed from the development of simplemodels to more complex ones, reflecting advancements in improving theaccuracy of damage location predictions.Fully-connected Neural Networks FCNN is one of the earliest, sim-plest, and effective deep learning structures and has been widely employedfor its capability in learning underlying systematic features. Figure. 2.1 isan illustration of a FCNN with one input layer, three hidden layers, and anoutput layer. Usually, a feature extraction step is needed before feeding thedata into the FCNN models. The extracted feature from the UGW signalcan either be in time domain and frequency domain.Sbarufatti et al. [36] defined features as several dominant eigenvalues ofa data matrix composed of wave signals collected from multiple transmitter-receiver paths. These features were used as inputs for the FCNN model topredict the damage location and length. The FCNN modelâs performance21in estimating damage location significantly depends on the crackâs length.With crack lengths of 5 mm and 10 mm, the average estimation error fordamage location coordinates can reach up to 30 mm. De Fenza et al. [37]proposed using features defined as the signal difference between damagedand undamaged specimens in the frequency domain as inputs for the FCNNmodel. This model could identify the damage location region in a plate-likestructure divided into 16 regions but failed to provide an accurate damagelocation coordinate. Feng et al. [38] utilized ToF data from three wavepropagation paths as inputs for an FCNN model to locate damage, achievingan average error of 2.8 mm in estimating the coordinates of 6 mm diametercircular damage. However, this algorithmâs limitation is its requirement fora large volume of ToF-location pair data for model training.Convolutional Neural Networks CNN is constructed with a hierar-chical structure to capture high-level features and to learn more complexmapping between input and output. The development of CNN boosts theapplication of computer vision and natural language processing, and alsoprovide a promising solution to the damage localization using UGW. A ba-sic CNN structure consists of convolutional layers, pooling layers, and fullyconnected layers. Figure. 2.2 illustrates the CNN structure proposed byLeCuN [98], which was well known as LeNet. The input of the LeNet is sup-posed to be 2D, such as images. The convolutional layer works as a filter,where the input from the previous layer is convolved with a kernel function.The kernel function is defined as a set of weights and scales the images bylocally multiplying its elements with the elements of the corresponding re-ceptive field of input. The output of the convolutional layer and poolinglayer is called feature-map.Following the LeNet, researchers have extensively developed the CNNwith many novel architectures. The main thrust in CNN innovation hasbeen made in depth and width exploration, feature-map exploitation, com-putation boosting, new block and link design, and attention mechanism [99].The most representative ones are AlexNet [100], VGG [101], and ResNet[102]. These CNNs are designed to process 2-dimensional data, such asimages. To use for damage localization with UGW. A natural way is touse time-frequency transform algorithm to convert time series wave signalsinto images in time-frequency domain. A new study conducted by Niri etal. [40] highlighted the superiority of the CWT as a method for convertingwave signals into the time-frequency domain, emphasizing its effectivenessin terms of both time and frequency resolution. Su et al. [103] reshaped22Figure 2.2: Architecture of LeNet and illustration of convolution operationand pooling operation. © Hongguang Yunthe 900 frequency points of the wave signals spectrum into 30 Ã 30 featurematrix as the input of a CNN model with two convolutional layers and twopooling layers. Features from three actuator-receiver paths were fed into theCNN model to locate a mass block. Although the CNN model was able toidentify and distinguish the size of the mass, it can only provide the regionof the mass on the plate instead of accurate coordinates.While preprocessing techniques allow for the conversion of time serieswave signals into images, this approach negates the inherent advantage ofCNN to leverage data characteristics without the need for a feature extrac-tion process. To address this, CNN models have been specifically developedto directly process time series wave signals, maintaining the efficiency anddepth of analysis offered by CNN without compromising their native capa-bility to analyze and learn from raw data directly. Munir et al. [104] builta basic CNN model, which consists of two convolution layers, one poolinglayer, and two fully-connected layers as shown in Figure. 2.3. The raw wavesignal was fed into the CNN model directly as an input. The authors alsobuilt a deep FCNN model with three hidden layers and three dropout layers.Hosseinabadi et al. [105] presented an effective wavelet network approachfor detecting structural damage location and severity using UGW signals,employing a three-stage algorithm to establish a fixed grid wavelet networkwith extracted damage-sensitive features as inputs and outperforming threeartificial neural network-based methods in both damage detection aspects.23Figure 2.3: Architecture of 1DCNN with basic convolutional, pooling, andfully-connected layer. © Hongguang YunRautela et al. [106] proposed to use a CNN to extract damage featuresfrom time-series wave signals to identify the damage on plate-like structures.However, their model can only estimate the rough region where the dam-age is present on the structure instead of providing accurate coordinates.It was reported that multi-headed CNNs, which feature multiple parallelCNN models, can learn high-level features more efficiently [107]. Zhang etal. [41] proposed a novel time-varying feature, defined as the energy ratioof the current signal to the baseline signal, to enhance damage localization.This feature, extracted from multiple transmitter-receiver paths, is inputinto a multi-headed CNN model. The model effectively provides damagecoordinates, and testing on an aluminum plate demonstrated its significantsuperiority over traditional ellipse-based damage localization methods. Liaoet al. [108] introduced a novel damage detection method for compositestructures, combining guided wave-based monitoring, Gramian angular fieldimage coding, and CNN. The proposed method features an improved datacompression algorithm for better localization accuracy, transforming dam-age indicators into 2D images for CNN analysis. Tested on datasets withadded noise, the method achieves a 3.59 % mean relative error in unseendatasets, showing superior generalization and accuracy compared to existingtechniques.2.3.2 Domain Generalized Damage LocalizationMany industrial applications fail to meet the xIxIsI criteria due to dis-crepancy between the distributions of training and testing data, a phe-24nomenon known as domain shift. To address this issue, methods designedfor such varying data distributions are categorized under domain generaliza-tion tasks. Enhancing deep learning models to maintain robust performancein these scenarios involves incorporating specific designs and components.These adaptations enable the models to adapt and perform well despitedomain shifts, ensuring their effectiveness across diverse and unpredictableenvironments and bridging the gap between theoretical assumptions andpractical applications.Most published work regarding domain generalization methods are fo-cused on computer vision tasks [109, 110], and there are few studies aim toaddress the fault diagnosis of rotating machinery [111, 112]. In the contextof composite damage identification, Pyle et al. [113] compared five domainadaptation methods on a notch damage characterization task, and foundthat the adversarial-based methods are most effective in sizing notch dam-ages with limited training data. Khan et al. [114] used a supervised machinelearning framework to identify different damage on composite structures andconcluded that linear discriminant analysis outperformed other classifiers re-garding reducing the overfitting issue. However, far too little attention hasbeen applied to damage localization for composite materials considering thedomain generalization problem.2.3.3 OOD Damage LocalizationOOD generalization addresses the issue of significant distribution differ-ences between training and testing data. In the context of damage localiza-tion, this involves predicting unseen damage locations that differ from thoseencountered during training, necessitating models that can not only fit thetraining data but also derive and apply the underlying physical principles ofdamage localization using UGW. However, deep learning studies that con-sider these data conditions are limited, especially in the UT domain. Thissituation underscores a critical gap in leveraging deep learning for OODgeneralization within the realm of damage localization.Limited studies have appeared in the last few years. Wang et al. [42]introduced a novel graph-in-graph convolutional network to enhance dam-age detection and localization using UGW signals. This was achieved byconstructing spatial-temporal feature representations and a global graph tohighlight differences among signals, thereby achieving superior accuracy andstability, even with limited and imbalanced datasets. Zhang et al. [43] de-veloped an interpretable prototypical network for damage identification us-ing UGW, embedding an attention module into the traditional CNN block25to feature superior generalization ability. This method was validated on amulti-class damage classification task, achieving an accuracy of over 95%.2.4 SummaryThis chapter investigates existing studies on composite damage localiza-tion. It begins by introducing various NDT techniques, including imaging-based and mechanical vibration-based methods. Imaging-based methodsachieve damage localization by scanning the specimen, with the main chal-lenge being the identification of internal and micro damage. Vibration-basedmethods, such as UT, rely on extracting ToFs from multiple sensors, whereaccurately estimating ToFs poses a significant challenge, especially whenusing traditional signal processing methods. This aspect is discussed inthe second section of this chapter. Furthermore, a comprehensive literaturereview on DNN-based methods for damage localization using UGW is con-ducted, covering the three application scenarios mentioned in Chapter 1.While most existing studies focus on the standard deep learning framework(xIxIsI), there is significantly less work in the areas of domain generalizationand OOD damage localization.26Chapter 3Composite DamageInspection using UltrasonicGuided Waves 23.1 OverviewTo examine UGW signal behaviour for composite damage localization,an impact damage inspection experiment using UGW was performed. First,this chapter presents the design and implementation of the composite dam-age inspection experiment using the nonlinear second harmonic generation(SHG) testing technique. Additionally, to demonstrate the generalizationability of the subsequent DNN methods for composite damage localizationusing the UGW technique, a public damage localization inspection using thelinear ultrasonic testing technique was introduced. These two inspectionsprovide the datasets for the DNN-based composite damage localization vali-dation. Furthermore, UGW signal visualization and analysis were conductedto provide an intuitive illustration of using UGW to identify composite dam-age.2Chapter 3 is based on work conducted in National Research Council CanadaAerospaceResearch Centre by me, with support from Marc Genest and Dr. Shashank Pant. I wasresponsible for conducting impact damage inspections using ultrasonic guided waves, im-mersion ultrasonic testing, and destructive microscope inspection. A version of chapter3 has been published. Yun H, Pant S, Genest M, Li L, Liu Z. Impact damage charac-terization in composite plate using ultrasonic guided waves. In2023 IEEE InternationalUltrasonics Symposium (IUS) 2023 Sep 3 (pp. 1-4). IEEE. I conducted all the methoddevelopment, experimental validation, and manuscript writing.273.2 Composite Damage Inspection usingNonlinear UGW3.2.1 Inspection OverviewThis inspection is designed to provide a data source for subsequent DNN-based damage localization in composite materials using UGW. Particularly,the nonlinear SHG technique was utilized to identify the composite dam-age. Compared to linear UGW technique, nonlinear UGW is more sensitivethe early damage in the composite, such as the matrix crack and delamina-tion [115]. The CFRP plates inspected were manufactured at the NationalResearch Council Canada Aerospace Research Centre. They were manufac-tured from IM7/977-3 prepreg material using autoclave machines, arrangedunidirectionally with dimensions of 2M4mm Ã 304I8mm Ã 4I3Mmm and astacking sequence of s0L4ML90Lâ 4M]4s. Table 3.1 lists the mechanical prop-erties of the inspected plate.Table 3.1: Mechanical properties of IM7/977-3 prepreg material for platemanufacture.Mechanical properties ValueDensity (g/cm3) 1.585E11 (GPa) 158E22 (GPa) 8.97G12 (GPa) 5.68v12 0.32v23 0.46Figure. 3.1 outlines the procedure for inspecting plate damage. The im-pact damage is induced using a mechanical impactor set to a specific energylevel. To prevent the specimen from rebounding upon impact, it is securedwith four clamps equipped with rubber tips. The impact is delivered by ahemispherical steel projectile with a diameter of 15.87 mm (0.625 inches).Nonlinear UGW inspection is performed both before, to acquire baselinesignals, and after the introduction of impact damage, to capture the inspec-tion signals. Additionally, an immersion ultrasonic C-scan is conducted toprovide ground truth regarding the location of the damages.28Figure 3.1: The procedure of the impact damage inspection using nonlinearUGW technique. © Hongguang Yun3.2.2 Detailed Inspection ProcedureTransducer Installation The transducers used are from the SMARTLayers series by Acellent Technologies, Inc. [1], consisting of a single piezo-electric (PZT) sensor. This sensor is embedded within a thin, flexible di-electric carrier film, as illustrated in Figure. 3.2. Each sensor in a sensornetwork is individually wired and then connected to the data-acquisitionhardware. The central frequency of the sensor is 5 MHz.Figure 3.2: The SMART Layers PZT transducer by Acellent Technologies,Inc.. [1]To install the transducer on the plate, the process involves three mainsteps: plate preparation, transducer attachment, and transducer potting.Figure. 3.3 outlines this procedure with illustrations of the installation prod-ucts. Specifically, for plate preparation, the surface is treated with MCA-229M-Prep Conditioner A and M-Prep Neutralizer 5A. This combination, act-ing as a mild phosphoric acid, accelerates the surface cleaning process, whichis then finalized with Ethyl Alcohol. For transducer attachment, the trans-ducer is adhered to the plate using M-Bond 200 adhesive, with M-Bond 200catalyst C employed to manage the adhesiveâs reactivity rate. Finally, fortransducer potting, a Silicone-based potting compound, Loctite SI 5140, isused to secure the transducer, enhancing its stability.Figure 3.3: Flowchart of the installation process for PZT transducers on theCFRP plate. © Hongguang YunNonlinear UGW Testing System The nonlinear UGW testing systemcomprises an Acellent ultrasonic actuator, an amplifier, an 8-channel Pi-coScope 4000A oscilloscope, and a data collection application developed inMATLAB, as depicted in Figure. 3.1. Eight PZT transducers, labelled T1-T8, were installed on each plate as illustrated in Figure. 3.4. A pitch-catchconfiguration was employed for the inspection, wherein one transducer actsas the transmitter to introduce the UGW signal into the medium, and theremaining transducers serve as receivers to collect the UGW signals.Nonlinear UGW Signal Frequency Selection To accurately locateBVID within the plate, this study employs the nonlinear SHG technique,which is widely recognized and proven effective for composite damage char-acterization. The SHG technique is able to identify micro-damage by gener-ating higher harmonics when UGW propagate through specimens [115]. AsUGW travels through a plate, regardless of whether the fundamental modeis symmetric or antisymmetric, the genesis of second-order waves remains30Figure 3.4: Illustration of CFRP plate with eight PZT transducers installed.© Hongguang Yunsymmetric with respect to the plateâs neutral centre. Consequently, exclud-ing material nonlinearity, only symmetric second harmonic Lamb modesare generated [31]. These modes have the capability for cumulative am-plification during their propagation. As noted by [31], for effective SHG,the fundamental and second harmonic frequency modes must propagate atidentical phase velocities. This condition is met due to the second harmonicwaveâs reliance on the squared magnitude of the first harmonic, allowingits intensificationand thus, growthonly when its phase velocity aligns withthat of the fundamental wave. For SHG to occur, both the fundamentaland second harmonic must travel at the same phase velocity to accumulatecomponents. Figure. 3.5 shows the phase velocity dispersion curve for theIM7/977-3 made plate. A pair of s0 and s1 modes at approximately 230kHzwas chosen for measurement, as both propagate at a speed of around 6280m/s.Nonlinear Composite Damage Inspection Dataset To compose adataset for the DNN models validation, three CFRP plates, labelled as #1,#2, and #3, of identical size were manufactured and inspected following the31Figure 3.5: Calculated phase velocity dispersion curve of the IMM/977-3specimen. Synchronous pair of modes s0 and s1 at a synchronous phasevelocity of around 6280 m/s. © Hongguang Yunprocess outlined in Figure. 3.1. Each plate was subjected to impact energiesof 19J, 17J, and 16.5J, respectively. Figure. 3.6, 3.7, and 3.8 displays theimmersion ultrasonic C-scan results for the three plates, where the yellowarea represents the plate, blue areas are the transducers, and the red area in-dicates the damage. In addition, to enrich the dataset, considering the costof manufacturing plates, metal disks of varying sizes were used to simulatedamage, as depicted in Figure. 3.9. Three disks, all of the same thickness,have diameters of 6.35 mm, 12.7 mm, and 19.05 mm, respectively. For eachinspection, one disk was attached to one of the nine designated locations onthe plate using an ultrasonic couplant. Table. 3.2 and Table. 3.3 providethe coordinates of the eight transducers and the nine designated locationsfor the disks on the CFRP plate, respectively. The eight transducers wereutilized one at a time as transmitters to carry out the inspection. Each in-spection involved five measurements to enrich the dataset. To summary, thedamage inspection includes both real impact damage and metal disk dam-age simulation. Together, these form the nonlinear UGW dataset, referredto as the NUGW dataset in the subsequent text.32Figure 3.6: Immersion ultrasonic C-Scan results of #1 CFRP plate after19J impact. © Hongguang Yun3.3 Public Composite Damage InspectionDataset using UGWTo more convincingly demonstrate the modelâs effectiveness, a publiclyavailable dataset known as the Open Guided Wave dataset [2] was alsoemployed for damage localization method validations. The reason for usingthis dataset is its utilization of a linear UGW technique, contrasting withour inspectionâs nonlinear approach. These two datasets can substantiatethe generalization ability of DNN methods. In the rest of this study, theinspection from the linear technique is referred to as the LUGW dataset.The LUGW inspection was conducted on CFRP plates with dimensionsof M00mm Ã M00mm Ã 2mm, arranged in a layup of s4ML0L â 4ML90L â4ML0L4ML90]s. Twelve PZT transducers, labelled T1-T12, were installed onthe plate in two parallel rows. Likewise, a disk on the plate was used tosimulate damage on 28 different locations as shown in Figure. 3.10. Apitch-catch configuration was used, where a 5-cycle Hanning-windowed toneburst at 60 kHz frequency was emitted by one transducer and received bythe others.33Figure 3.7: Immersion ultrasonic C-Scan results of #1 CFRP plate after17J impact. © Hongguang Yun3.4 Data Analysis of UGW SignalsThis section demonstrates the waveform of the two datasetsâ guidedwaves in the time domain. Moreover, given that the datasets employ lin-ear and nonlinear techniques respectively, the fast Fourier transform andtime-frequency transform method STFT was used to discern the differencesbetween these two techniques in the time-frequency domain.In this study, the baseline signal is defined as the UGW signal collectedwhen the plate is in an intact condition, meaning no damage has been in-flicted on the plate aside from manufacturing defects. The inspection signalis defined as the UGW signal collected when potential damage is presenton the plate. The term "transducer pair 1-2" denotes the use of transducer1 as the transmitter and transducer 2 as the receiver. The initial step insignal pre-processing involves baseline subtraction, which entails subtract-ing the baseline from the inspection signal. Subsequently, a second-orderButterworth band-pass filter is applied to the subtracted signal to eliminatenoise. The band-pass parameters were determined to avoid affecting thefundamental and second harmonic frequencies during filtering.Nonlinear UGW Dataset In our impact damage inspection, a 5-cycletone burst signal with a frequency of 230 kHz was employed to generate theguided waves. Figure. 3.11 shows the pitch-catch signals after a 19J impact34Figure 3.8: Immersion ultrasonic C-Scan results of #1 CFRP plate after16.5J impact. © Hongguang YunFigure 3.9: Metallic disks with varying size used as simulation damages. ©Hongguang Yunfrom transducer pair 1-4. The data sampling frequency was set at 20 MHz,with each measurement comprising 4000 data points. In the rest descrip-tion of this study, the UGW signals referenced are exclusively catch signalsfrom receivers unless stated otherwise. Figure. 3.12 presents the inspectionsignal from seven transducer pairs with a single damage at the centre ofthe plate, revealing that the ToFs from UGW from the transmitter to dif-ferent receivers vary, laying the foundation for damage localization. Figure.3.13 demonstrates the effect of the band-pass filter on the subtracted signal,clearly illustrating its significant role in noise reduction. To confirm the ef-ficacy of the nonlinear SHG technique, the received signal was examined inthe frequency domain. Figure. 3.14 demonstrates the frequency components35Table 3.2: Coordinates of eight transducers installed on the CFRP plate.Transducers index Coordinates (meter)1 (0.0635, 0.0508)2 (0.0653, 0.1270)3 (0.0653, 0.2032)4 (0.1524, 0.2032)5 (0.2413, 0.2032)6 (0.2413, 0.1270)7 (0.2413, 0.0508)8 (0.1524, 0.0508)Table 3.3: Coordinates of nine predefined disk damage locations on theCFRP plate.Damages index Coordinates (meter)1 (0.1143, 0.0889)2 (0.1143, 0.1270)3 (0.1143, 0.1651)4 (0.1524, 0.1651)5 (0.1905, 0.1651)6 (0.1905, 0.1270)7 (0.1905, 0.0889)8 (0.1524, 0.0889)9 (0.1524, 0.1270)of the UGW signal from the transducer pair 1-4, showing both fundamental(230 kHz) and second harmonic (460 kHz) components. This demonstratesthe effectiveness of the nonlinear UGW technique in identifying BVID com-posite damage. Furthermore, the STFT was applied to convert the signalinto the time-frequency domain. Figure. 3.15 depicts the time-frequencyrepresentation of the signal with damage on the plate, showing that in addi-tion to the fundamental frequency of 230 kHz, a second harmonic frequencyof 460 kHz is present, indicating damage within the plate and thereby val-idating the SHG techniqueâs utility in damage detection. However, due tothe complex nature of the frequency components, it is challenging to pin-point the exact timestamp at which the second harmonic component wasreceived, underscoring the limitations of using signal processing methods to36Figure 3.10: Configuration of 12 transducers and 28 different damage loca-tions of LUGW dataset [2]. Used with permission from Sage.determine the ToF for damage localization as discussed in Chapter 1.Linear UGW Dataset For the LUGW dataset, a 5-cycle tone burst sig-nal with a frequency of 60 kHz was employed to generate the guided waves.Figure. 3.16 shows the baseline pitch-catch signals from transducer pair1-7. The sampling frequency for the data was 10 MHz, with each measure-ment comprising 12000 data points. Similarly, Figure. 3.17 presents theinspection signal from six transducer pairs with a damage at the location5. Figure. 3.18 demonstrates the effect of the band-pass filter on the sub-tracted signal, both before and after its application. Figure 3.19 displaysthe frequency components of the linear UGW signal, showing only the fun-damental frequency component. It is worth noting that although there isno second harmonic component in the received signal, it is still sufficient toidentify the damage in this inspection using the UGW technique. This isbecause the damage induces changes in the magnitude and ToF of the UGWsignal. The STFT was utilized to convert the signal into the time-frequencydomain, with Figure. 3.20 depicting the time-frequency representation ofthe signal with damage on the plate. Unlike the results obtained using theSHG technique, it shows that the major component within the signal isthe fundamental frequency of 60 kHz. However, damage detection is stillfeasible by observing the differences between the baseline and inspectionsignals. The challenge of accurately locating damage is also evident withthis technique.37Figure 3.11: Pitch-catch configuration of the nonlinear UGW signals: toppanel displays the transmitted waveform; bottom panel shows the receivedwaveform. © Hongguang Yun38Figure 3.12: Visualization of the nonlinear UGW signals from seven receiverscollecting data from the same transmitter. © Hongguang YunFigure 3.13: Comparative visualization of received nonlinear UGW signal:before and after a band-pass filtering. © Hongguang Yun39Figure 3.14: Visualization of the nonlinear UGW signal in the frequencydomain, showing both fundamental and second harmonic frequencies. ©Hongguang YunFigure 3.15: Time-frequency representation of the nonlinear UGW signal,showing both fundamental and second harmonic wave packets. © Hong-guang Yun40Figure 3.16: Pitch-catch configuration of the linear UGW signals: top paneldisplays the transmitted waveform; bottom panel shows the received wave-form. © Hongguang Yun41Figure 3.17: Visualization of the linear UGW signals from six receiverscollecting data from the same transmitter. © Hongguang YunFigure 3.18: Comparative visualization of received linear UGW signal: be-fore and after a band-pass filtering. © Hongguang Yun42Figure 3.19: Visualization of the linear UGW signal in the frequency domain,showing only the fundamental harmonic frequency. © Hongguang YunFigure 3.20: Time-frequency representation of the linear UGW signal, show-ing the fundamental harmonic wave packet. © Hongguang Yun433.5 SummaryThis chapter details the procedure of conducting nonlinear UGW inspec-tions on CFRP plates for damage localization. Specifically, three CFRPplates manufactured by unidirectional material were prepared for inspec-tion, with each plate installed with eight PZT transducers. The nonlinearSHG technique was employed for the inspections, and immersion ultrasonicC-scan was conducted to establish ground truth. Additionally, a publicdataset employing a linear UGW technique was introduced for comparativeanalysis with the nonlinear UGW technique. The chapter also visualizedthe nonlinear and linear UGW signals following specific signal pre-processingsteps, with the STFT used to analyze the UGW signal in the time-frequencydomain. The nonlinear UGW technique revealed both fundamental and sec-ond harmonic frequencies, demonstrating the efficacy of nonlinear methodsin detecting damage. In contrast, the linear UGW identified damage solelythrough the fundamental frequency, yet still facilitated damage detectionvia signal amplitude comparison.44Chapter 4Multi-dimensional DataFusion for CompositeDamage Localization 34.1 OverviewThe objective of this study is to develop deep learning methods for lo-calizing damage in composite materials using UGW, focusing on scenarioswith varying data availability. DNN models depend significantly on thevolume and quality of training data to generalize effectively to new and un-seen data. A substantial discrepancy between training and testing datasetscan lead to decreased model performance due to potential distributionaldifferences. This is particularly relevant in industrial settings, where dataavailability can fluctuate based on time and cost.As mentioned in chapter 1, this study progresses from scenarios of am-ple data availability to those with limited data and increase the physicalknowledge embedding to the model accordingly. This chapter addressescomposite damage localization within a standard deep learning framework,assuming the training and testing data follow the xIxIsI. In other words, theavailable training data is presumed to comprehensively represent the condi-tions encountered during testing. The primary challenge here is to efficientlyincorporate all physical data inputs of varying dimensions from ultrasonicmeasurements into the DNN model, ensuring precise damage localization.3A version of chapter 4 has been published. Yun H, Feng K, Rayhana R, Pant S, Gen-est M, Liu Z. A multidimensional data fusion neural network for damage localization usingultrasonic guided wave. IEEE Transactions on Instrumentation and Measurement. 2023Aug 1. I conducted all the method development, experimental validation, and manuscriptwriting.454.2 Standard Deep Neural Network for DamageLocalization4.2.1 Problem DefinitionTo provide a clear picture of the damage localization task within differentdeep learning frameworks, a definition of the problem will be presented firstlyin this and the following chapters. First, the DNN models developed in thisstudy fall under the supervised learning paradigm. In supervised learning,the objective is to build a predictive model that maps inputs to outputs usinglabelled training data. Supervised learning tasks can be further classifiedinto classification and regression. A classification model outputs categoricalpredictions, while a regression model outputs continuous predictions. Thisstudy will employ a regression setting in this chapter and in Chapter 6,whereas Chapter 5 will discuss a multi-class classification task.This chapter will introduce the DNN models for damage localizing usingUGW under a standard training paradigm, where the training data and test-ing are in xIxIsI. Specifically, given the training dataset XS = sx), I I I ,xN ]S,where X includes N instances and each instance includes input data, includ-ing the UGW signals and other physical features. y = sy), I I I , yN ], wherey is the label vector of N instances. In the context of the damage localiza-tion, the label is the location of the damage. Given the xIxIsI problem, thetraining data XS and testing data XT are assumed sampled from the samedistribution.Meanwhile, the implicit functions are defined task-wise as T = {P (X),F(Â·)},where the P (X) is the marginal distribution of the data and F(Â·) is the pre-dictive function. The objective function used to calculate the error betweenthe prediction and the given label, and thus optimize the model, is called theloss function. The goal is to build a model that, when trained on trainingdata, can accurately locate damage in testing data.4.2.2 Multi-dimensional Input Data from UltrasonicTestingDamage localization using UGW relies on an active sensor network,where transmitters introduce guided waves into the medium, and receiverscollect the resulting signals. The presence of damage changes the character-istics of the waves, such as amplitude and phase, and can induce new wavepackets (mode conversion). By comparing the obtained signal with the base-line signal, the location of the damage can be determined accurately. In the46context of UGW for damage localization, two types of data are utilized formodelling. As shown in Figure. 4.1 (a), the UGW signals from multipletransmitter-receiver pairs represent high-dimensional data, while the cor-responding transducer coordinates represent low-dimensional data. Signalprocessing methods can extract additional low-dimensional data, such asdamage index and ToF, from the high-dimensional wave signals.Current damage localization methods can be categorized into two groupsbased on their use of available data. As illustrated in Figure. 4.1 (b), con-ventional methods aim to determine damage location by utilizing the coor-dinates and other low-dimensional data obtained through signal processingmethods. Two representative algorithms, namely the ellipse method [80]and RAPID [29], have been developed for damage localization. The ellipsemethod achieves damage localization by solving multiple hyperbola equa-tions built based on transducer coordinates and ToF. The RAPID methodevaluates the distance between the damage and the transducer by measur-ing the difference between the testing and baseline signals using correlationcoefficients as a damage index. An elliptical distribution function is thenused to model the likelihood of damage occurrence. The final prediction ofthe damage location is obtained by summing the weighted distribution fromseveral transmitter-receiver pairs.In contrast, existing deep learning-based studies, as depicted in Fig-ure. 4.1 (c), directly process high-dimensional wave signals or extractedfeatures using DNN models to predict the damage location. Existing deeplearning-based methods often neglect the use of low-dimensional transducercoordinates [41, 106]. However, both the transducer coordinates and wavesignals are necessary to theoretically determine the location of the damage.Failure to incorporate low-dimensional transducer coordinates can lead toa deterioration in the accuracy of damage localization models. Therefore,as shown in Figure. 4.1 (d), develop a DNN model that integrates high-dimensional wave signals and low-dimensional coordinates is necessary foraccurate damage localization.4.3 Multi-dimensional Data Fusion Model forDamage Localization4.3.1 Model ArchitectureThe scarcity of accurately localizing damage of current DNN-based meth-ods using lies in neglecting the use of transducer coordinates in model con-47Figure 4.1: Illustration of data structure and methods for UGW-based dam-age localization: (a) available low and high dimensional data, (b) conven-tional method with low-dimensional data used, (c) current DL-based methodwith high dimensional data used, and (d) proposed DL-based with multidi-mensional data incorporated. © Hongguang Yunstruction. This oversight can lead to models that overfits the training dataand lack the ability to generalize effectively to new scenarios. Addressingthis gap, this study introduces the multidimensional data fusion neural net-work, a novel framework that seamlessly integrates high-dimensional wavesignals with essential low-dimensional transducer coordinates.Central to our framework, illustrated in Figure. 4.2, is a parallel struc-ture combining an encoder module and a Random Fourier Feature (RFF)projection head. The encoderâs role is to extract relevant features fromthe wave signals, while the RFF projection head is explicitly designed toweave the transducer coordinates into the neural network, enriching themodelâs contextual understanding. The fusion of these multidimensionaldata sources is crucial for accurately predicting damage locations, a task ac-complished by a decoder based on a FCNN. This decoder capitalizes on the48Figure 4.2: The architecture of proposed multidimensional data fusion neu-ral network framework. © Hongguang Yuncomprehensive feature set aggregated from both the encoder and projectionhead.Specifically, this method utilizes a CNN-based encoder to extract dam-age features from various wave packets originating from multiple propaga-tion paths. In Figure. 4.2, it can be observed that multiple wave signalsfrom corresponding transmitter-receiver paths are treated as different inputchannels for the 1DCNN . To enhance the stability of the 1DCNN, resid-ual connections are implemented within the network. Residual connectionsserve as shortcuts in a CNN block to mitigate issues such as overfitting andvanishing gradients during the training process [116]. Figure. 4.3 illustratesthe CNN block structure. Additionally, batch normalization is incorporatedwithin the CNN block to accelerate the training process [117]. The rationaleof using a RFF projection to incorporate the low-dimensional data into themodel is elucidated in detail in the following subsection.4.3.2 Kernel RegressionDNN models are designed to process high-dimensional data such as im-ages. However, it has been observed that traditional DNN models often haveconvergence difficulties when processing low-dimensional data. Recently, theapproach of using one of the DNN methods, specifically FCNN, to processlow-dimensional data, particularly coordinate data, has become increasingly49Figure 4.3: The 1D convolutional block used in the encoder module. ©Hongguang Yunpopular [118]. This aligns well with the requirements of damage localizationproblems, where the incorporation of transducersâ coordinates into the DNNmodel is necessary.To understand the advantages of using FCNN for processing coordinates,a common method is to employ kernel regression to elucidate the behaviourof FCNN models. Exploring the mechanism of FCNN lays the foundation forthis study to find an effective way to incorporate both low-dimensional andhigh-dimensional data into DNN models. This section introduces kernelregression methods as the preliminary groundwork for understanding themechanisms of FCNN and their requirements for input.Given a dataset (X,y) = {(xi, yi)}Ni=), and xi â Rs, yi â R. Assumingthat a basic linear regression cannot fit this dataset properly, a feature mapÏ : Rs â Rm is used to project the original input into a high-dimensionalfeature space. Fitting the projected data with a l2 loss, the objective functionis given as [119]:L(w) :=Nâi=)12(Ï(xi)cw â yi)2 (4.1)where w is the weight of the regression model.For simplicity, use a matrix expression for y := (y), · · · , yN ) ,X :=50(x), · · · ,xN ) ,Φ := (Ï (x)) , · · · , Ï (xN )), and (4.1) can be expressed as:L(w) = 12âÎ¦w â yâ2 (4.2)The optimal solution of the w can be obtained by letting the deriva-tive of the objective function equal zero. With the corollary(Î¦cÎ¦)â Φc =Î¦c(Î¦Î¦c)â , the optimal solution wâ can be expressed as [120]:wâ =(Î¦cÎ¦)â Φc y = Î¦c(Î¦Î¦c)â y (4.3)The prediction of any data x can be obtained by the following function:f(x) = Ï(x)cwâ â Ï(x)cÎ¦c (ΦΦc )â)y (4.4)Since the feature map Ï is usually complicated, a kernel function k :RsÃRs â R with expression k(xi,xj) := Ï(xi)cÏ(xj) is defined to implicitlyachieve the projection goals. The kernel function and feature map have arelation: [ΦΦc]ij= Ï (xi)c Ï (xj) := k (xi,xj) ,[Ï(x)cÎ¦c]j= Ï(x)cÏ (xj) = k (x,xj)(4.5)Substituting 4.5 into 4.4, the prediction of x can be expressed as:f(x) = k(x,X)Kâ)y (4.6)where K is the N Ã N Gram matrix of X on the kernel function withentries Kij = k(xi,xj). The advantage of using a kernel function is free ofcalculating the complicated feature map in high-dimensional space.4.3.3 Random Fourier Feature ProjectionIn the field of deep learning, Ian Goodfellow et al. [121] made a signif-icant statement that a wide enough FCNN can represent any function. Inother words, the flexibility and robustness of FCNN make it suitable as adecoder for the damage localization problem. However, DNN methods aregenerally designed to process high-dimensional data, such as image pixels,and often face challenges in effectively learning low-dimensional data repre-sentations [118]. Recent studies found that it is difficult for an FCNN toconverge when it is directly fed with low-dimensional input, such as coordi-nates [122].51Development on DNN models for low-dimensional data were made bythe application of coordinate-based FCNN in a variety of images and scenereconstruction tasks [123, 124]. The FCNN is designed to receive low-dimensional coordinates as inputs and is trained to generate representa-tions of shape, density, and/or colour at each input location. However, anFCNN has difficulty in learning high frequency components when the inputis low-dimensional data [118]. To this end, most researchers proposed usinga projection head to map the low-dimensional data into high-dimensionalspace. Vaswani et al. [125] and Tancik [118] proposed to use ros features torepresent the positional information of words in a sentence. Therefore, toeffectively incorporate the low-dimensional transducer coordinates into theDNN model, this study proposes to implement innovative random Fourierfeature (RFF) projection into the damage localization problem using UGW.For a given input x, the output of a single fully-connected layer can beexpressed as [126]:f(x;w) = Ï(wcx+ b)(4.7)where, Ï is the activation function, w and b are weight and bias of the layer,respectively.An FCNN with L layers and an extra output layer can be expressed as:f(x;w) = f (L+)) · Ï(f (L) Ï(f (Lâ)) · · Â·Ï(f ())x)))(4.8)where fL are the weights of layer L, the bias of each layer is omitted.However, recent studies found that it is difficult for an FCNN to convergewhen it is directly fed with low-dimensional input, such as coordinates [122].To figure out the reason for this phenomenon, Jacot et al. [127] proposed touse a neural tangent kernel (NTK) to explain the behaviour of an FCNNâstraining process. Assume an FCNN f with parameters w initialized froma Gaussian distribution N has infinite width and the learning rate tends tozero, its behaviour would correspond to a kernel regression, whose kernel isan NTK and is defined as [127]:k (xi,xj) = Ewâ¼Nâ©âf (xi,w)âw,âf (xj ,w)âwâª(4.9)Following the theory proposed by [127], Lee et al. [128] revealed the pre-diction of the FCNN after t iteration training follows a Gaussian distributionwhose mean and variance can be expressed as:52µt(x) = K((x,X)Kâ)((X â tâÎ·K0t)yÏt(x) = f((x)âK((x,X)Kâ)((X â tâÎ·K0t) f((X) (4.10)where K( is the Gram matrix of the initial NTK kernel and f((X) is theinitial prediction of the FCNN on the dataset XThe prediction can be approximated as:fË(x)(t) â k(x,X)Kâ) (Iâ tâÎ·Kt)y (4.11)Since the Gram matrix of NTK is positive semidefinite [129], decompos-ing the K as K = QÎQT, the estimation of prediction error can be givenas:QT(fË(X)(t) â y)â QT ((Iâ tâÎ·Kt)y â y)= âtâÎ·ÎtQTy(4.12)Equation 4.12 discloses that the xth prediction error in QT(fË(X)(t) â y)will decay exponentially at rate ηλi. According to the research from Basri etal., [130], the high frequency components of a task are difficult for the FCNNto learn. For example, the details of the image in an image reconstructiontask will be lost.As discussed earlier, directly feeding low-dimensional data into a DNNmodel will cause difficulty to learn the high frequency components of atask. In the context of damage localization, the accurate coordinates of thedamage location will be difficult to predict. To address this issue, this paperproposes to use a RFF projection head to map low-dimensional transducercoordinates into a high-dimensional space before an FCNN to estimate thedamage location. The RFF head is defined as:p(v) = sp) cos(2ÏbT) v), p) sin(2ÏbT) v),I I I , pm cos(2ÏbTmv), pm sin(2ÏbTmv)]T(4.13)where pi and bi correspond to the amplitude and frequency of the RFFprojection, respectively.To summarize, as a widely used DNN structure for giving prediction,FCNN has difficulty in learning the high frequency components in low-dimensional data. In the context of UGW-based damage localization, theFCNN in a DNN model needs to process high-dimensional wave signals and53low-dimensional transducer coordinates simultaneously. To address this is-sue, the use of RFF projection mapping the low-dimensional transducer co-ordinates into a high-dimensional space can decrease the learning difficultyfor the following FCNN.4.4 Experimental Results and Discussion4.4.1 Data DescriptionAs mentioned in Chapter 3, to demonstrate the generalization abilityof the DNN models for composite damage localization using both linear ornonlinear UGW techniques, all DNN methods were tested on both NUGWand LUGW datasets.1. For the impact damage inspection, which utilizes nonlinear UGW tech-nique as depicted in Figure. 4.4, eight transducers were installed onthe plate, with transducers 1, 3, 5, and 7 being used individually astransmitters. When one transducer was used as the transmitter, theremaining seven transducers served as receivers. Similarly, each dataentry comprises UGW signals from seven transmitter-receiver propa-gation paths, corresponding to seven channels. Due to the lack of suf-ficient real impact damage data for training, the inspection performedon the three disks was utilized for this experiment. With 5 baselinesand 9 damage locations, a total of 540 data entries were collected toconstruct the NUGW dataset.2. For the LUGW dataset, which utilizes linear UGW technique as illus-trated in Figure. 3.10, twelve transducers were installed on the plate.Transducers 1 to 6 were used individually as transmitters, and trans-ducers 7 to 12 were used as receivers simultaneously. Consequently,each data entry consists of UGW signals from six transmitter-receiverpropagation paths, corresponding to six channels. With 30 baselinesand 28 damage locations, a total of 5040 data entries were collectedto construct the dataset.Under the xIxIsI assumption of the training and testing data, the entiredataset was randomly divided into training, validation, and testing datasets,with ratios of 80%, 10%, and 10%, respectively. The DNNmodels are trainedin a regression task with objective to provide the coordinates of the damages.A metric named mean normalized distance (MND) is employed as the metricfor evaluating the performance of the models.54Figure 4.4: Illustration of CFRP plate with eight PZT transducers installed.© Hongguang YuneND =âNn=)â(xs â xËs)2 + (ys â yËs)2N(4.14)where, xs, ys, xËs, yËs are the actual and prediction of damage location, re-spectively. And N is the number of data samples.4.4.2 Ablation StudiesThe backbone of the feature extractor is built on top of ResNet 18 [116]with batch normalization utilized. As an example, Table. 4.1 displays thedetailed architecture of the proposed method for LUGW dataset. Since thelength of the UGW signal in one inspection is different, the architecturerequires a small adjust accordingly, such as the first layer of the FCNN.55Table 4.1: Detailed architecture of proposed method with ResNet18 as encoder for LUGW dataset.Components Stage Layers (Kernel size, stride) Channels Input size Output sizeEncoderFirst Conv Conv 1D (4, 2)Max pool (2, 1) 8 6 x12288 8x3072Block 1[ Conv 1D (4, 2), BN, ReLUConv 1D (4, 1), BN, ReLUMax pool (2, 1)]Ã 3 8 8x3072 8x384Block 2[ Conv 1D (4, 2), BN, ReLUConv 1D (4, 1), BN, ReLUMax pool (2, 1)]Ã 3 16 8x384 16x48Pooling Adaptive average pool 16 16x48 16x32Projection head Fourier features Linear layer - 7x2 7x72- Merge Flatten features from encoderand projection head, then concatenate -16x32,7x72 1016Decoder MLP Fully connected layers with ReLU -1016 10241024 10241024 10241024 128128 256The key of the data fusion is the RFF projection, whose properties arecontrolled by hyper-parameters pi and bi. Empirical studies shown that pican be set as 1, while bi can be sampled from a Gaussian distribution witha mean of zero and a standard deviation determined as a hyper-parameter.To verify this property in the context of damage localization, an ablationstudy on these two hyper-parameters was firstly conducted by training mod-els with and without optimization of pi and bi using gradient descent. Sincethe LUGW dataset is larger compared to NUGW and is publicly accessi-ble, this ablation study was conducted on the LUGW dataset for betterreproducibility. Based on the proposed multidimensional framework with1DCNN as the encoder, four settings were designed: (1) Not optimizing piand bi, (2) optimizing bi, (3) optimizing pi, and (4) optimizing both pi andbi. In all four models, pi was initialized as 1, and bi was initialized by sam-pling from a Gaussian distribution with a mean of zero and eight differentstandard deviations.Figure. 4.5 illustrates the training loss logs of all four models for 150epochs. It can be observed that the model with optimized pi and bi us-ing gradient descent does not outperform the model with pi set as 1 andbi randomly sampled from a Gaussian distribution. Moreover, spikes areevident in the training process of the model with pi optimization, indicatingincreased instability in the learning process. Additionally, Fig. 4.6 presentsthe training MND logs of all four models, showing no noticeable differencebetween the models with and without bi optimization. The coefficients piand bi control the bandwidth of the RFF projection. The experimental re-sults demonstrate that the modelâs performance is significantly influencedby the setting of bi, and optimizing bi does not lead to improved performancecompared to randomly sampling its value from a Gaussian distribution. Toreduce the number of trainable parameters in the model, subsequent ex-periments in this paper adopt the approach of setting bi through randomsampling from a Gaussian distribution with a mean of 0, considering thestandard deviation Ï as a hyper-parameter.To further investigate the characteristics of setting the standard devia-tion of the Gaussian distribution, a hyper-parameter sweep experiment wasconducted by varying Ï from 1 to 64. Figure. 4.7 illustrates the trainingand testing MND results for all 64 settings. It can be observed that both thetraining and testing accuracy exhibit a similar periodic trend as Ï changes.A smaller value of Ï corresponds to a narrower bandwidth RFF projection.The periodic properties of the RFF indicate that a projection with a largerÏ may exhibit similar characteristics to that of a projection with a smallerÏ. However, due to the random sampling of values for bi from the Gaus-570 25 50 75 100 125 150Epoch0.000.020.040.060.080.100.12Loss(a)Not optimizing ai, biOptimizing biOptimizing aiOptimizing ai, bi0 25 50 75 100 125 150Epoch0.000.020.040.060.080.100.12Loss(b)Not optimizing ai, biOptimizing biOptimizing aiOptimizing ai, bi0 25 50 75 100 125 150Epoch0.000.020.040.060.080.100.12Loss(c)Not optimizing ai, biOptimizing biOptimizing aiOptimizing ai, bi0 25 50 75 100 125 150Epoch0.000.020.040.060.080.100.12Loss(d)Not optimizing ai, biOptimizing biOptimizing aiOptimizing ai, bi0 25 50 75 100 125 150Epoch0.000.020.040.060.080.100.12Loss(e)Not optimizing ai, biOptimizing biOptimizing aiOptimizing ai, bi0 25 50 75 100 125 150Epoch0.000.020.040.060.080.100.12Loss(f)Not optimizing ai, biOptimizing biOptimizing aiOptimizing ai, bi0 25 50 75 100 125 150Epoch0.000.020.040.060.080.100.12Loss(g)Not optimizing ai, biOptimizing biOptimizing aiOptimizing ai, bi0 25 50 75 100 125 150Epoch0.000.020.040.060.080.100.12Loss(h)Not optimizing ai, biOptimizing biOptimizing aiOptimizing ai, biFigure 4.5: Training loss of models with and without optimizing the pi andbi. All four models were trained 150 epochs with pi initialized as 1 and biinitialized at eight different scales of standard deviations Ï: (a) Ï = 0I1, (b)Ï = 1, (c) Ï = 2, (d) Ï = M, (e) Ï = 10, (f) Ï = 1N, (g) Ï = 32, (h) Ï = N4.© Hongguang Yunsian distribution, the results do not exhibit strict periodic behaviour. Forsubsequent experiments, Ï was set to 8.With the hyper-parameters of RFF determined, the ablation study wasconducted on both datasets at the model level regarding the multi-dimensionaldata fusion. Considering whether to incorporate low-dimensional data andhow to incorporate them, three variants of the model were examined: (1) the"None" model refers to the case where the low-dimensional data is not fedinto the model; (2) the vanilla projection is achieved by directly inputtingthe seven transducer coordinates into the neural network using neurons; and(3) the model with an RFF projection head.The evaluation metric, MND, is used to assess the performance of mod-580 25 50 75 100 125 150Epoch0.000.020.040.060.080.100.120.140.16MND / mm(a)Not optimizing ai, biOptimizing biOptimizing aiOptimizing ai, bi0 25 50 75 100 125 150Epoch0.000.020.040.060.080.100.120.140.16MND / mm(b)Not optimizing ai, biOptimizing biOptimizing aiOptimizing ai, bi0 25 50 75 100 125 150Epoch0.000.020.040.060.080.100.120.140.16MND / mm(c)Not optimizing ai, biOptimizing biOptimizing aiOptimizing ai, bi0 25 50 75 100 125 150Epoch0.000.020.040.060.080.100.120.140.16MND / mm(d)Not optimizing ai, biOptimizing biOptimizing aiOptimizing ai, bi0 25 50 75 100 125 150Epoch0.000.020.040.060.080.100.120.140.16MND / mm(e)Not optimizing ai, biOptimizing biOptimizing aiOptimizing ai, bi0 25 50 75 100 125 150Epoch0.000.020.040.060.080.100.120.140.16MND / mm(f)Not optimizing ai, biOptimizing biOptimizing aiOptimizing ai, bi0 25 50 75 100 125 150Epoch0.000.020.040.060.080.100.120.140.16MND / mm(g)Not optimizing ai, biOptimizing biOptimizing aiOptimizing ai, bi0 25 50 75 100 125 150Epoch0.000.020.040.060.080.100.120.140.16MND / mm(h)Not optimizing ai, biOptimizing biOptimizing aiOptimizing ai, biFigure 4.6: Training MND (mm) of models with and without optimizingthe pi and bi. All four models were trained 150 epochs with pi initializedas 1 and bi initialized at eight different scales of standard deviations Ï: (a)Ï = 0I1, (b) Ï = 1, (c) Ï = 2, (d) Ï = M, (e) Ï = 10, (f) Ï = 1N, (g) Ï = 32,(h) Ï = N4. © Hongguang Yunels. The data is split into 80% for training, 10% for validation, and 10%for testing. Table. 4.2 presents the results of evaluating three architectureson two datasets. It is evident that incorporating an RFF projection headcan significantly enhance prediction accuracy compared to the other mod-els, highlighting a testing accuracy under 2 mm for both datasets. Withthe vanilla projection, since the low-dimensional data is directly fed into ahigh-dimensional DNN model, it even produced worse results compared tothe baseline model. One possible reason is that the FCNN struggles to han-dle the differences in features between low-dimensional and high-dimensionalspace simultaneously. Overall, the results reveal the necessity to incorporatetransducersâ location and the superiority of the RFF projection.590 10 20 30 40 50 60 70Standard deviation of 0.0010.0020.0030.0040.0050.006MND / mmTrain resultTest resultFigure 4.7: Training and testing MND (mm) results of the proposed modelwith bi sampled from a Gaussian distribution at a different scale of standarddeviation. © Hongguang YunTable 4.2: Comparative results on the use of low-dimensional data on twodatasets.DatasetProjection headNone Vanilla RFFValidation Testing Validation Testing Validation TestingNUGW 2.69 3.65 4.23 4.89 1.09 1.22LUGW 2.59 5.86 3.06 6.46 1.23 1.564.4.3 Comparative Studies with SOTA MethodsThe proposed method was compared against six state-of-the-art (SOTA)models for UGW-based damage localization. As this chapter focuses on de-veloping DNN-based methods under the xIxIsI assumption, the comparativemethods were also chosen based on this criterion. The selected methods andthe reasons for their selection are detailed below:1. 1DCNN [106]. This is one of the first studies to locate compositedamage using a DNN method, utilizing a 1DCNN model to exploitthe characteristics of time-series UGW signals.2. TVDI+CNN [41]. This study designed a time-varying damage index(TVDI) as input for the CNN model. The TVDI was constructed in60the time-frequency domain by comparing the windowed coefficients ofthe inspection signal with those of the baseline signal. With the TVDIas the input of the DNN model, the temporal information of carriedby the signal was incorporated into the model.3. STFT+CNN [131]. Instead of focusing solely on the time domain,this study used the time-frequency representation of UGW signals, ob-tained via STFT, as input into a 2DCNN model. The time-frequencyrepresentation can reveal damage-related information in both the timeand frequency domains, facilitating the modelâs ability to exploit pat-terns in both domains.4. FCNN [132]. Unlike other methods using CNN, this study used FCNNas the model backbone with UGW signals as input to locate the dam-age. The FCNN, with its wide and deep layers, is able to simulatenonlinear system behaviour and can effectively identify data patternsin the damage localization problem.5. Se-ResNet-18 [133]. This is one of the most powerful and successful2DCNN models across deep learning applications with channel atten-tion mechanisms implemented. The attention mechanism enables themodel to find correlations among features, allowing the DNN modelto uncover deeper patterns within the UGW signals.6. CBAM-ResNet-18 [134]. The CBAM-ResNet-18 further improve theattention mechanism on the top of Se-ResNet-18 model with includingboth channel attention and spatial attention [134].Results on the NUGW Dataset Similar to the experiment setting inthe ablation study, the evaluation metric, MND, is used to assess the perfor-mance of models. The data is split into 80% for training, 10% for validation,and 10% for testing. Table. 4.3 lists the comparative results on the NUGWdataset. Overall, the proposed model achieves the best damage locationprediction with a relative error of approximately 1.2 mm. In contrast, allother methods have prediction errors exceeding 3 mm. Two CNN methodswith an embedded attention mechanism provide competitive results com-pared to the proposed methods, with accuracy still under 6 mm. However,the results of 1DCNN, TVDI CNN, and STFT CNN are much worse, espe-cially for the one with TVDI methods. The 1DCNN model was built withmuch fewer layers compared to other methods, which limits the ability ofthe model to learn the representation through the data. The TVDI CNN,61and STFT CNN with relatively poor results have in common is that theyuse hand-crafted features as inputs instead of the raw UGW signals. Suchresults indicate that, for training and testing data following the xIxIsI situa-tion, purely data-driven DNN models have the ability to learn the inherentcharacteristics of the data better than using hand-crafted features, whichare built on a limited understanding of the data and may miss importantinformation. Another noteworthy point is that the testing errors of theSOTA models are significantly larger than their validation errors, whereasthe proposed method maintains a more consistent prediction error on boththe validation and testing datasets. This consistency in performance sug-gests that models not incorporating transducer coordinates struggle to learnan effective representation for locating damage, potentially leading to over-fitting on the training dataset.Table 4.3: Comparative results of various DNN-based methods for damagelocalization on NUGW dataset.Method MND (mm)Validation Testing1DCNN [106] 9.58 15.3TVDI+CNN [41] 11.8 15.1STFT+CNN [131] 7.31 9.56FCNN [132] 6.50 7.17Se-ResNet-18 [133] 4.13 5.22CBAM-ResNet-18 [134] 3.37 4.33Ours 1.09 1.22Results on the LUGW Dataset The same experimental settings wereused for validation on the LUGW dataset. Table. 4.4 displays similar resultsto those on the NUGW dataset. The proposed method still provides the bestoverall accuracy compared to other methods. The observation for the com-parison of data-driven models and hand-crafted models remains the same.The effectiveness for both linear and nonlinear UGW techniques justifies thegeneralization ability of the proposed method for damage localization.4.4.4 Robustness Study Regarding the Data AvailabilityThe available data in the industrial applications are generally much lim-ited. Small dataset will affect the performance of the DNN models even the62Table 4.4: Comparative results of various DNN-based methods for damagelocalization on LUGW dataset.Method MND (mm)Validation Testing1DCNN [106] 8.57 15.3TVDI+CNN [41] 10.4 17.2STFT+CNN [131] 5.42 7.69FCNN [132] 3.91 8.37Se-ResNet-18 [133] 3.49 4.87CBAM-ResNet-18 [134] 3.30 5.63Ours 1.23 1.56training and testing data follow a xIxIsI condition, the volume of the data. Toinvestigate the effectiveness of the proposed method regarding different datavolume, an experiment considering the size of the dataset for training is con-ducted. Each method is assessed under three data split scenarios50%, 30%,and 10% of the data used for trainingcorresponding to decreasing amounts oftraining data and indirectly increasing levels of challenge. For each split, themethodsâ performance is measured on both validation and testing datasets.Results on the NUGW Dataset Table. 4.5 listed the performance ofvarious DNN models on LUGW dataset. The results indicate that generally,all methods perform better with more training data, as seen by lower MNDvalues at 50% training data compared to 10%. However, the performancedegradation varies across methods as the training data decreases. For the50% data usage scenario, the proposed method outperforms all other meth-ods, with the lowest MND of 4.41 on validation and 6.63 on testing. At the30% data usage, The proposed method maintains superiority with MND of7.11 on validation and 8.56 on testing. Even at the most challenging scenarioof 10% training data, our method continues to lead with an MND of 10.89on validation and 13.3 on testing, although the gap in performance narrows,as indicated by the more comparable MND values of other methods. DNNmodels with hand-crafted features show a marked increase in error as theamount of training data is reduced. The increase is steeper than that ofthe other methods, particularly in the testing data. Two attention-basedmethods demonstrate intermediate robustness, with MND that generallystay below those of the models with hand-crafted features but above the63proposed method.Table 4.5: Comparative results (MMD (mm)) of various DNN-based meth-ods for damage localization under three different data split settings onNUGW dataset.MethodData used in training (100% stands for 540 entries)50% 30% 10%Validation Testing Validation Testing Validation Testing1DCNN 14.4 22.7 14.6 30.4 27.4 31.9TVDI+CNN 12.9 28.5 15.6 34.1 25.7 40.8STFT+CNN 9.83 10.1 11.6 13.9 13.6 21.9FCNN 6.95 17.4 12.5 17.7 24.5 27.2Se-ResNet-18 6.96 8.69 10.1 16.2 13.6 17.9CBAM-ResNet-18 7.77 11.6 10.0 14.4 17.8 25.5Ours 4.41 6.63 7.11 8.56 10.89 13.3Results on the LUGW Dataset Still, Table. 4.6 displays the results ofvarious DNN methods on LUGW dataset. The results are consistent withthose on the NUGW dataset, with the hand-crafted DNN models deliveringthe worst performance, purely data-driven DNN models performing at anintermediate level, and the proposed method demonstrating the best per-formance across the different volumes of training data. In summary, theproposed method demonstrates the most effective performance across alltraining data volumes, suggesting it has superior generalization capabilitiesand robustness to data scarcity in this regression task on the LUGW dataset.Table 4.6: Comparative results (MMD (mm)) of various methods underthree different data split settings on LUGW dataset.MethodData used in training (100% stands for 5040 entries)50% 30% 10%Validation Testing Validation Testing Validation Testing1DCNN 10.7 19.0 14.1 26.7 25.1 34.8TVDI+CNN 12.0 27.6 15.1 32.5 25.1 39.9STFT+CNN 6.26 9.97 8.19 12.2 13.3 19.3FCNN 5.94 13.4 9.04 16.2 23.6 26.9Se-ResNet-18 4.82 8.00 8.31 12.7 11.0 16.8CBAM-ResNet-18 7.61 10.5 8.83 11.8 16.5 22.9Ours 3.79 5.03 4.06 5.56 10.0 12.4644.5 SummaryIn recent years, there has been a growing trend toward using DNNmodels to locate damage on plate-like composite structures using UGW.To achieve accurate damage localization, the transducer coordinates anddamage-related features from the UGW signals are two major components.Together, these factors determine the accuracy of damage location. How-ever, previously published studies did not incorporate the low-dimensionalcoordinates of transducers into the neural network, which reduced estimationaccuracy and limited the generalization ability for practical applications. Toaddress this, a multi-dimensional data fusion DNN model was developed toprocess both low-dimensional transducer coordinates and high-dimensionalUGW signals. Specifically, a CNN-based encoder was adopted to extractfeatures indicative of damage from the high-dimensional wave signals. Moreimportantly, the low-dimensional coordinates of transducers were integratedinto the neural network through an RFF projection head. An FCNN wasused to learn features from both the encoder and projection head to providethe final location prediction. The proposed method achieved outstandingresults, with an average absolute distance error of around 2 mm when ap-plied to a CFRP inspection dataset, and demonstrated better generalizationthan SOTA methods. The consistency of providing the best performance onboth datasets indicates the superior generalization ability of the proposedmethod for both linear and nonlinear UGW techniques. Furthermore, theproposed method was assessed under three data split scenarios50%, 30%,and 10% of the data used for trainingeach corresponding to a decreasingamount of training data and thus increasing the level of challenge. Theproposed method demonstrated the best performance across the differentvolumes of training data. In summary, the proposed method showed themost effective performance across all training data volumes, suggesting ithas superior generalization capabilities and robustness to data scarcity inthis regression task on the LUGW dataset.65Chapter 5Contrastive Learning forDomain GeneralizedComposite DamageLocalization 45.1 OverviewThe previous chapter addressed the implementation of deep learningmethods for damage localization, assuming identical data distributions fortraining and testing. However, in practical composite damage localizationtasks, a more commonly scenario involves discrepancies between the trainingand testing data distributions. For example, if the model is trained solelyusing a specific damage size, its performance could decrease when dealingwith damage sizes that deviate from the one used during the training phase.This discrepancy is often observed due to the high costs associated withdata acquisition in industrial settings. Training a model under one measure-ment configuration for application across various scenarios is a pragmaticapproach in industry. Such challenge, known as the domain generalizationproblem, involves bridging the gap between the training data (source do-main) and the testing data (target domain). A key strategy in addressingthis challenge with deep learning involves optimizing the model to enhanceits resilience to such variations. This chapter develops a novel domain gen-eralization framework that employs contrastive learning to improve damagelocalization under these conditions.4A version of chapter 5 has been published. Yun H, Wang R, Rayhana R, PantS, Genest M, Liu Z. WaveCLR: Contrastive learning of guided wave representations forcomposite damage identification. IEEE Transactions on Instrumentation and Measure-ment. 2024 Apr 8. I conducted all the method development, experimental validation, andmanuscript writing.665.2 Domain Generalization Problem in DamageLocalization5.2.1 Problem DefinitionIn the field of composite damage localization, discrepancies between thetraining and testing stages are often encountered due to the complex na-ture of UGW signal and the inspection environments. For instance, duringthe damage localization task, variations in damage size between the trainingand inference phases can introduce inconsistencies in the UGW data. If suchvariations are not accounted for during model development and training, therisk of overfitting increases, which can result in the model underperformingon unseen data. In the given example, although damage size may alteraspects of the guided wave signal, like its amplitude, it usually does notsignificantly impact essential features such as the ToF, which is vital foraccurate damage localization. To improve the modelâs performance amidstsuch distributional discrepancies, our strategy concentrates on enhancingthe modelâs resilience to these variations, effectively boosting its robustnessto change. To tackle this challenge, this study introduces a new domaingeneralization method in this chapter, named WaveCLR, for damage local-ization in composite materials using UGW. This method aims to improvethe modelâs generalization ability to the target domain by maximizing mu-tual inter-class distance and minimizing intra-class distance of features inthe source domain.A domain defined as D = {X ,Y} contains explicit data space X andcorresponding label space Y. In the meantime, the implicit functions aredefined task-wise as T = {P (X),F(Â·)}, where the P (X) is the marginaldistribution of the data and F(Â·) is the predictive function.Given the domain generalization problem, let the Ds represents single-source domain:Ds = {(xsi ,ysi )}nsi=) (5.1)where xsi â RNinput is the input with Ninput data points, ysi â RNc is thelabel with Nr conditions, and ns is the number of data entries in the sourcedomain. Likewise, the target domain that contains nc data entriesDc = {(xcj , ycj )}nTj=) (5.2)Given the possible input data variation, there is a distribution discrep-ancy between the source domain and the target domain: P(XS) Ì¸= P (Xc ).And it should be noted that the label spaces in the source and target do-mains are identical; that is, YS = Yc . The objective of this study is to67develop a DNN model capable of identifying damage within both the sourceand target domains, without the necessity of accessing target domain dataduring the training phase.5.2.2 Domain Generalization of Deep Neural NetworkModelsAlthough previous study has made significant achievements in applyingDNN to damage localization with UGW, it relies heavily on the assumptionthat the training and testing data follow the xIxIsI principle. Such prerequi-site can be achieved when there is large volume training data to representthe all data space. However, in the context of composite damage localiza-tion, training data is often limited due to the high costs and time-consumingnature of data collection. This limitation causes the available data may onlycover a portion of the potential inference scenarios. Consequently, there isprobably a distribution discrepancy between the data used in the trainingand inference stages.Neglecting such discrepancy could lead to overfitting on training dataand poor performance in the inference stage. As shown in Table. 5.1, con-sidering the training data and testing data have individual distributions inthe source domain and target domain, respectively. As discussed in Chapter4, the standard deep learning application require the data in source domainand target domain to follow the xIxIsI principle. The domain adaptationapplication lift the same distribution requirement but need unlabelled datafrom the target domain to train the model, whereas the domain generaliza-tion methods only need data from the source domain to train the model. Inaddition, there is data distribution difference between source domain andtarget domain data.5.3 Contrastive Learning for DomainGeneralization Damage LocalizationTo enhance the generalization capabilities of deep learning models, thisstudy proposes to mitigate the overfitting by maximizing the mutual inter-class distance and intra-class distance via contrastive learning. Figure. 5.1illustrates the difference of standard DNN model and proposed method: (a)represents the latent features learned by a standard DNN model, with a hy-perplane distinguishing between classes. However, such a hyperplane maybecome overfitted to the source domain in the absence of optimized inter-68Table 5.1: Comparison of different deep learning paradigms for data require-ments.Problem settings Same domaindata dist. 1Labelledsource dataTarget dataUnlabelled 2 labelledStandard DNN (xIxIsI) 33 3 74 7Domain adaptation 7 3 3 7Domain generalization 7 3 7 71 Require source domain and target domain follow xIxIsI principle.2 Unlabelled data required for training.3 3 represents the component required by the methods.4 7 represents the component not required by the methods.class and intra-class distance. Conversely, as shown in Fig. 5.1 (b), theWaveCLR model proposed herein not only facilitates basic classification butalso aims to increase both inter-class and intra-class feature distances, thusenhancing domain generalization. This expanded feature space is crucial forreducing misclassification in the target domain. The maximized distanceamong features increases the tolerance region for data distribution discrep-ancy caused by variations of the input, allowing for target domain data tobe correctly classified within this expanded feature space.5.3.1 Model ArchitectureContrastive learning is designed to distinguish between similar and dis-similar data pairs by pulling representations of similar ones closer and push-ing away those of dissimilar ones. However, a limitation of this approachis its typical dependency on substantial volumes of training data labelledacross a large array of data classes. Conventionally, a positive pair in con-trastive learning is defined by two data samples with identical labels, whereasa negative pair is characterized by samples with differing labels. It is usu-ally assumed that a mini-batch contains only one positive pair. Due to theprevalent use of random sampling in deep learning, a substantial diversity ofdata classes is required to increase the likelihood of having a single positivepair per mini-batch. Yet, in industrial applications, the amount of availabletraining data and the variety of label classes are often limited. This limita-tion becomes evident when a mini-batch size exceeds the number of classes,resulting in multiple positive pairs within a single mini-batch, thereby chal-69Damage A (a)ð!ð"ð!ð"(b)Damage B Damage CIncorrectly classified samplesFigure 5.1: Illustration of learned feature space of different deep learningparadigms. (a) Standard deep learning: models are trained for source do-main data fitting without considering maximizing mutual feature distance.(b) Proposed domain generalization methods. Model performance in the tar-get domain is improved by pulling inter-class and intra-class features awayfrom each other in the source domain as much as possible. © HongguangYunlenging the fundamental principles of contrastive learning loss.To address this issue, this paper proposes to use a stratified samplingstrategy in addition to the standard random sampling. To be specific, onesample from each class is selected to construct a mini-batch for both raw andaugmented data. The mini-batch size, determined by stratified sampling,matches the number of data classes Nr, ensuring the presence of only onepositive pair in a mini-batch. The stratified sampling maintains the func-tionality of contrastive learning mechanism in a small label space, therebyaccelerating the model learning process.Figure. 5.2 illustrates the proposed WaveCLR model, where a dual-pathway architecture is designed to process UGW signals for damage lo-calization. The raw signal is augmented to together constitute the sourcedomain data. Data in the source domain are then subjected to two distinctsampling methods: random sampling and stratified sampling.For the pathway with random sampling, a standard deep learning train-ing logic is adopted, where the raw data and augmented data are mixed andrandomly sampled as input. The input is then processed by a feature ex-tractor and a classifier, and optimized by a cross-entropy loss function. Thefeature extractor is built with six convolutional blocks, where each of them70consists of a series of convolutional layers as shown in Figure. 5.2. Thefeature extractor is expected to extract damage-related features from theinput data. The classifier is constructed with three fully connected layers.This pathway ensures that the model learns the primary classification task.The second pathway is designed to fine-tune the model in feature spacewith a contrastive learning paradigm. This involves a distinct process whereboth raw and augmented data undergo stratified sampling independently.Within this framework, corresponding data points with identical labels fromthese two mini-batches are designated as positive pairs for the purposes ofcontrastive learning. Subsequent to a shuffling operation, these mini-batchdatasets are fed into the model and processed by a feature extractor anda feature projection module. The feature projection module comprises twocomponents: an active head and a lazy head. The active head is optimizedby the back-propagation, whereas the lazy head is momentum updated byits legacy parameter and the active headâs parameter. The output featuresfrom both raw and augmented data are then concatenated and subjected tooptimization through contrastive learning loss functions.The feature projection is dedicated to transforming features into a spacewhere contrastive loss can be effectively applied, facilitating the model tomaximize the inter-class and intra-class distance. A detailed mechanism ofthis projection will be expressed in the following subsection. It is noteworthythat the parameters of the feature extractor is shared across both pathways.5.3.2 Mutual Inter-class Distance MaximizationIn a standard contrastive learning framework, given input Xi and Xjsampled from raw data and augmented data in source domain, the corre-sponding feature representations outputted by model are denoted as ri andrj as shown in Figure. 5.3 (a). The resulting feature representations arethen optimized by contrastive learning loss function. As shown in Figure.5.3 (b), within a single training iteration, contrastive loss push one positivepair away from all other negative pairs. In next iteration, this positive pairis randomly changed to another class, and the model try to push the posi-tive pairs away all other negative pairs again. By repeating this process, themodel learns to push each class away from another class. It is obvious thatthis framework simplifies a multi-class classification into a multiple binaryclassification tasks in the training process and thus cause two limitations: (1)Since the model is optimizing multiple different binary classification in everyiteration, it is difficult to converge. (2) This design fails to maximize themutual inter-class distance across all classes due to the binary classification71Raw input ððClass 1Class 2Class 3Class 4Class 5Class 6Class 7ShuffleConv 4 x 1 - 16Conv 4 x 1 - 16Batch norm & ReLUMaxpooling 2 x 1In channel featuresOut channel featuresResidual connectionMaximize inter-class andIntra-class feature distance Random samplingStratified samplingConvolutional feature extractorConvolutional feature extractorsharedFeature projectionClassifier Cross-entropy lossInput ððâ²Batch norm & ReLUInter-class and intra-class distance lossFCNN 512ReLUFCNN 32FCNN 512ReLUFCNN 32Active head Lazy headMomentum updateData augmentationSource domainFigure 5.2: Illustration of the WaveCLR architecture with a dual-pathwayin the training process. © Hongguang Yunsimplification.To this end, this paper proposes a two projection head structure witha momentum update strategy. As shown in Figure. 5.3 (c), the featureprojection p (â¢ : θp) consists of an active head and a lazy head. The activeheadHA (â¢ : θa) and lazy headHL (â¢ : θl) have same structures but distinctparameters θa and θl, respectively. The composition of the projection headcan be found in Figure. 5.2.Upon processing the inputs through f (â¢ : θf ), the feature extractor, thefeature projection yields two mini-batches of representations, each generatedby a distinct projection head:rp =HA (f (X))rl =HL (f (X))(5.3)where rp, and rl contain the batch of representations from the active headand the lazy head, respectively.As mentioned before, to force the model to optimize the mutual inter-class distance, triggered by a contrastive learning method called MoCo [135],this paper proposes to use a momentum update strategy on the parametersof the lazy projection head. The initial weights of the lazy head are dupli-cated from the active head, and the weights after t+1 iteration are updated72as:θt+1l = mθtl + (1âm)Î¸t+1a (5.4)where m â {0, 1} is the momentum coefficient. Through momentum up-dating, the lazy head integrates information from past iterations. As shownin Figure. 5.3 (d), during the training process, the optimized feature spacereflects the distances between current and past positive-negative pairs, fa-cilitating the maximization of the mutual distances across all classes. Thisdesign can benefits the model learning by: (1) The lazy head keeps it con-sistence on the feature representation, and (2) it accounts for the mutualdistance with the momentum update strategy.To calculate the contrastive learning loss, define a distance metric sim(u,v) =uâ¤vLâuââvâ to denote the o2 normalized dot product between u and v. Thedistance among the positive and negative pairs is estimated by calculatingthe normalized temperature-scale cross-entropy loss [136]:LCL = âNclâi=)logexp (sim (ri,p, ri,l) LÏ)â2Nclk=) 1[rk Ì¸=ri,a] exp (sim (ri,p, rk) LÏ)(5.5)where {rk} includes all representations from active head and lazy head, Nrlis the batch size for the contrastive learning, 1[rk Ì¸=ri,a] â {0, 1} is an indicatorfunction that equals to 1 if rk Ì¸= ri,p, and Ï is the temperature parameter tocontrol the loss scale.5.3.3 Intra-class Distance MaximizationThe proposed method effectively maximizes the mutual inter-class dis-tance. However, an inherent aspect of contrastive learning loss is its ten-dency to draw positive pairs closer, thereby reducing the intra-class featurespace. This reduction is sub-optimal for domain generalization, as it leadsto an overly condensed distribution within each class feature space. Such aprecise, yet narrowly defined feature space, reduces the tolerance to inputvariations from the target domain. To mitigate this issue, our study pro-poses to expand the intra-class feature space, thereby enhancing the modelâscapabilities for domain generalization.Given the normalized features rp,i and rp,j from the active projectionhead of raw data and augmented data, the Cosine similarity among all datasamples in a mini-batch can be computed as:Lsim = rp,i · rp,jmax (ârp,iâ2 · ârp,jâ2, ε) (5.6)73(a) (b)(c) (d)Figure 5.3: Illustration of traditional and proposed contrastive learningstructure and training mechanism: (a) traditional contrastive learning struc-ture, (b) learning mechanism of traditional contrastive learning, which sim-plifies multi-class classification to multiple binary classification (c) proposedWaveCLR structure, and (d) momentum update in proposed WaveCLR tomaximize mutual inter-class distance. © Hongguang Yun74where rp,i · rp,j denotes inner product. The denominator of the fractiontakes the maximum of the product and a small constant ε to ensure it isnot zero, where ε is a small positive constant that avoids division by zero.The equation measures the cosine of the angle between two representations,providing a similarity score between -1 and 1, where 1 indicates that thevectors are in the same direction, 0 indicates orthogonality.The rationale behind employing Cosine similarity, rather than other dis-tance metrics such as Euclidean distance, lies in its specific operational focus.Cosine similarity primarily seeks to expand the feature by rotating the rep-resentations in their own sub feature space, rather than force the featuresfrom one another. This approach helps to prevent the features from enteringinto the feature space of a different class. In other words, combined withthe aforementioned contrastive learning loss employed, this measure ensuresthat the inter-class features do not become closely aligned, thereby avoidingpotential misclassification.5.3.4 Training strategyTo optimize the performance of the proposed method, this study has de-veloped a tailored training strategy. The concurrent optimization of lossesfor both mutual inter-class and intra-class distances can lead to conflicts,potentially impeding model convergence. To address this issue, a two-stagetraining strategy is introduced to streamline the training process. In the ini-tial phase, the modelâs focus is on optimizing the mutual inter-class distanceloss and multi-class classification loss, while the intra-class distance loss istemporarily detached from the learning graph. Once the model achieves asatisfactory level of classification accuracy, the intra-class distance loss isthen attached to the learning graph. As the backbone of the model, thecross-entropy loss for classification will be optimized throughout the wholeprocess.It is critical to adjust the learning rate at different stages of the training.Initially, when the primary objective is to identify the optimal hyperplanefor class separation, a relatively higher learning rate is beneficial. Thisfacilitates rapid localization and adjustment towards the best classificationboundaries. Conversely, in the subsequent phase where the emphasis shiftsto fine-tuning the hyperplane within a localized optimal region, a reducedlearning rate is preferable to allow for more precise adjustments. In ourimplementation, the model was trained with a learning rate of 1tâ 3 in thefirst stage, followed by a reduced rate of 9tâ M in the second stage.Overall, there are three objective functions to optimize in the WaveCLR75model: classification loss, mutual inter-class distance loss, and intra-classdistance loss. The classification c (â¢ : θc) loss is calculated with the cross-entropy loss:LC = âE(xsr,ysr)âDs sLCT (c(f(xsr)),ysr)] (5.7)where LCT is the cross-entropy function. The whole objective function isexpressed as follows,L = LC + LCL + Lsim (5.8)5.4 Experimental Results and Discussion5.4.1 Data DescriptionTo validate the effectiveness of the proposed method in improving thedomain generalization performance for composite damage localization, thisstudy conducted two distinct experiments using the two dataset, each ad-dressing different aspects of input variation. In this chapter, the damagelocalization is defined in a multi-class classification task.1. For the impact damage inspection, the experiment was designed toassess the effects of minor location variations on damage localization.Still, transducer 1, 3, 5, and 7 were individually used as the trans-mitter to show the generalization ability of the model. As shown inFigure. 5.4, due to the lack of sufficient real impact damage datafor training, the inspection performed on the three disks was utilizedfor this experiment. To address the domain generalization challenge,a three-fold cross-validation approach was adopted based on damagesize: the training dataset included samples from only one damage size,while the validation dataset comprised data from the remaining twodamage sizes.2. For the LUGW dataset, the focus was shifted to examining the impactof varying damage sizes on the damage localization process. Transduc-ers 1 to 6 were used individually as transmitters, and transducers 7 to12 were used as receivers simultaneously. Different to the setting inthe Chapter 4, as illustrated in Figure. 5.5, in this setup, the 28 dif-ferent locations were categorized into 7 distinct regions. To define thedomain generalization problem, data corresponding to the same regionwere labelled identically, establishing four locations (domains) within76each region. A four-fold cross-validation method was employed. Thecategorization of the data is detailed in Table. 5.2, where one samplefrom each class was selected to constitute a data group. In the train-ing phase, only data from one group was utilized, while the remainingthree groups were reserved for the inference stage. This approach re-sulted in the definition of four distinct tasks, as delineated in Table.5.3.Figure 5.4: The composite plate being inspected with eight transducersmounted and three disk damages. © Hongguang YunTable 5.2: Definition of the data group according to the location of damages.Data group Source damage1 1, 5, 9, 13, 17, 21, 252 2, 6, 10, 14, 18, 22. 263 3, 7, 11, 15, 19, 23, 274 4, 8, 12, 16, 20, 24, 2877Figure 5.5: Configuration of 12 transducers and 28 different damage loca-tions [2]. Used with permission from Sage.Table 5.3: Definition of classification tasks according to the data groups inthe source and target domain.Task Source data group Target data group1 1 2, 3, 42 2 1, 3, 43 3 1, 2, 44 4 1, 2, 378To enhance the datasetâs diversity and generate positive data pairs formodel training, data augmentation was performed subsequent to filteringand baseline subtraction. The characteristics of the UGW, including ToFand amplitude difference, play a pivotal role in identifying damage in com-posite materials. Our data augmentation methods aimed to simulate inputvariations resulting from changes in testing conditions, while preserving thefundamental characteristics of the wave signals. Notably, the sequentialcomposition of wave packets reflects the propagation of signals across trans-ducers, boundaries, and damages, which is essential for damage localization.This study avoided augmentation techniques such as rotation or croppingthat could significantly alter the waveform of the wave signals. The meth-ods of data augmentation employed in this study are as follows, with eachmodification to the original wave signals being minor to closely replicate theinput variations:1. Gaussian noise. This involves adding noise to the wave signal, withthe noise being sampled from a Gaussian distribution with designatedmean and variance values.2. Magnitude wrap. The wave signal is multiplied by a wrap factor,which is sampled from a Gaussian distribution with designated meanand variance values.3. Time shift. The entire wave signal is shifted along the time axis by arandom length.Figure. 5.6 and 5.7 display the original wave signals and augmented wavesignals of two datasets. It is worth noting that the baseline and testing wavesignals were de-noised by a Butterworth band-pass filter.5.4.2 Ablation StudiesIn this study, an ablation study was performed to justify the effectivenessof the proposed WaveCLR. Since the main contribution of the proposedmethod is based on the contrastive learning paradigm, the baseline modelfor the ablation study is set as the 1D ResNet-18 with SimCLR architecture.Regarding the novel designs in the WaveCLR model: (1) stratified samplingstrategy, (2) momentum update, (3) intra-class distance maximization, and(4) training strategy, six models were built for comparison:1. A convolutional model based on 1D ResNet-18 for multi-class classifi-cation was built, and no contrastive learning mechanism added.79Figure 5.6: Illustration of ultrasonic guided waves of baseline and damagesignal from LUGW dataset with one transmitter and six receivers. © Hong-guang YunFigure 5.7: Illustration of ultrasonic guided waves of baseline and damagesignal from NUGW dataset with one transmitter and seven receivers. ©Hongguang Yun802. The SimCLR-based [136] contrastive learning component was addedto the model 1), the input were randomly sampled from the augmentedsource domain.3. Compared to model 2), the stratified sampling was used for contrastivelearning, and the momentum update was not applied.4. The stratified sampling and momentum update were used, which canrealize the mutual inter-class distance maximization.5. The WaveCLR model with a standard training strategy, where allthree objective functions were optimized simultaneously.6. The WaveCLR model trained with proposed two-stage training strat-egy.Table. 5.4 provides a detailed ablation study, assessing the impact ofvarious features on the performance of six models across two datasets, usingthe T1 transmitter. The cross mark denotes features that were excludedfrom a given model, while the check-mark indicates included features. Thebase model served as a baseline. Model 2, incorporating contrastive learn-ing without stratified sampling, displayed a decrease in accuracy on bothdatasets, whereas the model 3 with stratified sampling benefits the modelperformance. The inclusion of momentum update in Model 4 led to an in-crease in accuracy on both datasets. The most notable results were observedin Model 6, which integrated all features including the two-stage trainingprocess. This model achieved the highest accuracy of 92.411% on NUGWand 87.165% on LUGW, signifying the effectiveness of the proposed method.The model 5 with standard model yields a decrease due to the difficulty inoptimizing the inter-class and intra-class distance simultaneously.Table 5.4: Accuracy results (%) of ablation study on six models across twodatasets using transducer T1 as transmitter.Model Contrastive Stratified Momentum Intra-class Two-stage Accuracy on Accuracy onloss sampling update maximization training NUGW LUGW1 7 7 7 7 7 80.342±12.214 80.249±6.4282 3 3 7 7 7 80.387±10.484 72.135±8.2453 3 3 3 7 7 81.134±15.832 81.090±6.5744 3 3 7 3 7 84.866±8.987 83.494±2.2815 3 3 3 3 7 83.557±14.655 82.192±7.4686 3 3 3 3 3 92.411±4.725 87.165±3.878815.4.3 Comparative Studies with SOTA MethodsTo validate the effectiveness of WaveCLR, comparative studies withSOTA DNN-based models, considering domain generalization, were per-formed on two datasets. The methods description and selection criteriaare as follows:1. 1D ResNet-18[102]. Since the residual connection is treated as astandard paradigm in the deep neural network models, a 1D CNNmodel based on ResNet-18 architecture is built. The rest 5 methodsfor comparison are modified based on the 1D ResNet-18.2. MMD [137]. The maximum mean discrepancy (MMD) is a widelyused distance metric in domain generalization methods. It enables theneural network to learn a kernel that minimizes the data distributiondiscrepancy between the source and target domains. The MMD loss iscalculated on the representation extracted from the last FCNN layerbefore the output layer. The MMD-based model serves as the startingpoint for damage localization using DNN models with considerationof domain generalization.3. MK-MMD [138]. The multi-kernel MMD (MK-MMD) is an en-hanced version of MMD for domain adaptation. The representationsof the data from the last two FCNN layers are used to calculate theMK-MMD loss. The data distribution of the source and target do-mains is significantly aligned. However, this alignment may blur thedistinguishing characteristics between the source and target domains.4. CNN-C [139]. Since there are limited studies using DNN models forcomposite damage localization with consideration of domain general-ization, this study selected relevant studies in the fault diagnosis fieldfor comparison. CNN-C is an emerging DNN model proposed for faultdiagnosis, which considers domain generalization by utilizing centreloss to enhance intra-class clustering and enlarge inter-class distance.This model was implemented for composite damage localization in theproposed study.5. DGNN [140]. DGNN is another emerging adversarial domain gener-alization method that uses fake data to let the neural network learnmore generalized representations of the data. Metric learning is alsoused to pull data together within the class.826. SimCLR [136]. The improvement in domain generalization ability inthe proposed method is based on the contrastive learning mechanism.Thus, emerging DNN models featuring contrastive learning were alsoimplemented for composite damage localization in this study for com-parison. SimCLR is a representative method in the field of contrastivelearning. In this study, SimCLR was trained using a supervised learn-ing protocol for damage localization problems.The proposed WaveCLR was implemented for composite damage local-ization. One of the key components in the proposed method is inter-classfeature maximization with a moment updating strategy. The momentumparameter is the hyper-parameter in mutual inter-class loss optimization,which controls the influence of the previous iterationâs learning on the cur-rent iteration. Generally, the momentum value is highly related to the num-ber of classes. Assuming that data from each class contribute equally to themodel training, to make the model benefit from all data every Nr iteration,the momentum value should be set as 1â 1LNr.Results on the NUGW dataset The comparison experiments weretested with guided waves using different transducer as the transmitter in-dividually. Classification accuracy and F1 score were used as evaluationmetrics in this paper. The average and standard deviation of accuracy andF1 score are calculated.The Table 5.5 presents the comparative results of classification accuracyachieved by various methods on the NUGW dataset when utilizing differenttransducers as the transmitter. Among the methods evaluated, WaveCLRexhibits superior performance across all transmitters, underscoring its effec-tiveness. For Transmitter 1, WaveCLR achieves a noteworthy accuracy ofaround 92.411%, significantly outperforming other methods such as ResNet-18, MMD, and SimCLR. Two domain adaptation methods, MMD and MK-MMD even damaged the performance compared to the baseline. The SOTAdomain generalization method, DGNN achieves most closed accuracy com-pared to WaveCLR when using transmitter 7, while still behind WaveCLRacross all transmitter. The standard deviations associated with WaveCLRâsaccuracy suggests a consistent performance with less variability in results,reinforcing the robustness of the method. Table 5.6 gives the correspondingF1 score performance of various methods on the NUGW dataset. WaveCLRmethod demonstrates consistent performance compared to its counterparts,consistently achieving the highest F1 scores across all transmitters. The con-sistency in WaveCLRâs F1 scores indicates a robustness that is beneficial for83practical applications that requires stability in prediction.Table 5.5: Comparative accuracy (%) results of various DNN-based methodson NUGW dataset using different transducers as the transmitter.Methods Transmitter 1 Transmitter 3 Transmitter 5 Transmitter 7ResNet-18 80.342±12.214 76.637±6.259 76.637±19.852 77.530±15.034MMD 69.792±2.322 68.333±1.804 67.083±2.009 68.125±1.654MK-MMD 12.500±3.125 13.542±3.609 11.458±6.505 10.417±4.774CNN-C 83.333±28.868 77.083±20.091 86.458±23.454 87.500±11.267DGNN 85.119±12.489 81.696±18.097 83.661±19.070 88.869±8.464SimCLR 80.134±15.832 77.530±3.237 69.568±25.479 75.446±21.661WaveCLR 92.411±4.725 87.574±9.787 86.488±9.487 90.685±2.600Table 5.6: Comparative F1 score (%) results of various DNN-based methodson NUGW dataset using different transducers as the transmitter.Methods Transmitter 1 Transmitter 3 Transmitter 5 Transmitter 7ResNet-18 85.752±11.901 77.686±4.069 77.021±19.497 79.013±13.685MMD 67.301±2.635 64.260±1.316 64.847±4.529 65.574±5.593MK-MMD 26.460±35.810 3.571±0.002 66.749±9.324 73.056±7.933CNN-C 79.434±35.621 74.815±22.419 84.124±27.498 86.314±11.902DGNN 85.562±12.160 83.116±16.021 84.379±18.662 89.926±6.469SimCLR 80.958±15.332 77.678±2.823 69.687±24.956 76.511±19.619WaveCLR 92.504±4.627 88.016±9.369 87.577±8.841 90.685±2.507Results on the LUGW dataset The comparative analysis were alsoperformed on LUGW dataset with using six different transducers as thetransmitter individually. Table 5.7 lists the accuracy results of all modelson the data from the target domain. The result shows that WaveCLR sig-nificantly outperforms all other methods. Two domain adaptation methodsand two domain generalization methods under-performed relative to theResNet-18 baseline across the majority of transmitters. Specifically, onlyCNN-C and DGNN slightly improve model performance with transmitter5. Regarding the average accuracy value, WaveCLR consistently achievesabove 84%, with a standout performance using transmitter 5, achieving anaccuracy surpassing 91%. WaveCLR is distinguished by smaller standarddeviations relative to its counterparts, underscoring the methods robustness84across various transmitters. Table 5.8 details the corresponding F1 scoreof all models on the LUGW dataset. The results of the F1 score are inhigh agreement with the accuracy, and our model still outperforms all othermodels.Table 5.7: Comparative accuracy (%) results of various DNN-based methodson LUGW dataset using different transducers as the transmitter.Methods Transmitter 1 Transmitter 2 Transmitter 3 Transmitter 4 Transmitter 5 Transmitter 6ResNet-18 80.249±6.428 80.149±11.236 79.728±10.451 77.945±10.093 78.445±5.723 85.357±7.957MMD 78.345±4.046 77.643±7.310 74.342±10.416 79.486±15.675 78.874±11.764 86.049±6.120MK-MMD 78.861±3.937 80.371±6.522 73.535±8.841 77.923±17.268 78.691±12.278 83.594±5.902CNN-C 62.298±14.487 73.750±7.794 79.310±6.855 78.398±11.529 92.637±3.807 84.147±6.866DGNN 57.812±9.289 69.491±9.688 67.327±3.514 69.291±9.389 80.929±16.166 62.340±11.805SimCLR 82.091±6.782 75.520±7.507 74.559±12.071 75.521±15.502 82.740±7.309 78.926±3.787WaveCLR 87.165±3.878 87.642±4.382 84.776±6.385 84.515±3.180 91.927±4.174 88.902±4.148Table 5.8: Comparative F1 score (%) results of various DNN-based methodson LUGW dataset using different transducers as the transmitter.Methods Transmitter 1 Transmitter 2 Transmitter 3 Transmitter 4 Transmitter 5 Transmitter 6ResNet-18 79.796±6.101 77.903±14.507 78.160±11.965 76.451±11.231 77.306±5.688 84.249±9.211MMD 78.040±4.102 75.130±9.523 72.884±12.632 76.943±18.496 77.730±12.669 85.696±6.719MK-MMD 78.621±4.001 78.592±8.739 72.108±10.810 74.895±20.623 77.655±13.167 82.423±8.117CNN-C 56.200±17.568 69.847±11.547 77.504±8.850 76.839±12.297 92.254±4.137 82.207±8.718DGNN 56.091±9.255 68.271±9.733 65.469±4.478 68.269±9.788 80.793±16.220 60.062±12.537SimCLR 80.584±7.314 73.754±8.315 72.279±14.879 72.403±16.716 81.436±6.186 78.327±4.433WaveCLR 86.985±4.391 86.863±4.658 83.589±7.079 83.080±3.817 91.488±4.753 88.198±5.0405.5 SummaryIn industrial applications, variations in input data between the train-ing stage and the inference stage are frequently observed. Such variationscause data distribution discrepancies between the training data and testingdata, leading to a drop in model performance during use. Most DNN-basedmethods for composite damage localization failed to consider such varia-tions cause the poor performance in inference stage. To this end, this studyproposed a domain generalization method to address this issue for damagelocalization in a multi-class classification task. To reduce the effect of distri-bution discrepancy caused by input variations, a inter-class and intra-classfeature distance maximization mechanism was proposed. The maximizationof mutual inter-class distance was achieved through a contrastive learningframework, empowered by an innovative momentum update strategy. Mean-85while, the maximization of intra-class distance was accomplished by reduc-ing intra-class feature similarity. The efficacy of WaveCLR was validatedon both NUGW and LUGW datasets, which account for different typesof input variations: one considering minor location fluctuations and theother focusing on damage size variations within a multi-class classificationcontext. When compared with leading DNN-based models for compositedamage localization, WaveCLR has demonstrated superior performance androbustness. This is evidenced by achieving an average accuracy of around90% across comprehensive experiments, whereas other DNN-based methodshave accuracy below 85%.86Chapter 6Physics-informed NeuralNetwork forOut-of-distributionComposite DamageLocalization6.1 OverviewIn the preceding chapter, the domain generalization problem in compos-ite damage localization was examined, underlining the significant impact ofinput variation, such as varied damage sizes on the performance of deepneural network models during the inference stage. The proposed WaveCLRframework, based on contrastive learning, effectively mitigates the data dis-tribution discrepancies between the training and inference datasets.Notwithstanding the outstanding performance enhancements achievedby the WaveCLR framework, real-world applications of damage localizationmay encounter more stringent implementation conditions, where the infer-ence data could be completely OOD relative to the training data. Specifi-cally, damage may manifest at any location on the specimen, contradictingto those hypothesized during the training phase. This scenario is particu-larly common in industrial contexts, where limitations imposed by budgetaryconstraints and efficiency requirements result in a significantly limited avail-ability of training data.Addressing the scarcity of training data, this chapter presents the de-velopment of PINN. Consistent with the overall themes of this study, dataavailability and physical knowledge embedding, this chapter details the de-velopment of a PINN tailored for OOD damage localization. By decompos-ing the complex problem of damage localization into a two-stage framework,the proposed PINN comprises two synergistic components, each optimized87through objective functions imbued with explicit physical knowledge.6.2 Out-of-distribution Problem in DamageLocalization6.2.1 Problem DefinitionThe preceding chapter discussed the domain generalization problem incomposite damage localization, marked by data distribution discrepanciesbetween training and inference datasets. It demonstrated that even minorinput deviations could cause significant predictive changes. However, in real-world industrial applications, generalized application scenarios may presentmore extreme conditions. Here, the inference data could be entirely OODrelative to the training data.In this chapter, we address the OOD damage localization regressionproblem. Let us denote the set of training data as Xtrpin = {x), x2, I I I , xn},comprising UGW captured from multiple transducers, alongside measure-ment information such as transducer coordinates. These features follow adistribution Ptrpin(X,v), with vtrpin = {y), y2, I I I , yn} representing thecorresponding damage locations on the plate, expressed as coordinates.For the testing phase, Xttst and vttst correspond to the input dataand damage locations, drawn from a distribution Pttst(X,v), distinct fromPtrpin, i.e., Ptrpin Ì¸= Pttst. The objective is to learn a mapping functionf : X â v that predicts damage locations vttst on the composite platewith minimized loss over the testing distribution.As highlighted previously, obtaining sufficient data for training to covermost of the scenarios in inference is economically challenging. Neural net-work modelsâ dependence on the training data distribution constrain theirgeneralization capabilities. This limitation is pivotal, especially when thetask is regression, where model outputs will not be limited within severalclasses. Consequently, during the training stage, the model should be moreheavily constrained by embedded physical knowledge to give reasonable pre-dictions. To summarize, OOD damage localization involves identifying andprocessing data samples not represented in the training dataset, ensuringthe modelâs robustness and reliability in the face of unseen inputs.6.2.2 Revisiting Analytical MethodsFor a deeper understanding of the physical knowledge required for thedeep neural network model, a revisit to the traditional analytical methods for88damage localization is necessary. A primary and prevalent analytical methodis the ellipse method, which utilizes the ToF extracted from UGW for dam-age localization. Consider a sensor network with Np transmitter-receiverpaths capable of introducing and receiving guided waves. The ToF of theguided wave scattered by damage along the xth path (for x = 1, 2, III, Np) isthe sum of the wave propagation times from the transmitter to the damageand then to the receiver. The theoretical ToF is defined as:ci =â(xs â xit)2 + (ys â yit)2eg(f)+â(xs â xir)2 + (ys â yir)2eg(f)(6.1)where xs, ys represent the damage coordinates, xit, yit the transmitter co-ordinates, and xir, yir the receiver coordinates for the xth propagation path.eg denotes the group velocity of the scattered Lamb wave at the excitationfrequency f .By solving the inverse problem of equation 6.1, the damage location canbe determined from ToFs across multiple paths, as illustrated in Figure.6.1. Traditional signal processing methods, such as the continuous wavelettransform, are employed to extract damage-related wave packets for ToFcalculation. However, two primary challenges arise in applying the ellipsemethod:1. The complexity of ToF calculation arises from the dispersive nature ofguided waves, making it challenging to extract damage-related wavepackets using signal processing methods. Consequently, the ToF maybe calculated inaccurately.2. The solution to equation 6.1 is highly ill-posed; its accuracy heavilyrelies on precise ToF calculation, and the solving process is prone toinstability. Minor input changes can lead to significant output varia-tions.As a result, the ellipse method is difficult to apply in the real industrialapplication, thereby highlighting the need for deep learning-based methodsfor reliable damage localization.6.3 Physics-informed Neural Network for OODDamage LocalizationPINN represent a scientific deep learning paradigm, integrating physicslaws as inductive biases within neural network structures. By embedding89Figure 6.1: Illustration of the ellipse method for damage localization: tran-sitioning from a single transmitter-receiver path to multiple paths. © Hong-guang Yunphysical knowledge into the neural network, the learning process is directed,ensuring predictions adhere to physical laws. A PINN typically incorporatesthe physical modelâs structure into the neural networkâs loss function.For the OOD damage localization, the task necessitates a model capableof extrapolation beyond the confines of the training data. To realize thisobjective, the model can not be purely training data dependent but trainedon the physics that governs the problem. Despite the ellipse methodâs lim-itations, it provides crucial physical insights into UGW based damage lo-calization, which this research seeks to harness through a deep learningframework.The transition from raw UGW signals to the final damage location pre-diction is too intricate for a generic deep learning model due to the dualchallenges of extracting physical features from complex waveform and solv-ing highly nonlinear equations. Thus, this study proposes a PINN withcomponents aligned with distinct physical objectives. Following equation6.1, damage localization is divided into two phases. The first phase involvesextracting damage-related information from the UGW signals. The secondphase computes the damage location with extracted information. This de-sign yields multiple benefits:1. The model is constructed with specific components, each with clearobjectives, simplifying the modelâs inherent complexity and thus mit-igating overfitting.2. Embedding substantial physical knowledge reduces the complexity in90the feature space, allowing for model training with a limited datasetand enhancing generalization.6.3.1 Model ArchitectureTo construct an effective PINN model for OOD damage localization,the methodology involves two critical tasks: constructing model compo-nents that correspond to specific physical processes and defining objectivefunctions grounded in physical laws to direct the learning process. In thecontext of model construction, damage localization can be decomposed intotwo subtasks: determining the ToF and estimating the final damage loca-tion. Addressing these subtasks involves designing components within thePINN that can handle the corresponding objectives. For the feature extrac-tor, instead of using actual ToFs as training labels, which can be challengingto calculate, a novel approach using guided wave travel distance, or damage-related Distance of flight (DoF), is proposed. The DoF, which is easier tocompute, serves as a proxy for ToF and obviates the need for accurate time-of-arrival identification. The DoF is defined as the sum of the distance fromthe transmitter to the damage and the distance from the damage to thereceiver. The DoF can be easily calculated as:Si =â(xs â xip)2 + (ys â yip)2 +â(xs â xis)2 + (ys â yis)2 (6.2)Figure. 6.2 illustrates the architecture of proposed PINN model. The ar-chitecture encompasses two primary components: the DoF extractor, whichprocesses signals in parallel to yield DoF features optimized via an L1 lossfunction, and the location solver, which compiles these features to predictdamage location. The location solver, as the second component of the sys-tem, synthesizes the DoF data obtained from all transducer pairs. SinceDoF represents the cumulative distance from the damage to both the trans-mitter and receiver, it can be represented as an elliptical trajectory on theplate. The actual damage location corresponds to the intersection point ofthese ellipses. Thus, the location solver employs a two-stage computationalapproach within the PINN model. Initially, it computes the ellipse featurefrom each DoF independently for all transmitter-receiver pairs. These ellipsefeatures are then concatenated and input into the final prediction module,which estimates the damage coordinates. The optimization of these pre-dictions is conducted via a mean squared error loss function, assessing thefidelity of the predicted damage coordinates in relation to the true val-ues. This architecture is methodically crafted to progressively incorporate91Figure 6.2: Architecture of the proposed physics-informed neural networksfor damage localization. © Hongguang YunFigure 6.3: Illustration of the standard recurrent block in recurrent neuralnetworks. © Hongguang Yunphysics-informed algorithm within the neural network computation, withthe goal of achieving precise OOD damage localization.6.3.2 Distance-of-flight ExtractorThe DoF extractor aims to estimate the distance travelled by UGWfrom each transmitter-receiver pair. Usually, the CNN model has kernelwith limited length, such as a standard 3x3 kernel, making it has localizedreceptive field. This characteristic limits its ability to connect data pointsfar away to each other. By contrast, as shown in Figure 6.3, recurrent neuralnetwork (RNN) is designed for time-series data such as UGW due to theircyclic connections, allowing them to maintain a dynamic contextual windowover the input sequence. During one cycle, the RNN feeds the output froma previous time step as input to the network to influence predictions at thecurrent time step. The output of each time step will also be stored as hiddenstate as the input of the next layer, which enables the network to memorizethe temporal contextual information.Given the UGW signal with the baseline subtracted, it is assumed that92the wave packets in the received signal are purely scattered by the damageand boundary. The most critical among these complex wave packets is thefirst one, which is directly related to the damage and is essential for de-termining the DoF. Therefore, the model is required to correlate the initialwave packet emitted by the transmitter with the first damage-related wavepacket received. Moreover, the model must possess the capability to iden-tify the local maxima within a wave packet, which is crucial for accuratelypinpointing the data points between two wave packets. However, traditionalRNN models face challenges in capturing long-term dependencies within se-quences, limiting their ability to maintain correlations over long sequences[141].To address this challenge, this study proposes to implement Long Short-Term Memory (LSTM) networks as the core of the DoF extractor. LSTMarchitecture is characterized by a recurrent structure, yet it distinguishes itwith RNN by a more complex design within its memory blocks. As shownin Figure 6.4, in addition to the hidden state in the RNN, a notable featurein the LSTM is the cell state line that runs horizontally at the top of thememory block diagram, ensuring the retention of long-term dependenciesfrom the beginning to the end of the data sequence. These memory blocksadeptly manage the update and storage of the cell state through specializedunits known as gates. There are three gates, specifically the forget gate,the input gate, and the output gate, manipulate the information flow withina memory block. Each gate performs a unique function in the sequentialupdating process of the cell state, ensuring the effective management ofinformation within the LSTM network.Given a sequence of UGW from a transmitter-receiver pair, denoted asx = (x), xt, III, xNt), the corresponding hidden states and cell states are rep-resented as h = (h), ht, III, hNt), r = (r), rt, III, rNt), respectively. As shownin Figure. 6.4, the first function in one memory block involves determiningwhich information is insignificant in DoF calculation and should be forgot-ten. The forget gate, taking xt and htâ) as inputs, outputs a factor valuebetween 0 and 1 for each number in the cell state rtâ). A value of 1 signifiesthe retention of information, while a value of 0 indicates its removal. Inthe context of the determine DoF with UGW, the noise and non-damagerelated wave packets should be forgotten. The forget gate operates througha sigmoid function, where the factor vector is calculated as:ft = Ï (ff · shtâ), xt] + bf ) (6.3)where ff and bf are weight and bias of a linear transformation of the93Figure 6.4: Illustration of forget, input, and output gates mechanism in thelong short-term memory neural networks. © Hongguang Yunconcatenated xt and htâ), and Ï is the sigmoid function.The subsequent step involves determining which new information fromthe current input should be incorporated into the cell state, a process thatengages two mechanisms. Firstly, a tanh function evaluates which informa-tion to add to the cell state. Concurrently, a sigmoid function evaluatesthe significance level of each piece of newly introduced information. Thisprocess is calculated as:xt = Ï (fi · shtâ), xt] + bi)CËt = tanh (fC · shtâ), xt] + bC)(6.4)where fi, bi, fr, br are weights and bias of linear transformation.Following the forget and input gate, the old cell state can be updatedaccordingly. The updated cell state is calculated as:Ct = ft â Ctâ) + xt â CËt (6.5)Finally, the memory block concludes with the output gate, which decidesthe content to be forwarded to the succeeding memory block, combining thepresent input with the data from the cell state. The output gate is calculatedas:ot = Ï (fo shtâ), xt] + bo) (6.6)where fo, bo are weight and bias of a linear transformation.94Figure 6.5: Architecture illustration of the feature extractor of the proposedphysics-informed neural networks. © Hongguang YunAbove are the information flow within a LSTM layer. In addition, theLSTM layer will update the hidden state as follow for the next layer usage.ht = ot â tanh (Ct) (6.7)This study implemented a design comprising six stacked LSTM layer,followed by two fully-connected layers, to estimate the DoF for the locationsolver. The DoF extractor employs a parallel structure, featuring multipleblocks for each UGW signal as shown in Figure. 6.5.Given the presence of multiple transmitter-receiver pairs and the goal tominimize the discrepancy between the predicted and actual DoF for eachpair, the L1 loss equation tailored for this setup can be expanded to accountfor the contributions from all pairs:Ls =1N I I INpNâj=)Npâi=)|sij â sËij |, (6.8)where N is the total number of input in a mini-batch, Np is the number oftransmitter-receiver pairs in each input, sij is the actual DoF in the y â thpair UGW in the x â th input, and sËij is the predicted DoF by the modelfor the same.6.3.3 Location SolverThe location solverâs inputs are the DoFs provided by the DoF extrac-tor. As previously discussed, a single DoF determines an ellipse on theplate, with the actual damage location necessitating the integration of mul-tiple DoFs. Accordingly, the location solver employs a cascade architecture95Figure 6.6: Architecture illustration of the location solver of the proposedphysics-informed neural networks. © Hongguang Yunto align with mathematical principles. As shown in Figure. 6.6, each DoF isprocessed by an individual feature processing header to extract ellipse fea-tures in parallel. Subsequently, these features from all transmitter-receiverpairs are concatenated. Drawing on methodologies from earlier Chapter 4,a wide fully-connected layer is utilized for final coordinate prediction. Thisstudy implements an fully-connected layer with two hidden layers and anoutput layer to estimate the final damage location.It is should be noted that as the DoF is low-dimensional data, it issuggested to project the DoF to the high-dimensional space to acceleratethe convergence of the following FCNN according to the work validatedin Chapter 4. The final output from the location solver is the predictedcoordinate of the damage. This prediction is optimized through a meansquared error loss as:Lr =1NNâi=)(yËi â yi)2, (6.9)where yËi denotes the predicted location coordinates for the xâ th example,and yi stands for the actual damage location coordinates.96Combining the objective functions regarding the DoF and the final dam-age location gives the total loss functions as:L = λLs + Lr (6.10)where the λ is a learnable weight for the loss function balance.6.4 Experimental Results and Discussion6.4.1 Data DescriptionThe proposed method was validated using both LUGW and NUGWdatasets. The data utilized for training and testing were designed under theOOD damage localization framework. Specifically,â For the NUGW dataset, this study employed disk damage to trainthe model, aiming to predict actual damage introduced by the impact.Still, transducer 1, 3, 5, and 7 were individually used as the transmitterto show the generalization ability of the model.â For the LUGW dataset, this study employed damage located in theperipheral area for model training, aiming to predict damage in rel-atively central locations. Specifically, as illustrated in Figure. 6.7,damages positioned at locations 9 to 16 were designated for testing,while the remaining locations were utilized for training. The rationalebehind this setup involves enclosing an area with peripheral points toencompass the entire area, thereby reducing inspection time and costs.6.4.2 Ablation StudiesThe novelty of the proposed methods lies in two aspects: physicallyinformed objective functions and specialized model structures designed ac-cordingly. To validate the effectiveness of these innovations, an ablationstudy was conducted, defined by (1) distance loss, (2) parallel structure,and (3) the RFF proposed in Chapter 4.Results on the NUGW Dataset Table. 6.1 presents the detailed set-tings of the ablation study and its corresponding outcomes. The "Average"column reports the mean MND results when using different transducers asthe transmitter individually. Conversely, the "Best" column highlights the97Figure 6.7: Illustration of the training and testing data split of LUGWdataset [2]. Reproduced with permission from Sage.best prediction outcomes across the six transmitters. Still, the cross markdenotes features that were excluded from a given model, while the check-mark indicates included features. The foundational model, with all threefeatures excluded, yields the worst performance, indicating the significantoverfitting to the training dataset. This model lacks discriminative abilitywith assigning identical damage location predictions for any input within thetest results. In contrast, incorporating distance loss into the modelâs learningmechanism enhanced its ability to differentiate between damage locations,thereby improving performance. Further augmentation of the model withadditional proposed features progressively elevated its overall efficacy. Thefully-featured model achieved the highest scores in both the best and av-erage results categories, affirmatively demonstrating the effectiveness of theproposed PINN approach. Additionally, the utility of the RFF introducedin Chapter 4 was reaffirmed here.Results on the LUGW Dataset Table. 6.4 presents the correspondingresults on LUGW dataset. Still, the foundational model cannot distinguishthe damage location through the input and give the worst results. In con-trast, the embedding of the distance loss enables to model to differentiatebetween damage locations. The application of the parallel structure and98Table 6.1: MND (m) results ablation studies on four models on NUGWdataset.Model index Distance loss Parallel structure RFF Average Best1 7 7 7 0.131 0.1312 3 7 7 0.099 0.0813 3 3 7 0.090 0.0834 3 3 3 0.078 0.059Table 6.2: MND (m) results ablation studies on four models on LUGWdataset.Model index Distance loss Parallel structure RFF Average Best1 7 7 7 0.110 0.1102 3 7 7 0.094 0.0713 3 3 7 0.087 0.1094 3 3 3 0.084 0.070RFF projection improve the modelâ performance consistently.6.4.3 Comparative Studies with SOTA MethodsTo validate the superiority of the proposed method over SOTA meth-ods, comparative studies were conducted with five DNN-based methods fordamage localization using UGW across two datasets. In this chapter, theselected DNN methods focus on improving OOD performance compared tothe studies in previous chapters. The comparison methods are as follows:1. 1D ResNet-18 [102]. Still, the 1D ResNet-18 is implemented as it isthe standard paradigm for various deep learning applications.2. 2D ResNet-18 [137]. ResNet-18 is also implemented with a 2D input,where the UGW signal is reshaped into a 2D format, similar to animage. This model, along with the 1D ResNet-18, serves as the baselinefor the OOD composite damage localization.3. TVDI-CNN [41]. The TVDI-CNN study, known for its significant im-pact on damage localization, employs a time-varying damage index asinput, coupled with a CNN backbone that features a parallel structure.99This approach was tested for its predictive accuracy on data unseenduring training. A notable distinction from OOD damage is that thevalidation process underscored the modelâs effectiveness in scenarioswhere the test damage locations are closely situated to the trainingdamage locales, rather than in completely different locations.4. Graph-in-graph (GiG) [42]. The GiG study represents a cutting-edgeapproach in the field of damage localization using UGW, leveraginga GNN model. The innovative use of a graph structure to encapsu-late the relationships among transducers set the GiG sets apart fromother studies. The GiG was built using a global-local framework whereUGW features are initially learned within a local graph, and subse-quently, UGW data from various transducer-receiver pairings are in-terconnected within a global structure.5. CA-CNN [43]. CA-CNN is an improved CNN-based method that in-corporates a channel attention mechanism into the model. The modelused in this paper was constructed with a ResNet-18 as the backbone.Results on the NUGW Dataset All methodsâ MND training errorswere kept under 0.002 m to facilitate an equitable comparison. Table.6.3 displays the comparative MND results achieved by various methodson the NUGW dataset, utilizing different transducers as the transmitter.The ResNet models, both 1D and 2D, serve as reasonable baselines for thisexperiment, with prediction errors around 0.1 m and 0.12 m, respectively.Notably, results from using transducer 1 with ResNet 1D demonstrate a pre-diction error under 0.1 m. However, both TVDI-CNN and GiG exhibit sig-nificantly worse performance, with larger MND errors. Particularly, GiGâsresults exceed the specimenâs dimensions, which is 0.3048 m, indicating themodelâs failure to incorporate physical knowledge from the data, resultingin arbitrary predictions. Although both methods achieve commendable per-formance with MND prediction errors under 0.002 m, they are overfitted tothe training data due to the lack of appropriate physical priors to constrainthe model. CA-CNN yields relatively satisfactory results, with its best per-formance using transducer 1 as the transmitter. The proposed PINN modeloutperforms all other methods across all transducers, achieving the lowestMND error at 0.06 m.Results on the LUGW Dataset Table. 6.4 displays the comparativeMND results achieved by various methods on the LUGW dataset, utilizing100Table 6.3: Comparative MMD (m) results with various DNN-based methodson NUGW dataset.Transmitter 1 3 5 71D ResNet-18 0.088 0.113 0.103 0.0992D ResNet-18 0.111 0.126 0.113 0.145TVDI-CNN 0.339 0.299 0.315 0.266GiG 1.453 1.222 1.994 1.824CA-CNN 0.084 0.155 0.096 0.092Ours 0.065 0.058 0.077 0.080different transducers as the transmitter. The ResNet models, both 1D and2D, serve as reasonable baselines for this experiment, with prediction errorsaround 0.15 m. Notably, results from using transducer 1 with ResNet 1Ddemonstrate a prediction error under 0.1 m. However, both TVDI-CNNand GiG exhibit significantly worse performance, with larger MND errors.Particularly, GiGâs results exceed the specimenâs dimensions, which is 0.5m, indicating the modelâs failure to incorporate physical knowledge fromthe data, resulting in arbitrary predictions. Although both methods achievecommendable performance with MND prediction errors under 0.002 m, theyare overfitted to the training data due to the lack of appropriate physicalpriors to constrain the model. CA-CNN yields relatively satisfactory results,with its best performance using transducer 6 as the transmitter, yet thebest error still exceeds 0.1 m. The proposed PINN model outperforms allother methods across all transducers, achieving the lowest MND error at0.07 m. It is noteworthy that the model shows enhanced performance whenutilizing transducers 1 and 6, located closer to the plateâs corners, comparedto those situated in the middle. This suggests that configurations withcorner transducers have superior differentiation capabilities due to largerToF differences among transmitter-receiver pairs than configurations usingmiddle transducers.6.4.4 Comparative Studies with Analytical MethodsTo further validate the superiority of the proposed method, it was com-pared with traditional analytical methods across two datasets. As mentionedearlier in this chapter, the ellipse method was replicated for the comparativestudy on both datasets. The ToFs for the first damage-related packet wereextracted using the continuous wavelet transform.101Table 6.4: Comparative MMD (m) results with various DNN-based methodson LUGW dataset.Transmitter 1 2 3 4 5 61D ResNet-18 0.096 0.125 0.113 0.14 0.164 0.1142D ResNet-18 0.145 0.144 0.136 0.121 0.129 0.118TVDI-CNN 0.229 0.290 0.225 0.178 0.438 0.205GiG 2.403 1.472 2.167 0.924 1.308 1.138CA-CNN 0.164 0.185 0.136 0.122 0.131 0.118Ours 0.070 0.078 0.089 0.092 0.101 0.076Results on the NUGW Dataset For the NUGW dataset, as with thecomparative study involving DNN methods, the ellipse method was assessedfor locating actual impact damage on three plates. As depicted in Figure.6.8, the actual damage locations are marked by red dots. The blue marksrepresent predictions made using the ellipse method with the calculatedToFs. The results indicate that even with carefully calculated ToFs thelocalization is imprecise. Figure. 6.9 shows the results provided by theproposed PINN method, where the overall prediction accuracy was enhancedcompared to the ellipse method.102Figure 6.8: Illustration of the actual damage location and predicted location using ellipse method on NUGWdataset. © Hongguang Yun103Figure 6.9: Illustration of the actual damage location and predicted location using proposed PINN model onNUGW dataset. © Hongguang Yun104Results on the LUGW Dataset For the LUGW dataset, as with thecomparative study involving DNN methods, the ellipse method was assessedfor locating damages 9 to 16. As depicted in Figure. 6.10, the actual damagelocations are marked by red dots. The blue marks represent predictionsmade using the ellipse method with the calculated ToFs. The results indicatethat even with carefully calculated ToFs the localization is imprecise. Figure.6.11 shows the results provided by the proposed PINN method, where theoverall prediction accuracy was enhanced compared to the ellipse method.Notably, for damage 11, the predicted and actual locations are very close.105Figure 6.10: Illustration of the actual damage location and predicted location using ellipse method on LUGWdataset. © Hongguang Yun106Figure 6.11: Illustration of the actual damage location and predicted location using proposed PINN model onLUGW dataset. © Hongguang Yun1076.5 SummaryEnabling DNN models for OOD tasks presents a major challenge in in-dustrial applications where training data is often limited and rich physicalknowledge governs the tasks. This situation also applies to the problem ofcomposite damage localization using UGW. To this end, this study proposesa PINN model that is capable of learning inherent knowledge with limiteddata by building physics-informed objective functions. Specifically, the dam-age localization problem was essentially broken down into two stages. Thefirst stage involved extracting ToF from UGW using an LSTM network, op-timized by a distance loss. The second stage calculated the location with theextracted ToF. The proposed PINN method was validated on two datasetsand outperformed both SOTA DNN-based methods and traditional analyti-cal methods. Specifically, the proposed method achieved relative predictionerrors of 0.078m and 0.084m on the NUGW and LUGW datasets, respec-tively, while other DNN models had relative errors exceeding 0.1m. Notably,most compared DNN methods failed to demonstrate distinguishing abilityfor OOD damage localization, as evidenced by providing the same loca-tion prediction for damage distributed in different locations on the plate.To summary, by leveraging known physics, PINN can make accurate pre-dictions with less data, thus addressing scenarios where data collection isexpensive or infeasible. Objective functions that leverage physics knowledgeenable the model to generalize better to unseen scenarios by adhering to thesame physical laws that govern the training data.108Chapter 7Conclusion7.1 Concluding RemarksThis study explores the application of DNN for localizing damage incomposite materials, a critical issue in industries such as aerospace, windenergy, and shipbuilding. Most existing DNN-based methods for compositedamage localization overlook the characteristics of UGW signals and thephysical knowledge governing the localization problem. To address this,two major challenges were identified in this study: (1) Data in industrialapplications is often limited, leading to potential differences in data distri-bution between training and testing phases. (2) The physical knowledgeunderlying damage localization differs from that in conventional deep learn-ing applications, such as image recognition, requiring specific considerationswhen building the model. To address these challenges of limited data andthe need for embedding physical knowledge, three specialized DNN mod-els tailored for specific industrial scenarios were introduced: xIxIsI damagelocalization, domain generalized damage localization, and OOD damage lo-calization. Specifically:Chapter 4 developed DNN-driven damage localization under the xIxIsIassumption for training and testing data. This study identified the maindrawback of existing studies in improving localization accuracy as the fail-ure to effectively incorporate low-dimensional transducer coordinates intoDNN models. To this end, a multi-dimensional data fusion DNN model wasproposed, featuring a random Fourier feature projection head to map low-dimensional input into a high-dimensional feature space, thus enabling theDNN model to exploit the damage pattern more efficiently and effectively.The proposed method was validated as effective, reducing the damage lo-cation error to around 2 millimetres, whereas existing methods gave theprediction errors over 4 millimetres. Moreover, compared to other DNNmodels that do not incorporate transducer location information, the pro-posed model exhibits superior learning ability when the size of training datawas reduced to under 500.The investigation of building a DNN model under the xIxIsI assumption109establishes a solid foundation for data-driven methods for composite dam-age localization using UGW. This outcomes indicate that in real industrialapplications, once the comprehensive training data was obtained, a purelydata-driven DNN model has the ability to distinguish data patterns in thedamage localization problem. Therefore, efforts can be focused on buildingan efficient model with all input data incorporated instead of constructinga complicated model.Chapter 5 considered a scenario where data distribution discrepanciesbetween training and testing data are caused by variations in input, suchas the size of the damage or changes in inspection conditions. To improvethe modelâs resilience to such discrepancies, a mechanism for maximizingthe distance between inter-class and intra-class features was proposed. Thismaximized feature space enables the model to deliver more consistent pre-dictions despite input variations. The proposed model is validated on twodatasets considering different input variations: varying damage sizes andslight damage location deviations. When compared with leading DNN-basedmodels for composite damage localization, WaveCLR has demonstrated su-perior performance and robustness. This is evidenced by achieving an av-erage accuracy of around 90% across comprehensive experiments, whereasother DNN-based methods have accuracy below 85%.It is worth noting that the idea of expanding the feature space to improvethe modelâs robustness can also be implemented to address other data dis-tribution discrepancies caused by factors from the inspection environmentor sensors, such as the temperature during inspection. The logic behind theproposed method can be interpreted in two steps. First, identify the result-ing effect of the factor on the UGW signals, such as amplitude reduction,and use data augmentation techniques to mimic these effects. Then, trainthe model by implementing WaveCLR to improve the modelâs robustness todiscrepancies caused by the factor.Chapter 6 explored a more generic scenario where the testing data areOOD compared to the training data. In other words, the testing damagelocations were completely different from those in the training stage. Inspiredby the traditional analytical ellipse method for damage localization, a PINNmodel was built, leveraging the inherent physical knowledge governing thisproblem. Experimental results showed that existing DNN-based methodsstruggle with OOD damage localization, demonstrating no distinctive pre-diction ability by producing almost identical outputs regardless of the input.Instead, the proposed method can predict OOD damage within a 0.1 meterrelative error in distance.The idea of breaking down the complex problem into multiple smaller110tasks and building an objective function with corresponding physical knowl-edge can not only improve the modelâs generalization ability but also en-hance the interpretability of the model. The study of building a PINNmodel for OOD damage localization demonstrated the promising potentialof implementing DNN models for more difficult industrial tasks, where theavailable training data is limited and the application scenario is complicated.While this study made significant strides in advancing DNN-driven dam-age localization in composite materials, it is important to acknowledge sev-eral limitations. Firstly, the proposed models require baseline UGW signalscollected when there are no damages in the materials. Secondly, the methodassumes that the testing signal, after subtracting the baseline signal, con-tains only the components induced by the damage in the materials, withoutconsidering the reflected signals from the boundaries of the materials. Ad-ditionally, the study predominantly focuses on scenarios with single damageinstances, whereas real-world applications often involve multiple, simulta-neous damages. Finally, the current models, although incorporating somephysical knowledge, may not fully capture all the complexities of UGWpropagation in various materials and environmental conditions. There is aneed for more comprehensive integration of physical principles and advancedsimulation techniques to enhance model robustness and reliability further.To conclude, this thesis presented an in-depth study on DNN-drivendamage localization using UGW. It systematically investigated strategies toaddress various levels of data disparity between training and testing data.Without accounting for these differences and the underlying physical knowl-edge, DNN models are prone to significant overfitting on training data, re-sulting in a marked decrease in performance in practical applications. Inaddition, the proposed mechanisms in three scenarios can be incorporatedinto customized DNN models according to the requirements of real applica-tions. For example, the RFF projection can be used to map low-dimensionalstructural information into the model, and the feature space maximizationmechanism can be incorporated into the model in a parallel structure toimprove the modelâs robustness to variations. This work highlighted thecritical role of data in constructing DNN models and the necessity of de-signing models that incorporate specific physical knowledge pertinent to thetask. By emulating the analytical process of solving the damage localiza-tion problem in the construction of the learning objective function, the DNNmodel is guided towards learning the physical principles underlying the datarather than merely fitting the data. With the implementation of a DNN-driven damage locator utilizing UGW, damage localization in the field canbe made more efficient, thereby reducing maintenance time and costs.1117.2 Future WorksFor composite damage localization using DNN models, although sig-nificant development has been achieved in this study, it is important toacknowledge that further work is needed to advance this field. Specifically,OOD Damage Localization The accuracy of the OOD damage localiza-tion still needs improvement. While this study has explored the possibilityof building specific objective functions to learn the inherent physical knowl-edge, there are several avenues for further research and development.One promising direction is to modify specific DNN operators amongneurons to embed physical knowledge directly into the model. This couldinvolve developing custom activation functions [142], loss functions, or reg-ularization techniques [143] that explicitly incorporate physical constraintsand principles. By embedding this knowledge into the model architecture,we can constrain the learning process to operate within a defined space thataligns with the known physical behaviour of composite materials. Addi-tionally, exploring transfer learning could provide valuable insights into im-proving the OOD framework [144]. By pre-training models on related tasksor utilizing knowledge from similar domains, we can enhance the modelâsability to generalize to new and unseen damage scenarios. This approachcould be particularly beneficial in industrial applications where collectingextensive labelled datasets is challenging.Multiple Damage Localization This study assumes the presence ofonly one damage in the inspected material. However, practical scenariosoften involve multiple damages. Future research should address the uncer-tainties associated with the number of damages and devise methods capableof managing such complexities.One promising direction for future work is to develop algorithms that candetect and localize multiple damages simultaneously. This would involve en-hancing the model architecture to account for multiple damage signaturesand implementing sophisticated signal processing techniques to distinguishbetween overlapping signals from different damage sites. By incorporatingthese capabilities, the model can be made more robust and applicable toreal-world scenarios where multiple damages are present. Additionally, ex-ploring advanced deep learning techniques such as multi-instance learningand ensemble methods could provide valuable insights into managing mul-tiple damages [145] Multi-instance learning allows the model to learn froma set of instances, each potentially containing multiple damage signatures,112while ensemble methods can combine the predictions from multiple mod-els to improve overall accuracy and reliability. Furthermore, incorporatingprobabilistic models and uncertainty quantification techniques can also helpin addressing the complexities associated with multiple damages [97]. Bymodelling the uncertainties and variabilities in damage detection and local-ization, we can develop more resilient systems that can better handle theinherent ambiguities in practical scenarios. This approach would not onlyimprove the accuracy of damage localization but also provide confidence in-tervals and probabilistic estimates, enhancing the interpretability and trust-worthiness of the results. Finally, extending the dataset to include scenarioswith multiple damages and varying damage configurations is essential fortraining and validating the models. Creating comprehensive datasets thatrepresent a wide range of damage scenarios will enable the development ofmore generalized models.Optimal Sensor Network Configuration The current study lacks ananalysis of the optimal configuration of sensor networks, such as determiningthe most cost-effective number of transducers and their optimal placementon the plate. Future research should focus on this aspect to establish bench-marks in this field, guiding the construction of DNN models.One approach to addressing this issue is to develop optimization algo-rithms that can determine the best sensor configurations for various damagelocalization scenarios [146â148]. By incorporating factors such as cost, ac-curacy, and coverage, these algorithms can identify the minimum numberof transducers needed and their optimal placement to maximize detectionefficiency. This would not only reduce costs but also enhance the practical-ity of deploying these systems in industrial applications. Simulation studiescan also play a crucial role in optimizing sensor networks. By creating de-tailed models of the inspection environment and simulating different damagescenarios, researchers can test various sensor placements and configurationsvirtually. These simulations can help identify the most effective configura-tions before implementing them in real-world applications, saving time andresources.113Bibliography[1] P. Acellent Technologies, âSmart layer sensors - products - acellenttechnologies,â 2023. â pages xv, 29[2] J. Moll, J. Kathol, C.-P. Fritzen, M. Moix-Bonet, M. Rennoch, M. Ko-erdt, A. S. Herrmann, M. G. Sause, and M. Bach, âOpen guided waves:online platform for ultrasonic guided wave measurements,â StructuralHealth Monitoring, vol. 18, no. 5-6, pp. 1903â1914, 2019. â pages xvi,xviii, xix, 33, 37, 78, 98[3] A. A. Baker, Composite materials for aircraft structures. AIAA, 2004.â pages 1[4] L. Setlak, R. Kowalik, and T. Lusiak, âPractical use of compositematerials used in military aircraft,â Materials, vol. 14, no. 17, p. 4812,2021. â pages[5] B. Parveez, M. Kittur, I. A. Badruddin, S. Kamangar, M. Hussien,and M. Umarfarooq, âScientific advancements in composite materialsfor aircraft applications: a review,â Polymers, vol. 14, no. 22, p. 5007,2022. â pages 1[6] P. Brøndsted, H. Lilholt, and A. Lystrup, âComposite materials forwind power turbine blades,â Annu. Rev. Mater. Res., vol. 35, pp. 505â538, 2005. â pages 1[7] S. S. P. Reddy, R. Suresh, H. MB, and B. Shivakumar, âUse of compos-ite materials and hybrid composites in wind turbine blades,â MaterialsToday: Proceedings, vol. 46, pp. 2827â2830, 2021. â pages[8] P. Brondsted, R. P. Nijssen, and S. Goutianos, Advances in wind tur-bine blade design and materials. Woodhead Publishing, 2023. â pages1[9] L. Tang, J. Zhang, Y. Tang, J. Kong, T. Liu, and J. Gu, âPolymermatrix wave-transparent composites: A review,â Journal of MaterialsScience & Technology, vol. 75, pp. 225â251, 2021. â pages 1114[10] M. Kalanchiam and M. Chinnasamy, âAdvantages of composite ma-terials in aircraft structures,â International Journal of Aerospace andMechanical Engineering, vol. 6, no. 11, pp. 2428â2432, 2012. â pages1[11] M. Mrazova, âAdvanced composite materials of the future in aerospaceindustry,â Incas bulletin, vol. 5, no. 3, p. 139, 2013. â pages 1[12] A. Johnston, Composite in Canada. National Research CouncilCanada, 2023. â pages 1[13] S. Innovation and E. D. Canada, âState of canadas aerospace indus-try,â Jul 2023. â pages 1[14] W. Tan, B. G. Falzon, L. N. Chiu, and M. Price, âPredicting low ve-locity impact damage and compression-after-impact (cai) behaviour ofcomposite laminates,â Composites Part A: Applied Science and Man-ufacturing, vol. 71, pp. 212â226, 2015. â pages 1[15] J. Babu, T. Sunny, N. A. Paul, K. P. Mohan, J. Philip, and J. P.Davim, âAssessment of delamination in composite materials: a re-view,â Proceedings of the Institution of Mechanical Engineers, PartB: Journal of Engineering Manufacture, vol. 230, no. 11, pp. 1990â2003, 2016. â pages 1[16] C. Yi-yong, D. Li-ping, and Z. Kai, âAnalysis of preventive mainte-nance program improvement for in-service aircraft,â Int. Counc. Aero-naut. Sci, vol. 1, no. 1, pp. 1â6, 2002. â pages 1, 2[17] C. K. Liew, M. Veidt, N. Rajic, K. Tsoi, D. Rowlands, and H. Morton,âInspections of helicopter composite airframe structures using con-ventional and emerging nondestructive testing methods,â Journal ofTesting and Evaluation, vol. 39, no. 6, pp. 1011â1022, 2011. â pages1[18] G. Dobmann, L. Debarberis, and J.-F. Coste, âAging material evalua-tion and studies by non-destructive techniques (ames-ndt)a europeannetwork project,â Nuclear Engineering and Design, vol. 206, no. 2-3,pp. 363â374, 2001. â pages 1[19] C. Kralovec and M. Schagerl, âReview of structural health monitoringmethods regarding a multi-sensor approach for damage assessment ofmetal and composite structures,â Sensors, vol. 20, no. 3, p. 826, 2020.â pages 1115[20] B. Le Crom and M. Castaings, âShear horizontal guided wave modesto infer the shear stiffness of adhesive bond layers,â The Journal of theAcoustical Society of America, vol. 127, no. 4, pp. 2220â2230, 2010. âpages 2, 16[21] M. Castaings, âSh ultrasonic guided waves for the evaluation of inter-facial adhesion,â Ultrasonics, vol. 54, no. 7, pp. 1760â1775, 2014. âpages[22] S. Shan, M. Hasanian, H. Cho, C. J. Lissenden, and L. Cheng, âNewnonlinear ultrasonic method for material characterization: Codirec-tional shear horizontal guided wave mixing in plate,â Ultrasonics,vol. 96, pp. 64â74, 2019. â pages 2, 16[23] P. S. Lowe, T. Scholehwar, J. Yau, J. Kanfoud, T.-H. Gan, and C. Sel-cuk, âFlexible shear mode transducer for structural health monitoringusing ultrasonic guided waves,â IEEE Transactions on Industrial In-formatics, vol. 14, no. 7, pp. 2984â2993, 2017. â pages 2[24] V. E. Nazarov and A. M. Sutin, âNonlinear elastic constants ofsolids with cracks,â The Journal of the Acoustical Society of Amer-ica, vol. 102, no. 6, pp. 3349â3354, 1997. â pages 3[25] J. H. Cantrell, âUltrasonic harmonic generation from fatigue-induceddislocation substructures in planar slip metals and assessment of re-maining fatigue life,â Journal of Applied Physics, vol. 106, no. 9,p. 093516, 2009. â pages[26] M. Meo, U. Polimeno, and G. Zumpano, âDetecting damage in com-posite material using nonlinear elastic wave spectroscopy methods,âApplied composite materials, vol. 15, no. 3, pp. 115â126, 2008. âpages[27] W. Wang, âA two-stage prognosis model in condition based mainte-nance,â European Journal of Operational Research, vol. 182, no. 3,pp. 1177â1187, 2007. â pages 3[28] Z. Su, L. Ye, and Y. Lu, âGuided lamb waves for identification ofdamage in composite structures: A review,â Journal of sound andvibration, vol. 295, no. 3-5, pp. 753â780, 2006. â pages 3[29] X. Zhao, H. Gao, G. Zhang, B. Ayhan, F. Yan, C. Kwan, and J. L.Rose, âActive health monitoring of an aircraft wing with embedded116piezoelectric sensor/actuator network: I. defect detection, localizationand growth monitoring,â Smart materials and structures, vol. 16, no. 4,p. 1208, 2007. â pages 3, 18, 47[30] M. Kehlenbach and S. Das, âIdentifying damage in plates by analyz-ing lamb wave propagation characteristics,â in Smart NondestructiveEvaluation for Health Monitoring of Structural and Biological Systems,vol. 4702, pp. 364â375, SPIE, 2002. â pages 3, 17[31] C. Bermes, J.-Y. Kim, J. Qu, and L. J. Jacobs, âNonlinear lamb wavesfor the detection of material nonlinearity,â Mechanical Systems andSignal Processing, vol. 22, no. 3, pp. 638â646, 2008. â pages 3, 31[32] F. Li, G. Meng, K. Kageyama, Z. Su, and L. Ye, âOptimal motherwavelet selection for lamb wave analyses,â Journal of intelligent ma-terial systems and structures, vol. 20, no. 10, pp. 1147â1161, 2009. âpages 3[33] P. Wang, W. Zhou, and H. Li, âA singular value decomposition-basedguided wave array signal processing approach for weak signals withlow signal-to-noise ratios,â Mechanical Systems and Signal Processing,vol. 141, p. 106450, 2020. â pages 3[34] J. Zhou, Z. Li, and J. Chen, âDamage identification method based oncontinuous wavelet transform and mode shapes for composite lami-nates with cutouts,â Composite Structures, vol. 191, pp. 12â23, 2018.â pages 3[35] H. J. Lim and H. Sohn, âOnline fatigue crack prognosis using nonlinearultrasonic modulation,â Structural Health Monitoring, vol. 18, no. 5-6,pp. 1889â1902, 2019. â pages 3[36] C. Sbarufatti, G. Manson, and K. Worden, âA numerically-enhancedmachine learning approach to damage diagnosis using a lamb wavesensing network,â Journal of Sound and Vibration, vol. 333, no. 19,pp. 4499â4525, 2014. â pages 3, 21[37] A. De Fenza, A. Sorrentino, and P. Vitiello, âApplication of artificialneural networks and probability ellipse methods for damage detectionusing lamb waves,â Composite Structures, vol. 133, pp. 390â403, 2015.â pages 3, 22117[38] B. Feng, D. J. Pasadas, A. L. Ribeiro, and H. G. Ramos, âLocating de-fects in anisotropic cfrp plates using tof-based probability matrix andneural networks,â IEEE Transactions on Instrumentation and Mea-surement, vol. 68, no. 5, pp. 1252â1260, 2019. â pages 3, 22[39] G. Simone, F. C. Morabito, R. Polikar, P. Ramuhalli, L. Udpa, andS. Udpa, âFeature extraction techniques for ultrasonic signal classi-fication,â International Journal of Applied Electromagnetics and Me-chanics, vol. 15, no. 1-4, pp. 291â294, 2002. â pages 3[40] E. D. Niri and S. Salamone, âA probabilistic framework for acousticemission source localization in plate-like structures,â Smart materialsand structures, vol. 21, no. 3, p. 035009, 2012. â pages 3, 22[41] S. Zhang, C. M. Li, and W. Ye, âDamage localization in plate-likestructures using time-varying feature and one-dimensional convolu-tional neural network,â Mechanical Systems and Signal Processing,vol. 147, p. 107107, 2021. â pages 3, 24, 47, 60, 62, 63, 99[42] S. Wang, Z. Luo, P. Shen, H. Zhang, and Z. Ni, âGraph-in-graphconvolutional network for ultrasonic guided wave-based damage de-tection and localization,â IEEE Transactions on Instrumentation andMeasurement, vol. 71, pp. 1â11, 2022. â pages 4, 25, 100[43] H. Zhang, J. Lin, J. Hua, T. Zhang, and T. Tong, âAttention-basedinterpretable prototypical network towards small-sample damage iden-tification using ultrasonic guided waves,âMechanical Systems and Sig-nal Processing, vol. 188, p. 109990, 2023. â pages 4, 25, 100[44] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, âIma-genet: A large-scale hierarchical image database,â in 2009 IEEE con-ference on computer vision and pattern recognition, pp. 248â255, Ieee,2009. â pages 4[45] J. Montesano, Z. Fawaz, and H. Bougherara, âUse of infrared thermog-raphy to investigate the fatigue behavior of a carbon fiber reinforcedpolymer composite,â Composite structures, vol. 97, pp. 76â83, 2013.â pages 13[46] B. Szymanik, T. Chady, and K. GorÄ cy, âNumerical modelling andexperimental evaluation of the composites using active infrared ther-mography with forced cooling,â Quantitative InfraRed ThermographyJournal, vol. 17, no. 2, pp. 107â129, 2020. â pages118[47] M. A. Machado, M. I. Silva, A. P. Martins, M. S. Carvalho, and T. G.Santos, âDouble active transient thermography,â NDT & E Interna-tional, vol. 125, p. 102566, 2022. â pages 13[48] F. Ciampa, G. Scarselli, S. Pickering, and M. Meo, âNonlinear elasticwave tomography for the imaging of corrosion damage,â Ultrasonics,vol. 62, pp. 147â155, 2015. â pages 14[49] M. Lizaranzu, A. Lario, A. Chiminelli, and I. Amenabar, âNon-destructive testing of composite materials by means of activethermography-based tools,â Infrared Physics & Technology, vol. 71,pp. 113â120, 2015. â pages 14[50] H. Xie, H. Fang, X. Li, L. Wan, P. Wu, and Y. Yu, âLow-velocityimpact damage detection and characterization in composite sandwichpanels using infrared thermography,â Composite structures, vol. 269,p. 114008, 2021. â pages 14[51] Z. Wei, H. Fernandes, H.-G. Herrmann, J. R. Tarpani, and A. Os-man, âA deep learning method for the impact damage segmentationof curve-shaped cfrp specimens inspected by infrared thermography,âSensors, vol. 21, no. 2, p. 395, 2021. â pages 14[52] S. Farmaki, D. A. Exarchos, I. K. Tragazikis, T. E. Matikas, and K. G.Dassios, âA novel infrared thermography sensing approach for rapid,quantitative assessment of damage in aircraft composites,â Sensors,vol. 20, no. 15, p. 4113, 2020. â pages 14[53] J. Padiyar M, L. Zanotti Fragonara, I. Petrunin, J. Raposo, A. Tsour-dos, I. Gray, S. Farmaki, D. Exarchos, T. E. Matikas, and K. G.Dassios, âFast, accurate, and reliable detection of damage in aircraftcomposites by advanced synergistic infrared thermography and phasedarray techniques,â Applied Sciences, vol. 11, no. 6, p. 2778, 2021. âpages 14[54] K. Balasubramaniam, S. Sikdar, D. Ziaja, M. Jurek, R. Soman, andP. Malinowski, âA global-local damage localization and quantifica-tion approach in composite structures using ultrasonic guided wavesand active infrared thermography,â Smart Materials and Structures,vol. 32, no. 3, p. 035016, 2023. â pages 14119[55] R. Gupta, D. Mitchell, J. Blanche, S. Harper, W. Tang, K. Pancholi,L. Baines, D. G. Bucknall, and D. Flynn, âA review of sensing tech-nologies for non-destructive evaluation of structural composite mate-rials,â Journal of Composites Science, vol. 5, no. 12, p. 319, 2021. âpages 14[56] H. Lopes, F. Ferreira, J. A. Dos Santos, and P. Moreno-García, âLocal-ization of damage with speckle shearography and higher order spatialderivatives,â Mechanical Systems and Signal Processing, vol. 49, no. 1-2, pp. 24â38, 2014. â pages 15[57] R. Bossi and V. Giurgiutiu, âNondestructive testing of damage inaerospace composites,â in Polymer composites in the aerospace in-dustry, pp. 413â448, Elsevier, 2015. â pages 15[58] L. Yang and Y. Hung, âDigital shearography for nondestructive eval-uation and application in automotive and aerospace industries,â J.Hologr. Speckle, vol. 1, no. 248, pp. 69â79, 2004. â pages 15[59] S. Gholizadeh, âA review of non-destructive testing methods of com-posite materials,â Procedia structural integrity, vol. 1, pp. 50â57, 2016.â pages 15, 16[60] L. Zhang, Z. W. Tham, Y. F. Chen, C. Y. Tan, F. Cui, B. Mu-tiargo, and L. Ke, âDefect imaging in carbon fiber composites byacoustic shearography,â Composites Science and Technology, vol. 223,p. 109417, 2022. â pages 15[61] V. Arumugam, C. S. Kumar, C. Santulli, F. Sarasini, and A. J. Stanley,âA global method for the identification of failure modes in fiberglassusing acoustic emission,â Journal of Testing and Evaluation, vol. 39,no. 5, p. 1, 2011. â pages 15[62] S. Gholizadeh, Z. Leman, B. Baharudin, et al., âA review of the ap-plication of acoustic emission technique in engineering,â Struct. Eng.Mech, vol. 54, no. 6, pp. 1075â1095, 2015. â pages 15, 16[63] G. R. Kirikera, V. Shinde, M. J. Schulz, A. Ghoshal, M. Sundare-san, and R. Allemang, âDamage localisation in composite and metal-lic structures using a structural neural system and simulated acousticemissions,â Mechanical Systems and Signal Processing, vol. 21, no. 1,pp. 280â297, 2007. â pages 16120[64] F. A. Leone Jr, D. Ozevin, J. Awerbuch, and T.-M. Tan, âDetectingand locating damage initiation and progression in full-scale sandwichcomposite fuselage panels using acoustic emission,â Journal of Com-posite Materials, vol. 47, no. 13, pp. 1643â1664, 2013. â pages 16[65] S. Sikdar, P. Mirgal, S. Banerjee, and W. Ostachowicz, âDamage-induced acoustic emission source monitoring in a honeycomb sand-wich composite structure,â Composites Part B: Engineering, vol. 158,pp. 179â188, 2019. â pages 16[66] M. Eaton, M. Pearson, W. Lee, and R. Pullin, âAccurate damagelocation in complex composite structures and industrial environmentsusing acoustic emission,â in Journal of Physics: Conference Series,vol. 628, p. 012105, IOP Publishing, 2015. â pages 16[67] S. Sikdar, W. Ostachowicz, and J. Pal, âDamage-induced acousticemission source identification in an advanced sandwich compositestructure,â Composite Structures, vol. 202, pp. 860â866, 2018. âpages 16[68] D. Li, J.-H. Nie, H. Wang, J.-B. Yan, C.-X. Hu, and P. Shen, âDamagelocation, quantification and characterization of steel-concrete compos-ite beams using acoustic emission,â Engineering Structures, vol. 283,p. 115866, 2023. â pages 16[69] C. Barile, C. Casavola, G. Pappalettera, and P. K. Vimalathithan,âDamage characterization in composite materials using acoustic emis-sion signal-based and parameter-based data,â Composites Part B: En-gineering, vol. 178, p. 107469, 2019. â pages 16[70] J. Du, H. Wang, L. Cheng, Y. Bi, and D. Yang, âDamage localization,identification and evolution studies during quasi-static indentation ofcfrp composite using acoustic emission,â Polymers, vol. 15, no. 24,p. 4633, 2023. â pages 16[71] M. Barbosh, K. Dunphy, and A. Sadhu, âAcoustic emission-baseddamage localization using wavelet-assisted deep learning,â Journal ofInfrastructure Preservation and Resilience, vol. 3, no. 1, p. 6, 2022. âpages 16[72] V. Vy, Y. Lee, J. Bak, S. Park, S. Park, and H. Yoon, âDamagelocalization using acoustic emission sensors via convolutional neural121network and continuous wavelet transform,â Mechanical Systems andSignal Processing, vol. 204, p. 110831, 2023. â pages 16[73] J. Zhao, W. Xie, D. Yu, Q. Yang, S. Meng, and Q. Lyu, âDeep transferlearning approach for localization of damage area in composite lami-nates using acoustic emission signal,â Polymers, vol. 15, no. 6, p. 1520,2023. â pages 16[74] W. Zhou, Z.-b. Pan, J. Wang, S. Qiao, L.-h. Ma, J. Liu, X.-y. Ren,and Y.-z. Liang, âReview on acoustic emission source location, dam-age recognition and lifetime prediction of fiber-reinforced composites,âJournal of Materials Science, vol. 58, no. 2, pp. 583â607, 2023. âpages 16[75] A. S. Birks and R. E. Green, âNondestructive testing handbook 7:ultrasonic testing,â ASNT Handbook, 1991. â pages 16[76] J. Frouin, S. Sathish, T. E. Matikas, and J. K. Na, âUltrasonic linearand nonlinear behavior of fatigued tiâ6alâ4v,â Journal of materialsresearch, vol. 14, no. 4, pp. 1295â1298, 1999. â pages 17[77] Z. Su, C. Zhou, M. Hong, L. Cheng, Q. Wang, and X. Qing, âAcousto-ultrasonics-based fatigue damage characterization: Linear versus non-linear signal features,â Mechanical Systems and Signal Processing,vol. 45, no. 1, pp. 225â239, 2014. â pages 17[78] K.-Y. Jhang, âNonlinear ultrasonic techniques for nondestructive as-sessment of micro damage in material: a review,â International journalof precision engineering and manufacturing, vol. 10, no. 1, pp. 123â135, 2009. â pages 17[79] A. Raghavan and C. E. Cesnik, âGuided-wave signal processing usingchirplet matching pursuits and mode correlation for structural healthmonitoring,â Smart Materials and Structures, vol. 16, no. 2, p. 355,2007. â pages 18[80] P. Tua, S. Quek, and Q. Wang, âDetection of cracks in plates usingpiezo-actuated lamb waves,â Smart Materials and Structures, vol. 13,no. 4, p. 643, 2004. â pages 18, 47[81] Y. Lu, L. Ye, and Z. Su, âCrack identification in aluminium platesusing lamb wave signals of a pzt sensor network,â Smart Materialsand Structures, vol. 15, no. 3, p. 839, 2006. â pages 18122[82] M. Niethammer, L. J. Jacobs, J. Qu, and J. Jarzynski, âTime-frequency representations of lamb waves,â The Journal of the Acousti-cal Society of America, vol. 109, no. 5, pp. 1841â1847, 2001. â pages18[83] H. Jeong and Y.-S. Jang, âWavelet analysis of plate wave propagationin composite laminates,â Composite Structures, vol. 49, no. 4, pp. 443â450, 2000. â pages[84] N. E. Huang, Z. Shen, S. R. Long, M. C. Wu, H. H. Shih, Q. Zheng,N.-C. Yen, C. C. Tung, and H. H. Liu, âThe empirical mode decompo-sition and the hilbert spectrum for nonlinear and non-stationary timeseries analysis,â Proceedings of the Royal Society of London. Series A:mathematical, physical and engineering sciences, vol. 454, no. 1971,pp. 903â995, 1998. â pages 18[85] S. R. Ara, S. K. Bashar, F. Alam, and M. K. Hasan, âEmd-dwt basedtransform domain feature reduction approach for quantitative multi-class classification of breast lesions,â Ultrasonics, vol. 80, pp. 22â33,2017. â pages 18[86] S. Cantero-Chinchilla, J. Chiachío, M. Chiachío, D. Chronopoulos,and A. Jones, âA robust bayesian methodology for damage localiza-tion in plate-like structures using ultrasonic guided-waves,â Mechan-ical Systems and Signal Processing, vol. 122, pp. 192â205, 2019. âpages 18, 20[87] C. Xu, Z. Yang, B. Qiao, and X. Chen, âA parameter estimation basedsparse representation approach for mode separation and dispersioncompensation of lamb waves in isotropic plate,â Smart Materials andStructures, vol. 29, no. 3, p. 035020, 2020. â pages 18[88] H. Sun, L. Peng, S. Wang, S. Huang, and K. Qu, âDevelopment offrequency-mixed point-focusing shear horizontal guided-wave emat fordefect inspection using deep neural network,â IEEE Transactions onInstrumentation and Measurement, vol. 70, pp. 1â14, 2020. â pages18[89] M. Thiene, Z. S. Khodaei, and M. Aliabadi, âOptimal sensor place-ment for maximum area coverage (mac) for damage localization incomposite structures,â Smart materials and structures, vol. 25, no. 9,p. 095037, 2016. â pages 19123[90] F. Deng, X. Zhang, N. Yu, and L. Zhao, âAn improved rapid imagingmethod of defects in composite plate based on feature identificationby machine learning,â Sensors, vol. 22, no. 21, p. 8413, 2022. â pages19[91] G. Yan, âA bayesian approach for damage localization in plate-likestructures using lamb waves,â Smart Materials and Structures, vol. 22,no. 3, p. 035012, 2013. â pages 19, 20[92] C.-T. Ng, âBayesian model updating approach for experimental iden-tification of damage in beams using guided waves,â Structural HealthMonitoring, vol. 13, no. 4, pp. 359â373, 2014. â pages 20[93] C. Fendzi, N. Mechbal, M. Rebillat, M. Guskov, and G. Coffignal,âA general bayesian framework for ellipse-based and hyperbola-baseddamage localization in anisotropic composite plates,â Journal of In-telligent Material Systems and Structures, vol. 27, no. 3, pp. 350â374,2016. â pages 20[94] S. He and C.-T. Ng, âA probabilistic approach for quantitative identi-fication of multiple delaminations in laminated composite beams usingguided waves,â Engineering Structures, vol. 127, pp. 602â614, 2016. âpages 20[95] S. He and C.-T. Ng, âGuided wave-based identification of multiplecracks in beams using a bayesian approach,â Mechanical Systems andSignal Processing, vol. 84, pp. 324â345, 2017. â pages 20[96] H. Huo, J. He, and X. Guan, âA bayesian fusion method for compositedamage identification using lamb wave,â Structural Health Monitoring,p. 1475921720945000, 2020. â pages 20[97] M. Zhao, Y. Huang, W. Zhou, and H. Li, âBayesian uncertainty quan-tification for guided-wave-based multidamage localization in plate-like structures using gibbs sampling,â Structural Health Monitoring,vol. 20, no. 6, pp. 3092â3112, 2021. â pages 20, 113[98] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, âGradient-basedlearning applied to document recognition,â Proceedings of the IEEE,vol. 86, no. 11, pp. 2278â2324, 1998. â pages 22[99] A. Khan, A. Sohail, U. Zahoora, and A. S. Qureshi, âA survey of therecent architectures of deep convolutional neural networks,â ArtificialIntelligence Review, vol. 53, no. 8, pp. 5455â5516, 2020. â pages 22124[100] A. Krizhevsky, I. Sutskever, and G. E. Hinton, âImagenet classificationwith deep convolutional neural networks,â Advances in neural infor-mation processing systems, vol. 25, pp. 1097â1105, 2012. â pages22[101] M. D. Zeiler and R. Fergus, âVisualizing and understanding convolu-tional networks,â in European conference on computer vision, pp. 818â833, Springer, 2014. â pages 22[102] K. He, X. Zhang, S. Ren, and J. Sun, âDeep residual learning forimage recognition,â in Proceedings of the IEEE conference on computervision and pattern recognition, pp. 770â778, 2016. â pages 22, 82, 99[103] C. Su, M. Jiang, S. Lv, S. Lu, L. Zhang, F. Zhang, and Q. Sui, âIm-proved damage localization and quantification of cfrp using lamb wavesand convolution neural network,â IEEE Sensors Journal, vol. 19,no. 14, pp. 5784â5791, 2019. â pages 22[104] N. Munir, H.-J. Kim, J. Park, S.-J. Song, and S.-S. Kang, âConvo-lutional neural network for ultrasonic weldment flaw classification innoisy conditions,â Ultrasonics, vol. 94, pp. 74â81, 2019. â pages 23[105] H. Z. Hosseinabadi, B. Nazari, R. Amirfattahi, H. R. Mirdamadi, andA. R. Sadri, âWavelet network approach for structural damage iden-tification using guided ultrasonic waves,â IEEE Transactions on In-strumentation and Measurement, vol. 63, no. 7, pp. 1680â1692, 2014.â pages 23[106] M. Rautela, J. Senthilnath, J. Moll, and S. Gopalakrishnan, âCom-bined two-level damage identification strategy using ultrasonic guidedwaves and physical knowledge assisted machine learning,â Ultrasonics,vol. 115, p. 106451, 2021. â pages 24, 47, 60, 62, 63[107] V. Suresh, P. Janik, J. Rezmer, and Z. Leonowicz, âForecasting solarpv output using convolutional neural networks with a sliding windowalgorithm,â Energies, vol. 13, no. 3, p. 723, 2020. â pages 24[108] Y. Liao, X. Qing, Y. Wang, and F. Zhang, âDamage localization forcomposite structure using guided wave signals with gramian angu-lar field image coding and convolutional neural networks,â CompositeStructures, vol. 312, p. 116871, 2023. â pages 24125[109] K. Zhou, Y. Yang, T. Hospedales, and T. Xiang, âLearning to generatenovel domains for domain generalization,â in European conference oncomputer vision, pp. 561â578, Springer, 2020. â pages 25[110] Z. Huang, H. Wang, E. P. Xing, and D. Huang, âSelf-challenging im-proves cross-domain generalization,â in European Conference on Com-puter Vision, pp. 124â140, Springer, 2020. â pages 25[111] H. Zheng, R. Wang, Y. Yang, Y. Li, and M. Xu, âIntelligent faultidentification based on multisource domain generalization towards ac-tual diagnosis scenario,â IEEE Transactions on Industrial Electronics,vol. 67, no. 2, pp. 1293â1304, 2019. â pages 25[112] Y. Zhang, Z. Ren, S. Zhou, K. Feng, K. Yu, and Z. Liu, âSupervisedcontrastive learning-based domain adaptation network for intelligentunsupervised fault diagnosis of rolling bearing,â IEEE/ASME Trans-actions on Mechatronics, vol. 27, no. 6, pp. 5371â5380, 2022. â pages25[113] R. J. Pyle, R. L. Bevan, R. R. Hughes, A. A. S. Ali, and P. D.Wilcox, âDomain adapted deep-learning for improved ultrasonic crackcharacterization using limited experimental data,â IEEE Transactionson Ultrasonics, Ferroelectrics, and Frequency Control, vol. 69, no. 4,pp. 1485â1496, 2022. â pages 25[114] A. Khan and H. S. Kim, âClassification and prediction of multidam-ages in smart composite laminates using discriminant analysis,â Me-chanics of Advanced Materials and Structures, vol. 29, no. 2, pp. 230â240, 2022. â pages 25[115] M. Deng, âCumulative second-harmonic generation of lamb-modepropagation in a solid plate,â Journal of applied physics, vol. 85, no. 6,pp. 3051â3058, 1999. â pages 28, 30[116] K. He, X. Zhang, S. Ren, and J. Sun, âDeep residual learning forimage recognition,â in 2016 IEEE Conference on Computer Visionand Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June27-30, 2016, pp. 770â778, IEEE Computer Society, 2016. â pages 49,55[117] S. Ioffe and C. Szegedy, âBatch normalization: Accelerating deep net-work training by reducing internal covariate shift,â in Proceedings of126the 32nd International Conference on Machine Learning, ICML 2015,Lille, France, 6-11 July 2015, vol. 37, pp. 448â456, 2015. â pages 49[118] M. Tancik, P. Srinivasan, B. Mildenhall, S. Fridovich-Keil, N. Ragha-van, U. Singhal, R. Ramamoorthi, J. Barron, and R. Ng, âFourierfeatures let networks learn high frequency functions in low dimen-sional domains,â Advances in Neural Information Processing Systems,vol. 33, pp. 7537â7547, 2020. â pages 50, 51, 52[119] A. Berlinet and C. Thomas-Agnan, Reproducing kernel Hilbert spacesin probability and statistics. Springer Science & Business Media, 2011.â pages 50[120] T. Hofmann, B. Schölkopf, and A. J. Smola, âKernel methods in ma-chine learning,â The annals of statistics, vol. 36, no. 3, pp. 1171â1220,2008. â pages 51[121] I. Goodfellow, Y. Bengio, and A. Courville, Deep learning. MIT press,2016. â pages 51[122] N. Rahaman, A. Baratin, D. Arpit, F. Draxler, M. Lin, F. Hamprecht,Y. Bengio, and A. Courville, âOn the spectral bias of neural networks,âin Proceedings of the 36th International Conference on Machine Learn-ing, vol. 97 of Proceedings of Machine Learning Research, pp. 5301â5310, PMLR, 09â15 Jun 2019. â pages 51, 52[123] P. A. Bricman and R. T. Ionescu, âCoconet: A deep neural networkfor mapping pixel coordinates to color values,â in International Con-ference on Neural Information Processing, pp. 64â76, Springer, 2018.â pages 52[124] B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ra-mamoorthi, and R. Ng, âNerf: Representing scenes as neural radiancefields for view synthesis,â in European conference on computer vision,pp. 405â421, Springer, 2020. â pages 52[125] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N.Gomez, Å. Kaiser, and I. Polosukhin, âAttention is all you need,âAdvances in neural information processing systems, vol. 30, 2017. âpages 52[126] H. Taud and J. Mas, âMultilayer perceptron (mlp),â Geomatic ap-proaches for modeling land change scenarios, pp. 451â455, 2018. âpages 52127[127] A. Jacot, C. Hongler, and F. Gabriel, âNeural tangent kernel: Con-vergence and generalization in neural networks,â in Advances in Neu-ral Information Processing Systems 31: Annual Conference on NeuralInformation Processing Systems 2018, NeurIPS 2018, December 3-8,2018, Montréal, Canada, pp. 8580â8589, 2018. â pages 52[128] J. Lee, L. Xiao, S. Schoenholz, Y. Bahri, R. Novak, J. Sohl-Dickstein,and J. Pennington, âWide neural networks of any depth evolve aslinear models under gradient descent,â Advances in neural informationprocessing systems, vol. 32, 2019. â pages 52[129] M. J. Wainwright, High-dimensional statistics: A non-asymptoticviewpoint, vol. 48. Cambridge University Press, 2019. â pages 53[130] B. Ronen, D. Jacobs, Y. Kasten, and S. Kritchman, âThe convergencerate of neural networks for learned functions of different frequencies,âAdvances in Neural Information Processing Systems, vol. 32, 2019. âpages 53[131] G. Azuara, M. Ruiz, and E. Barrera, âDamage localization in com-posite plates using wavelet transform and 2-d convolutional neuralnetworks,â Sensors, vol. 21, no. 17, p. 5825, 2021. â pages 61, 62, 63[132] S.-H. Park, J.-Y. Hong, T. Ha, S. Choi, and K.-Y. Jhang, âDeeplearning-based ultrasonic testing to evaluate the porosity of additivelymanufactured parts with rough surfaces,âMetals, vol. 11, no. 2, p. 290,2021. â pages 61, 62, 63[133] J. Hu, L. Shen, and G. Sun, âSqueeze-and-excitation networks,â inProceedings of the IEEE conference on computer vision and patternrecognition, pp. 7132â7141, 2018. â pages 61, 62, 63[134] S. Woo, J. Park, J.-Y. Lee, and I. S. Kweon, âCbam: Convolutionalblock attention module,â in Proceedings of the European conference oncomputer vision (ECCV), pp. 3â19, 2018. â pages 61, 62, 63[135] K. He, H. Fan, Y. Wu, S. Xie, and R. Girshick, âMomentum contrastfor unsupervised visual representation learning,â in Proceedings of theIEEE/CVF conference on computer vision and pattern recognition,pp. 9729â9738, 2020. â pages 72[136] T. Chen, S. Kornblith, M. Norouzi, and G. Hinton, âA simple frame-work for contrastive learning of visual representations,â in Interna-128tional conference on machine learning, pp. 1597â1607, PMLR, 2020.â pages 73, 81, 83[137] L. Guo, Y. Lei, S. Xing, T. Yan, and N. Li, âDeep convolutionaltransfer learning network: A new method for intelligent fault diagnosisof machines with unlabeled data,â IEEE Transactions on IndustrialElectronics, vol. 66, no. 9, pp. 7316â7325, 2018. â pages 82, 99[138] B. Yang, Y. Lei, F. Jia, and S. Xing, âAn intelligent fault diagnosisapproach based on transfer learning from laboratory bearings to loco-motive bearings,â Mechanical Systems and Signal Processing, vol. 122,pp. 692â706, 2019. â pages 82[139] Y. Yang, J. Yin, H. Zheng, Y. Li, M. Xu, and Y. Chen, âLearn gen-eralization feature via convolutional neural network: A fault diagno-sis scheme toward unseen operating conditions,â IEEE Access, vol. 8,pp. 91103â91115, 2020. â pages 82[140] X. Li, W. Zhang, H. Ma, Z. Luo, and X. Li, âDomain generalizationin rotating machinery fault diagnostics using deep neural networks,âNeurocomputing, vol. 403, pp. 409â420, 2020. â pages 82[141] Y. Bengio, P. Simard, and P. Frasconi, âLearning long-term dependen-cies with gradient descent is difficult,â IEEE transactions on neuralnetworks, vol. 5, no. 2, pp. 157â166, 1994. â pages 93[142] P. Maczuga and M. PaszyÅski, âInfluence of activation functions onthe convergence of physics-informed neural networks for 1d wave equa-tion,â in International Conference on Computational Science, pp. 74â88, Springer, 2023. â pages 112[143] A. Krishnapriyan, A. Gholami, S. Zhe, R. Kirby, and M. W. Mahoney,âCharacterizing possible failure modes in physics-informed neural net-works,â Advances in Neural Information Processing Systems, vol. 34,pp. 26548â26560, 2021. â pages 112[144] A. Rai and M. Mitra, âA transfer learning approach for damage diag-nosis in composite laminated plate using lamb waves,â Smart Materialsand Structures, vol. 31, no. 6, p. 065002, 2022. â pages 112[145] Y. Tian, W. Hao, D. Jin, G. Chen, and A. Zou, âA review of latestmulti-instance learning,â in Proceedings of the 2020 4th InternationalConference on Computer Science and Artificial Intelligence, pp. 41â45, 2020. â pages 112129[146] S. Cantero-Chinchilla, J. Chiachío, M. Chiachío, D. Chronopoulos,and A. Jones, âOptimal sensor configuration for ultrasonic guided-wave inspection based on value of information,â Mechanical Systemsand Signal Processing, vol. 135, p. 106377, 2020. â pages 113[147] S. Mustapha, Y. Lu, C.-T. Ng, and P. Malinowski, âSensor networksfor structures health monitoring: Placement, implementations, andchallengesa review,â Vibration, vol. 4, no. 3, pp. 551â585, 2021. âpages[148] H. An, B. D. Youn, and H. S. Kim, âA methodology for sensor numberand placement optimization for vibration-based damage detection ofcomposite structures under model uncertainty,â Composite Structures,vol. 279, p. 114863, 2022. â pages 113[149] G. E. Hinton and S. Roweis, âStochastic neighbor embedding,â Ad-vances in neural information processing systems, vol. 15, 2002. âpages 131130AppendicesAppendix A: Feature Space Visualization of KeyMechanismsTo provide an intuitive understanding of the specific mechanisms pro-posed in this study, t-Distributed Stochastic Neighbour Embedding (t-SNE)visualization [149] is utilized to compare differences before and after the keycomponent is used at a feature level. t-SNE facilitates the visualization ofDNN features by reducing the learned features from the deep learning modelto two dimensions. Specifically, the benefit of using the RFF projection wasfirst illustrated. The contributions of several key components in the Wave-CLR model to improving generalization ability are also presented in thissection.A.1 Visualization of RFF Projection BenefitTheoretically, the RFF projection head maps the low-dimensional co-ordinates into high-dimensional feature space working as a kernel function.The projected features in high-dimensional space should be distinguishable.To give a more intuitive demonstration of this property, the t-SNE is usedto illustrate the projected features. The t-SNE illustrates high-dimensionaldata in a two-dimensional plane, where similar data points remain close anddissimilar data points remain distant by evaluating the similarities amongdata points.The well-trained model with 1DCNN as the encoder was used to pro-vide the projected features. Feature maps of all twelve different transducercoordinate after the RFF projection head are extracted as input for the t-SNE model. Figure. 7.1 shows the distribution of RFF projected featuresof twelve transducers in a two-dimensional plane. Twelve clusters have clearinter-class distinction and intra-class association. These results validate thatthe RFF projection can ensure features distinguishable and are learnable fordeep neural networks.131Figure 7.1: Illustration of the distribution of RFF projected coordinatesusing t-SNE. © Hongguang Yun132A.2 Visualization of WaveCLR Mechanismt-SNE was also utilized to visualize the contributions of key noveltiesin the WaveCLR framework, including mutual inter-class feature distancemaximization, intra-class feature maximization, and a two-stage trainingstrategy. The latent features were extracted from the feature extractor ofthe model, which achieved 100% accuracy in the source domain. This vi-sualization demonstrates the impact of the proposed contrastive learningmechanism by comparing the latent space distributions derived from differ-ent model configurations on the OGW dataset. The models compared forfeature differences are the same as those studied in the ablation study inChapter 5. The six models being inspected are listed below again for easyreference.Regarding the novel designs in the WaveCLR model: (1) stratified sam-pling strategy, (2) momentum update, (3) intra-class distance maximization,and (4) training strategy, six models were built for comparison:1. The 1D ResNet18 baseline with no contrastive learning mechanismadded.2. The SimCLR-based contrastive learning component was added to model1), and the input was randomly sampled from the augmented sourcedomain.3. Compared to model 2), the stratified sampling was used for contrastivelearning, and the momentum update was not applied.4. The stratified sampling and momentum update were used to realizethe mutual inter-class distance maximization.5. The WaveCLR model with a standard training strategy, where allthree objective functions were optimized simultaneously.6. The WaveCLR model is trained with the proposed two-stage trainingstrategy.Each point in Figure. 7.2 represents a unique data sample, with identicalcolours signifying samples that share the same label. Figure. (a) displays thefeature distribution from the baseline CNN model. It can be seen that thereis more than one cluster for data with the same label. This phenomenonarises due to the modelâs overfitting on raw data and augmented data intodistinct clusters. Figure. (b) shows the limitations of contrastive learningin a small dataset without stratified sampling. This limitation becomes133evident when more than one positive pair is present in a mini-batch. Incontrast, Figure. (c) illustrates that, with contrastive learning, the model iscapable of differentiating each class from others but still overfitted on the rawand augmented data clusters. The effect of incorporating the momentumupdate is shown in Figure. (d), where the maximization of mutual inter-class distance in the feature space is evident. Figures (e) and (f) comparethe complete WaveCLR model employing different training strategies. Thesuperiority of the proposed two-stage training strategy in facilitating modelconvergence is validated. Furthermore, Figure. (f) demonstrates that, withthe maximization of intra-class distance, the class clusters remain distinctand exhibit expansion, in contrast to Figure. (d), whose intra-class featuresare densely clustered.134(a) (b) (c)(d) (e) (f)Figure 7.2: t-SNE visualization of latent feature spaces of key components in WaveCLR model. Features withdifferent labels are distinguished by different colours. (a) Shows the ResNet18 model as the baseline. (b) showsthe model with standard contrastive learning but without stratified sampling. (c) shows stratified sampling usedon the top of the model (b). (d) the model with momentum update shows the maximized mutual inter-classdistance. (e) and (f) shows the feature space with the complete WaveCLR model with and without a two-stagetraining strategy. It is best viewed in colour. © Hongguang Yun135Appendix B: Destructive Inspection of CompositePlateA destructive inspection was performed on a manufactured plate to pro-vide an actual view of the internal damage caused by impact. The inspectionwas carried out on plate #1, which was subjected to an impact energy of 19Jat the centre of the plate. As shown in Figure. 7.3, there is no visible surfacedamage on the side that was impacted. On the back side of the impactedarea, BVID was caused by the breakthrough energy. The plate was cutthrough the centre. Figure. 7.4 illustrates the cross-sectional view showingthe depth dimension after cutting the plate. The 32-ply layout of the platecan be easily identified throughout the figure. Several damages distributedin different layers of the plate, including delaminations and matrix cracks,are visible. Figure. 7.5 shows an enlarged detail of the damages.136Figure 7.3: Illustration of the front and back view of the area introduced byan 19J impact of plate #1. © Hongguang Yun137Figure 7.4: Illustration of the cross-section view showing the depth dimen-sion after cutting plate #1. © Hongguang Yun138Figure 7.5: Detailed illustration of the internal damages: delamination andmatrix cracks. © Hongguang Yun139
Thesis/Dissertation
eng
Electrical Engineering
Vancouver : University of British Columbia Library
Attribution-NonCommercial-NoDerivatives 4.0 International
Graduate
Liu, Zheng (Professor of engineering)
Deep neural networks for damage localization in composite materials with ultrasonic guided waves
Text