Applied Science, Faculty of
Engineering, School of (Okanagan)
DSpace
UBCO
Shahrestani, Afshin
Master of Applied Science - MASc
University of British Columbia
Pavement crack detection and treatment is one of the vital tasks in road infrastructure management. Failure to do so in a proactive manner increases maintenance costs and results in a drop in the structural integrity of the pavement. Deep learning techniques promise to automate pavement crack detection, aiding proactive highway maintenance. This approach reduces costs, time, and human bias in defect analysis. One barrier to large-scale adoption of deep learning techniques in this area is the scarcity of high-quality, and balanced training datasets. Inherent data imbalances, where certain crack types predominate over others, lead to detection bias and overfitting in machine learning models. This is exacerbated by variations in crack types, skewing the model training towards more frequently occurring cracks, thereby diminishing efficacy in identifying less common types of cracks. Data augmentation is one of the possible solutions to these issues. Mode Collapse, training instability, and lack of agency in data generation were identified as the main challenges associated with previous pavement crack augmentation methods. To overcome the challenges, the thesis implements two Generative Adversarial Network (GAN) architectures based on the models Wasserstein GAN (WGAN), and Conditional WGAN (C-WGAN) developed in previous literature. These models aim to remedy the shortcomings of previous data augmentation methods used in pavement crack analysis. The research utilizes annotated images from the Crack500 and CrackForest datasets, categorized into transverse, longitudinal, block, and alligator crack types. These images are used as the training data for both the GANs and a baseline classifier, aimed at measuring the impact of synthetic crack images in augmented datasets. The findings demonstrate the effectiveness of WGAN and C-WGAN in addressing the limitations of previous augmentation methods, generating high-quality, diverse synthetic images. The GAN-augmented models achieved an average classification score improvement of 5\% over the baseline. C-WGAN offered the advantage of user-specific image generation. Both models proved to be effective data augmentation tools for pavement crack datasets, each with distinct advantages and potential trade-offs. This research contributes to the field by providing robust GAN-based solutions for enhancing pavement crack detection and classification.
Using Wasserstein Distance andConditionality for Stable and SelectiveData Augmentation in PavementCrack AnalysisbyAfshin ShahrestaniB.Sc., Ferdowsi University of Mashhad, 2022A THESIS SUBMITTED IN PARTIAL FULFILLMENT OFTHE REQUIREMENTS FOR THE DEGREE OFMASTER OF APPLIED SCIENCEinThe College of Graduate Studies(Civil Engineering)THE UNIVERSITY OF BRITISH COLUMBIA(Okanagan)September 2024© Afshin Shahrestani 2024The following individuals certify that they have read, and recommend tothe College of Graduate Studies for acceptance, a thesis entitled:Using Wasserstein Distance and Conditionality for Stable and Selective DataAugmentation in Pavement Crack Analysissubmitted by Afshin Shahrestani in partial fulfillment of the requirementsfor the degree of Master of Applied Science.Dr. Suliman Gargoum, School of EngineeringSupervisorDr. Anas Chaaban, School of EngineeringSupervisory Committee MemberDr. Zheng Liu, School of EngineeringSupervisory Committee MemberDr. Yong Gao, Department of Computer ScienceUniversity ExamineriiAbstractPavement crack detection and treatment is one of the vital tasks in road in-frastructure management. Failure to do so in a proactive manner increasesmaintenance costs and results in a drop in the structural integrity of thepavement. Deep learning techniques promise to automate pavement crackdetection, aiding proactive highway maintenance. This approach reducescosts, time, and human bias in defect analysis. One barrier to large-scaleadoption of deep learning techniques in this area is the scarcity of high-quality, and balanced training datasets. Inherent data imbalances, wherecertain crack types predominate over others, lead to detection bias andoverfitting in machine learning models. This is exacerbated by variationsin crack types, skewing the model training towards more frequently occur-ring cracks, thereby diminishing efficacy in identifying less common types ofcracks. Data augmentation is one of the possible solutions to these issues.Mode Collapse, training instability, and lack of agency in data generationwere identified as the main challenges associated with previous pavementcrack augmentation methods. To overcome the challenges, the thesis im-plements two Generative Adversarial Network (GAN) architectures basedon the models Wasserstein GAN (WGAN), and Conditional WGAN (C-WGAN) developed in previous literature. These models aim to remedythe shortcomings of previous data augmentation methods used in pavementcrack analysis.The research utilizes annotated images from the Crack500 and CrackForestdatasets, categorized into transverse, longitudinal, block, and alligator crackiitypes. These images are used as the training data for both the GANs anda baseline classifier, aimed at measuring the impact of synthetic crack im-ages in augmented datasets. The findings demonstrate the effectiveness ofWGAN and C-WGAN in addressing the limitations of previous augmenta-tion methods, generating high-quality, diverse synthetic images. The GAN-augmented models achieved an average classification score improvement of5% over the baseline. C-WGAN offered the advantage of user-specific imagegeneration. Both models proved to be effective data augmentation tools forpavement crack datasets, each with distinct advantages and potential trade-offs. This research contributes to the field by providing robust GAN-basedsolutions for enhancing pavement crack detection and classification.iiiLay SummaryDetecting and classifying pavement cracks is crucial for road maintenanceand safety. Deep learning can automate this process, reducing costs andeliminating biases in manual assessments. However, the adoption of thesetechniques often suffers from a lack of high-quality, balanced training data,leading to biased models with poor predictions. This thesis introduces newaugmentation models that generate synthetic images to enhance trainingdatasets. These models were trained to generate realistic pavement crackimages and were then tested in classification tasks. Results showed that thegenerated images significantly improved model performance and data bal-ance by adding diversity and balance to the original dataset, demonstratingthe effectiveness of the proposed models in enhancing pavement crack de-tection.ivPrefaceThis thesis is based on the research conducted under the supervision of Dr.Suliman Gargoum and the contents of the work have been reviewed by him.The work presented in this thesis has been submitted to conferences andjournals below:• Shahrestani et al. (2024) “Pavement Crack Image Generation andData Balancing Using Deep Adversarial Generative Networks”, pro-ceedings of the Transportation Association of Canada. Accepted June2024.• Shahrestani et al. (2024) “Conditional Pavement Crack Data Gen-eration for Selective Data Augmentation Using GANs”, IEEE Transac-tions on Intelligent Transportation Systems Journal. Submitted July2024.Generative AI tools were only used in part during the preparation of thedraft for the thesis to improve the quality of writing and the flow of infor-mation, as the English language is not the author’s first language.vTable of ContentsAbstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iiLay Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ivPreface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vTable of Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . viList of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viiiList of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ixAcknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . xDedication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41.3 Knowledge Gap . . . . . . . . . . . . . . . . . . . . . . . . . 71.4 Research Objective . . . . . . . . . . . . . . . . . . . . . . . 81.5 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . 91.6 Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102 A Review of Previous Works . . . . . . . . . . . . . . . . . . 122.1 Deep Learning Methods in Pavement Crack Analysis . . . . 132.2 State of Data Augmentation in Pavement Crack Analysis . . 15vi2.2.1 Remarks . . . . . . . . . . . . . . . . . . . . . . . . . 233 Selective Pavement Crack Generation Using C-WGAN-GP 253.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . 253.1.1 Deep Neural Networks . . . . . . . . . . . . . . . . . 253.1.2 Generative Adversarial Networks . . . . . . . . . . . 283.2 Data Augmentation Using DCGAN . . . . . . . . . . . . . . 293.3 Proposed Methodology . . . . . . . . . . . . . . . . . . . . . 383.3.1 WGAN-GP . . . . . . . . . . . . . . . . . . . . . . . . 383.3.2 C-WGAN-GP . . . . . . . . . . . . . . . . . . . . . . 444 Data Description and Testing Framework . . . . . . . . . . 494.1 Dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 494.1.1 Annotation and Classification . . . . . . . . . . . . . 544.2 Testing Framework . . . . . . . . . . . . . . . . . . . . . . . 554.2.1 Model Training Conditions and Hyperparameters . . 574.2.2 Training and Testing Environments . . . . . . . . . . 595 Data Augmentation Performance and Discussion . . . . . . 605.1 Quality of Synthetic Images . . . . . . . . . . . . . . . . . . 605.2 Impacts on Pavement Crack Classification . . . . . . . . . . 666 Conclusion, Limitations, and Future Work . . . . . . . . . . 736.1 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . 736.2 Limitations and Future Work . . . . . . . . . . . . . . . . . . 756.3 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78viiList of Tables2.1 Comparison Of data augmentation methods in previous studies 225.1 Disadvantages and Advantages of The Data AugmentationModels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 665.2 A Comparison Weighted Average Classification Metrics ForData Augmentation Models . . . . . . . . . . . . . . . . . . . 675.3 Performance Metrics for Alligator, Longitudinal, and Trans-verse Cracks . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68viiiList of Figures3.1 A Representation of a Kenrel Operation in Convolutional Layers 273.2 A GAN model general overview . . . . . . . . . . . . . . . . . 283.3 DCGAN Architecture . . . . . . . . . . . . . . . . . . . . . . 333.4 DCGAN Training Loop . . . . . . . . . . . . . . . . . . . . . 363.5 WGAN-GP Architecture . . . . . . . . . . . . . . . . . . . . . 423.6 WGAN-GP Architecture . . . . . . . . . . . . . . . . . . . . . 474.1 Pavement Crack Image Samples for Each Main Crack Type . 514.2 Crack500 and CrackForest Dataset Samples . . . . . . . . . . 534.3 Testing Framework Overview . . . . . . . . . . . . . . . . . . 555.1 DCGAN Generated Data Samples . . . . . . . . . . . . . . . 625.2 WGAN-GP Generated Data Samples . . . . . . . . . . . . . . 635.3 C-WGAN-GP Generated Data Samples . . . . . . . . . . . . 645.4 Baseline and Augmented models’ Confusion Matrices . . . . . 70ixAcknowledgementsI would like to thank my supervisor, Dr. Suliman Gargoum, who providedme with this opportunity and guided me in my research.I am extremely grateful to my colleagues, Ali Faisal, Yeganeh Morshedzadeh,and Ahmad Khataan, who were both my friends and mentors and taughtme so much in the past two years.This research was possible owing to the funding provided by NSERC Allianceand Discovery and the support of Digital Alliance Canada.Lastly, I need to express my deepest gratitude to my family and friends,who gave me their unconditional support over the years. Their belief inme, sacrifices in support of my goal, and putting up with my sometimesunpleasant personality, allowed me to reach the position I am in right nowand the future I have ahead of me. I will be forever indebted to you andhope to someday repay a fraction of what was done for me.xDedicationDedicated to my father, my mother, my brother Armin, and my fianceNadia, who made this journey so much easier than it could have been. I ameternally thankful for your support along the way.xiChapter 1Introduction1.1 BackgroundPavement cracks are common distresses that could be a sign of reducedfunctionality of roads. Among many different reasons, some key causes ofcracks include heavy traffic, and drastic weather and temperature change[22]. Failure to apply timely maintenance procedures also speeds up the de-terioration process. Pavement cracks can lead to faster wear down of pave-ment structural integrity as water penetrates the surface of the pavementand deteriorates the underlying material. These pavement deteriorationssignificantly increase the risk to road users and future maintenance costs.According to statistics provided by BCMoti, the cost of resurfacing one kilo-meter of in-use highway after 12 years is $65,000 while this cost increases to$400,000 if left unmaintained for 8 more years [20]. High-severity cracks canalso compromise road safety by causing vehicles to lose traction with theroad surface, especially in harsh weather conditions. This risk also extendsto both pedestrians and cyclists [34]. As such, proactive management ofpavement cracking through early detection and diagnosis is of key impor-tance to preserving the road infrastructure. Depending on the pavement andenvironmental conditions, different types of cracks form in the asphalt. Aspavement cracks differ based on the conditions of their formation and requiredifferent treatments, it is important to correctly categorize and differentiatethem in pavement crack detection and classification tasks.Various pavement detection and classification methods have been proposed11.1. Backgroundand used over the years. Manual inspection was the traditional approach infinding the affected regions. These methods include visual inspections wherean inspector typically walks through pavement sections and crack patternsare mapped and assigned severity [32]. This process can be time-consuming,and laborious, and requires an engineering technical background which in-troduces unwanted subjectivity in crack assessments [32]. The subjectivitypresent in the assessments could lead to different detected cracks or sever-ity ratings from the same area, leading to inconsistent results and faultydecision-making. The need for better inspection methods led to the de-velopment of several automatic pavement crack classification and detectiontechniques. These techniques can be generally separated into 2 categoriesbased on their used data type: (1) 3D point cloud processing methods, and(2) 2D image processing and deep learning-based methods.Point clouds are a collection of points in a three-dimensional space. Thesepoints represent the external surface of an object or the environment andare commonly generated using 3D scanning technologies such as LiDAR.3D point cloud processing techniques have gained better results in com-parison with traditional methods. These techniques typically process pointclouds either alongside captured 2D images or by themselves, using variousalgorithms including machine learning and deep learning. Some of the meth-ods used in 3D data pavement crack detection include clustering algorithms[10], random sample consensus (RANSAC) sliding window [11], and Prin-ciple Component Analysis [35]. In some studies, 3D point clouds are usedin 2D image extraction to generate pavement images that can later on beused in 2D pavement crack detection methods. 3D point cloud processingtechniques boast high accuracy with low incorrect classification, however,there are external factors that could impact their results. The most impor-tant factor in the quality of the gathered point clouds is the LiDAR sensorused in the operation. Depending on the accuracy of the LiDAR sensor, thegathered point clouds and the crack analysis results could be impacted nega-tively. High-accuracy LiDAR sensors ideal for detection tasks are extremelyexpensive and unattainable for most tasks. Furthermore, environmental con-21.1. Backgroundditions such as rain and snow can hinder the scanning process or negativelyimpact the gathered data.Image-processing hybridization methods included the use of image-processingtechniques to find the correct parameters in detecting pavement cracks.Early work in pavement detection uses threshold-based methods to detectcracks by exploiting the difference in the color intensity between crack andnon-crack points [7, 14, 42]. Discontinuity-based algorithms were also uti-lized to exploit crack edges of different grey levels and trace crack regionsusing image gradient magnitude information [4, 17]. Some authors alsoutilized different techniques to automatically detect pavement cracks usingimage processing where the approaches include utilizing region-based activecontour models [25], unsupervised multi-scale algorithms with statisticalevaluation [24], and vision-based slicing methods with thresholding [2]. Al-though image-processing-based methods have garnered some success, theyhave a lot of limitations, including their results. Threshold-based methods’performance in detecting cracks often deteriorates as variations in lighting,pavement colors, and textures introduce high false positives and negatives[8]. While discontinuity-based algorithms can detect crack edges, they mightstruggle with accurately segmenting complex crack patterns, and their per-formance might degrade in the presence of noise or irregular crack shapes[23].Studies have attempted using unsupervised machine learning techniques todetect cracks in pavements through texture manipulation, different classi-fiers, and random forests [13, 29, 40, 44]. One limitation of such an approachis the inability to directly use images and extract features. The recent ad-vancements in deep learning techniques and available computational pow-ers, allowed researchers to leverage deep neural networks to detect pavementcracks [12, 19, 45]. These techniques address some of the challenges faced byother methods since they can automate crack detection processes, reducingsubjectivity and significantly speeding up the assessment process. Moreover,deep learning models can learn intricate features in crack patterns which31.2. Motivationlead to accurate detection and classification of various crack types, sizes,and shapes [51]. They also can adapt to new types of cracks without theneed for manual parameter tuning and this adaptation in return enhancesthe methods’ adaptability and usability. Although deep learning methodsperformed excellently in research, they are highly dependant on data qual-ity, and diversity of the training dataset at hand. The performance of deeplearning techniques depends on the quality of the samples in the dataset,dataset balance, and sample variety and distribution. Problems in balance,diversity, and quality of the dataset could lead to unwanted or misleadingperformance and results by the deep learning models1.2 MotivationDataset imbalance refers to a situation wherein the distribution of classes orcategories within a dataset is skewed, with certain classes being significantlymore prevalent than others. In the context of deep learning model train-ing, dataset imbalance introduces challenges that can impede the model’sability to generalize effectively. Firstly, imbalanced datasets can lead tobiases in model predictions, where the model tends to favor the majorityclass(es) over the minority class(es). This bias is particularly pronouncedin classification tasks, where the model may struggle to accurately classifyinstances belonging to the minority class(es). Consequently, the model’s per-formance metrics, such as accuracy and precision, may be disproportionatelyinfluenced by the majority class, leading to an inaccurate representation ofthe model’s true performance. Furthermore, imbalanced datasets can ex-acerbate the problem of overfitting, whereby the model learns to memorizethe patterns present in the majority class(es) while neglecting the minorityclass(es). As a result, the model’s ability to generalize to unseen data, espe-cially data from the minority class(es), is compromised. This phenomenonis particularly detrimental in scenarios where accurate predictions for theminority class(es) are of utmost importance, such as in medical diagnosisor fraud detection. Dataset imbalance poses significant challenges to deeplearning model training by introducing biases, compromising generalization,41.2. Motivationand potentially leading to inaccurate or unreliable predictions, particularlyfor minority class(es). Addressing dataset imbalance is therefore crucial forensuring the robustness and effectiveness of deep learning models in variousapplications.The quality of input samples significantly influences the performance of deeplearning models. In image analysis, the efficacy of a model is dependent uponits compatibility with the resolution, level of detail, and feature complexityinherent in the images. For instance, models designed for high-resolutionimages with intricate details may struggle to extract meaningful featuresfrom low-resolution or blurry images. This mismatch in model and imagefeatures can lead to overfitting or the model focusing on undesirable de-tails of the input. Moreover, the presence of noise in images, arising fromfactors such as sensor imperfections or environmental conditions, poses aformidable challenge to model training. Noise can obscure relevant infor-mation and introduce spurious patterns, thereby undermining the model’sability to accurately learn and generalize from the data. Additionally, char-acteristics such as the ratio of background to foreground elements and thecomposition of color channels in images have notable influences on modeltraining outcomes. Imbalanced ratios of background to foreground can biasthe model towards dominant features, potentially leading to inaccuracies inclassification or detection tasks. Similarly, variations in color channels, in-cluding differences in lighting conditions or color distributions, necessitaterobust model architectures capable of discerning pertinent features acrossdiverse color spaces.The distribution and diversity of dataset samples within the same categoryplay a critical role in determining model performance in practical applica-tions. A single class within the dataset may encompass a range of sam-ples exhibiting significant intra-class variation. These samples might sharefeatures with those from other classes, necessitating that the model effec-tively distinguish and accurately classify them. Therefore, the distributionof samples within a class, particularly when there is substantial variabil-51.2. Motivationity, is crucial. These intrinsic variations pose challenges in data collection,as not all representative samples may be available during the initial datasetcompilation. However, it is imperative that the model is capable of correctlyrecognizing and processing these diverse samples when they are presented asinputs. Ensuring a balanced and comprehensive representation of sampleswithin each class enhances the model’s ability to generalize and performaccurately across varied real-world scenarios.These issues are especially prevalent in pavement crack classification anddetection tasks, as gathering an adequate dataset requires high diversityin samples and balance between the different crack types. The chances ofoccurrence for a specific crack type are not equal. Depending on the geo-graphical location of the road, its use case, and environmental conditionssuch as weather, lack of proper drainage systems, and defects in initial pave-ments, specific types of cracks have higher chances of occurrence and willbe more common in the road segment. Even within the same crack types,owing to the more complex crack shapes there could be a high variety amongsamples. The data-gathering process is costly, time-consuming, and cumber-some given that it is not possible to selectively choose what kind of cracksare present in the collected dataset. During data gathering, there is a highchance some crack types or shapes might be absent from the site, as notall crack types or shapes have the same probability of occurrence, makingcreating of generalized, balanced, diverse, and high-quality datasets next toimpossible.Pavement crack tasks are also sensitive to the general circumstances of datagathering, such as camera angle, image size, and the ratio of non-pavementimages in the images. Block, Transverse, and longitudinal cracks are exam-ples of crack types that are mostly affected by these data-gathering condi-tions since they have defined characteristics in terms of their angle in thepavement and their direction and rotations in the images will cause mis-classification in the models. Most of the data-gathering conditions shouldremain the same for augmenting an existing dataset to keep the results of61.3. Knowledge Gapthe models accurate.To address these issues, different solutions have been employed over theyears. Synthetic data augmentation techniques are a powerful technique inthe field of computer vision that addresses the challenge of limited trainingdata by artificially expanding the dataset through various transformations[28]. It involves creating new instances of pavement images with cracks byapplying diverse modifications. These modifications can encompass changesin lighting conditions, angles, scales, and other factors that replicate real-world variations encountered in pavement environments. By augmentingthe dataset, the models used for crack detection can be more robust andaccurate across a wider range of scenarios, which ultimately improves theirperformance and generalization capabilities [28]. With the boom in the ad-vancement of deep learning models, more research has been conducted ondeep generative networks and their utilization as augmentation methods.Over the past few years, there has been an increasing number of studies fo-cusing on the lack of balance in datasets and research on applying generativemodels, such as Generative Adversarial Networks (GANs) and Autoencoderarchitectures [46][33][27].1.3 Knowledge GapAlthough GANs have been utilized in life-like and synthetic pavement crackimage generation, dataset balancing, and augmentation, there have beenvery limited attempts at training GAN models capable of generating specificcrack types based on user needs. Furthermore, there are common issuessurrounding GANs as generative models that have not been explored andremedied in previous research. Also, the results of the generation of eachdifferent crack type, with its own characteristics and shape, are different andneed in-depth analysis to ascertain the impact of the generative model fordifferent crack types.Current approaches in data augmentation for pavement crack dataset bal-ancing lack agency in their data generation. Agency here refers to the71.4. Research Objectivemodel’s ability to generate specific crack types based on the users’ inputsand requests. Although current approaches are capable of generating life-like synthetic pavement cracks, the existence and number of occurrences ofspecific crack types are unpredictable. This leads to significant time spenton post-generation filtering in populating certain crack types in the dataset.Furthermore, this lack of agency could allow the generation of unexpectedor non-existent crack types by the model.The application of generative models like GANs in pavement crack data aug-mentation brings with it several common but critical issues, notably traininginstability and mode collapse [5]. Training instability can result in the gener-ation of images that do not accurately represent real-world conditions, whilemode collapse might lead to the overrepresentation of certain crack types atthe expense of others. This phenomenon occurs when the generator startsproducing a limited variety of outputs, often repeating the same patterns,thus failing to capture the broader diversity of crack types required for com-prehensive training. This repetitive output not only restricts the diversityrequired in the training dataset but may also cause the model to revert toproducing basic, non-informative images such as plain gray backgrounds.These issues significantly diminish the potential of deep learning models togeneralize well across real-world scenarios, making it imperative to developtechniques that stabilize training and prevent mode collapse to ensure theefficacy and reliability of pavement maintenance models. The focus of previ-ous research done in pavement crack data generation has been on the qualityof the generated outputs, but the occurrence of the aforementioned prob-lems, their impact on model training and data augmentation, and ways toremedy them have not been explored, leaving a gap in the current work.1.4 Research ObjectiveThe goal of this research is to fill the knowledge gaps listed in the previoussection by (i) addressing mode collapse and training instability issues in thecurrent state of data augmentation in pavement crack analysis, (ii) present-81.5. Contributionsing a novel generative network capable of generating crack images based onlabels, and (iii) analyzing different crack types in their compatibility withdeep generative and classification models.To achieve a stable training procedure without the occurrence of mode col-lapse, this thesis attempts to use the Wasserstein distance as an alternativeloss function to Binary Cross Entropy in the GAN model for pavement crackanalysis tasks. Wasserstein distance is another measurement for calculatingthe similarity of two distributions and implementing this distance into themodel should provide better gradients in training, and in turn, reduce therisk of mode collapse and stabilize the training.The thesis also attempts to add agency to the generated outputs by im-plementing conditionality into the GAN model. Conditional GAN architec-tures are capable of establishing connections between the input conditionsand generated images, which could allow for the selective generation of pave-ment cracks.Finally, the thesis aims to understand the impacts of adding synthetic imagesof different crack types on crack classification accuracy by training a classifierand comparing the performance of multiple models.1.5 ContributionsThis work contributes to several existing research gaps in the literature. Thefollowing paragraphs give a summary of these contributions.1. This thesis adopts the Wasserstein Generative Adversarial Networkwith Gradient Penalty (WGAN-GP) as the generative model for dataaugmentation in pavement crack tasks. We show implementingWasser-stein distance as the loss function of the model through GradientPenalty reduces training instability and solves mode collapse presentin DCGANs (Deep Convolutional Generative Adversarial Networks),the most common GAN architecture used in previous works.91.6. Outline2. The implemented WGAN-GP model was further modified by addingconditionality to the architecture, creating a Conditional WassersteinGenerative Adversarial Network with Gradient Penalty (C-WGAN-GP), improving the dataset balancing, by giving agency in the gen-erated model output and targeted data generation for specific cracktypes.3. The thesis also conducts a comparative study between DCGAN,WGAN-GP, and C-WGAN-GP, showing their generated outputs, their indi-vidual impact on classification metrics for different crack types, andtheir advantages and disadvantages. To train and compare the models,two established benchmark datasets, Crack500 and CrackForest areannotated and classified into four crack types: alligator, transverse,longitudinal, and block cracks for classification and data augmenta-tion tasks. The effects of data augmentation for each crack type onthe classification metrics, what impact it had on misclassification, andincreasing sample variety inside the class, are then discussed.1.6 OutlineThe remainder of the thesis is organized as follows:• Chapter 2 provides a review of related literature pavement crack anal-ysis tasks, and data augmentation in pavement crack datasets usingdeep learning models, highlighting their contributions and presentingresearch gaps addressed in the thesis.• Chapter 3 gives a small background to deep neural networks andGANs, and then presents the methodology of the implemented modelsin this thesis. The methodology discusses the previously used DCGANmodel, the implemented WGAN-GP and C-WGAN-GP models, theirloss functions, training loop, and architecture, and how adopting theseGAN models fix the issues with the DCGAN architecture.101.6. Outline• Chapter 4 discusses the data and testing framework used in this re-search. It first introduces the different pavement crack types, then goesover the details of the datasets used for training the GAN models, theexisting crack types and their count, and how they were separated fortraining and testing. Then, the testing framework, its implementa-tion, all the classification and GAN model hyperparameters, and thetesting environments are detailed.• Chapter 5 conducts a comprehensive analysis of each GAN model,including a discussion on the generated images by each GAN modelon each specific crack type, their overall diversity and quality, GANmodel stability during training, and mode collapse occurrence. Then,the GAN models are used to augment the original dataset, and classi-fication metrics and results are presented and compared.• Chapter 6 summarizes the key contributions of the proposed modelsand this thesis, while also mentioning the limitations of conductedresearch and future steps in elevating the research.11Chapter 2A Review of Previous WorksManual crack detection in pavement maintenance is subjective and time-consuming, encouraging the development of automated methods to enhancethe process. The adoption of deep learning models has significantly revolu-tionized pavement crack detection by offering solutions that aim to accel-erate the detection process, minimize manual labor, and improve accuracy.These tasks in pavement crack analysis primarily involve classification, de-tection, and segmentation. Classification in this context requires sortingdata into multiple categories, allowing the classifier to determine whetherthe pavement image depicts a crack (binary classification) or to identify thetype of pavement defect among several categories. Classification is crucialfor subsequent detailed analysis and prioritizing repair strategies.On the other hand, the complexities of these tasks vary considerably andpose unique challenges. Detection involves identifying and localizing crackswithin an image through bounding boxes, which not only highlight the lo-cation of the cracks but also classify them into types, such as longitudinal,transverse, or alligator cracks. This step is particularly challenging due tothe diverse appearances of cracks caused by different materials, ages of pave-ment, and environmental conditions. Segmentation goes further by conduct-ing a pixel-level analysis, where each pixel in an image is labeled as eitherpart of a crack or the background. Segmentation demands a high level ofprecision as it aims to accurately outline the crack, enabling more preciseassessments and interventions. The success of each task heavily relies onthe availability of comprehensive and accurately labeled datasets. These122.1. Deep Learning Methods in Pavement Crack Analysisdatasets must represent a wide range of crack types and conditions to trainmodels that are robust and effective across real-world scenarios. The devel-opment of such datasets is a significant undertaking that requires meticulousplanning and resources, emphasizing the need for collaborative efforts in theresearch community to advance the field of pavement maintenance throughmachine learning.2.1 Deep Learning Methods in Pavement CrackAnalysisIn [45], Wang & Hu showed that deep learning classifiers can be utilized ina unique way to get the rough crack skeleton of the image. The image wasdivided into 32x32 and 64x64 square grids, and the classifier model was ap-plied to each square, classifying it into the crack or non-crack classes. Hence,generating a very crude crack map of the image. This idea is fundamentalto pixel-based segmentation algorithms developed later.Attention-based deep learning methods have been gaining attraction in re-cent years. Traditional object detection models are not designed for cracksand their specific characteristics. An automated detection method was pro-posed by Ren et al. [37], using these attention methods to improve crackdetection accuracy. This paper used YOLOv5, a fast and accurate objectdetection model in images, and modified it with several attention methods,including SENet, ECANet, CBAM, and CoordAtt. The authors also gath-ered their own pavement crack dataset, containing four different types ofcracks, pothole, alligator, longitudinal, and transverse. The results fromthe comparison of the baseline models and the attention-infused variationsshowed that YOLOv5-CoordAtt increased the precision of the detection pro-cess by more than 4 percent.To remedy the challenges in pavement crack detection including inconsistentcrack intensities, and background complexity, Yang and their colleagues [47]introduced a novel network architecture named Feature Pyramid and Hi-132.1. Deep Learning Methods in Pavement Crack Analysiserarchical Boosting Network (FPHBN) for pavement crack detection. Thisnetwork integrates context information into low-level features using a featurepyramid approach. It also hierarchically employs nested sample reweightingduring training to balance the contributions of both easy and hard sam-ples to the loss function. Additionally, a novel measurement called averageintersection over union (AIU) is introduced for crack detection.A convenient and cost-effective approach was proposed in Hu et al. [19] todetect and localize cracks using street-view images of pavements. A datasetconsisting of 400 images from five cities is collected and labeled for trainingpurposes. The dataset is then utilized to train YOLOv5, the latest versionof the YOLO network, for target detection. The results demonstrate thatthis network achieves a mAP (mean average precision) of over 70% and adetection time of 152ms, surpassing the performance of the classical methodYOLOv3. Given the ease of collecting images, this method offers a suitablemeans to evaluate pavements.Although Deep Convolutional Neural Networks (DCNNs) detect cracks withgood accuracy, they have very high pixel-level annotation data require-ments. To overcome this challenge, a study conducted by Al-Huda et al.[3] proposed a weakly supervised model that works with image-level anno-tations. A pavement crack classification network provides class activationmaps (CAMs) with multiple image sizes. Then the proposed framework fusesthese CAMs to generate high-quality annotations and a new segmentationnetwork (U2ˆ-net) with Triple Attention is trained on the initial annotations.Further, an incremental annotation refinement (IAR) is used for iterativelyoptimizing the segmentation network and refining the segmentation mask.The method proposed by Feng et al. [12] combines the advantages of themultitarget single-shot multi-box detector (SSD) convolutional neural net-work model and the U-Net model. The SSD detector model complimentsthe U-Net model by reducing the search space for the U-net and therebyreducing the inaccuracy of the segmentation model. The proposed methodcan provide the category information for pavement cracks as well as accu-142.2. State of Data Augmentation in Pavement Crack Analysisrate positioning and geometric parameter information, which can be useddirectly for evaluating the pavement condition.To mitigate the noise sensitivity issues of traditional image processing tech-niques, which could lead to missing cracks due to crack texture variety, andcomplex lighting conditions, DeepLabv3+, a popular segmentation network,was adopted and enhanced by Sun et al. [41]. A multi-scale attention mod-ule is added to the decoder of DeepLav3+ to generate an attention maskthat can assign dynamic weights to low and high-level features. This provesto be better than having fixed weights across different features and helpsDMA-Net achieve state-of-the-art results on various benchmarks.2.2 State of Data Augmentation in PavementCrack AnalysisAs mentioned before, one of the biggest challenges with deep learning mod-els is the availability of training data. Deep learning models require dataabundance to train properly and achieve acceptable metrics. Depending onthe tasks, domain, and the model itself, the required amount and quality ofdata changes. Classic image processing techniques are the most basic formof data augmentation applied to images. However, in some areas, such aspavement crack images, these techniques can be problematic, as techniqueslike rotation or mirroring can lead to fundamental changes in the character-istics of the cracks that do not appear in real life. Deep generative modelscan be a useful alternative for pavement crack data. Deep generative modelscan create quality artificial images that keeps these fundamental character-istics, such as the length, positioning, and direction of the crack. Here, wewill discuss some of the previous work done in different pavement crack tasksand the attempts at data augmentation and data balancing.GANs have been previously utilized to help remedy the lack of balanced anddiverse datasets in pavement crack tasks. A good example of the utilizationof DCGANs is the work by Xu et al. [46]. The augmented data would help152.2. State of Data Augmentation in Pavement Crack Analysiswith insufficient available data for the training of deep learning models.This paper used a variation of DCGAN to generate pavement crack images,solving the insufficient training data, and then used them in a classificationmodel. The data was captured by a UAV, and for the classifier, a VGG-net based CNN was deployed. The evaluation of the original data modeland the augmented one showed an increase of more than 10 percent in thereported accuracy, pushing the accuracy to over 90 percent. The researchersalso experimented with a class evaluation map (CAM) which could act as adetector for cracks in the images but was not accurate enough.Pei et al. [33] addressed the low availability of data for pavement crackdetection using deep learning models by employing a Deep ConvolutionalGenerative Adversarial Network (DCGAN). They chose a sample of labeledcrack images and performed several preprocessing steps to improve the de-tection task. The preprocessed images were then given to a variationalautoencoder (VAE), which created the encoded input data of the DCGAN.DCGAN generated new synthetic pavement crack images, which were fedinto a Faster R-CNN network, one of the established object detection deeplearning models. The evaluation of this new data-augmented model showedan improvement compared to the base model with non-augmented data.In Ma et al. [27] the difficulties of data acquisition and defect countingin pavement crack detection were also discussed in the literature. Theirsolution to these issues was a deep learning approach with a GAN networkas the image generator, PCGAN, and a modified version of YOLOv3, asingle-shot image object detection algorithm, for the detection and crackcounting, called YOLO-MF. YOLO-MF has acceleration and median flow(MF) algorithms. The YOLOv3 model detects the cracks and MF keepstrack of them in moving video frames. The cracks are tracked over thevideo frames but are counted only once so the total number of cracks can becalculated by the end of footage. Their proposed detection method achievedbetter results than other detection methods, such as Faster R-CNN and SSD,in small and adjacent frames. Also, their acceleration algorithm achieved162.2. State of Data Augmentation in Pavement Crack Analysissix times faster processing time compared to traditional models.Although GAN-based crack detection models have shown improvement inthis field, they have also introduced extra computational costs and overhead.Liu et al. [26] proposed a lightweight GAN structure for pavement crackdetection and classification. They experimented with Squeeze and Expand(SE), multiscale convolution (MC), and depth-wise separable convolution(DSC), to improve the computational cost for the network. Their resultsindicated that GAN-based models with SE in their fully connected layer, andMC and DSC in their transpose convolutional layers significantly reduced thenumber of parameters in the model, speeding up the training and inferenceprocess, while maintaining acceptable classification accuracy.CrackGAN was introduced by K. Zhang et al. [49], a DCGAN network mod-ified for pavement crack detection and segmentation tasks. This study triesto solve the “All Black” phenomenon, where the unavailability of accurateground truth (GT) images, leads to misclassification of pixels in the imagesas a background. CrackGAN is a crack-patch-only (CPO) supervised gener-ative adversarial network, that generates input data from non-accurate GTfor a Fully Convolutional Network (FCN)-based pavement cracks segmen-tation model. The proposed approach was tested on several datasets andachieved competitive and state-of-the-art results.In T. Zhang et al. [50], the authors tried to solve the problem of insufficienttraining data by modifying the DCGAN architecture for the pavement do-main and using it to generate synthetic training data. This modified model,APC-GAN, changes the architecture to adapt to the specific characteristicsof pavement crack images, such as their size or the need for bigger kernelsizes in the convolutional layers. To segment the cracks in the pictures, anattention-infused U-Net, named AttuNet was added to the proposed frame-work. They used the open-source DeepCrack dataset to test their modelsand achieved better evaluation metrics, such as mIoU, F1 Score, and Recall,compared to traditional data augmentation and segmentation methods.172.2. State of Data Augmentation in Pavement Crack AnalysisThe low availability of data in pavement crack tasks can also be improvedby incorporating the crack data in other domains. This information can betranslated between domains to solve the issues with low availability prob-lems. Branikas et al. [6] made use of CycleGAN, a generative adversarialnetwork that learns the mappings between two domains, as a novel dataaugmentation method. This approach generates challenging images for seg-mentation tasks that are underrepresented in the target domain. A U-Netwas used as the segmentation algorithm in the evaluations of the augmenteddata. The model was trained with the original and augmented datasets,showing the augmented models to be superior in crack segmentation tasks.In a more recent work, Hui et al. [48], used 5 open-source road datasetsin an attempt to augment data for two common types of cracks namely,longitudinal and transverse. The study incorporated a conditional genera-tive adversarial network modified with the self-attention mechanism to solvechallenges with the inability to selectively generate images and limited crackdata. By conditioning the generator on category information, the SAGANmodel was able to generate high-quality data crack images while focusingon targeted categories.182.2.StateofDataAugmentationinPavementCrackAnalysisLiterature AugmentationMethod UsedData Key Points andLimitationsXu et al. 2022[46]DCGANA dataset of pavementcrack images gatheredby UAVUsed VGG-net as theclassifier.Different crack types are notseparated.Mode collapse and traininginstability not addressed.Lack of agency in datageneration.Pei et al. 2021[33]DCGAN with VAE -Faster R-CNN is used as thedetection method.A combination of VAE andDCGAN for generatingpavement crack images.Did not focus on agency ingeneration or differentpavement crack types.Ma et al. 2022[27]DCGAN -Used a modified version ofYOLOv3 as the detectionalgorithm.192.2.StateofDataAugmentationinPavementCrackAnalysisTable 2.1 continued from previous pageLiterature AugmentationMethod UsedData Key Points andLimitationsAchieved better detectionresults than other methods atthe time with the dataaugmentation.Did not focus on agency ingeneration or differentpavement crack types.Liu et al. 2023[26]Modified DCGAN withSE, MC, DSC-Modified DCGANarchitecture to make itlighter and faster.Maintained acceptableperformance with the changein number of parameters.Did not focus on agency ingeneration or differentpavement crack types.K. Zhang et al.2021 [49]DCGAN -Addressed the “All Black”phenomenon with acrack-patch-only approach.202.2.StateofDataAugmentationinPavementCrackAnalysisTable 2.1 continued from previous pageLiterature AugmentationMethod UsedData Key Points andLimitationsDid not focus on agency ingeneration or differentpavement crack types.T. Zhang et al.2023 [50]DCGAN DeepCrackUsed AttuNet as thesegmentation model.Changed the DCGANparameters to better fitpavement characteristics.Did not focus on agency ingeneration or differentpavement crack types.Branikas et al.2023 [6]CycleGAN -Used CycleGAN to transferfeatures from concrete cracksto the pavement domain.U-Net was used as thesegmentation algorithm.212.2.StateofDataAugmentationinPavementCrackAnalysisTable 2.1 continued from previous pageLiterature AugmentationMethod UsedData Key Points andLimitationsDid not focus on agency ingeneration or differentpavement crack types.Hui et al. 2023[48]Conditional GAN withSelf-attention(SAGAN)5 datasets includingCrack500 andCrackForestSolved lack of agency in datageneration.Used SA to increase outputimage qualityDid not mention modecollapse and traininginstability.Focused only on simplercrack types.Table 2.1: This table includes a summary of the previousresearch done in data augmentation for pavement crack anal-ysis with their advantages and disadvantages.222.2. State of Data Augmentation in Pavement Crack Analysis2.2.1 RemarksAs evident in the literature review, multiple studies have attempted to usedeep learning models for pavement crack detection applications. These stud-ies gained better results compared to their traditional counterparts, such asvision-based methods with thresholding and image-processing techniques,while maintaining their inherent benefits, such as scalability, flexibility, androbustness.By looking at the data augmentation research done in pavement crack anal-ysis, we can see the increased interest in this area of research, an indicatorof dataset quality and balance’s importance. However, to the best of theauthor’s knowledge, aside from one recent paper, no study has attemptedto solve training instability and mode collapse in pavement crack data gen-eration models, introduce agency to the generation outputs, and discussthe effects of data generation on specific crack types. Even though Huiet. al. explored the use if conditional GAN to address selective data gen-eration, self-attention is employed for higher output quality and does nottarget training instability and possible mode collapse in the outputs. Fur-thermore, the study focuses on transverse and longitudinal cracks which are(a) simpler in terms of their visual features and are easier to differentiate,(b) are both dominant types of cracks. More complex crack types such asalligator cracks are not included in the research. In fact, most previous stud-ies used the same general GAN architecture with minor changes to matchtheir dataset, while the biggest difference in their approaches came from thetechnique or detection/classification/segmentation model used in pavementcrack analysis.This work tries to fill in the gaps existing in the previous studies conductedon data augmentation for pavement crack analysis, namely mode collapse,training instability, lack of agency in data generation, and crack-type com-patibility with augmentation while comparing the newly implemented ap-proaches to the existing studies. A DCGAN architecture suitable for theused dataset in this thesis is implemented as a representation of the most232.2. State of Data Augmentation in Pavement Crack Analysiscommonly used method in the literature. This model is then compared withthe implemented approaches, WGAN-GP and C-WGAN-GP, in this paperon classification metrics, training time, and computational cost, and theiradvantages and disadvantages are discussed.24Chapter 3Selective Pavement CrackGeneration UsingC-WGAN-GPTo address the mode collapse phenomena, training instability, and lack ofagency issues associated with GANs and the current state of data aug-mentation techniques, an understanding of GAN family of models, neuralnetworks, and their building blocks is required. Sections 3.1 and 3.2 aimto provide this understanding by explaining the operations of deep neuralnetworks, the building the underlying mechanics of GANs works, and whatallows them to learn features from images and then generate synthetic out-puts. The rest of this chapter is dedicated to discussing the details of theGAN models, the impact of loss function on the training of GAN models,what leads to mode collapse and training instability in DCGAN, and theproposed model architecture to solve these issues.3.1 Background3.1.1 Deep Neural NetworksDeep neural networks (DNNs) are a class of fundamental models in modernmachine learning and can be traced back to the emergence of the back-propagation algorithm [38] which allowed for efficient training of multi-layerneural networks. DNNs are characterized by their multiple processing lay-253.1. Backgrounders which enable extraction and transformation of features from raw data.This architecture allows DNNs to learn complex patterns and relationshipswithin vast amounts of data, making DNNs applicable to various tasks, fromobject detection in images to text generation.Each processing layer in the DNNs architecture is made up of building blockscalled neurons. Each neuron has an associated set of weights and is chargedwith extracting certain features from its given input. As the model is trained,the extracted features become more refined, transforming them into moreuseful representations. The learning process in DNNs involves adjusting theweights of the connections between neurons across different layers. Thisadjustment is typically achieved through backpropagation as part of an op-timization algorithm such as stochastic gradient descent and ADAM. In thetraining phase, the model is typically fed large amounts of labeled data,trying to find the right combination of weights and connections for neuronsthat results in the desired outputs. Throughout the training, the differencebetween the output data and the desired outputs are constantly measuredand quantified through a loss function. The gradients of the loss value calcu-lated using the loss function are then backpropagated into the model layers,guiding the adjustments to the weights. This iterative process allows thenetwork to minimize errors improve its performance over time and enhanceits ability to accurately solve complex tasks such as image classification andobject detection.There are different types of layers in DNNs, each with its own unique func-tionality designed for specific purposes. Convolutional layers are one ofthem. Convolutional layers are specialized layers in DNNs designed primar-ily for processing data with a known grid-like topology where the geometricalrelations between features in the data are important [39]. This characteris-tic of CNNs makes them suitable in processing pavement crack image datain the form of 2D images. These layers implement a mathematical oper-ation called convolution, which involves sliding a filter or kernel over theinput data to produce a feature map. Convolution extracts local feature263.1. BackgroundFigure 3.1: A Sample figure of a 3 x 3 Convolution Layer Kernel operatingon a longitudinal crack image, extracting relevant features. The Conv layerhas multiple kernels each creating their own output. The concatenation ofthese outputs will form the output of the Conv layer.representations from the input, such as edges or textures in the context ofimages, which are critical for tasks like object detection and classification.Convolutional layers are typically accompanied by other layers and com-ponents such as activation functions, which introduce non-linearities intothe model making it capable of learning more complex patterns, and pool-ing layers, reducing the spatial dimensions of the feature maps to decreasethe computational load and enhancing the detection of features at multiplescales. A sample 3 x 3 kernel operating on a pavement crack image is pre-sented in figure 3.1. The introduction of convolutional layers has changedthe field of AI and more specifically computer vision forever. ConvolutionalNeural Networks (CNNs), which employ multiple convolutional layers, havebecome state-of-the-art in a wide array of tasks and applications in the fieldof computer vision. CNNs’ ability to learn spatial hierarchies of features di-rectly automatically and efficiently from data has significantly reduced theneed for manual feature extraction, a labor-intensive process in traditionalapproaches.273.1. Background3.1.2 Generative Adversarial NetworksGenerative Adversarial Networks (GANs) are a family of DNN models in-troduced in 2014 by Ian Goodfellow and his colleagues [15]. This family ofmodels is capable of generating high-quality outputs with the same char-acteristics and features as their given input. The conceptual foundation oftheir creation is rooted in a scenario where the two neural networks insidethe model, the discriminator, and the generator, are engaged in a game-likecontinuous adversarial process. The generator tries to create outputs thatare indistinguishable from the input, while the discriminator is in chargeof differentiating the generated and real data from each other. Throughoutthis process, the generator becomes better at generating higher-quality out-puts, and the discriminator learns to identify the fakes more effectively, thusprogressively improving each other. The training process of GANs is seenin figure 3.2.Figure 3.2: A GAN model general overview with the flow of information.The generator initially generates the fake images and the discriminator triesto distinguish the real images from the fake images. The generator anddiscriminator are then trained based on the results of the discriminator.GANs have been utilized in the generation of various data types, with the283.2. Data Augmentation Using DCGANmost common one being images. There have been many versions of GANsintroduced over the years, each attempting to improve on the previous ones,solving existing issues of, or adding new features to the models. DCGANs[36] incorporated convolutional and convolutional transpose layers into theirarchitecture to better process and generate images, stabilizing the trainingprocess. They have had a significant influence in demonstrating the powerof deep convolutional networks as generators. WGANs [5] changed the typi-cal loss function of the GAN models to make a more stable training processpossible. CycleGANs [52] allowed for image-to-image translation withoutneeding paired examples, with some of their most notable use cases be-ing in style transfer and image colorization. Conditional GANs [31] addedconditionality to the architecture, making targeted image and data genera-tion possible by using input conditions. Progressive Growing of GANs [21]incrementally increases the complexity and resolution of the generator anddiscriminator during training, which helps in producing very high-resolutionimages and improves the training stability.With this brief background and introduction to DNNs and GANs, it will beeasier to understand the DCGAN architecture, the Wasserstein distance lossfunction, and the modifications required to make conditionality possible.3.2 Data Augmentation Using DCGANA GAN is a deep learning model architecture that can generate new ar-tificial data based on real input data and learned features [15]. After theintroduction of GAN, other versions of this architecture emerged over theyears, focusing on different aspects of the generation process. One of thesemodels was Deep Convolutional Generative Adversarial Network (DCGAN)[36].Researchers showed that DCGAN and its deep convolutional adversarialpair can learn hierarchical features from images and scenes and proved itselfto be a great image-generation tool. The DCGAN architecture consists of agenerator and a discriminator network. The generator learns to transform293.2. Data Augmentation Using DCGANlatent vectors or random noise into synthetic data that resembles real data.The discriminator is tasked with differentiating the generated fake data andthe real data from each other by classifying them. The optimization processof DCGAN can be described as a minimax game. The discriminator aimsto minimize the loss function associated with its classification task, whilethe generator aims to maximize the loss function associated with fooling thediscriminator. This gives rise to a competitive dynamic between the twonetworks [15].Loss FunctionGANs utilize two probability distributions, one for the generated outputof the generator model Pg, and one for the distribution of the real imagesPr. Their goal is to make sure these two distributions are close to eachother, indicating the output is high-quality and similar to the original im-ages. Jensen–Shannon divergence (JS-divergence) is the loss function usedin most GAN models, including the DCGAN. JS-divergence is a symmet-ric measure derived from Kullback-Leibler divergence (KL-divergence) [30].KL-divergence is a measure of calculating the divergence of one distribu-tion from another. The KL-divergence is not a symmetric method, meaningKL(P ∥ Q) is not necessarily equal to KL(Q ∥ P ), with KL(P ∥ Q) be-ing the KL-divergence of distribution P from distribution Q. KL-divergencelacks some of the defining features of distance measurement, such as lackof symmetry and not holding true in triangle inequality. JS-divergence isdefined as the symmetrical version of KL-divergence, which has the char-acteristics of a distance measurement. As such JS-divergence is used inmost GANs including DCGAN as the measurement to calculate the dis-tance between the generated and real image distributions. The formula forJS-divergence can be seen in equation 3.1:JS(P ∥ Q) = 12KL(P ∥M) + 12KL(Q ∥M) (3.1)With M being the average of the Q and P distributions. In implementa-303.2. Data Augmentation Using DCGANtion, Binary Cross-Entropy loss is used as the loss function to calculatethis similarity. Binary Cross-Entropy is a common loss function for binaryclassification tasks and calculates the probability distributions of the pre-dicted and true labels. In the case of DCGANs, the real crack images wouldbe part of the true class while the synthetic fake images form the secondlabel. The use of cross-entropy in the implementation of the model implic-itly applies the JS-divergence minimization in the training. In this thesis,the loss function in the DCGAN architecture will be called cross-entropy,both representing the JS-divergence and binary cross-entropy loss to reducethe confusion caused by the interchangeable names. Mathematically, theobjective of the discriminator can be written as follows:maxVD(D,G) = Ex∼pr [logD(x)] + Ez∼pd(z)[log(1−D(G(z)))] (3.2)In equation 3.2, x is the real data from the distribution Pr; z is the noisevector generated randomly from the Gaussian distribution Pd(z); D and Gare the discriminator and the generator networks, consisting of convolutionaland deconvolutional networks, respectively. The discriminator’s goal is tocorrectly classify the original images as real and the generated images as fake.This is evident in the equation with the first part of the equation calculatingthe loss on real images and the second part of the equation is calculatingthe binary cross-entropy on generated images. Equation 3.3 below showsthe value function for the generator:maxVG+(D,G = Ez∼Pd(z)) = Ez∼pz(z)[logD(G(z))] (3.3)The generator’s goal is to create images that the discriminator classifies asreal. In the generator loss function, the generated images are passed tothe discriminator and are classified. If the discriminator classifies them asreal images, the generator has been successful in fooling the discriminator.313.2. Data Augmentation Using DCGANThe binary cross-entropy loss not only provides insight into how well eachnetwork is performing while driving both of them towards improvement.ArchitectureThe discriminator network architecture is made from convolutional blocksthat analyze the initial image and extract rich features from them. Each ofthese blocks includes a 2D-convolutional layer with a kernel size of 4, strideof 2, and padding of 1. A kernel is a small matrix that slides over the inputfeatures and performs the convolution operation on them. The dimensionsof this matrix are defined by the kernel size hyperparameter. This lets themodel find the relations between each input feature and the features near it,understanding their geometrical relations. Stride defines the number of stepsthe kernel takes on the input when processing it. A stride of 2 means thekernel moves two steps at a time. A stride of 2 allows the reduction of inputsize without the use of pooling functions, resulting in the model learning itsown down-sampling function. Padding is the process of adding extra pixelsto the sides of the input. Padding of 1 adds one layer of zero features to eachside of the input, preventing information loss in the convolution operations.After the convolution layer, a batch normalization operation is performedin the convolution block to stabilize the network, speed up the convergencein the model, and improve the generalization. Finally, leakyReLU is usedas the activation function for the block. The discriminator used in thisDCGAN model consists of 6 convolutional blocks, with the first one, wherein original image is the input, missing the batch normalization. At the end,a convolutional layer is with output channel size of 1, kernel size of 4, strideof 1, and padding of 0 is used to give a single value as the output of thenetwork. A sigmoid function is then applied to the output, converting thevalue into a logit acceptable by the network’s loss function. Binary CrossEntropy is the chosen loss function for the network. This loss function givesthe probability of the input image being a real pavement crack image or anartificially created one, generated by the generator network.323.2.DataAugmentationUsingDCGANFigure 3.3: This figure includes the architectures for the discriminator and the generator in DCGAN imple-mentation. The discriminator takes in images as input and validates their originality while the generator createspavement crack images based on its random noise vector.333.2. Data Augmentation Using DCGANThe generator uses the inverse architecture of the discriminator network,generating artificial pavement crack images that intend to fool the discrim-inator by taking a random sequence of numbers with a size of 100. Thearchitecture of the generator consists of transposed convolution blocks. Theblocks include convolution transpose layers, batch normalization layers, andReLU activation functions. Convolution transpose layers apply up-samplingon their input through kernels and convolution transpose (or deconvolution)operations. They increase the spatial resolution of input feature maps in theneural network. The first convolution transpose block uses a convolutiontranspose layer with kernel size of 4, stride of 1, and padding of 0. This isfollowed by a batch normalization layer and a ReLU activation layer. Afterthat, the convolution transpose blocks use a kernel size of 4, stride of 2, andpadding of 1. Finally, the network is finished with a convolution transposelayer, and a tanh activation function. Through this architecture, the inputrandom vector is converted to a 256 * 256 grayscale image. Figure 3.3 showsthe DCGAN architecture.Training LoopThe training loop of DCGANs consists of discriminator and generator train-ing, intending to refine the generator to continuously create more realisticimages, while the discriminator learns to better differentiate fake and realimages. In the first step, both models are initialized. The generator startsfrom a random noise vector and learns to generate images, while the dis-criminator learns to classify images as real or fake. The training proceeds inmini-batches, with each mini-batch consisting of noise-vector-generated im-ages and real pavement crack images. In each mini-batch, the discriminatoris trained, and the training proceeds to the generator. A high-level view ofthis process is shown in figure 3.4. The following is a detailed breakdown ofthis process:• Discriminator Training:1. Fake images are generated by giving the random noise vector to343.2. Data Augmentation Using DCGANthe generator.2. The discriminator processes the real images and tries to classifythem as real. The output labels are compared against the truelabel and the loss value is calculated.3. The discriminator processes the fake images and tries to classifythem as fake. The output labels are compared against the fakelabels and the loss value is calculated.4. The two calculated loss values are summed up to calculate thetotal loss of the discriminator. Then, the discriminator parame-ters are updated through backpropagation using the loss value tominimize the loss.• Generator Training:1. Random noise vectors are given to the generator to produce fakeimages.2. The fake images are passed along to the discriminator, but thegoal in this step is to fool the model by labeling the generatedimages are real.3. The discriminator processes the fake images, and the loss valueis calculated.4. The generator parameters are updated using the calculated lossvalue to minimize the loss and improve the model’s ability to gen-erate images that are more likely to be classified as real images.This training loop is repeated until a certain number of epochs or specificconditions are met in the generated outputs. Each iteration of the trainingloop enhances the discriminator’s abilities to discern fake images from realimages, while the discriminator becomes more adept in fooling the generatorwith higher-quality generated images.353.2. Data Augmentation Using DCGANFigure 3.4: The training loop of the DCGAN model can be seen here.First, the discrimination training block is executed and the discriminatorgets updated. Then, the updated discriminator is used to assess the validityof a new set of generated images and the generator’s weights are updated.Limitations of Using DCGAN in Pavement Crack DatasetsCross Entropy, the traditional loss function used in GANs, measures the dif-ferences between the predicted and true label distributions. This loss is cal-culated through a logarithmic function which can lead to worse model con-vergence and unexpected behaviors in gradient, causing training instabilityand mode collapse. This phenomenon usually happens when the predictionsare far from the target, exacerbating the gradient. When the discriminatoris performing too well, producing output values really close to 0 and 1, thegradients provided by the discriminator can vanish. Since the generator usesthe discriminator to calculate its loss, this small gradient makes the updates363.2. Data Augmentation Using DCGANmade to the model parameters too small, essentially halting the learning ofthe generator. With the mentioned behavior in the gradient, unwanted situ-ations might arise during the training of the GAN model. DCGAN is one ofthe GAN models that utilize cross-entropy in its training as the loss functionand is also affected with these problematic behaviors. The sharpness in gra-dient caused by the loss function can lead to halting of the training in thegenerator or significant swings in the learning process, causing oscillationand instability in training. The oscillations lead to changes in generator anddiscriminator behaviors and outputs without them achieving convergenceand reaching equilibrium. The resulting images from the generator in thesesituations are usually distorted and noisy. This was evident in the trainingof the DCGAN model on pavement crack images where the generator, af-ter producing improved outputs during its training epochs, would suddenlyhave distorted all-gray images as outputs. Training instability makes themodel behavior unpredictable and could waste long training epochs with nousable outputs.Another issue arising from this training instability and unexpected gradientbehavior in the model is mode collapse. Mode collapse is a failure scenarioin GAN models where the generator is only capable of producing a limitedvariety of outputs. Instead of capturing the full diversity of the input datasetand creating images that contain the same variety in them, the generatoronly outputs identical images or images with minor differences, regardless ofthe variety and changes in its input noise vector. Since the generated imagesare still similar to the inputs and meet the goodness criteria for cross-entropyloss used in the GAN model, the generator halts in its improvement. Withthe halted improvement in the generator, the GAN model fails in its missionto produce diverse outputs containing the variety present in the inputs. Thisis especially bad for data balancing purposes as all the output images arethe same and would only worsen the overfitting caused by an inadequatedataset.These issues are further exacerbated by the input dataset to the model. If373.3. Proposed Methodologythe input dataset is imbalanced, lacks variety in its samples, or has com-plex images that would take a long time for the generator to replicate, thepossibility of mode collapse increases. When the discriminator overpowersthe generator in the earlier stages of training, the generator could get forcedinto a narrow path where it might focus on a specific set of features presentin the images that exploit discriminator weaknesses. Also, with the lackof gradient diversity in gradients and meaningful updates to the generatorowing to factors such as cross-entropy loss, the generator does not get thechance to explore other possible outputs and instead focuses on replicatinga limited set of images.c, it is possible for the discriminator to produce high certainty in the earlierstages, leading to unvaried gradients. This is a direct cause of training in-stability and mode collapse in the model. Also, resizing the images to fit theinput requirements of the GAN and classification models can cause featureloss in the data, making convergence and stable training in the GAN modeleven harder. To combat these issues and design a GAN model more suitablefor pavement crack image generation, Wasserstein GANs and Wassersteinloss as an alternative to DCGAN and binary cross entropy loss in pavementcrack augmentation are explored in the next section.3.3 Proposed Methodology3.3.1 WGAN-GPWGAN was created as an alternative to the DCGAN model to solve someof the challenges faced in training DCGAN models by Arjovsky [5]. Theinception of this model was derived from the need to address training in-stability and mode collapse as a result of the cross-entropy loss functionused in DCGANs. WGAN utilized the Wasserstein distance, also knownas Earth’s mover distance as its loss function, bringing better convergenceproperties, usable gradient always, and less emphasis on discriminator andgenerator balance, to the training process [5]. The networks’ architecturein the model follows the same scheme as in DCGANs with minor changes383.3. Proposed Methodologymade to accommodate the new loss function.Loss FunctionWasserstein distance provides a measure of cost required to transform onedistribution to another [43]. A popular conceptualizing metaphor for thisdistance is how much energy is required to convert a pile of dirt with aspecific shape (first distribution), to another shape (second distribution).The theoretical formula for this distance is present in equation 3.4.W(P,Q) = infγ∈Γ(P,Q)∫X×X∥x− y∥ dγ(x, y) (3.4)In this formula, Q and P represent the two probability distributions over ametric space X. γ(x, y) is the transport plan specifying how much “mass”from point x in distribution P should be transported to point y in Q. Theset Γ(P,Q) includes all of these possible transport plans. Here, ∥ x − y ∥is the cost function, typically represented by the Euclidean distance be-tween point x and y, measuring the cost of moving the mass from pointx to y. The integral calculates the total cost of transporting mass accord-ing to plan γ. Finally, the infimum, the greatest lower bound, over allpossible transport plans in Γ(P,Q) is taken to find the minimum cost totransform P to Q. To make Wasserstein distance computable in practice,Kantorovich-Rubinstein duality is used to make the calculation more feasi-ble. Kantorovich-Rubinstein duality states that when the probability spaceis a metric space, for any L¿0:W(P,Q) =1Lsup∥f∥L≤L(Ex∼Pr [f(x)]− Ez∼P (z)[f(g(z))])(3.5)In Equation 3.5, the supremum is taken over all functions of f that are L-Lipschitz. Here, L, the Lipschitz constant is 1. This supremum gives theWasserstein distance between real and generated data distributions. f(x) isthe discriminator, or critic in WGAN, output for the real data, and fw(g(z))393.3. Proposed Methodologyis the discriminator output for the generated data by the generator. Thediscriminator tries to maximize distance, differentiating the real and fakedata, while the generator tries to minimize this equation by generating datasimilar to the real ones. f(x) obeys the Lipschitz constraint. The Lips-chitz condition is a mathematical condition that ensures the continuity andboundedness of a function and is as follows:|f(x1)− f(x2)| ≤ L.|x1 − x2| (3.6)Where f is the discriminator function, x is the input point and L is theis the Lipschitz constant. This helps the gradient of the function beingbounded, leading to a more stable optimization. Using the equation 3.7, theloss formula for the discriminator and generator in WGAN would be:LDiscriminator = Ex∼Pr [f(x)]− Ez∼P (z)[f(g(z))] (3.7)Maximizing this function is akin to approximating the supremum in theKantorovich-Rubinstein duality.Enforcing 1-Lipschitz constraintOne of the important aspects of implementing this loss function is enforc-ing the 1-Lipschitz constraint. This condition prevents rapid changes in thefunction and is critical in having a valid estimation of Wasserstein distance.The original implementation of WGAN used weight clipping as the con-straint holder. In weight clipping, weights of the discriminator are boundedwithin a certain range such as [-c and c]. This is essentially a brute-forcemethod to prevent rapid changes in the discriminator, that validates the1-Lipschitz condition and prevents exploding gradient in training. This isa relatively simple method to implement and understand. However, limit-ing the weights in the discriminator severely hinders its ability to learn andbetter distinguish between fake and real images. Furthermore, arbitrary403.3. Proposed Methodologyclipping of weights might lead to more training instability, which increasesthe need for hyperparameter tuning [16]. To remedy this problem, the Gra-dient Penalty (GP) was introduced, leading to the creation of WGAN-GP.GP allows for a more stable training network than WGAN and requires verylittle hyperparameter tuning [16]. The idea behind this method is to add apenalty term to the loss function based on the gradient norm of the discrim-inator. This term penalizes the norm of the gradient of the disseminator’soutput with respect to its input. The penalty ensures that the gradientnorm does not exceed 1 (thus maintaining the Lipschitz condition) acrossthe batch of real and generated samples. The equation below shows the fullloss function with a gradient penalty:L = Ex∼Pr [f(x)]− Ez∼pz [f(g(z))] + λExˆ[(∥∇xˆf(xˆ)∥2 − 1)2] (3.8)The equation consists of the original WGAN loss plus the gradient penaltyterm. Pxˆ is the distribution obtained by uniformly sampling a straight linebetween the original data distribution and the generated data distribution,Pr and Pg(z). λ is the penalty coefficient and is used to add weight tothe gradient penalty. GP tends to stabilize the training by ensuring thediscriminator’s gradient always remains meaningful and consistent, avoidingissues such as exploding and vanishing gradients. Unlike weight clipping,this penalty term does not restrict the weights by a constant value anddynamically changes the penalty term based on generated and real data.Finally, GP helps the convergence in the model.413.3.ProposedMethodologyFigure 3.5: This figure includes the architectures for the discriminator and the generator in WGAN-GP imple-mentation. The discriminator takes in images as input and validates their originality while the generator createspavement crack images based on its random noise vector. The discriminator does not use a Sigmoid function atits end and uses Instance Normalization instead of Batch Normalization. The output of the discriminator is givento the Gradient Penalty calculation and Wasserstein distance loss function.423.3. Proposed MethodologyArchitectureWGAN mostly follows the same architecture for its discriminator and gen-erator as DCGAN, with the difference in the use of instance normalizationinstead of batch normalization in the discriminator. Instance normalization,as opposed to batch normalization, performs the normalization operationonly on the channels and pixels of the instance, not the whole batch. Thisway, the spatial relations between the pixels are preserved which is useful forcalculating the Wasserstein distance. The convolution blocks in the discrim-inator use the same hyperparameters as the DCGAN discriminator, beginkernel size of 4, stride of 2, and padding of 1, followed by a convolutionlayer with kernel size of 4, stride of 1, and padding of 0. Since WGAN-GPdoes not use the Wasserstein distance instead of the binary cross entropy tocalculate the loss of the GAN model, the last layer of the discriminator doesnot use sigmoid as an activation function and directly outputs the calculatedvalue.The generator follows the same architecture as the DCGAN architecture.Convolution transpose blocks, made up of convolution transpose layers withkernel size of 4, stride of 2 (stride of 1 for the first block), and paddingof 1 (padding of 0 for the first block), followed up by batch normalizationlayers and ReLU activation layers. In the WGAN-GP model itself, duringthe training, the discriminator or the critic network, is trained more thanthe generator. In this implementation, the discriminator is first trained for5 loops and then the generator is trained for 1. During the training of thediscriminator, the loss value on real and artificial images is calculated andthen gets added to the gradient penalty value. Figure 3.5 gives an overviewof the WGAN-GP architecture.Training LoopTraining the WGAN-GP is like any other GAN model with the generatorand the discriminator trying to best each other at every epoch. The differ-ence between DCGAN and WGAN-GP in their training is in the number433.3. Proposed Methodologyof epochs each network, the discriminator and the generator, are trained inthe loop. In DCGAN, the overpowering of the generator by the discrim-inator was one of the causes of training instability and mode collapse. Adiscriminator with early good results hinders the gradient backpropagationand halts the updates of the generator. With the change in loss function, thediscriminator in WGAN-GP would have much smoother and more meaning-ful gradients. Even if the discriminator performs significantly well, the lossvalue and the gradient would allow the generator’s training to continue asnormal.The limitation of sensitivity in training parameters such as individual modelepochs are not present in WGAN-GP, allowing for the discriminator to betrained more epochs compared to the generator. The discriminator acts asthe approximator for Wasserstein distance, and the more it is trained, themore accurate the approximated value is. In the initial stages of training,unlike DCGANs, a stronger discriminator can ensure the updates to thegradient are guiding the generator in the right direction. With the mentionedadvantage of WGAN-GP compared to DCGAN, it is a common practice tore-do the training steps of the discriminator multiple times before performingthe same for the generator, with the most commonly used ratio being 5:1.As seen in this section, WGAN-GP elevates the capabilities of the DCGAN.It changes the loss function and parts of the model architecture and im-proves the stability during training. However, this new model still lacks theability to selectively generate pavement crack images based on specified cat-egories. In the next section of this chapter, conditional GANs, and addingconditionality as a means of agency to the WGAN-GP architecture will bediscussed.3.3.2 C-WGAN-GPAlthough GANs are great for data generation, there is no control over thegenerated output. Conditional GANs [31] were introduced to allow condi-tioning the network with additional information, such as the class of the443.3. Proposed Methodologyinput, meaning the network can learn the difference between different in-puts with the help of the additionally provided information. This way, themodel can generate specific outputs based on the given information dur-ing inference. In the case of pavement cracks, each crack has a specifictype and simply creating lines on pavement can lead to the generation ofunrealistic crack images. Pavement cracks of different types emerge undercertain conditions, and classifying or detecting them accurately can be agood indicator of such characteristics. Having cracks that do not belong toany classes could lead to their misclassification, incorrect results, and faultydecision-making. To create a conditional WGAN-GP model, the WGAN-GP discriminator and generator architecture needs to be modified, allowingthe input of classes, and merging this data with the image data.ConditionalityConditional GANs incorporate additional information alongside the typicalinputs to the discriminator and the generator. This additional informationcan guide the discriminator to establish a relationship between the inputimage and the given label and understand their connection. The generatormixes this input with the random noise vector and through this new input,learns to generate images that have this established connection with theinput. The discriminator in the conditional GAN is not only in charge ofauthenticating the originality of the image but also checking the correspon-dence of the input image to the given label. It should be noted that theoutput of the discriminator remains the same, being a value between 0 and1 in DCGANs and the raw model output in WGAN-GP. However, the labelhas intrinsically impacted the score or output during the data processing,using the established relations.Loss FunctionC-WGAN-GP uses the same loss function as WGAN-GP, meaning an ap-proximation of Wasserstein distance, with the Lipschitz constraint enforcedthrough gradient penalty. The equation for the discriminator loss function453.3. Proposed Methodologycan be seen in equation 3.9:L = Ex∼Pr [f(x)]− Ez∼pz [f(g(z))] + λExˆ[(∥∇xˆf(xˆ)∥2 − 1)2] (3.9)L is the discriminator loss, Ex∼Pr [f(x)] and Ez∼pz [f(g(z))] are the mean ofthe discriminator outputs for real and fake images, λ is the penalty coeffi-cient. The changes in the model architecture to accommodate image labelsalongside input does not impact the loss function. The model essentiallycombines the label encodings and either random noise vector or pavementcrack image without changing the WGAN-GP loss function.ArchitectureIn the conditional GAN models, the label of the data that needs to begenerated needs to be passed along the latent space vector from which theimage is generated. To achieve this, the label must be embedded in a formthat can be concatenated with the latent vector and processed through thearchitecture. Depending on the model architecture and the preferences ofthe programmer, there are different ways to do this embedding. In thisthesis, this embedding is performed through a convolutional transpose layerthat performs a deconvolution operation on the single input label. In thegenerator of the two other networks, the first convolutional transpose blockuses 128 * 16 kernels, meaning the number of channels in the next layerwould be 128 * 16. The second convolutional transpose block uses 128 *32 kernels and then continues from that to create the 256 * 256 desiredimage. In C-WGAN-GP, the second block does not use deconvolution tocreate the 128 * 32 channels, but the input latent vector and label are bothgoing through 128 * 16 kernels first simultaneously, and then the output ofthese layers go through a leaky ReLU, finally getting concatenated together.This creates the 128 * 32 channels in the data. The rest of the network isthe same as WGAN-GP. 6 more layers of convolutional transpose blocksfollowed by a final tanh activation function operate on the data to createthe desired output, based on the input label.463.3.ProposedMethodologyFigure 3.6: The architecture for the C-WGAN-GP is mainly the same architecture of the WGAN-GP model withthe added convolution and convolution-transpose blocks at the beginning of networks to impose conditionality.The discriminator takes in an image and the corresponding image label as input and validates their originalitywhile the generator creates pavement crack images based on its random noise vector and a given label for theimage to be generated.473.3. Proposed MethodologyThe discriminator in C-WGAN-GP also follows the same architecture asWGAN-GP’s discriminator. The main difference between them is the pres-ence of a label alongside the image as an input to the model. Similar tothe C-WGAN-GP’s generator there are two convolution layers in the be-ginning that simultaneously take the image and label, process them, andconcatenate them into 128 * 32, the entry for the next layer of the network.After the concatenation, 6 convolution blocks exist, with the last one justbeing a convolution layer that outputs a number, that is used in calculatingthe Wasserstein distance in for the loss function. The modified WGAN-GParchitecture, forming the C-WAGN-GP model is present in figure 3.6.Training LoopThe main difference in the training loop of WGAN-GP and C-WGAN-GPis the need for passing each input image crack label alongside the im-age/random noise vector to the discriminator/generator. The rest of thetraining loop is the same for both networks. Passing the label to the gener-ator indicates the generated image should be from the specified crack type.48Chapter 4Data Description andTesting FrameworkThis chapter dives into the details of the used datasets, how they were an-notated and categorized, and the tools used in the process. It also discussesthe testing framework for the aforementioned GAN models, their effective-ness in data augmentation, training conditions, and the experiment testingenvironment.4.1 DatasetTo train the DCGAN, WGAN-GP, and C-WGAN-GP models and test theireffectiveness and feasibility, a diverse dataset of pavement crack images wasrequired. One of the goals of this research was to check the effectivenessof the proposed GAN models in the augmentation of each crack type. Assuch, it was important to have a dataset that not only was suitable fortraining GAN models but also contained a diverse set of pavement cracksfrom various crack types. Pavement cracks can generally be divided intoseven categories [1]. The description of these cracks can be found below andtheir sample images are present in figure 4.1:1. Alligator or Fatigue Cracking:Alligator cracks are load-related deterioration resulting from a weak-ened base course or subgrade, too little pavement thickness, overload-ing, or a combination of these factors in the asphalt that resemble the494.1. Datasetskin of an alligator with interconnected cracks.2. Block Cracking:Block cracking presents as a series of large, typically one-foot or larger,rectangular cracks on the surface of asphalt pavement. These crackstend to cover extensive areas and may emerge even where traffic isabsent. Block cracking arises primarily from the shrinkage of asphaltpavement induced by temperature fluctuations.3. Longitudinal Cracking:Longitudinal cracks run parallel to the pavement’s centerline and canarise from various factors such as poorly constructed joints, shrinkageof the asphalt layer, cracks propagating from underlying layers, orlongitudinal segregation due to improper paver operation. Unlike someother crack types, longitudinal cracks are not directly attributable toloading stresses.4. Edge Cracking:Edge cracks are longitudinal cracks that emerge within one or two feetof the outer perimeter of pavement surfaces. These cracks develop dueto inadequate support at the edges of the pavement structure.5. Transverse Cracking:Transverse cracks form approximately perpendicular to the pavement’scenterline and are typically caused by the shrinkage of the asphalt layeror the reflection of existing cracks. These cracks are not induced byload-related stresses.6. Reflection Cracking:Reflection cracks develop over joints or cracks in concrete pavementor overlays on deteriorated asphalt pavement due to movement in theolder pavement layers.504.1. Dataset7. Slippage CrackingSlippage cracks take on a crescent shape and occur due to low-strengthasphalt mix or poor bonding between pavement layers. They resultfrom the forces exerted by vehicle turning or braking movements.Figure 4.1: Pavement crack image samples for each crack type are includedin this figure. Edge Cracks are not distinguished by their shape but by theconditions of their creation and as such were not included in the figure.The data used in the experiments of this thesis came from Crack500 andCrackForest datasets. Neither of the datasets had enough instances of eachcrack or variety for adequate training of the GAN models. To remedy this,514.1. DatasetCrack500 and CrackForest datasets were merged together, since their imageshave similar shapes and characteristics, and differences in the datasets wouldnot cause uncontrolled behaviors in the experiments. A full description ofthe CrackForest and Crack500 images and their annotation and classificationare given below:• Crack500– Crack500 consists of 500 images of size 2000 x 1500 pixels takenby cell phones on the main campus of Temple University. Theimages have pixel-level annotations and are separated into train,validation, and test sets. The dataset was pre-annotated usingpixel-level annotations for segmentation tasks and included vari-ous types of cracks such as transverse, longitudinal, and alligatorcracks. Since the focus of this thesis was on crack detection andclassification the images were re-annotated and individually clas-sified into transverse, longitudinal, alligator, and block cracks.• CrackForest– CrackForest includes 118 pavement crack images of size 480 x320 with background noise such as shadows. CrackForest also in-cludes the crack path and annotations for each dataset, however,these annotations do not match the popular annotation formatssuch as COCO and YOLO and therefore require re-annotatingof all images. The cracks in the dataset included 3 crack types,transverse, longitudinal, and alligator cracks. Each image wasindividually annotated and classified into four crack types, trans-verse, longitudinal, alligator, and block cracks, and then mergedwith the Crack500 images to form a unified dataset.Samples of the images from each of the four crack types present in thedatasets can be seen in figure 4.2. The images in the two datasets werecombined to create one uniform dataset of pavement crack images. Some ofthe images from the datasets were removed due to the presence of several524.1. DatasetFigure 4.2: Samples of different crack types from Crack500 and CrackForestdatasets are presented in this figure. CrackForest images mostly have thesame aspect ratio and resolution, while Crack500 images have both portraitand landscape images with various aspect ratios. The shapes of cracks andthe angle at which their images were taken are similar in these datasets,making them compatible for merging.534.1. Datasetcrack types in one image, which would cause issues for the classifier. In theend, we had a total of 483 images. This included 134 images with alligatorcracks, 179 images with longitudinal cracks, 164 images with transversecracks, and 6 images with block cracks. Due to the small number of blockcrack images in the dataset, it was not possible to separate them or train aclassifier on them, however, they were still included in the GAN training tosee if the model is able to generate appropriate samples even in low-resourceenvironments. These images were then separated into train and test portionswith a 20 percent split, to be used in the classifier, and the GAN trainingprocess. Aside from Crack500 and CrackForest datasets, 2 other publiclyavailable datasets, GAPS and DSPS23, were used in the initial steps of theresearch but they were unsuitable due to their general attributes, such asshape, crack placements, and background.4.1.1 Annotation and ClassificationTo annotate the images for detection tasks, all of the images in Crack500and CrackForest were re-annotated with YOLO and COCO format bound-ing boxes. The tool used in the annotation process was CVAT. CVAT (Com-puter Vision Annotation Tool) is an open-source web-based tool designedto facilitate the annotation of images and videos for computer vision ap-plications [9]. CVAT provides tools for annotating images using boundingboxes and polygons for image and video object detection and segmentation.In the case of this thesis, this tool was used to create bounding boxes foreach crack present in the 655 available images according to their crack type.After the annotation, each image was checked for presence of more than onedominant crack type inside them based on their bounding boxes and thenseparated into the aforementioned crack types.544.2. Testing Framework4.2 Testing FrameworkThe testing framework used in this thesis consists of three modules: (i) thepreprocessing module, (ii) the GAN data augmentation module, and (iii) theclassification module. A high-level overview of this framework is presentedin figure 4.3. The implemented GAN networks are evaluated through a seriesof classification tasks. A classifier is trained on a selected dataset of alligator,transverse, and longitudinal cracks. DCGAN, WGAN-GP, and C-WGAN-GP models were all trained under the same conditions with the same datato generate pavement crack images from images of the aforementioned threecrack types.A classification baseline was created by training the classifier on the originaldataset. To see the impact of data augmentation on the classification task,the classification metrics of the classifier trained on the GAN-augmenteddatasets and the original baseline are compared together. After training theclassifier on each dataset, the augmentation results, quality of images, andgeneral limitations of the proposed GAN networks are discussed on eachindividual crack type based on their defining characteristics.Figure 4.3: A high-level overview of the testing framework is presented here,consisting of preprocessing, data augmentation, and classification steps.Preprocessing ModuleThe preprocessing module includes the common preprocessing steps for eachof our GAN models and the final classification model. Preprocessing is a cru-554.2. Testing Frameworkcial step in all deep-learning models. Some of the preprocessing steps takenfor these models are similar, while model-specific aspects are also considered.The first step in preprocessing was to grayscale the image. The input for allmodels is images, which can be either 3-channel RGB images or 1-channelgrayscale ones. After training and testing our models on both 3-channelRGB and 1-channel grayscale images, 1-channel grayscale images achievedbetter classification metrics in comparison during early testing. This mightbe due to the nature of the pavement images where most of the aspectsof the road are in black and white and having three channels in the imagemight not improve the outputs, especially the artificially generated imagesby the GAN. All the images were resized to 256 * 256 followed by normaliza-tion, and center-cropping. Most deep learning models require a fixed shapeamong their inputs and normalization reduces the risk of exploding gradi-ents, adding more stability to the training process. Reducing instability byutilizing techniques such as normalization helps ensure a controlled environ-ment for testing the different GAN models without uncontrolled behaviors.GAN Data Augmentation ModuleThis module was created to test the effectiveness of GAN-generated imagesin pavement crack dataset balancing and augmentation while providing themeans of comparison of the DCGAN, WGAN-GP, and C-WAGN-GP mod-els. In the data augmentation module, the three GAN models were imple-mented based on the architecture mentioned in the previous chapter. Thearchitectures designed for the GAN models take 256 * 256 images as inputsand output images with the same dimensions. As such, they are suitable forthe preprocessing steps performed in this work.Classification Module Finally, in the classification module, ResNet18(45),an implementation of the famous ResNet architecture, was used as the clas-sifier architecture based on the size of the gathered images and the perfor-mance of the model. Other ResNet architectures were explored as a classifierbut due to the more complex architecture and the small size of images in thiswork, overfitting occurred during the model training. This architecture is564.2. Testing Frameworkdesigned with 18 layers to process the images and output the desired results.This network was trained using (real and GAN-generated synthetic images)to categorize the given crack images into alligator, transverse, and longitu-dinal classes. The model was initially trained on the original un-augmenteddataset to create the baseline for metrics.4.2.1 Model Training Conditions and HyperparametersIn total, there are three GAN models, DCGAN, WGAN-GP, and C-WGAN-GP, and three classification models, including the baseline classifier, that aretrained as part of the experiments in this thesis. Each of the three GANmodels was trained on the pavement crack dataset for 10000 epochs, with alearning rate of 0.00005, beta of 0.5, latent space size of 100, and batch sizeof 32. The hyperparameter used here performed adequately with our data,with batch size depending on the size of the images and available memory.Different numbers of samples for data augmentation were tested with 200images achieving the most tangible improvement. More images could causeoverfitting in the classifier.GAN models can use a metric called Inception Score (IS) as a measure ofdiversity and realism. However, this metric is calculated by a pre-trainedinception model, which is trained on the ImageNet dataset. ImageNet pri-marily contains images of natural objects (animals, landscapes, everydayitems), which are vastly different from pavement crack images. As a re-sult, the Inception model will not be able to meaningfully classify pavementcracks into distinct classes, because pavement cracks do not belong to thecategories that the Inception model was trained on. During training, gen-erated output quality was used as a measure of improvement in the model.Quality in this regard refers to the similarity in characteristics of the gener-ated images and original images. These characteristics are the clarity of thecrack, lack of noise in the generated image, and how well the crack shapeis formed in the pavement background. There is no quantifiable metric tomeasure the quality of generated pavement crack images and higher qualitydoes not necessarily translate to better augmentation results, as diversity574.2. Testing Frameworkin generated images and containing key features is more impactful. There-fore, the quality mentioned regarding the generated images of the models isnot a quantifiable measure and after observing the generation of pavementcrack images by the model, the model instance with the highest quality anddiversity was chosen as the final generator of that model.Although changes in loss function are normally an indicator of model im-provement during training, the GAN loss function is a method of pushingthe discriminator and generator to better learn their task and cannot bea good measure of model improvement or desirability of generated output,and as such, regular monitoring of generator output is required. This alsomakes incorporating techniques such as early stopping in finding the mostadequate instance of the GAN model not feasible.The ResNet18 classifier was trained for 2000 epochs, with a batch size of 16,a learning rate of 0.0002, and a beta of 0.5. The classifier used the originalpavement crack dataset for creating the baseline and then was subsequentlytrained on each of the three GAN-augmented datasets. The original datasetwent through a stratified train test split with 20 percent of the data used asthe validation set while 80 percent of it formed the training set. The train-ing conditions of the classifier, including its validation set, were kept thesame for the baseline and all the GAN models. This would allow the exper-iments to take place in a controlled environment, meaning the only changesseen in the results of the classifier could be attributed to the changes in thetraining set. In each epoch, the training was monitored on the validationdataset to check its performance based on accuracy. The training initiallyconsistently increased accuracy but then, due to major similarities betweentraining and validation data, overfitting would occur, resulting in fluctua-tions in validation metrics. Several techniques were implemented to get thebest model. In the first step, early stopping based on increases in accuracywas implemented. If the accuracy on the validation set did not improve for200 and 300 epochs, the training would stop, and the best model would bereturned. This increased the metrics of the model, but in some instances,584.2. Testing Frameworkthe model still had the potential to better itself in later epochs. Hence, wetested training the model for the full duration and keeping the best modelbased on accuracy or loss. Training loss was not an ideal metric for moni-toring the performance as overfitting would cause the loss to decrease whilethe validation accuracy did not necessarily increase, and in fact, in mostcases, it worsened. Choosing the best model based on accuracy did resultin the best overall model, keeping track of it during the whole training andreturning it at the end.4.2.2 Training and Testing EnvironmentsAll the code for the models and preprocessing steps were written in Python.PyTorch was used as the deep learning library to create, train, and testthe GAN and classification models. ResNet18 was defined using its de-fault implementation in PyTorch, while all the GAN models were designedand implemented from scratch. To train the models, an Nvidia 4080 12GBgraphics card and a V100 40 GB graphics card was used. On both of theseGPUs, the maximum image size for the generated images we could fit ontothe model was 256 x 256. Initially, the 4080 graphics card was used but dueto the long training time of the model, which could take weeks of constantprocessing, a V100 GPU from CCDB was used. This sped up the training bymore than 10 times. The classifier was trained on the Nvidia 4080 graphicscard since the training duration was not too long.59Chapter 5Data AugmentationPerformance and DiscussionIn this chapter, the training process and results of DCGAN, WGAN-GP,and C-WGAN-GP are compared and discussed. Samples from each GANmodel’s output in their different training steps and the final generated out-puts for each crack type are compared together and discussed. After that,the impact of data augmentation using the GAN on the model’s ability toaccurately classify pavement cracks is assessed and compared to the baseline.5.1 Quality of Synthetic ImagesBefore discussing the quality of the generated images and comparing thetrained GAN models, diversity should be defined. In this thesis, diversityrefers to the difference and variety present in the samples of a class. Forexample, higher diversity in the generated alligator cracks means the gen-erated samples had varied shapes and details which resembles the varietypresent in the actual crack shapes in the real world. Since diversity can-not be measured by a metric, visual inspection was used to ascertain if thegenerated samples had a high or low level of diversity. While this could in-troduce bias and subjectivity in the models’ comparison, a high level of careand sensitivity was used in this visual inspection to minimize the resultingsubjectivity.DCGAN, was the first data augmentation model trained on the data under605.1. Quality of Synthetic Imagesthe testing framework conditions and hyperparameters. After observing thegenerated images by the model in each 500 epochs, epoch 8,000 proved tohave the best-quality generated images. The model was still trying to formthe full shape of the crack and pavement until epoch 7,000. From epoch9,000, the images started to blur and collapse. Even though the qualityof synthetic images was decent, there was little variation in them. Dueto the similarity in the shapes of cracks and their background, DCGAN’sloss function, and discriminator overpowering the generator, model collapsehappened in the training. This caused the model to only be able to replicatea very limited set of cracks. After filtering through 5,000 generated images,the final images did not have enough diversity to fix the data imbalancein the dataset. This showed the normal DCGAN might not be a goodarchitecture choice for pavement crack generation. The failure in adequatedata generation by DCGAN also would not allow for experimenting withits effects using the classifier, since the replicated crack images would onlyintroduce more bias and imbalance to the already imbalanced dataset.The images generated by the DCGAN model mostly consisted of longitudi-nal and transverse cracks, meaning the generator in this GAN model failedto capture the more complex features of alligator cracks. In the generatedimages, most of the outputs were gray and noisy featureless pictures showingthe model’s failure in convergence. Since there were only a small number ofblock cracks in the dataset, it was not expected that the model would beable to generate any block cracks, which in the end, it could not. Samplesof the generated images by the DCGAN can be seen in figure 5.1.In WGAN-GP, the discriminator can be trained more than the genera-tor to better guide the generator in the correct direction. For each epochthat the generator was trained, the discriminator or critic was trained for5 epochs. The training was monitored at every 100 epochs and the modelshowed consistent improvement in its generated image quality until the 9000epoch, with minor improvements until the 10000 epoch. The generated im-ages had high variety with life-like pavement cracks. Then 10000 generated615.1. Quality of Synthetic ImagesFigure 5.1: This figure contains pavement crack images generated by theDCGAN architecture at different point in training. By epoch 2000, themodel is still trying to process and understand the features of the pavementcrack images, and by 6000 there is improvement as basic silhouettes of thecracks are formed. However, by the final epoch, the model is suffering frommode collapse and can only replicate a limited number of images.images were filtered through and categorized.The images generated by WAGN-GP had sufficient variety and were of high-quality, which made them very similar to the original dataset. The modelwas able to generate the 4 given crack types to it with high levels of detail.Even block cracks, with only 6 samples in the GAN model training datasetwere replicated, although the images generated for this crack type did nothave the diversity present in other crack types. The complexity of crackpatterns in alligator cracks were perfectly captured. The longitudinal andtransverse cracks, despite their relative simplicity in shape, have challengesin their creation since the background/crack ratio in the images are skewedtowards the background. The WGAN-GP model managed to overcome this625.1. Quality of Synthetic ImagesFigure 5.2: This figure contains pavement crack images generated by theWGAN-GP architecture at different point in training. The first two columnsshow samples from the early and mid stages of training where the pavementcrack images are taking shape. In the final instance of training, the modelis capable of generating longitudinal, transverse, alligator, and block cracks,similar to the samples present in the original dataset.challenge and generate diverse longitudinal and transverse cracks. Samplesof the generated images by the WGAN-GP can be seen in figure 5.2.Lastly, the C-WGAN-GP model was trained to augment the dataset.C-WGAN-GP needed the labels of each input crack image beside the ac-tual image. The model was trained with alligator, transverse, longitudinal,and block crack labels and their associated images. The training time foreach epoch of this network was considerably lengthier compared to WGAN-GP and DCGAN. Like WGAN-GP, for each epoch that the generator wastrained, the discriminator or critic was trained for 5 epochs. The train-ing showed consistent improvement until the 5000th epoch. In this state,the model was perfectly able to generate life-like crack images from the re-635.1. Quality of Synthetic Imagesquested category with few incorrect crack types. After the 5000th epoch,the training started fluctuating without much improvement which resultedin the model producing gray pixelated colors as backgrounds. This is aninstance of non-convergence that GAN models can suffer from in the laterepochs of their training. Since epoch 5000 in the training was capable ofproducing the desired images from input labels, it was used to generate theC-WGAN-GP images. It should be noted that the models here were alltrained for 10000 epochs to have a controlled experiment environment andin real-world scenarios, the models would reach their local or global optimumpoints in different training intervals.Figure 5.3: This figure contains pavement crack images generated by theC-WGAN-GP architecture at different point in training. The model wascapable of generating longitudinal, transverse, and alligator cracks, basedon the given label to the generator. The generated cracks are similar inshape and characteristics to the original dataset images and are of high-quality and diversity.645.1. Quality of Synthetic ImagesC-WGAN-GP was able to generate high-quality alligator, transverse, andlongitudinal images from the input labels correctly. The generated imagescontained some noise; however, the images still included all the defining fea-tures of their respective crack types and were of high diversity. For eachcrack type, the model was asked to generate 1000 images of that type. Themodel was able to produce the requested crack types with little to no wrong-generated images. Since the input to the model contains a random noisevector, the outputs of the model have a randomness that prevents the cal-culation of a constant number of incorrect instances in each set of generatedimages based on the given label. However, in each generation, these incor-rect instances were less than 1 to 2 percent of the total generated images.In the case of block cracks, the C-WGAN-GP managed to create silhouettesof block cracks based on the 6 input images in earlier epochs but failed toimprove and continue their generation by epoch 5000. Based on the perfor-mance of the model with the other crack types, it is reasonable to assume themodel is capable of generating appropriate block cracks given more trainingsamples. Overall, C-WGAN-GP images had more diversity compared toWGAN-GP-generated images. This could be attributed to the label presentin the input which guided the model to only generate a specific crack type orto the lesser number of trained epochs for the used model instance, which,although produced slightly lower-quality images, allowed for more varietyin generated samples due to less convergence in the model. Samples of thegenerated images by the C-WGAN-GP can be seen in figure 5.3.Overall, the proposed models, WGAN-GP, and C-WGAN-GP, were able togenerate diverse and high-quality pavement crack images of each categorythey were sufficiently trained on. C-WGAN-GP also managed to accuratelygenerate each requested crack type based on the given label with minorincorrect generated images, with a higher intra-class diversity compared tothe other GAN models. DCGAN, on the other hand, fell to mode collapseand training instability which prevented it from generating diverse or high-quality outputs, making it unusable in data augmentation in our datasets.A general comparison of these models based on the generated data can be655.2. Impacts on Pavement Crack Classificationfound in table 5.1.Base-lineDCGAN WGAN-GP C-WGAN-GPAdvan-tages- - Training stabilityHigh quality aug-mented imagesLess prone tomode collapseSelective image genera-tionLess prone to modecollapseTraining stabilityHigh quality augmentedimagesDisad-vantages- Training instabilityProne to mode col-lapsePost-processing re-quiredNo control over imageclassLong trainingtimeHigh computa-tional overheadNo control overimage classMore complexloss functionLonger training timeHigher computationaloverheadMore complex loss func-tionMore complex data pro-cessing due to addedlabelsTable 5.1: The table gives a summary of the general advantages anddisadvantages of each GAN model based on their training conditionsand generated outputs. Both WGAN-GAP and C-WGAN-GP were su-perior to DCGAN, with C-WGAN-GP having the advantage of agencyin outputs over WGAN-GP.5.2 Impacts on Pavement Crack ClassificationThe first step in assessing the performance of different GAN models as dataaugmenters was to establish the classification baseline. A baseline modelwas trained on real images only of alligator, transverse, and longitudinalcracks. The metrics gained by the classifier in classifying alligator, trans-verse, and longitudinal cracks are 79.16 accuracy, 77.66 recall, precision78.04, and 77.62 f1-score using best model selection. Since DCGAN did notperform adequately in data generation and suffered from mode collapse, thefirst model used for augmentation of data was WGAN-GP. The dataset aug-mented using WGAN-GP managed to gain the score an accuracy of 83.33,recall of 82.46 recall, precision of 82.46, and f1-score of 82.46. C-WGAN-665.2. Impacts on Pavement Crack ClassificationGP performed almost identical to WGAN-GP and its augmented datasetmanaged to gain the following metrics: 83.33 accuracy, 81.76 recall, 82.52precision, and 81.80 f1-score. These metrics can be found in table 5.2. Indi-vidual per-class metrics can give insight into how well the augmentation byeach GAN model affected their classification and can indicate their under-standing and performance in the generation of their images. These metricscan be found in table 5.3.Base-lineWGAN-GP C-WGAN-GPAccu-racy79.16 83.33 83.33Precision 78.04 82.46 82.52Recall 77.66 82.46 81.76F1-Score 77.62 82.46 81.8Table 5.2: This table includes a full metrics comparison of the baselineand the GAN-augmented dataset. WGAN-GP and C-WGAN-GP gainednear identical results in metrics while DCGAN with significant improve-ments compared to the baseline. DCGAN failed to generate adequateand diverse pavement crack images and was not used in augmentation.The results of the classification show the augmented datasets using the GAN-generated images were successful in increasing all of the classification metricscompared to the baseline. Figure 5.4 contains the confusion matrix of thebaseline and the augmented models for comparison. Analyzing this con-fusion matrix shows Alligator Cracks saw the biggest improvement inclassification results after the data augmentation. Alligator cracks have themost complexity out of the discussed crack types. This improvement wasmost apparent in WGAN-GP-generated images, which were of high-qualityand life-like characteristics and had high variety amongst them. The gen-erated images helped the classification model better differentiate the othercrack types from alligator cracks and introduced more intra-class diversityto the alligator category. This augmentation managed to increase the cor-rectly classified alligator cracks while reducing the misclassification in otherclasses. C-WGAN-GP also improved the classification scores in alligator675.2. Impacts on Pavement Crack Classificationcracks, with almost identical results as WGAN-GP, while adding the easeof on-demand generation of these cracks. C-WAGN-GP was also able togenerate high-quality and diverse alligator cracks based on the given labelwith 1 to 2 percent incorrect images in each set of generations. C-WGAN-GP also sped up the augmentation and generation post-processing steps byeliminating the need for manual filtering through all of the generated im-ages to populate the alligator class. By looking at the generated images byboth models, the proposed GAN architectures managed to learn the defin-ing features of alligator cracks and correctly use them in their outputs. Theclassification results on alligator cracks demonstrate the capability of thenew proposed GAN models in capturing the defining features of the mostcomplex crack types and successfully generating indistinguishable life-likesynthetic images.AlligatorBaseline WGAN-GP C-WGAN-GPPrecision 69.56 70.37 77.27Recall 59.25 70.37 62.96F1-Score 63.99 70.37 69.38LongitudinalBaseline WGAN-GP C-WGAN-GPPrecision 88.88 86.11 85Recall 88.88 86.11 94.44F1-Score 88.88 86.11 89.47TransverseBaseline WGAN-GP C-WGAN-GPPrecision 75.67 90.90 85.29Recall 84.84 90.90 87.87F1-Score 80 90.90 86.56Table 5.3: A full comparison of the micro per-crack type metric of eachdataset and their classifier based on Precision, Recall, and F1-Score.Almost all metrics across all crack types have been increased by thedata augmentation. C-WGAN-GP gained better metrics on longitudinalcracks while WGAN-GP performed better on transverse and alligatorcracks.685.2. Impacts on Pavement Crack ClassificationLongitudinal Cracks also gained improvements in classification scores af-ter the augmentations. C-WGAN-GP augmentations managed to increasethe number of correctly classified longitudinal cracks by two. WGAN-GPcouldn’t improve the longitudinal cracks in first glance, but the overall in-crease in correct classification in other crack types leads to better classifi-cation of longitudinal cracks too. The superiority of C-WGAN-GP in thiscase could be a result of more diversity in generated samples compared toWGAN-GP images.Transverse Cracks were more impacted by WGAN-GP compared to C-WGAN-GP. WGAN-GP managed to increase the number of correctly clas-sified samples in transverse cracks while decreasing the number of misclassi-fications in other classes. C-WGAN-GP also improved the transverse cracksclassification results. Although the impact of augmentation on individualcrack types was discussed in the above paragraphs, as mentioned in the al-ligator cracks section, the impact of augmentation cannot be limited to onespecific category. Improvement in the classification of one crack type leadsto fewer misclassifications of other classes by the model and will raise theoverall accuracy, precision, recall, and f1-score of all classes. Both data aug-mentation models were able to capture the defining features of each cracktype and generate appropriate sample images for them. The proposed mod-els had different impacts on different crack types, both having a positiveoverall impact on all of the crack types, with some being affected more thanothers.Although WGAN-GP’s accuracy metrics were almost identical to C-WGAN-GP, it is important to highlight the significance of the manual input requiredto sort WGAN-GP results that using the proposed conditional GAN over-comes. To augment the original dataset with WGAN-GP, 10,000 pavementcrack images were created and then filtered through. To create these images,they had to be created in batches of 2000 images, due to the computationalresources the process required and would at least take 15 minutes per batch.This number is dependent on the specifications of the computational sys-695.2. Impacts on Pavement Crack ClassificationFigure 5.4: The confusion matrix for the baseline classifier, WGAN-GPaugmented dataset, and C-WGAN-GP augmented dataset are presentedhere. The confusion matrices contain the number of correct and incorrectclassifications of each crack type. As can be seen, the overall number oftrue positives, and in turn, the correctly classified cracks are increased inboth WGAN-GP and C-WGAN-GP, with WGAN-GP having an advantagein alligator and transverse cracks, while C-WGAN-GP performed better inlongitudinal cracks.tem, model size and complexity, and image size. Since each crack type hada different generation probability, to augment a specific crack type witha lower chance of appearance, one might have to filter through thousands705.2. Impacts on Pavement Crack Classificationof images to get the desired dataset of diverse synthetic cracks. Filteringthrough 10000 generated images took between one and a half to two and halfhours. Although this filtering led to more categorized synthetic pavementcrack images than required for the purpose of experimenting the augmenta-tion effects, C-WGAN-GP managed to generate pre-categorized pavementcrack images of the same amount without any manual input. The outputs ofa perfectly trained WGAN-GP model with the highest possible quality anddiversity would still be unusable without post-processing and manual anno-tation steps. After their manual annotation, WGAN-GP generated outputsneed to be checked to see if enough samples of the imbalanced crack typesare contained within them so the original dataset can be augmented. C-WGAN-GP outputs, on the other hand, can be directly augmented into thedataset and easily balance any class required by the user. This difference ingeneration and processing time would further increase if the non-conditionalmodel is used in augmenting a crack type with a lower chance of appearance.The focused data generation in C-WGAN-GP was rigorously tested withvarious numbers of generated images from the three different available cracktypes. C-WGAN-GP was able to successfully create the targeted crack typeswith 1 to 2 percent incorrect generated instances in 1000 samples of a class.Unlike DCGAN, WGAN-GP and C-WGAN-GP did not fall into mode col-lapse or heavy training instability before their convergence. Both of thesemodels showed consistent improvements until their convergence points andhad a diverse generated image pool with intra and inter-class variety. Thegenerated pavement cracks by both WGAN-GP and C-WGAN-GP mod-els were similar in shape, and characteristics to the images in the originaldataset. This high similarity allowed the synthetic images to be easily aug-mented into the real images without the fear of model bias toward the gen-erated images. This was incorporated in the classification experiment byonly using real pavement crack images in the testing set and gaining im-provements in the results. The experiments performed on the WGAN-GPand C-WGAN-GP models showed both models to be capable of handlingcomplex shapes and features of the crack images, correctly understanding715.2. Impacts on Pavement Crack Classificationtheir defining characteristics, and generating similar images based on them.All three discussed crack types had a good diversity in the shape of cracksand their placement in the generated images. Alligator cracks generated bythe WGAN-GP model had a slight edge in quality over the C-WGAN-GPgenerated ones, but the conditional samples had a higher diversity in crackshapes.72Chapter 6Conclusion, Limitations, andFuture WorkThe objective of this thesis was to (i) research on mode collapse and train-ing instability present in data augmentation techniques for pavement crackdatasets and solve them in a new GAN data augmentation model, (ii) addagency in the generated output of the GAN model, something lacking inmost previous studies, and (iii) compare the performance of the proposedGAN models and the widely-used DCGAN architecture, in generated imagequality, the diversity of outputs, training stability, and classification met-rics for pavement crack analysis tasks. Two alternative GAN architectures,WAGN-GP and C-WGAN-GP were implemented as part of the thesis thatachieved the outlined objectives. Here, the key contributions of this thesis,alongside the present limitations and avenues of research for future work arediscussed.6.1 ContributionsThis thesis has made several key contributions to the balancing of pavementcrack datasets through AI-generated synthetic images. These contributionsinclude:1. The adoption of a stable GAN model architecture for the generationof pavement crack images called WGAN-GP, which uses Wassersteindistance and Gradient Penalty in its loss calculation, which reduces736.1. Contributionsinstability during training and prevents mode collapse occurrences.2. By implementing conditionality in to the WGAN-GP architecture, C-WGAN-GP was created. This new model can generate pavement crackimages from specific crack types based on control parameters definedby the user, such as the output label. This model benefits from theadvantages of WGAN-GP and removes the lengthy post-processingsteps of filtering and re-generated images to populate specific cracktypes. The added labels to the input can also increase the diversity ofthe generated images.3. Lastly, the proposed models achieved far better results compared tothe DCGAN architecture, when trained under the same conditions andon the same data in data augmentation for pavement crack datasets.The DCGAN model suffered from mode collapse and training insta-bility, only being able to generate a small subset of replicated low-quality outputs, which would not increase the diversity or balance inthe datasets. WGAN-GP and C-WGAN-GP, on the other hand, didextract the defining features of the pavement crack images of differ-ent types and created high-quality, life-like, and diverse outputs thatincrease accuracy, precision, recall, and f1-score of classification taskscompared to an established baseline without augmentations.The experiments on the two implemented GAN models demonstrated theircapabilities in generating life-like pavement crack images from various cracktypes and increasing the accuracy of analysis tasks by balancing and aug-menting existing datasets. This solves the inherent challenges with the pave-ment data gathering, and the unequal chance of occurrences in cracks, whileremoving the need for costly and time-consuming data gathering sessionsenforced on transportation agencies. C-WGAN-GP in particular is able togenerate specific crack types it is trained on, based on the demands of the en-gineers. Data diversity added to the dataset by using GAN-generated imagescan the model better understand each crack type and correctly classify anddetect unfamiliar cracks. Also, the GAN models can be fine-tuned and re-746.2. Limitations and Future Worktrained based on the requirements of the tasks, making them future-proof.These advantages could allow for cost-effective and faster maintenance ofroads and automate major parts of their survey and analysis, through proac-tive detection of defects. Furthermore, the bias present in the results of theseanalyses would be much less compared to field-data gathering and manualpavement rating.6.2 Limitations and Future WorkAlthough augmentation with C-WGAN-GP and WGAN-GP proved to bebeneficial in dataset balancing and classification metrics, there are limita-tions and downsides too. Training the GAN models is time-consuming andrequires substantial computational power. WGAN-GP and C-WGAN-GPin particular, due to the longer discriminator training time and gradientpenalty calculations impose significantly more computational costs com-pared to DCGANs. However, with the ever-increasing cloud computationalinfrastructures, training such models is becoming easier and more affordable.The size of the generated images is limited to the memory capacity of thetraining system RAM, and VRAM. Resizing the images and changing theiraspect ratio can cause information loss or change in some of the features ofthe cracks, resulting in unintended results for data generation and classifi-cation or detection. Based on the comparison of the generated images andthe resized original dataset, it appears information loss has not occurred inthe samples used for the thesis. In this thesis, the maximum size achievedon the available hardware was 256 x 256 pixels. Future research could alsoexplore implementing different architectures such as progressive generativeadversarial networks (46). Progressive GANs increase image size graduallyduring training by starting with a low-resolution image and progressivelyadding layers to both the generator and the discriminator. This approachhelps in stabilizing training and improving the quality of the generated im-ages.C-WGAN-GP uses the crack labels as inputs to add agency to the model’s756.3. Conclusiongeneration process, however, the current form of input labels does not di-rectly impact the output of the discriminator and loss function. This way,the model is not directly impacted by generating the wrong crack typesbased on the input label. Since the discriminator output in both DCGANand WGAN-GP architectures approximate JS-divergence and Wassersteindistance, the discriminator loss function cannot be easily changed to incor-porate labels in the output. However, there are other methods that couldbe integrated into C-WGAN-GP to change this. Research on auxiliary clas-sifier generative adversarial network (AC-GAN) [18] in the pavement cracktasks, which use a separate classifier for the generated images in the GANarchitecture to produce a loss that is directly impacted by the generatedimages’ classes can prove helpful in increasing the accuracy and agency inthe model. Nevertheless, the methods proposed by the thesis were successfulin answering the identified challenges and gaps in data augmentation tech-niques for pavement cracks. The selective generation of pavement cracksand training stability present in C-WGAN-GP can positively impact futurepavement crack analysis tasks, reduce the cost, and speed up the process ofhighway maintenance.6.3 ConclusionDeep learning technology has the potential to improve the efficiency of pave-ment surface analysis associated with manual road inspections. Neverthe-less, the lack of a diverse and balanced training dataset that can be usedto train deep learning algorithms represents a major obstacle. This the-sis explores the recent attempts at pavement crack dataset balancing andaugmentations and focuses on the gaps present in their research, namelyinstability in training, mode collapse, and lack of agency in generated re-sults present in Deep Convolutional Generative Adversarial Networks (DC-GANs), the state-of-the-art models in current literature. Two data aug-mentation models, the Wasserstein Generative Adversarial Network withGradient Penalty (WGAN-GP), and the Conditional Wasserstein Genera-tive Adversarial Network with Gradient Penalty (C-WGAN-GP) are imple-766.3. Conclusionmented as alternatives to DCGAN that fix the aforementioned problemswhile producing high-quality and diverse pavement crack images applicableto dataset balancing and augmentation. The theory, architecture, trainingloop, and weaknesses of DCGAN, WGAN-GP, and C-WGAN-GP are fullydiscussed and compared with each other, and then the augmentation per-formance of each model is tested on 2 established pavement crack datasetsthrough observation and classification tasks. C-WGAN-GP and WGAN-GPgained better results compared to DCGAN and gained near identical resultsto one another. WGAN-GP remedied training instability and mode collapseoccurrence in model training, while C-WAGN-GP added conditionality toit, allowing for the selective generation of life-like pavement crack images,removing the need for post-processing in data augmentation.77Bibliography[1] Understanding asphalt pavement distresses – five distresses explained -Asphalt magazine, October 2009.[2] Amila Akagic, Emir Buza, Samir Omanovic, and Almir Karabegovic.Pavement crack detection using Otsu thresholding for image segmenta-tion. May 2018.[3] Zaid Al-Huda, Bo Peng, Riyadh Nazar Ali Algburi, Saghir Alfasly, andTianrui Li. Weakly supervised pavement crack semantic segmentationbased on multi-scale object localization and incremental annotation re-finement. Applied Intelligence, 53(11):14527–14546, June 2023.[4] Rabih Amhaz, Sylvie Chambon, Jerome Idier, and Vincent Baltazart.Automatic Crack Detection on Two-Dimensional Pavement Images: AnAlgorithm Based on Minimal Path Selection. Trans. Intell. Transport.Syst., 17(10):2718–2729, October 2016.[5] Martin Arjovsky, Soumith Chintala, and Le´on Bottou. WassersteinGAN, December 2017. arXiv:1701.07875 [cs, stat].[6] Efstathios Branikas, Paul Murray, and Graeme West. A Novel DataAugmentation Method for Improved Visual Crack Detection Using Gen-erative Adversarial Networks. IEEE Access, 11:22051–22059, 2023.Conference Name: IEEE Access.[7] Peter Bullen. Adaptive reuse and sustainability of commercial build-ings. Facilities, 25:20–31, February 2007.78Bibliography[8] Fu-Chen Chen, Mohammad Jahanshahi, Rih-Teng Wu, and ChrisJoffe. A texture-Based Video Processing Methodology Using BayesianData Fusion for Autonomous Crack Detection on Metallic Surfaces: Atexture-based video processing. Computer-Aided Civil and Infrastruc-ture Engineering, 32, February 2017.[9] CVAT ai Corporation. Computer Vision Annotation Tool (CVAT), July2024.[10] Pablo del R´ıo-Barral, Mario Soila´n, Silvia Mar´ıa Gonza´lez-Collazo, andPedro Arias. Pavement Crack Detection and Clustering via Region-Growing Algorithm from 3D MLS Point Clouds. Remote Sensing,14(22):5866, January 2022. Number: 22 Publisher: MultidisciplinaryDigital Publishing Institute.[11] Qiao Dong, Sike Wang, Xueqin Chen, Wanqi Jiang, Ruiqi Li, andXingyu Gu. Pavement crack detection based on point cloud data anddata fusion. Philosophical Transactions of the Royal Society A: Math-ematical, Physical and Engineering Sciences, 381(2254):20220165, July2023. Publisher: Royal Society.[12] Xiaoran Feng, Liyang Xiao, Wei Li, Lili Pei, Zhaoyun Sun, Zhidan Ma,Hao Shen, and Huyan Ju. Pavement Crack Detection and SegmentationMethod Based on Improved Deep Learning Fusion Model. MathematicalProblems in Engineering, 2020:e8515213, December 2020. Publisher:Hindawi.[13] Yusuke Fujita, Koji Shimada, Manabu Ichihara, and YoshihikoHamamoto. A method based on machine learning using hand-craftedfeatures for crack detection from asphalt pavement surface images.March 2017.[14] Miguel Gavilan, David Balcones, Oscar Marcos, David Ferna´ndez-Llorca, Miguel-Angel Sotelo, Manuel Ocan˜a, Pedro Aliseda, PedroYarza, and Alejandro Amı´rola. Adaptive Road Crack Detection System79Bibliographyby Pavement Classification. Sensors (Basel, Switzerland), 11:9628–57,December 2011.[15] Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, DavidWarde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Gen-erative Adversarial Networks, June 2014. arXiv:1406.2661 [cs, stat].[16] Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin,and Aaron Courville. Improved Training of Wasserstein GANs, Decem-ber 2017. arXiv:1704.00028 [cs, stat].[17] Nhat-Duc Hoang and Lam Nguyen. A Novel method for asphalt pave-ment crack classification based on image processing and machine learn-ing. Engineering With Computers, Accepted, In Press, April 2019.[18] Liang Hou, Qi Cao, Huawei Shen, Siyuan Pan, Xiaoshuang Li, andXueqi Cheng. Conditional GANs with Auxiliary Discriminative Clas-sifier, June 2022. arXiv:2107.10060 [cs].[19] Guo X. Hu, Bao L. Hu, Zhong Yang, Li Huang, and Ping Li. Pave-ment Crack Detection Method Based on Deep Learning Models. Wire-less Communications and Mobile Computing, 2021:e5573590, May 2021.Publisher: Hindawi.[20] Ministry of Transportation and Infrastructure. Planning, Strategic &Economic Reports & Studies - Province of British Columbia. LastModified: 2024-07-03 Publisher: Province of British Columbia.[21] Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. Progres-sive Growing of GANs for Improved Quality, Stability, and Variation,February 2018. arXiv:1710.10196 [cs, stat].[22] Narges Kheradmandi and Vida Mehranfar. A critical review and com-parative study on image segmentation-based techniques for pavementcrack detection. Construction and Building Materials, 321:126162,February 2022.80Bibliography[23] Stephen L. H. Lau, Edwin K. P. Chong, Xu Yang, and Xin Wang. Au-tomated pavement crack segmentation using U-net-based convolutionalneural network. IEEE Access, 8:114892–114899, 2020. Publisher: IEEE,Institute of Electrical and Electronics Engineers.[24] Baoxian Li, Kelvin C. P. Wang, Allen Zhang, Yue Fei, andGiuseppe Sollazzo. Automatic Segmentation and Enhancementof Pavement Cracks Based on 3D Pavement Images. Jour-nal of Advanced Transportation, 2019(1):1813763, 2019. eprint:https://onlinelibrary.wiley.com/doi/pdf/10.1155/2019/1813763.[25] Shuai Li, Yang Cao, and Hubo Cai. Automatic Pavement-Crack De-tection and Segmentation Based on Steerable Matched Filtering andan Active Contour Model. Journal of Computing in Civil Engineering,31(5), September 2017.[26] Zhuo Liu, Shuo Pan, Zhiwei Gao, Ning Chen, Feng Li, Linbing Wang,and Yue Hou. Automatic intelligent recognition of pavement distresseswith limited dataset using generative adversarial networks. Automationin Construction, 146:104674, February 2023.[27] Duo Ma, Hongyuan Fang, Niannian Wang, Chao Zhang, Jiaxiu Dong,and Haobang Hu. Automatic Detection and Counting System for Pave-ment Cracks Based on PCGAN and YOLO-MF. IEEE Transactionson Intelligent Transportation Systems, 23(11):22166–22178, November2022. Conference Name: IEEE Transactions on Intelligent Transporta-tion Systems.[28] Shadrack Fred Mahenge, Stephen Wambura, and Licheng Jiao. RCNN-GAN: An Enhanced Deep Learning Approach Towards Detection ofRoad Cracks. In 2022 The 6th International Conference on Computeand Data Analysis, ICCDA 2022, pages 91–99, New York, NY, USA,May 2022. Association for Computing Machinery.[29] Senthan Mathavan, Vaheesan Kanapathippillai, Akash Kumar, Chan-jief Chandrakumar, Khurram Kamal, Mujib Rahman, and Martyn81BibliographyStonecliffe-Jones. Detection of pavement cracks using tiled fuzzy Houghtransform. Journal of Electronic Imaging, 26:1, September 2017.[30] M. L. Mene´ndez, J. A. Pardo, L. Pardo, and M. C. Pardo. The Jensen-Shannon divergence. Journal of the Franklin Institute, 334(2):307–318,March 1997.[31] Mehdi Mirza and Simon Osindero. Conditional Generative AdversarialNets, November 2014. arXiv:1411.1784 [cs, stat].[32] British Columbia Ministry of Transportation and Infrastructure. Pave-ment Surface Condition Rating Manual. 6 edition, 2022.[33] Lili Pei, Zhaoyun Sun, Liyang Xiao, Wei Li, Jing Sun, and He Zhang.Virtual generation of pavement crack images based on improved deepconvolutional generative adversarial network. Engineering Applicationsof Artificial Intelligence, 104:104376, September 2021.[34] Ardilson Pembuain, Sigit Priyanto, and Latif Suparma. The Effectof Road Infrastructure on Traffic Accidents. pages 176–182. AtlantisPress, October 2019.[35] Anh Thu Phan and Thi Huynh. Pavement Crack Extraction Methodfrom Mobile Laser Scanning Point Cloud. Advances in Civil Engineer-ing, 2022:1–11, July 2022.[36] Alec Radford, Luke Metz, and Soumith Chintala. Unsupervised Rep-resentation Learning with Deep Convolutional Generative AdversarialNetworks, January 2016. arXiv:1511.06434 [cs].[37] Junhua Ren, Guowu Zhao, Yadong Ma, De Zhao, Tao Liu, and JunYan. Automatic Pavement Crack Detection Fusing Attention Mecha-nism. Electronics, 11(21):3622, January 2022. Number: 21 Publisher:Multidisciplinary Digital Publishing Institute.[38] David E. Rumelhart, Geoffrey E. Hinton, and Ronald J. Williams.Learning representations by back-propagating errors. Nature,82Bibliography323(6088):533–536, October 1986. Publisher: Nature PublishingGroup.[39] Ju¨rgen Schmidhuber. Deep learning in neural networks: An overview.Neural Networks, 61:85–117, January 2015.[40] Yong Shi, Limeng Cui, Zhiquan Qi, Fan Meng, and Zhensong Chen.Automatic Road Crack Detection Using Random Structured Forests.IEEE Transactions on Intelligent Transportation Systems, 17(12):3434–3445, December 2016. Conference Name: IEEE Transactions on Intel-ligent Transportation Systems.[41] Xinzi Sun, Yuanchang Xie, Liming Jiang, Yu Cao, and Benyuan Liu.DMA-Net: DeepLab With Multi-Scale Attention for Pavement CrackSegmentation. IEEE Transactions on Intelligent Transportation Sys-tems, 23(10):18392–18403, October 2022. Conference Name: IEEETransactions on Intelligent Transportation Systems.[42] Ahmed Mahgoub Ahmed Talab, Zhangcan Huang, Fan Xi, and LiuHaiMing. Detection crack in image using Otsu method and multi-ple filtering in image processing techniques. Optik, 127(3):1030–1033,February 2016.[43] Ce´dric Villani. The Wasserstein distances. In Ce´dric Villani, editor,Optimal Transport: Old and New, pages 93–111. Springer, Berlin, Hei-delberg, 2009.[44] Weixing Wang, Mengfei Wang, Hongxia Li, Heng Zhao, Kevin Wang,Changtao He, Jun Wang, Sifan Zheng, and Jiabin Chen. Pavementcrack image acquisition methods and crack extraction algorithms: Areview. Journal of Traffic and Transportation Engineering (EnglishEdition), 6(6):535–556, December 2019.[45] Xianglong Wang and Zhaozheng Hu. Grid-based pavement crack analy-sis using deep learning. In 2017 4th International Conference on Trans-portation Information and Safety (ICTIS), pages 917–924, August 2017.83Bibliography[46] Boqiang Xu and Chao Liu. Pavement crack detection algorithm basedon generative adversarial network and convolutional neural networkunder small samples. Measurement, 196:111219, June 2022.[47] Fan Yang, Lei Zhang, Sijia Yu, Danil Prokhorov, Xue Mei, and HaibinLing. Feature Pyramid and Hierarchical Boosting Network for Pave-ment Crack Detection. January 2019.[48] Hui Yao, Yuhan Wu, Shuo Liu, Yanhao Liu, Hua Xie, Hui Yao,Yuhan Wu, Shuo Liu, Yanhao Liu, and Hua Xie. A pavement cracksynthesis method based on conditional generative adversarial net-works.Mathematical Biosciences and Engineering, 21(1):903–923, 2024.Cc license type: cc by Number: mbe-21-01-038 Primary atype: Math-ematical Biosciences and Engineering Subject term: Research articleSubject term id: Research article.[49] Kaige Zhang, Yingtao Zhang, and Heng-Da Cheng. CrackGAN: Pave-ment Crack Detection Using Partially Accurate Ground Truths Basedon Generative Adversarial Learning. IEEE Transactions on IntelligentTransportation Systems, 22(2):1306–1319, February 2021. ConferenceName: IEEE Transactions on Intelligent Transportation Systems.[50] Tianjie Zhang, Donglei Wang, Amanda Mullins, and Yang Lu. Inte-grated APC-GAN and AttuNet Framework for Automated PavementCrack Pixel-Level Segmentation: A New Solution to Small TrainingDatasets. IEEE Transactions on Intelligent Transportation Systems,24(4):4474–4481, April 2023. Conference Name: IEEE Transactions onIntelligent Transportation Systems.[51] Xinkai Zhang, Bo Peng, Zaid Al-Huda, and Donghai Zhai. Interna-tional Journal of Image, Graphics and Signal Processing(IJIGSP). In-ternational Journal of Image, Graphics and Signal Processing(IJIGSP),14(5):28.[52] Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A. Efros. Un-84Bibliographypaired Image-to-Image Translation using Cycle-Consistent AdversarialNetworks, August 2020. arXiv:1703.10593 [cs].85
Thesis/Dissertation
eng
Civil Engineering
Vancouver : University of British Columbia Library
Attribution-NoDerivatives 4.0 International
Graduate
Gargoum, Suliman
Using Wasserstein distance and conditionality for stable and selective data augmentation in pavement crack analysis
Text