Science, Irving K. Barber Faculty of (Okanagan)
Computer Science, Mathematics, Physics and Statistics, Department of (Okanagan)
DSpace
UBCO
Graham-Knight, John Brandon
Doctor of Philosophy - PhD
University of British Columbia
Focus on Neural Network research has largely been on statistical performance, as Deep Learning is an optimization problem. This often overshadows properties needed for adoption, such as robustness and fairness. Early adversarial attacks showed minor input changes could cause incorrect predictions. Fairness is obscured by the black-box nature of neural networks, which are typically evaluated by a single performance statistic. This thesis explores uncertainty quantification motivated by improving fairness and robustness. A use case on fair artificial intelligence (AI) explores the Breast Screening Program at the BC Cancer Agency with the evaluation of a commercial AI. Model performance was evaluated using the area under the curve of receiver-operating characteristic. Compared to radiologists, it underperforms on follow-up periods up to two years but overperforms for longer periods. It performs well overall on a Canadian population, but performs poorly in certain subgroups including BI-RADS densities, patients with a self-reported family history of cancer, and mammograms showing calcifications. A second use case evaluates a car detection algorithm using the KITTI dataset, where the model shows a lack of robustness for small perturbations within sensor error. These use cases motivate the thesis. Uncertainty is evaluated by comparing the performance and diversity of 144 models from a grid search of network parameters for semantic segmentation of the CityScapes dataset. Many models show no significant performance difference, and performance and diversity across models can be predicted. This diversity can be used a measure of uncertainty in model ensembles. Uncertainty quantification during training is often used for online labeling and pruning mislabeled examples. However, these methods are usually computationally expensive and require multiple stages or training runs. The final chapter applies conformal prediction for efficient online data curation in a single training run. It proposes a data-centric method for dynamically weighting examples using prediction set size. Evaluated on CIFAR-10 for classification and CityScapes for segmentation, results show improved accuracy and improved Jaccard Index with small computational overhead.
Efficient Online Uncertainty ManagementConformal-In-The-Loop for Measuring and ActioningUncertainty at Training TimebyJohn Brandon Graham-KnightB.A., The University of Victoria, 2005B.Eng., The University of Victoria, 2010M.Sc., The University of British Columbia, 2021A DISSERTATION SUBMITTED IN PARTIAL FULFILLMENT OFTHE REQUIREMENTS FOR THE DEGREE OFDOCTOR OF PHILOSOPHYinThe College of Graduate Studies(Computer Science)THE UNIVERSITY OF BRITISH COLUMBIA(Okanagan)May 2025© John Brandon Graham-Knight, 2025The following individuals certify that they have read, and recommend tothe College of Graduate Studies for acceptance, a dissertation entitled:Efficient Online Uncertainty Management:Conformal-In-The-Loop for Measuring andActioning Uncertainty at Training Timesubmitted by John Brandon Graham-Knight in partial fulfilment of therequirements of the degree of Doctor of PhilosophyDr. Patricia Lasserre, Irving K. Barber Faculty of ScienceCo-SupervisorDr. Homayoun Najjaran, University of VictoriaCo-SupervisorDr. Yves Lucet, Irving K. Barber Faculty of ScienceSupervisory Committee MemberDr. Mohamed Sami Shehata, Irving K. Barber Faculty of ScienceSupervisory Committee MemberDr. Apurva Narayan, University of Western OntarioSupervisory Committee MemberDr. Julian Cheng, Faculty of Applied ScienceUniversity ExaminerDr. Amber Simpson, Queenâs UniversityExternal ExamineriiAbstractFocus on Neural Network research has largely been on statistical perfor-mance, as Deep Learning is an optimization problem. This often overshadowsproperties needed for adoption, such as robustness and fairness. Early adver-sarial attacks showed minor input changes could cause incorrect predictions.Fairness is obscured by the black-box nature of neural networks, which aretypically evaluated by a single performance statistic.This dissertation explores uncertainty quantification motivated by im-proving fairness and robustness. A use case on fair artificial intelligence (AI)explores the Breast Screening Program at the BC Cancer Agency with theevaluation of a commercial AI. Model performance was evaluated using thearea under the curve of receiver-operating characteristic. Compared to radiol-ogists, it underperforms on follow-up periods up to two years but overperformsfor longer periods. It performs well overall on a Canadian population, butperforms poorly in certain subgroups including BI-RADS densities, patientswith a self-reported family history of cancer, and mammograms showingcalcifications. A second use case evaluates a car detection algorithm usingthe KITTI dataset, where the model shows a lack of robustness for smallperturbations within sensor error.These use cases motivate the dissertation. Uncertainty is evaluated bycomparing the performance and diversity of 144 models from a grid searchof network parameters for semantic segmentation of the CityScapes dataset.Many models show no significant performance difference, and performanceand diversity across models can be predicted. This diversity can be used ameasure of uncertainty in model ensembles.Uncertainty quantification during training is often used for online label-ing and pruning mislabeled examples. However, these methods are usuallycomputationally expensive and require multiple stages or training runs. Thefinal chapter applies conformal prediction for efficient online data curationin a single training run. It proposes a data-centric method for dynamicallyweighting examples using prediction set size. Evaluated on CIFAR-10 for clas-sification and CityScapes for segmentation, results show improved accuracyand improved Jaccard Index with small computational overhead.iiiLay SummaryArtificial Intelligence (AI) robustness in safety-critical applications, likeself-driving cars, is crucial. AI must handle unpredictable and hazardoussituations to ensure the safety of passengers, pedestrians, and other roadusers. Robust AI can manage sensor failures, environmental changes, andunexpected driver behavior. Similarly, AI in healthcare can improve patientoutcomes, streamline operations, and offer personalized treatments. However,risks arise if not carefully implemented and monitored. Bias in AI fromtraining data can reflect healthcare disparities, leading to unequal outcomes.Ensuring fairness requires diverse datasets and ongoing model evaluation.Equity in AI ensures all patients benefit from technological advancements,regardless of background or socioeconomic status.Uncertainty quantification (UQ) is essential in AI for managing the un-certainty in models and predictions. It aids decision-making by providingconfidence measures, allowing for fallback mechanisms or human interven-tion. In resource-limited scenarios, UQ prioritizes actions based on outcomecertainty, such as determining which medical cases need further testing. UQhelps assess and manage AI risks, preparing organizations for adverse out-comes. During development, UQ highlights poor model performance areas,guiding improvements for more robust models.This dissertation introduces definitions and metrics of uncertainty, thenexplores two use cases including fairness of a commercial AI applied to breastcancer screening and the robustness of self-driving cars. We then exploreuncertainty as diversity by producing many models with different architectureswith the same data, and show that this is a promising UQ method. Finally,we apply conformal prediction for efficient online data curation in a singletraining run, showing improved performance in unbalanced datasets.ivPrefacePortions of this work have been collected from the below papers.Chapter 3 has been published as:Graham-Knight, J.B., Liang, P., Lin, W., Wright, Q., Shen, H., Mar, C., Sam,J., and Rajapakshe, R. External Testing of a Commercial AI Algorithm forBreast Cancer Detection at Screening Mammography. Radiology: ArtificialIntelligence 2025 7:3. https://doi.org/10.1148/ryai.240287.This study was performed under University of British Columbia â BC CancerResearch Ethics Board H20-03683.My contribution to this research was the execution of the experiment,data analysis, initial writing of the manuscript, and revising / editing themanuscript.Chapter 5 has been published as:Graham-Knight, J. B., Bond, C., Najjaran, H., Lucet, Y. and Lasserre,P. Predicting and explaining performance and diversity of neural networkarchitecture for semantic segmentation. Expert Systems with Applications214 (2023), 119101. https://doi.org/10.1016/j.eswa.2022.119101.My contribution to the research was identification and design of the research,programming execution of the method, data analysis, initial writing of themanuscript, and revising / editing the manuscript.Generative AI was used to help edit this document for brevity, spelling andgrammar.vTable of ContentsAbstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iiiLay Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ivPreface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vTable of Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . viList of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ixList of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiAcknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . xvDedication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xviChapter 1: Introduction . . . . . . . . . . . . . . . . . . . . . . . 1Chapter 2: Formal Definition and Background . . . . . . . . . 42.1 Model-Centric AI . . . . . . . . . . . . . . . . . . . . . . . . . 52.2 Data-Centric AI . . . . . . . . . . . . . . . . . . . . . . . . . 102.3 Uncertainty as Probability . . . . . . . . . . . . . . . . . . . . 182.4 Uncertainty as Similarity . . . . . . . . . . . . . . . . . . . . . 212.5 Uncertainty as Diversity . . . . . . . . . . . . . . . . . . . . . 242.6 Uncertainty as Robustness and Sensitivity . . . . . . . . . . . 282.7 Conformal Prediction . . . . . . . . . . . . . . . . . . . . . . . 322.8 Uncertainty and Fairness . . . . . . . . . . . . . . . . . . . . . 34Chapter 3: Epistemic Uncertainty in Breast Cancer ScreeningMammograms . . . . . . . . . . . . . . . . . . . . . . 363.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 363.2 Materials and Methods . . . . . . . . . . . . . . . . . . . . . . 393.3 Study Setting . . . . . . . . . . . . . . . . . . . . . . . . . . . 39vi3.4 Dataset Composition . . . . . . . . . . . . . . . . . . . . . . . 403.5 AI Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . 413.6 Statistical Analysis and Comparison with Radiologist Perfor-mance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 423.7 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 433.8 The Impact of Follow-Up Duration . . . . . . . . . . . . . . . 473.9 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 503.10 Calcifications and Epistemic Uncertainty . . . . . . . . . . . . 533.11 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55Chapter 4: Measuring Aleatoric Uncertainty with WhiteboxPerturbations . . . . . . . . . . . . . . . . . . . . . . 574.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 574.2 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . 594.3 Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 614.4 Results and Discussion . . . . . . . . . . . . . . . . . . . . . . 634.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68Chapter 5: Predicting and explaining performance and diver-sity of neural network architecture for semanticsegmentation . . . . . . . . . . . . . . . . . . . . . . 695.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 695.2 Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 715.3 Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 715.3.1 Model Overview . . . . . . . . . . . . . . . . . . . . . 725.3.2 Explaining Performance and Diversity . . . . . . . . . 765.4 Results and Discussion . . . . . . . . . . . . . . . . . . . . . . 765.4.1 Model Performance . . . . . . . . . . . . . . . . . . . . 775.4.2 Model Diversity . . . . . . . . . . . . . . . . . . . . . . 815.4.3 Explaining Performance and Diversity . . . . . . . . . 825.5 Limitations and Future Work . . . . . . . . . . . . . . . . . . 865.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87Chapter 6: Conformal-in-the-Loop . . . . . . . . . . . . . . . . 896.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 906.2 Conformal Prediction . . . . . . . . . . . . . . . . . . . . . . . 926.2.1 Proposed CitL Method . . . . . . . . . . . . . . . . . . 936.2.2 Evaluation Benchmarks . . . . . . . . . . . . . . . . . 956.2.3 Experimental Setup and Implementation Details . . . 966.2.4 Training Setup . . . . . . . . . . . . . . . . . . . . . . 97vii6.2.5 Evaluation Metrics . . . . . . . . . . . . . . . . . . . . 986.2.6 Reproducibility . . . . . . . . . . . . . . . . . . . . . . 996.3 Results and Discussion . . . . . . . . . . . . . . . . . . . . . . 996.3.1 Preliminary Experiment: Selecting the Alpha Value . 996.3.2 Classification Results . . . . . . . . . . . . . . . . . . . 1006.3.3 Segmentation Results . . . . . . . . . . . . . . . . . . 1016.3.4 Computational Overhead . . . . . . . . . . . . . . . . 1036.3.5 Example Weighting . . . . . . . . . . . . . . . . . . . . 1036.3.6 Example Pruning . . . . . . . . . . . . . . . . . . . . . 1086.3.7 Considerations for Selecting Alpha . . . . . . . . . . . 1096.3.8 Model Calibration . . . . . . . . . . . . . . . . . . . . 1116.4 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111Chapter 7: Conclusion and Future Work . . . . . . . . . . . . . 115Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117viiiList of TablesTable 3.1 Results of Commercial AI For Mammography Screeningat 1-year Follow-Up . . . . . . . . . . . . . . . . . . . . 45Table 3.2 Radiologist and AI Confusion Matrix . . . . . . . . . . 50Table 3.3 1-year overall follow-up classification metrics, radiolo-gist workload, and patient recalls. AI values are bina-rized at the manufacturerâs recommended threshold of0.10. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50Table 4.1 Comparison of Model Performances for AP11 with IoU0.70 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66Table 4.2 Comparison of Model Performances for AP40 with IoU0.70 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66Table 4.3 Comparison of Model Performances for AP11 with IoU0.50 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66Table 4.4 Comparison of Model Performances for AP40 with IoU0.50 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67Table 5.1 Parameters Explored. A grid search is performed overall possible combinations, with one parameter selectedfrom each set, for a total of 144 models. . . . . . . . . 72Table 5.2 Class and example counts . . . . . . . . . . . . . . . . 74Table 5.3 Training Example Counts . . . . . . . . . . . . . . . . 75Table 5.4 Architecture Parameters for Best Performing Models. . 79Table 5.5 Tukey Test Values for Best Model Pairs . . . . . . . . 80Table 5.6 Explanation quality (R2). Aggregate metrics excludeoverall fit. . . . . . . . . . . . . . . . . . . . . . . . . . 82Table 6.1 Performance metrics (accuracy for CIFAR-10 and mIoUfor CityScapes) for the CitL method with optimal αvalues compared to the baseline model across differentdatasets. â represents the difference in performancebetween CitL and the baseline. . . . . . . . . . . . . . 99ixTable 6.2 Classwise Accuracy for the imbalanced CIFAR-10 datasetacross various levels of synthetic noise. The table showsthe baseline performance, performance with the methodapplied, and the resulting improvement for each class.Light red represents the worst-performing class (air-plane in all cases) whose accuracy is improved by themethod at all noise levels. Light green represents thebest-performing class (ship or truck). . . . . . . . . . . 102Table 6.3 Classwise IoU for the CityScapes dataset with the besttraining run using α = 0.02. The table shows thebaseline performance, performance with the methodapplied, and the resulting improvement for each class. 102Table 6.4 Percentage overhead of our CitL relative to the baseline,showing mean, standard deviation (Std), minimum(Min), and maximum (Max) values across datasets. . . 103xList of FiguresFigure 2.1 Binary Entropy (orange), and Entropy (yellow, Y=0;red, Y=1) . . . . . . . . . . . . . . . . . . . . . . . . 19Figure 2.2 Conformal Prediction, applied on top of heuristic no-tions of uncertainty to produce rigorous marginal cov-erage. . . . . . . . . . . . . . . . . . . . . . . . . . . . 32Figure 3.1 Example features reported by radiologists at mammo-graphic screening. . . . . . . . . . . . . . . . . . . . . 37Figure 3.2 Example Mammogram. CC and MLO views are takenof both the left and right breast for the radiologist toreview. . . . . . . . . . . . . . . . . . . . . . . . . . . 38Figure 3.3 Radiologist Report. Radiologists examine mammo-grams to identify imaging features asymmetry, archi-tectural distortion, mass, and calcifications, as well asother information such as breast density. If any of the4 imaging features are observed, the patient is recalledfor follow-up. . . . . . . . . . . . . . . . . . . . . . . . 38Figure 3.4 Data Flow Diagram. 150,301 screens and 150,254outcome data records were found for our study daterange. After matching these two sources and filteringon screen date and 4 standard views, 136,700 screensare used in this study. . . . . . . . . . . . . . . . . . . 41Figure 3.5 Overall Results. Circle indicates radiologist operatingpoint and diamond indicates AI operating point atmanufacturerâs specified threshold (0.10). . . . . . . . 44Figure 3.6 Stratified AUC - Patient Risk Factors. Circle indicatesradiologist operating point and diamond indicates AIoperating point at manufacturerâs specified threshold(0.10). . . . . . . . . . . . . . . . . . . . . . . . . . . . 46Figure 3.7 Stratified AUC - Mammographic Findings . . . . . . 47xiFigure 3.8 Results at 3-year Follow Up. Circle indicates radiol-ogist operating point and diamond indicates AI op-erating point at manufacturerâs specified threshold(0.10). Effect of follow-up Length on sensitivity for AI,radiologist, and hypothetical double reading scenarios. 48Figure 3.9 Categories identified with differing subgroup perfor-mance (3-year follow-up). Circle indicates radiologistoperating point and diamond indicates AI operatingpoint at manufacturerâs specified threshold (0.10). . . 49Figure 3.10 Confidence of the commercial AI tested in Chapter 3before and after cropping to the expected AI size forimages with radiologists observed calcification, otherfeatures, and no cancer. . . . . . . . . . . . . . . . . . 54Figure 3.11 AI Confidence for a single mammogram before crop-ping (left) and after cropping (right), for an imagewith calcifications present. . . . . . . . . . . . . . . . 55Figure 4.1 Overview of the perturbation model. . . . . . . . . . . 59Figure 4.2 Pre-Trained Model Structure and Perturbation Method. 61Figure 4.3 The ConvNet to Generate Perturbations. . . . . . . . 62Figure 4.4 Histogram of L2 Norm Magnitude (left) and Percent-ages of the Original Vectorâs L2 Norm (right) . . . . . 67Figure 5.1 Activation Functions . . . . . . . . . . . . . . . . . . . 73Figure 5.2 mIoU by Class . . . . . . . . . . . . . . . . . . . . . . 78Figure 5.3 Performance of the top 16 Models. . . . . . . . . . . . 78Figure 5.4 Count of Similar Models by Rank . . . . . . . . . . . 79Figure 5.5 Diversity by Class . . . . . . . . . . . . . . . . . . . . 81Figure 5.6 Explanation Quality, stratified over 19 classes in CityScapesdataset. Note that the y-axis is scaled to allow moreaccurate viewing. . . . . . . . . . . . . . . . . . . . . 82Figure 5.7 Overall Importance of Parameters . . . . . . . . . . . 83Figure 5.8 Stratified Importance of Parameters . . . . . . . . . . 83Figure 5.9 Overall Performance Difference Effect on Diversity . . 84Figure 5.10 Stratified Activation Score . . . . . . . . . . . . . . . 84Figure 5.11 Stratified Depth Score . . . . . . . . . . . . . . . . . . 85Figure 5.12 Stratified Residual Units Score . . . . . . . . . . . . . 85Figure 5.13 Stratified Width Score . . . . . . . . . . . . . . . . . . 85Figure 5.14 Stratified Width Ratio Score . . . . . . . . . . . . . . 86xiiFigure 6.1 An overview of the proposed Conformal-in-the-Loop(CitL) framework. conformal prediction is used to gen-erate prediction sets during self-supervised machinelearning. Training examples are weighted by uncer-tainty; validation examples are used to calibrate theuncertainty model. . . . . . . . . . . . . . . . . . . . . 94Figure 6.2 Segmentation mIoU on the validation set for variousvalues of α. The baseline performance is indicated bythe dotted blue line. . . . . . . . . . . . . . . . . . . . 100Figure 6.3 Segmentation mIoU for values of α [0.01, 0.10]. Thebest mIoU of 71.1 is achieved at α = 0.02. BaselinemIoU (66.1) is represented by the dotted blue line. . . 100Figure 6.4 Classification accuracy on imbalanced CIFAR-10 usingour CitL method and the baseline with cross-entropyloss (blue dotted line) and focal loss (orange dottedline). The box plots show 10 α values ranging from0.10 to 0.19. . . . . . . . . . . . . . . . . . . . . . . . 101Figure 6.5 Average per training-step time of CitL over baseline.Multiple CitL values represent different α hyperpa-rameters. (CIFAR-10 11.2 ± 0.8%, CityScapes 4.0 ±0.4%) . . . . . . . . . . . . . . . . . . . . . . . . . . . 103Figure 6.6 Weight range and uncertainty by training step andnoise level. . . . . . . . . . . . . . . . . . . . . . . . . 104Figure 6.7 Mean weight by class across training epochs for theimbalanced CIFAR-10 dataset with no synthetic labelnoise. Each subfigure shows the distribution of meanclass weights at different training epochs, illustratinghow the algorithm adjusts class weights over time. . . 105Figure 6.8 Mean weight by class across training epochs for theCityScapes dataset for best value of α = 0.02. Eachsubfigure shows the distribution of mean class weightsat different training epochs, illustrating how the algo-rithm adjusts class weights over time. . . . . . . . . . 106Figure 6.9 Amount Pruned by Training Step and Alpha. . . . . . 107Figure 6.10 Quantile by Training Step and Alpha. . . . . . . . . . 108xiiiFigure 6.11 Imbalanced CIFAR-10 classification performance at10% noise level. The blue dotted line represents thebaseline using Cross Entropy Loss; the red dotted linerepresents the baseline using Focal Loss. The methodoutperforms both baselines at all tested values of α.At α = 0.20, the method prunes â 2.5% of examples.Performance declines when α â¥ 0.21. . . . . . . . . . 109Figure 6.12 Mean weight by class across training epochs for theCityScapes dataset at worst value for α = 0.09. Eachsubfigure shows the distribution of mean class weightsat different training epochs, illustrating how the algo-rithm adjusts class weights over time. . . . . . . . . . 110xivAcknowledgementsI would like to acknowledge the hard work and support of my supervisors,Drs. Patricia Lasserre, Yves Lucet, and Homayoun Najjaran.I would also like to thank Dr. Rasika Rajapakshe, who leads the BCCancer Early Detection Group and was the PI on Chapter 3.Chapter 3 was supported through a Mitacs Accelerate International-Abroad grant (IT19427). The funding for extracting the mammographyimage dataset used in this study was provided by Canadian Cancer Societygrant (BC-RG-15-1).We acknowledge the support of the Natural Sciences and EngineeringResearch Council of Canada (NSERC), [funding reference number ALLRP561264 - 21]. Cette recherche a été financée par le Conseil de recherches ensciences naturelles et en génie du Canada (CRSNG), [numéro de référenceALLRP 561264 - 21].This research was enabled in part by support provided by Compute On-tario, WestGrid, and Compute Canada, and Yves Lucetâs NSERC Discoverygrant RGPIN-2018-03928.xvDedicationThis dissertation is dedicated to my wonderful wife Cassie Jones whobrought perspective to life and wouldnât let me give up when I wanted to.Hopefully Griff and Arlo, my two wonderful boys, will one day understandwhy these things take so long.xviChapter 1IntroductionUncertainty quantification in machine learning is a critical field focused onunderstanding, measuring, and expressing confidence in the predictions anddecisions made by AI models. As AI systems become increasingly integratedinto high-stakes domains, from medical diagnosis to autonomous vehicles, theability to transparently communicate their level of certainty is paramount.This involves distinguishing between two primary types of uncertainty: modeluncertainty and data uncertainty. Model uncertainty reflects the modelâsignorance or lack of knowledge due to an inappropriate model architectureor poor inductive biases. It is a reducible uncertainty, meaning that withmore diverse and representative data, or a better model, this uncertaintycan often be diminished. In contrast, data uncertainty arises from inherentrandomness or noise in the data itself, such as sensor limitations or naturalvariability in the phenomenon being modeled. This type of uncertainty isgenerally considered irreducible, regardless of how much data is collected orhow sophisticated the model becomes.The implications of these uncertainties directly impact the robustness andfairness of AI systems. For example, in an AI for cancer diagnosis, high modeluncertainty might arise if the model was trained predominantly on data fromone demographic, leading to less reliable predictions for underrepresentedgroups. Similarly, in self-driving cars, model uncertainty might occur when thevehicle encounters an entirely novel road sign or an unusual weather conditionnot seen during training, affecting its robustness and potentially leading tounsafe decisions. Data uncertainty could stem from sensor noise in foggyconditions or the unpredictable behavior of other drivers. Understanding andquantifying these uncertainties is essential to building more trustworthy andreliable AI.Many contemporary AI models, especially those used for classificationtasks, provide a point prediction; this score, often derived from the out-put of a softmax function in neural networks, is intended to represent themodelâs certainty in its prediction; a higher score intuitively suggests greaterconfidence. However, a significant challenge, particularly prevalent in DeepNeural Networks, is poor model calibration. Calibration refers to how well the1predicted confidence scores align with the actual likelihood of the predictionbeing correct. Unfortunately, DNNs are often overconfident, meaning theirreported high confidence scores do not reliably translate to a high probabilityof correctness. This discrepancy arises because the optimization processesused to train these networks primarily focus on maximizing accuracy, notnecessarily ensuring that the output probabilities are a true reflection of theunderlying uncertainty.Effectively managing uncertainty is crucial for the implementation ofrobust and fair AI. Model uncertainty can be estimated using methodslike ensembling, where multiple models are trained independently and theirprediction variance is used as a proxy for uncertainty. If different models inan ensemble provide widely varying predictions for a given input, it signalshigh model uncertainty. Furthermore, active learning can request labelingfor highly uncertain data points, thus improving the model in critical areas.Curriculum learning, presenting training data in a structured and meaningfulorder, can also help models learn more effectively and potentially reducemodel uncertainty by building a stronger foundational understanding.The motivation for this dissertation is presented in Chapters 3 and 4. Thefirst presents work done at BC Cancer on external testing of a commercial AIfor breast cancer screening. Overall results are in line with those reported bythe manufacturer and confirmed in previous studies. However, we identifiedseveral groups where the AI underperforms, particularly in images withcalcifications compared to those without. These calcifications are known tobe associated with cancer, and any calcification observed by radiologists isreferred for follow-up. We found that this underperformance is likely dueto the AI resizing the image before processing, an example of uncertaintyintroduced by the model. Chapter 4 analyzes the robustness of a popularalgorithm for car detection from 3D point clouds using the KITTI datasetby generating perturbations within sensing error and observing degradationof model performance. These sensor limitations are an example of datauncertainty.Chapter 5 introduces a method to measure and mitigate uncertainty ina general way by promoting well-performing ensembles. While promotingdiversity across models is commonly done by varying data in training, orvarying hyperparameters of models, we promote this diversity by varyingmodel architecture. Importantly, this chapter shows that when varyingarchitecture, many of these models show no statistically significant differencein performance; moreover, both model performance and diversity across thesemodels can be promoted and predicted, which could greatly speed up modeltraining loops in ensemble methods.2Class imbalance is identified as a major limitation in Chapter 5, wheresome minority classes like bicycle, train, and bus dramatically underperformeasier classes such as sky and road. Interestingly, there appears to be differentreasons for poor performance. Bicycle, for example, has a reasonable numberof examples but very small components in the image; train and bus, on theother hand, are very large components in an image but seen very rarely. Thisclass imbalance is also present in the calcifcations observed in the breastcancer screening AI data, where 2,321 examples (1.7%) have calficicationsbut 134,379 do not. Interestingly, 39.0% of the cancers (328) are found inthese 2,321 images with calcifications.Calibration-based methods for measuring uncertainty have recently gainedpopularity because of their computational efficiency and ability to produceuncertainty estimates for individual examples. However, in Chapter 3 itis found that the AI confidence score between images with calcifications isindistinguishable from images without. While we can increase the confidencescore for images with calcifications by cropping to the size expected bythe AI, this is impractical as it requires manually analyzing the image toensure important features are retained. Chapter 6 attempts to reconcilethis uncertainty in the training phase, by simultaneously weighting harderexamples and pruning mislabeled examples. This chapter achieves success inimproving performance in minority classes on the CityScapes dataset whichwas observed in Chapter 5.In summary, this dissertation investigates the critical role of uncertaintyquantification in enhancing the robustness and fairness of AI systems. Wedemonstrate, through studies on breast cancer screening and autonomousvehicle perception, that understanding and mitigating model and data uncer-tainty can address challenges such as demographic underperformance, sensorlimitations, and class imbalance, leading to more reliable and equitable AI.3Chapter 2Formal Definition andBackgroundThe world of computer vision was forever changed when Alex Krizhevskywon the 2012 ImageNet Large-Scale Visual Recognition Challenge. Not onlydid he win, he had an error rate of only 15.3%, 10.8% lower than the runner-up. He achieved this feat through deeper models, which were tractable for thefirst time through the use of Graphical Processing Units (GPU) [98]. Thisbegan a new era of Artificial Intelligence (AI) research, which sought to solveharder problems with larger, more computationally expensive networks.The goal in machine learning is to produce a generalizable function whichbest approximates the ground truth to a given task. When noise is considered,the ground truth can be represented asY = f(X) + ϵ, (2.1)where f(X) is the learnable portion of the ground truth, and ϵ is non-learnablenoise. A perfect model would learn the underlying function without modelingany noise. In practice, however, it is very difficult to separate the underlyingfunction from the noise. Every learning approach makes a trade-off betweenthese two pieces; in general, repeated exposure to examples will include morenoise in the learned function.Overfitting is learning an overly complex function which is able to perfectlyrepresent the training set but is unable to generalize to any other dataset.When a function is overfit, it learns the noise from the training set, whichappears to be an improvement while training. However, since the noise acrossdata sets is highly unlikely to be the same, the end result is worse whenapplied to new data. There are various techniques to avoid overfitting, but theonly way to measure a methodâs effect on a model is to test its generalizabilitythrough validation. Caruna, Lawrence, and Giles studied the overfitting ofneural networks and concluded that larger models are more likely to overfitthan smaller, properly sized models, as the larger network can model a morecomplex function. They also found that the larger network initially learns afunction similar to a well-sized network, and then additional training leads42.1. MODEL-CENTRIC AIto overfitting. Therefore, they suggested using early stopping, which usesvalidation to stop training when the model reaches minimal loss in a data setnot seen in training [36].Underfitting results in a model that does not maximize learning fromthe training set. Generally, underfitting is a result of being too cautious ofover-fitting. However, it is difficult to measure compared to overfitting, whichcan be calculated as the difference between the performance of the trainingset and an unseen validation set. To quantify underfitting, one would needto compare the performance of the training set with a hypothetical perfectmodel. If such a model existed, it would be more beneficial to use it, makingthe learning process unnecessary. Unfortunately, this is not the case in mostsituations. Another way to look at it is that to prove that a model is notunderfit, one would have to test all possible models, which is an impossibletask.2.1 Model-Centric AIProducing a well-performing model using artificial neural networks (ANN)is a complicated process dependent on the training data D, network architec-ture γ, as well as the optimizer and associated hyperparameters λ. Learnedparameters Î are then modified to minimize loss, a scalar value calculatedas a function of Y and YË given a defined loss functionL = L(Y, YË). (2.2)The loss value should be small when YË â Y. When YË and Y matchexactly, the loss should be at the global minimum; when YË and Y differsignificantly, the loss value should be large. Loss is used in the training processand as such is subject to any limitations enforced by the training processemployed; notably, they must be differentiable to allow for backpropagation.Each layer l â L of an ANN can be seen as a mapping function betweenits inputs and outputs,âl â L, fl : Xl 7â fl(Xl;Î, Î³),YËl = fl(Xl;Î, γ).(2.3)This mapping function takes the formYËl = Ïl[ââaâL(YËa Â·Îla) + bl], (2.4)52.1. MODEL-CENTRIC AIwhere Îla are the weights connecting layer a to l, bl is a bias weight, and Ïlis the activation function at layer l. For a feed-forward network, the input isdependent only on previous layersXl =âa<lYËa Â·Îla. (2.5)Combining equations (2.3), (2.4) and (2.5) givesfl = Ïl[âa<l(YËa Â·Îla) + bl], (2.6)where (Ïl, |Îla|) â γ.The full network is a series of these layerwise functionsYË = F (X;Î, γ) = (fl â¦ flâ1 â¦ ... â¦ f1)(X;Î, γ), (2.7)where architecture γ â Î describes: the network width |Î| at each layer; thetotal number of layers L; and the activation function Ïl at each layer.The goal of the training process is to solve the optimization problem.inf1|D|â(X,Y)âDL(YË(X),Y), (2.8)using examples taken from a set of independent and identically distributeddata points: D = {Xd,Yd}|D|d=1. From equation (2.7), the loss functionbecomesL(YË,Y) = L(F (X;Î, γ),Y), (2.9)where the scaling constant 1|D| can be ignored. The resulting optimizationproblem, including norm regularization, becomesinfÎâA,Î³âÎ,Î»âÎ1|D|â(X,Y)âDL(F (X;A, γ),Y) +ânÎ»nâ¥Îâ¥pn , (2.10)where pn specifies the dimension of the norm, commonly taken from {0, 1, 2,â}.Hyperparameters λ â Î are numerous, including: the amount of loss regu-larization (e.g. L2); learning rate; batch size; stopping criteria; and manymore.This problem is intractable. As a shortcut, it is common to select asufficiently large network architecture γ with regularization; for certainproblem domains, it is also possible to apply inductive bias to restrict the62.1. MODEL-CENTRIC AIarchitecture and weight combinations, such as convolutional neural networks(ConvNets) widely used in computer vision (CV) tasks. Supervised learningwith backpropagation is usually used to produce parameters Î â A, whichminimizes empirical risk in the set of examples D. Since this problem isgenerally non-convex, this process may find a local minimum only; however,by varying initialization weights Îâ², used as the starting point for training,or employing warm restart techniques, it is possible to explore several localminima.An early barrier to scaling DNNs was the vanishing gradient problem.DNNs are trained through backpropagation using the chain rule. When theneural network becomes very deep, repetitive applications of the chain rulelead to a gradient that either explodes quickly to infinity or quickly vanishesto zero [16, 70]. Glorot and Bengio showed through experimentation thatthe correct distribution of the initial weights can cause the activation valuehistogram to spread out and remain relatively similar across all layers. Theysuggest a randomized weight initialization bounded by the properties of thenetwork:W â¼ U[ââ6ânj + nj+1,â6ânj + nj+1], (2.11)where nj is the number of neurons in layer j, and U is the uniform distribution[70]. Interestingly, the authors equate the exploding gradient issue withactivation values clustered near 0 [70]. However, more recent work has shownthat, for extremely deep networks, correct initialization may not be sufficientto keep the gradient stable. He et al. attempted to rectify this issue byadding skip layers or summing the inputs of a layer with its convolutionaloutput to produce the final output. As a result, rather than optimizing theconvolutional output of the layer F(x), optimization is performed on F(x)+x[83]. Using the residual learning building block, the authors explored theviability of making very deep neural networks. They concluded that the erroron the CIFAR-10 test set was minimized when 110 layers were used [83].Scaling neural network models involves complexities due to the quadraticcomputational demands associated with increasing width, depth, and imagesize. Tan and Le demonstrated that each dimension offers diminishingreturns, with depth significantly impacting performance due to the additionof nonlinearities through activation functions [180]. Increasing networkwidth involves a computationally intensive process of performing multiplechannel convolutions; the output channels are then stacked. The number ofconvolution operations is O(N Â·M) â O(n2) in network width, where N is72.1. MODEL-CENTRIC AIthe number of filters in the previous layer and M is the number of filters inthe current layer.Neural Architecture Search Neural Architecture Search (NAS) is anAutoML task that seeks to balance the efficiency and performance of DeepLearning (DL) models. It has achieved success, leading to the widespreadadoption of models such as MnasNet [178] and EfficientNet [181]. Using thesemethods, platform-aware and systematic scaling of network width, depth, andinput resolution has led to architectures which are state-of-the-art while beingorders of magnitude faster than what came before. However, these methodsdo not consider ensembling as a possible scaling dimension, which is a knownmethod to improve generalizability and reduce variance in neural networks.The advantages of averaging ensembles are highly dependent on the diversityof the underlying predictions, which is not guaranteed. Averaging shows thegreatest benefit when predictions from the base learners are of similar qualitybut are not correlated as much as possible [72, 80].Learning a model architecture directly on a large data set can be alengthy process. Reinforcement learning can be used as a basis for a neuralarchitecture search (NAS) strategy. Zoph and Le [212] applied NAS with RLto CIFAR-10 Penn Treebank and the PTB character language modeling task,exceeding previous baselines in all cases [212].NASNet introduced the idea of creating building blocks designed on asmall data set to a larger data set [213]. The learned aspect of the designincluded elements such as which lower layer(s) each higher layer took as input,the transformations applied at that layer, and how to merge multiple outputsat each layer. The Efficient Neural Architecture Search (ENAS) approachdiscovers architectures by learning to search for an optimal subgraph withina large graph. The controller is trained with a policy gradient to select asubgraph that maximizes the expected reward of the validation set. Themodel corresponding to the subgraph is trained to minimize cross-entropyloss. ENAS employs multiple child models that share parameters. ENASachieved comparable performance, while being much faster than NASNet[141].Genetic algorithms were also explored for optimizing the neural archi-tecture; most notably, Lamarckian evolution, where learned parameters arepassed to offspring, was employed by LEMONADE [55]. LEMONADE per-formed multi-objective optimization, balancing computationally expensiveparameters such as model performance with easy-to-evaluate parameterssuch as model size. To do so, they maintain a population of non-dominatedparent models; every iteration then samples parent networks with respectto some probability distribution based on cheap objectives and generates82.1. MODEL-CENTRIC AIchild networks by applying network operators. A second sampling selects asubset of children based on cheap objectives, and only in this subset are theexpensive objectives evaluated [55].Differentiable ARchiTecture Search (DARTS) relaxed the network ar-chitecture search space to be continuous. The architecture can then beoptimized with respect to the performance of its validation set by gradientdescent, instead of searching for a discrete set of candidate architectures.The data efficiency of gradient-based optimization, as opposed to inefficientblack-box search, allows DARTS to achieve competitive performance withthe state of the art using orders of magnitude fewer computational resources.To accomplish this, they defined the network architecture as a set of nodes(feature maps) with connections between then; these connections are made upof candidate operations (e.g., convolution, max pooling) and a zero operationwhich means the nodes are disconnected. The weighting of these operationsis learned through gradient descent, and the dominant operation for eachconnection is promoted through softwax over all possible operations [114].Ensembling Ensembling as a possible scaling method for neural networkswas considered in [76]. Ensemble methods use multiple learning algorithms toachieve better predictive performance than any of the individual algorithmscould provide on their own. This usually requires more computation thanevaluating the prediction of a single model, but this computation scales basedon number of models. Ensemble learning can be seen as a way to compensatefor inadequate learning algorithms by doing extra computation, or as analternative to doing more learning on one non-ensemble system. It is oftenmore efficient to use the same increase in compute, storage, or communicationresources to improve accuracy across two or more methods, rather than justone. Fast algorithms such as decision trees are commonly used in ensemblemethods (e.g. random forests), although slower algorithms can also benefitfrom ensemble techniques.Averaging diverse predictions mitigates noise and reduces the impact ofoutliers. The theoretical justification of these techniques is well understood,and the practical benefits are undeniable. However, the advantages of aver-aging are heavily dependent on the diversity of the underlying predictions,which is not guaranteed. Averaging shows greatest benefit when predictionsfrom the base learners are of similar quality, but are uncorrelated as much aspossible [72, 80].Generally, ensembles are more effective when there is a significant diver-sity among the models. Therefore, many ensemble methods seek to promotediversity among the models they combine. Surprisingly, more random algo-rithms (like random decision trees) can create a stronger ensemble than very92.2. DATA-CENTRIC AIdeliberate algorithms (like entropy-reducing decision trees). Using a varietyof strong learning algorithms has been shown to be more effective than usingtechniques that attempt to dumb-down the models in order to promote diver-sity. Diversity can also be increased in the training stage of the model usingcorrelation for regression tasks or using information measures such as crossentropy for classification tasks. Like other scaling dimensions, ensembles showdiminishing returns with predictive performance asymptotically approachingsome practical maximum [76].First applied to neural networks by Schwenk and Bengio in 1998, gradientboosting can be used to produce many correlated models such that the sumof their losses is minimized; successive models are trained using the residualloss of previous models. This can be repeated for many iterations, with eachnew learner correcting the mistakes of past learners [64, 65, 161]. Recentwork by Bardirli et al. applied boosting techniques to neural networks for thetasks of binary classification, regression, and ranking, surpassing XGBoost inall cases [9].Boosting is commonly applied to binary classification problems throughdecision trees. Decision trees produce a single scalar output, where the lossis easily calculable. More recent work by Bardirli et al. applied boostingtechniques to neural networks for the tasks of binary classification, regression,and ranking. With neural networks, they achieved results which surpassedthe decision tree baseline [9].2.2 Data-Centric AIThe quadratic relationship between computational complexity and rep-resentation capacity of DNNs was alleviated by Vaswani et al. in theirfoundation paper introducing Attention [186]. Introduced in 2017 as amethod to replace Recurrent Neural Networks (RNN) for Natural LanguageProcessing (NLP) tasks, the concept was soon adopted by computer visionpractitioners [51, 182]. Given an input sequence of n tokens of dimensionsd X â RnÃd, three functions are used to produce the feature representationquery (Q), the key (K) and the value (V), calculated as [186]:Q = XWQ, K = XWk, V = XWV . (2.12)Attention is then calculated asY â² = D(Q,K, V ) = softmax(QKTâdq)V. (2.13)102.2. DATA-CENTRIC AINotably, the embeddings of Q, K, and V are usually implemented as simple,cost-efficient linear functions, rather than repeated convolution operationsin multiple filters. This allows the representation capacity of the model toscale linearly in the number of self-attention heads. These heads can alsobe calculated in parallel, if computational capacity allows, or in series forcomputationally constrained deployments. In self-attention, the embeddingsare derived from the same source, providing an uncertainty-aware methodfor selecting which input regions are most suited to producing a reliableoutput. Importantly, self-attention allows the model to select the input mostsimilar to the data observed during training. Self-attention, achieved state-of-the-art results on the difficult CityScapes dataset [44, 182]. Their methodpassed multiple augmentations of the same image, focusing on changes whereconvolutional neural networks are sensitive, such as scale and rotation.At the same time that attention was being developed, researchers exploredthe efficiencies of sharing a single feature extractor between multiple tasks.An early example of this, Panoptic-Deeplab used a shared encoder andunshared multiscale context and decoder branches to produce both semanticand instance segmentations simultaneously [42]. Multi-head attention andmulti-headed predictors came together to produce the first foundation models,which were trained on massive datasets and often multiple tasks. BetweenApril 2023 and January 2024, models for zero-shot segmentation [94], visualquestion answering [113], zero-shot classification and embedding [197], andzero-shot object detection [119, 126] were released. There were also massiveimprovements in multimodal models, such as GPT-4 and Google Gemini.The introduction of larger datasets required for training foundation modelsintroduces a new problem: how do we create and maintain large enoughdatasets? In only 25 years, deep learning has gone from grayscale, 10-class28x28 digit recognition in 1998 [105] to models, trained on a large portion ofthe internet, which arguably pass the Turing test [1, 90].Data-centric AI (DCAI) is the field of engineering data to constructan AI system, and is often the most effective way to improve performancein practical ML applications. Although experienced data scientists havelong practiced this manually through trial and error and intuition, DCAIlooks at the improvement of data as a systematic engineering discipline.There are many possible tasks within the DCAI paradigm, for example dataaugmentation, automated collection, and automatic labeling [208].Learning with Noisy Labels Learning with noisy labels is a commonproblem in real-world applications of artificial intelligence. One possibleapproach is to learn the easy examples first, and learn the hard exampleslater, delaying the mislabelled data until much later in the learning process.112.2. DATA-CENTRIC AIBengio et al. proposed curriculum learning as a way to incrementally increasethe complexity of the data samples used during the training process. This isa natural way to carry out curriculum learning, as it mirrors how humanslearn [15]. Curriculum learning strategies have been successfully applied inall areas of machine learning, for a variety of tasks. However, the need toclassify the samples from easy to hard, as well as the correct pacing functionto introduce more difficult data, can limit the use of curriculum approaches[169].MentorNet is a data-driven system that supervises training a CNN calledStudentNet. It can be trained to either approximate a predefined curriculumor discover new ones from data. The data-driven curriculum is adjusted basedon StudentNet feedback. During training, MentorNet controls the timing andattention to learn each sample, while StudentNet makes predictions aloneat test time. This collaborative learning paradigm allows the curriculum tobe determined by both the teacher and the student. This algorithm can beparallelized to train deep CNNs on large datasets, and its convergence hasbeen verified on large-scale benchmarks [89].Loss re-weighting is another possible approach to learning with noisylabels, and was one of the earliest to be explored in a deep learning context.The authors in [152] noted that there is a natural difficulty in identifyingmislabelled examples, since it is difficult to separate epistemic from aleatoricuncertainty; put another way, it is difficult to determine if an example issemantically incorrect whether the learned model is wrong. The authors ofthat paper add an additional stage to the normal optimization procedure,where the gradient is calculated on a known clean, unbiased set of data; anon-linear combination of the training minibatch and clean data gradients isused for optimization [152]. Post-hoc loss re-weighting is possible based onthe false positive (p+1) and false negative (pâ1) out-of-sample error rates. Inpractice, these error rates are found using cross-validation. The re-weightingfunction is [133]LË(t, y) := (1â pây)L(t, y)â pyL(t,ây)1â p+1 â pâ1 . (2.14)This is an unbiased estimator that is easy to calculate, but can makeconvex loss functions nonconvex and increases the Lipschitz constant (i.e.sensitivity) of the function. This method was tested on a simple classificationproblem, with corrupted labels in 20% and 40%; in both cases, the originalfunction was recovered with good accuracy [133].Boosting algorithms decrease bias of machine learning models by increas-ing the loss penalty for hard examples, a process called online hard-example122.2. DATA-CENTRIC AImining. However, it is often unclear if the label is incorrect or the model isunable to properly identify the example. This can be alleviated by creating aloss function which ignores examples below a threshold, for example by [89]gm ={Li â¤ λ1 λ2 = 0min(max(0, 1â LiâÎ»1Î»2 ), 1) λ2 Ì¸= 0.(2.15)When λ2 = 0, the function selects examples with loss below λ1. When λ2 Ì¸= 0,the function selects examples with loss below λ1+λ2 and weights all sampleslinearly by 1â LiâÎ»1Î»2 . One possible version of this is Self-Paced, where theweighting of an example increases with loss until some maximum, after whichit saturates. This is similar to a learned curriculum using [89], where theweight of the example increases with loss to some critical point, where itdecreases dramatically, leading to the example being ignored.The now seminal work on the subject of learning with noisy data isConfident Learning (CL). CL is a model-agnostic meta-learning algorithmwhich identifies and removes noisy data prior to training. First, CL identifiesclass-conditional label noise error rates through cross-validation. Second,CL removes the most difficult examples by rank-and-prune, with loss on thecross-validated model as the ranking criteria. The primary approach usedby CL is data pruning through co-teaching. At its simplest, this is cross-validation of a dataset, where lack of generalizability across datasets indicatesincorrect labels (see Algorithm 1 from [39]). There are known extensionsto this algorithm that operate iteratively [138]. Finally, CL re-weights theloss for each class to compensate for the removed examples, and re-trainson the entire dataset [137]. CL was devised to reconcile labeling errors indatasets, and has achieved good results. CL assumes that the input data arenot corrupted, but that the labels are incorrect with some class-conditionalprobability η. So long as η â¤ 12 , the data contains some useful information;however as η â 12 , training requires more samples and more time [5]. CLflips the concept of boosting by downweighting or even excluding exampleswith high loss. The assumption is that hard examples are out-of-domain andpolluting the final model.Learning with Missing Labels When dealing with extremely largequantities of data, the main bottleneck of supervised learning is often thetime and effort to collect data. Obtaining labeled data for a learning problemcan be very costly, sometimes requiring a knowledgeable human, and othertimes necessitating computationally expensive experiments. This can make itdifficult to obtain large, completely labeled training sets, while the acquisitionof unlabeled data is comparatively inexpensive. The reality of Deep Learning132.2. DATA-CENTRIC AIis that, given the quantities of data required to train extremely large models,working with unlabeled or poorly labeled data is an unfortunate reality.Semi-supervised learning reduces the labeling effort by combining a smallnumber of labeled data points with unlabeled data.Mathematically, semi-supervised learning uses a set of l independentlyand identically distributed examples x1, . . . , xl â X with their correspondinglabels y1, . . . , yl â Y and u unlabeled examples xl+1, . . . , xl+u â X. Thiscombination of labeled and unlabeled data is used to surpass the performancethat can be achieved by removing the unlabeled data. This technique canbe divided into two categories: transductive learning and inductive learning.Transductive learning is used to infer the correct labels for the given unlabeleddata xl+1, . . . , xl+u only, while inductive learning is used to infer a functionmapping from X to Y.To make use of unlabeled data, there must be some correlation with theunderlying data distribution. Semi-supervised learning algorithms are basedon one or more of the following assumptions: (1) points that are close togetherare more likely to have the same label. This is also assumed in supervisedlearning and leads to a preference for simple geometric decision boundaries; inthe case of semi-supervised learning, the smoothness assumption also impliesthat decision boundaries should be in areas of low density, where few pointsare close to each other but in different classes; (2) the data tends to formdistinct clusters, and points in the same cluster are more likely to have thesame label; (3) the input data can be approximated in much lower dimensionthan the input space; in this case, learning can use distances and densitiesdefined in the lower-dimensional space. This lower-dimensional representationcan help avoid the curse of dimensionality.Co-training is a machine learning technique that is used when there isa limited amount of labeled data and a large amount of unlabeled data.The algorithm requires two views of the data, each of which is composedof two different sets of features that provide complementary informationabout the instance. The two views should be conditionally independentand each should be sufficient to accurately predict the class of an instance.Co-training first learns a separate classifier for each view using the labeledexamples. The most confident predictions of each classifier on the unlabeleddata are then used to iteratively construct additional labeled training data[19]. Krogel and Scheffer demonstrated that co-training is only beneficial ifthe feature sets are independent; if the classifiers agree on all unlabeled data,i.e. they are dependent, labeling the data does not create new information.An experiment in which the classifier dependence was greater than 60%showed worse results [99].142.2. DATA-CENTRIC AIActive learning is a form of machine learning in which an algorithm canrequest labels from a user or other source of information. The authoritivesource of the labels is called a teacher or oracle. When there is an abundance ofunlabeled data, but manual labeling is expensive, active-learning algorithmscan query the user/teacher for labels. This type of iterative supervisedlearning is known as active learning, and it can often require fewer examplesto learn a concept than normal supervised learning. However, there is a riskthat the algorithm will be overwhelmed by uninformative examples. Activedeep learning employs a predictor F and a selector S of which samples tolabel. The selector S can consider many factors, such as the variance in theprediction of F given a data sample, the diversity of the underlying data, oradversarial robustness [115].ADL methods often balance uncertainty and diversity when selectingwhich data to introduce. Diversity must be high enough to introduce newinformation to the model, but high uncertainty indicates the example is out-of-distribution. Often this requires a two-stage training approach, where thefirst stage identifies examples with appropriate uncertainty and the secondstage seeks diversity. On the complicated task of instance segmentation,the authors of [204] explored margin and entropy as uncertainty metrics,and oversample examples with high uncertainty. To measure diversity, theycalculate the cosine similarity between example instances and form a bipartitegraph where the edges are kept only if diversity is above a defined threshold.They then prune the graph to preserve the maximum number of edges for afixed number of nodes. Annotations are provided for the new examples usingmajority vote aggregation over an ensemble of learners [204].Learning with Class Imbalance Beyond label noise, another criticalchallenge in real-world datasets is class imbalance [2, 46]. This imbalancearises when certain classes, often referred to as âheadâ classes, are representedby a large number of samples, while others, or âtailâ classes, are underrep-resented. Datasets with such imbalance tend to bias models toward themajority classes, leading to underrepresentation and under-learning of minor-ity classes. Imbalance-related biases are well-documented: models trained onsuch data generally optimize for majority-class accuracy, resulting in reducedperformance on minority categories. Common approaches to address classimbalance include resampling and reweighting, which adjust either the sampledistributions or the class-specific weights to encourage learning from minorityclasses.Class imbalance is a pervasive issue in many machine learning tasks,where the majority class is overrepresented in training data, often leadingmodels to underperform on minority classes. This imbalance causes models152.2. DATA-CENTRIC AIto overpredict the majority class, resulting in frequent misclassification ofminority instances, as noted in several studies [12, 13]. Solutions for addressingclass imbalance generally fall into two main categories: data-centric andalgorithmic approaches [96].Data-centric methods focus on adjusting the training data distribution.Undersampling the majority class, though effective, risks discarding criticalfeatures, which can lead to a loss of valuable information. Clustering-basedundersampling has been proposed to mitigate this by removing redundantor noisy samples while retaining informative examples [10, 100, 122, 200].Oversampling the minority class can also help [79, 173], but it can increase therisk of overfitting and prolong training. Synthetic data generation techniques,such as SMOTE [53, 135], and data augmentation [41, 207] are commonlyused to introduce new information, effectively balancing the data withoutmerely duplicating samples.On the other hand, algorithmic approaches often modify the loss functionto place greater emphasis on learning from minority samples. One prominentmethod is Focal Loss, which incorporates a focusing parameter, γ, to pri-oritize difficult examples, and a per-class weighting factor, α, to counteractimbalance [110]. While α is usually determined by the class distribution, itcan overlook the variability in sample quality. The focusing parameter isfrequently set to 2 but can be fine-tuned to optimize model performance.Recent advances have introduced additional loss functions tailored for im-balanced datasets. For instance, Label-Distribution-Aware Margin (LDAM)loss encourages larger margins for minority classes, aiming for better gen-eralization in imbalanced settings [35]. Another effective approach is theDistribution-Balanced Loss, which has shown significant improvements inmulti-label text classification by re-weighting the contributions of positiveinstances during training [195].In many real-world scenarios, noisy labels and class imbalance coexist,presenting a multifaceted problem that single-focus methods struggle tosolve. However, techniques aimed at mitigating label noise typically assumea balanced dataset, ignoring the complexities that arise in imbalanced data.Similarly, methods addressing class imbalance generally presume that labelsare accurate, failing to account for noise in the underrepresented classes. Insuch cases, label noise can exacerbate learning bias: when minority classescontain mislabeled samples, models may treat hard-to-learn but clean samplesas noise, neglecting genuine examples from the tail classes. This overlapcomplicates the modelâs ability to distinguish between true minority samplesand noisy samples from majority classes.Uncertainty is a concept that involves dealing with incomplete or imperfect162.2. DATA-CENTRIC AIinformation. In a machine learning project, there are numerous sourcesof uncertainty, such as the data values, the sampling method, and thelimitations of models created from the data. Aleatoric uncertainty is a typeof unknown that changes with each experiment, usually because of noisydata or limitations of the sensor. This does not mean that the information isnot present; it just means that the experiment does not detect it. Epistemicuncertainty, on the other hand, is not introduced by the experiment, but byhow the model learns. These could include a model that does not take intoaccount certain effects or is induced from incomplete data. Common sources ofuncertainty in machine learning are noise in observations, incomplete coverageof the domain, or an imperfect model of the problem. The variability in theobservations could be due to natural causes or it could be due to an error inthe measuring device.In all practical applications of machine learning, there will always bemissing observations; if this were not the case, an approximation model wouldnot be necessary. In addition, there is often an error in the model and theprocedure used to train it, including the selection and preparation of the dataand the choice of training hyperparameters. It is also common to deliberatelyselect models that are simpler but more robust to the specifics of the data,as opposed to complex models that may be highly specialized to the trainingdata. As such, we might choose a model known to make errors in the trainingdata set with the expectation that the model will generalize better to newcases and have better overall performance.Uncertainty can vary in a system over time. Machine learning commonlyassumes that the data used for training, testing, and inference is independentand identically distributed. However, this assumption may be compromised asthe target distribution evolves, leading to what is known as a distribution shift.Distribution shifts are generally divided into two types: covariate shift andsemantic shift. Covariate shift refers to changes in the input data distributionX over time, often due to factors such as environmental conditions such aslighting or proximity. Semantic shift involves changes in the distribution ofY, which may include the addition or removal of classes, or alterations inthe proportion of existing classes. It is important to note that covariate shiftis typically used to assess the generalization and robustness of models, withthe label space Y remaining constant at test time. In contrast, identifyingsemantic distribution shifts generally requires recognizing new classes [201].172.3. UNCERTAINTY AS PROBABILITY2.3 Uncertainty as ProbabilityA multiclassification logistic deep learning model takes the input data Xand produces a vector of probabilities of the example that corresponds toeach class. The final unbounded logits layer is clamped and weighted usingthe SoftMax functionsoftmax(zi) =eziâNj=1 ezj. (2.16)Binary cross-entropy is a commonly used loss function which encourages theclassifier to minimize outliers,L(y, yË) = â 1NNâi=1[yi log(yËi) + (1â yi) log(1â yËi)]. (2.17)The end result is a probability vector whereâNi=1 yi = 1. There are manyways to measure the uncertainty of such a classifier using only this probabilityvector. Given that the sample under consideration is in domain, it would beexpected that one of the classifier outputs would be at or near 1. Thus, abasic measure of uncertainty would be to calculateS = 1âmax(y), (2.18)where max denotes the highest probability of the output vector, and S isgenerally a heuristic non-conformity score that is generally higher whenuncertainty is higher [158]. However, such a method ignores the uncertaintyof the remaining classes. Imagine a four-class logistic classifier that pro-duces the following predictions for two different examples: {0.7, 0.3, 0.0, 0.0},{0.7, 0.1, 0.1, 0.1}. Intuitively, the latter should have higher uncertainty thanthe former, given the former is able to eliminate two classes as possibili-ties. However, from the basic uncertainty calculation above, both would beassigned the same score. We can improve this method by calculating thedifference between the highest and second-highest scores. Considering thetwo higest values in the probability vector gives S = max(y) â max2(Y ),where max2 denotes the second highest probability score of the output vector,and satisfies the above intuition. This takes the uncertainty one step further,but extending the counter-example above to consider the equality of the 3rd,4th, or so-on element becomes tiresome. Seeking a more comprehensive way,since each element ranges from [0,1], it is possible to calculate entropy,S = âkNâi=1yi log yi. (2.19)182.3. UNCERTAINTY AS PROBABILITY0 0.2 0.4 0.6 0.8 100.20.40.60.81xBinaryEntropy0246One-SidedEntropyEntropyFigure 2.1: Binary Entropy (orange), and Entropy (yellow, Y=0; red, Y=1)Entropy is interpretable as the surprise or uncertainty of the variable. In-tuitively, if the variable were to be very confident, one classification wouldbe at or near 1, and the remainder near 0. From Figure 2.1, S for theseclassifications would all be 0, resulting in overall entropy of 0. However, if theclassifier is highly uncertain, many classifications would be non-zero, resultingin a high entropy score.Uncertainty of new data points can also be quantified based on pastobservations using Frequentist or Bayesian statistics, with each able to providea Confidence Interval or Credible Interval (CI), respectively. CIs are measuresof uncertainty around effect estimates, and observed data outside of the CI canbe interpreted as unusual or surprising. Frequentist inference assumes that anull hypothesis is true before data collection and assesses how unexpectedthe new data are under this hypothesis. The data is surprising if there isa low probability of obtaining another data set as extreme by chance. TheP value does not indicate the hypothesis correctness or effect significance;high P values do not suggest that there is no effect. Rather, it representsthe probability of obtaining another data set as extreme by chance [62]. Afrequentistsâ CI defines the confidence that the unknown estimate would liewithin the lower and upper limits of the interval, based on hypothesizedrepeats of the experiment.Bayesian statistics stem from Bayesâ theorem, which calculates the proba-192.3. UNCERTAINTY AS PROBABILITYbility of an event based on prior knowledge. Bayesâ theorem inverts conditionalprobability using the probabilities of underlying variables. Specifically, theconditional probability of event A given B isP (A|B) = P (B|A) · P (A)P (B). (2.20)A key difference between frequentist and Bayesian inference is the use ofpriors, here denoted as P (A); frequentists rely solely on gathered data. Priorscan be based on expert opinions, historical data, or both. Defining a priorcan be challenging or ambiguous, and outcomes may depend on the selectedprior, potentially allowing manipulation. To address this, an âuninformativeâprior can be used, or multiple priors can be tested [62]. A Bayesian CI assignsa probability that the unknown estimate would lie within the interval, giventhe evidence provided by the observed data.Model Calibration Probabilistic uncertainty quantification methodsstruggle with poor model calibration. Recent studies show that modernneural networks with high predictive capability are poorly calibrated andproduce unreliable predictions. For instance, in medical diagnostics, a modelpredicting a 90% disease likelihood based on symptoms and history butsuch events happen approximately 80% of the time. Formally, a model isconsidered well calibrated if it satisfiesP (Y = 1 | PË (Y = 1) = p) = p. (2.21)The main source of calibration errors are the size of the overparameterizedDNN architectures. While increasing the depth and width of neural networkshelps to obtain high predictive power, it also negatively increases calibrationerrors by overfitting [78].Data-related issues can also lead to a poorly calibrated model, such asnoisy labels, as introduced in Section 2.2. Data imbalance, where one classor outcome is significantly more prevalent than others, can lead to biasedpredictions and poor calibration. This is because the model may becomeoverly confident in predicting the majority class while being less certain aboutthe minority classes. Focal loss has shown its effectiveness in calibratingmodels that trained with imbalanced datasets [110]. Finally, data bias, wherethe collection and sampling of data does not represent the true population,arises due to historical disparities, societal prejudices, or systemic inequalities.Facial recognition datasets, for exmaple, often exhibit racial and gender biases[28]. A simple way to mitigate bias is to oversample the underrepresentedclasses, often using Data Augmentation to introduce diversity [206].202.4. UNCERTAINTY AS SIMILARITY2.4 Uncertainty as SimilarityIt has been shown that any piecewise affine classifier function can generateextremely high-confidence predictions when far from the training data. Thisphenomenon is common in highly sensitive and overparameterized convolu-tional, residual, and ReLU networks [85]. In such a case, it is not possible tostrictly quantify the uncertainty by looking at the output of the network. An-other option is to analyze in feature space or input embedding, and comparesimilarity. Similarity is a commonly used metric in out-of-distribution (OOD)detection algorithms. Since neural networks can produce high confidencepredictions even when they are obviously wrong, methods relying on theclassifierâs output are unable to detect these inputs as OOD [85]. Prototypicalmethods generate a generalized representation of features which representin-distribution classes, then calculate the distance of examples to these proto-types at inference time. However, generating this prototype and selecting anappropriate distance metric is a challenging task and an open area of research.The simplest form of vector similarity is the Euclidean distance, or L2-norm,â¥X1,X2â¥ =ââââ nâi=1(X1,i âX2,i)2. (2.22)Euclidean distance is a good measure in lower dimensions; however, in higherdimensions, the metric becomes less useful, as small perturbations in manydimensions can lead to large Euclidean distances. In images, for example,a small change in brightness over the entire image can lead to very largedistance measurements. It is also highly dependent on the length of thevectors being compared; small angle changes for vectors near the origin aresignificantly less impactful than small angle changes for large vectors.The dot product measures the length of one vector projected onto another,X1 Â·X2 =nâi=1X1,iX2,i = â¥X1â¥â¥X2â¥ cos(Î), (2.23)where Î is the angle between the vectors. Like the Euclidean distance, the dotproduct depends on the length of the initial vectors, and in higher dimensionsit is an unreliable metric. Cosine Similarity removes this dependence oninitial vector-length by dividing it outX1 Â·X2â¥X1â¥â¥X2â¥ = cos(Î). (2.24)212.4. UNCERTAINTY AS SIMILARITYThe metrics mentioned previously are unsuited to application in thepixel-domain due to the high dimensionality and environmental sensitivity,which render pixel comparisons extremely volatile. However, promisingapproaches that use distance-based methods utilize DNNs to derive featureembeddings and detect OOD samples by calculating the distances betweenthese embeddings and the centroids or prototypes of ID classes [107, 127, 163].Four common algorithms for generating and comparing in the feature domainusing similarity are presented below.K-Nearest Neighbours K-Nearest Neighbours (KNN) is a transductivealgorithm that produces predictions directly from the source data withoutderiving an initial governing function. The training instances are representedas vectors within a multi-dimensional feature space, each tagged with a classlabel. The training stage involves only the storage of these feature vectorsand their corresponding class labels. During the classification stage, k, aconstant defined by the user, is used, and an unlabeled test point is classifiedby assigning it the most common label among the k nearest training samplesto that point. The optimal selection of k varies with the dataset; typically,higher k values decrease the impact of noise on classification but blur thedistinctions between classes. Effective k values can be determined usingvarious heuristic methods or hyperparameter optimization. The performanceof the KNN algorithm can be significantly reduced by noisy or irrelevantfeatures, or if the scales of the features do not align with their significance[95].T-distributed stochastic neighbor embedding T-distributed stochas-tic neighbor embedding (t-SNE) is a technique to reduce the dimensionalityof high-dimensional data to two or three dimensions. This nonlinear methodmaps each datapoint to a low-dimensional space such that similar objectsare represented by nearby points and dissimilar objects by distant pointswith high probability. The t-SNE algorithm consists of two stages. Initially,it creates a probability distribution over pairs of high-dimensional objects,assigning higher probabilities to similar objects and lower probabilities todissimilar objects. Then, it defines a similar probability distribution overthe points in the low-dimensional map and minimizes the divergence of thedistance distribution across representations. In the original paper [185], thesimilarity metric used in the higher dimension is the Euclidean distancebetween objects, but this can be any distance metric. While it is possible fort-SNE plots to find âclustersâ where none exist, with the right parameters,t-SNE is usually able to recognize clusters that are well defined.t-SNE aims to learn a d-dimensional mapping of {X1, . . . ,XN} â {Y1, . . . ,YN}(with Yi â Rd and d typically chosen as 2 or 3) that reflects the similarities222.4. UNCERTAINTY AS SIMILARITYbetween unknown data points as well as possible. A heavy-tailed Studentt-distribution is used to measure similarities between low-dimensional pointsin order to allow dissimilar objects to be modeled far apart on the map.The locations of the points yi in the map are determined by minimizing theKullbackâLeibler divergence of the distribution P from the distribution Q,that isKL (P â¥ Q) =âi Ì¸=jpij logpijqij, (2.25)where pij is the calculated similarity between pair xi and xj in the inputspace and qij is the similarity between pair yi and yj in the lower-dimensionalspace.Minimizing the KL divergence with respect to the points yi is carried outusing gradient descent. The result of this optimization is a map that reflectsthe similarities between the high-dimensional inputs.Siamese Neural Networks Siamese networks learn by comparing dif-ferent input instances to obtain discriminative features for downstream tasks,such as matching, classification and regression. Furthermore, they provide away to learn without labels, as they learn by comparison. Finally, Siamesenetworks can be used to enhance data in few-shot learning. By comparingtraining pairs, Siamese networks can generate pseudo-labels for unlabelleddata based on similarity to labelled data, which is essential for improvingaccuracy and robustness in few-shot learning [108].In a typical Siamese framework, learned functions F1, . . . , Fn deal withinput instances X1 . . .Xn, respectively. Assuming the simplest case of twobranches, the results of the feature extraction are represented by YË1 = F1(X1)and YË2 = F1(X2). In classification tasks, a comparison function Fc(YË1, YË2)is used to produce a diversity score D between the two predictions.The goal of siamese networks is that similar samples should have a similarrepresentation from all branches, whereas dissimilar samples should havedifferent representation. Given some diversity function D, the loss L is thencalculated asLD = 12YsD(YË1, YË2)2 + 12(1âYs)(M âD(YË1, YË2))2, (2.26)where M is a hyperparameter, and Ys is defined based on the ground-truthlabels: Ys = 1 if the groundtruth labels match, and Ys = 0 if they do notmatch. Thus, when the labels match, diversity is minimized; when they donot match, diversity is maximized [109].232.5. UNCERTAINTY AS DIVERSITYIt is common for the two branches F1 and F2 to share the same networkstructure and/or weight information W. However, this is not required, andmany algorithms deviate from this practice [108].Single Deterministic Networks Single Determinstic Networks are de-rived from Radial Basis Functions (RBF) and operate by building prototypicalrepresentations at the centroid of a class in feature space. These prototypesare learned and evaluated in a single training pass. During training, thedistance to the correct class is minimized, while the distance to all otherclasses is maximized. The class centroids are updated using an exponentialmoving average of the feature vectors of the data points belonging to thatclass. Two-sided Jacobian regularization simultaneously promotes smoothnessfor better training and sensitivity to prevent multiple inputs from collapsingto the same output. Using these methods, it is possible to achieve accuracycomparable to probabilistic softmax models (at least on simple problemssuch as MNIST versus FashionMNIST and CIFAR-10 versus SVHN) witha reliable measure of uncertainty. At inference, the class is taken as theclosest centroid to the embedded prediction and the uncertainty is quantifiedby the distance to the closest centroid. Most importantly, this method iscomputationally efficient and achieved in a single training pass [184].2.5 Uncertainty as DiversityUncertainty can also be defined as diversity between several predictionsgenerated from the same input data. These diverse predictions can then becombined, usually by averaging, to reduce the variance of the final prediction.The performance of a model is measured by a function M(YË,Y) on anindependent and identically distributed set unused in training. In particular,many different predictions often produce statistically similar results for thesame ground truth. Specifically, for two predictions,F(YË1, YË2) = F(F1(X;Î1, γ1), F2(X;Î2, γ2)), (2.27)where Î are the learnable parameters and γ1 is the architecture of the model.The function F is used to combine the underlying predictions; in practice,F is usually the average. For this pair of models, the combined performancecan be modeled as the performance of the underlying functions and thediversity between their predictionsM(F(YË1, YË2),Y) = f [M(YË1,Y),M(YË2,Y),D(YË1, YË2)], (2.28)242.5. UNCERTAINTY AS DIVERSITYwhere M is a function to determine the correctness of a prediction relativeto the ground truth, and D is a function measuring the diversity betweenpredictions.Trivial diversity can be achieved between an accurate prediction anda poor one; as such, diversity must be balanced with performance. Manysources of diversity have been explored in the literature, most commonlyvarying the data between models [26, 27]. Several works have also exploredthe suitability of varying activation function in ensembles [120, 132].Data-Resampling Validation, which is used to assess the suitabilityof a model, requires that some of the available data be withheld from thetraining process. This can have a detrimental effect on the training procedure,particularly if the data is limited. In a small data set, it is possible thatnot all features are present in many images; therefore, when removing datafrom the training set to use in the validation set, features can be eliminatedcompletely and never seen in the training. Cross-Validation is an evaluationtechnique where multiple trained models are produced; each model holdsback a different portion of the overall dataset for validation. The remainingdata in each fold is used in the training set. Each fold produces a differentmodel, and by combining the models after training, each piece of data canbe accurately represented in the final results. It is important to note thateach fold requires a full training run, which can be time consuming if manyfolds are used. On the other hand, if too few folds are employed, then a largeportion of the data set is held back in each fold, which can reduce the qualityof each trained fold [27].Bagging, also known as bootstrap aggregation, is a resampling techniquethat creates diverse samples by randomly selecting data points from thetraining dataset with replacement. This means that the same instancecan be chosen multiple times. These bootstrap samples are then trainedindependently and in parallel with weak or base learners. The final predictionis determined by combining the base learners by averaging, voting, or othermeans [26]. Similar to cross-validation, bagging can be time-consuming aseach weak learner requires a full training run. Research has shown thatthe weak learner must be constructed with an appropriate size to modelthe complexity of the problem; without suitable performance from the baselearners, techniques to grow through combinations of weak learners will notbe as effective as a single large learner [76, 77].Hyper-Deep Ensembles Deep ensembles promote diversity by changingthe optimized local minimum with different parameter initialization. Hy-perparameter ensembles produce diversity by changing the hyperparametersof the model. Hyper-deep ensembles simultaneously vary initialization and252.5. UNCERTAINTY AS DIVERSITYhyperparameters. When constructing hyperparameter ensembles, [192] usesa greedy algorithm first proposed by [37]. First, a set of models is trained apriori ; the model with the best performance is selected as a starting point.The ensemble is grown iteratively by adding in the model that best improvesperformance. When constructing hyper-deep ensembles, [192] employ a ran-dom search [17] to produce a performant model from varied hyperparameters;then they use these hyperparameters to generate multiple models from dif-ferent initializations. Finally, they apply the same greedy ensemble-growingalgorithm to select the best model. This is an expensive process, whichrequires many models at each iteration.Multi-Headed Networks Training and inferencing multiple modelsis computationally costly. Ensembles are particularly ill-suited to neuralnetworks for two main reasons. First, neural networks are generally moreexpensive to calculate than smaller models, such as decision trees. Second,splitting the data across models exacerbates overfitting, which neural networksare prone to do. Recent work has introduced the concept of creating ensemblesin a single training run. The authors of [190] present a mini-batch-friendlymethod to produce a shared weight matrix for all ensemble members, anda rank one independent weighting matrix for each ensemble member; theseare then combined through the Hadamard product to produce multiplepredictions. The authors of [210] employ a common feature extractor, fullyconnected shared layers, and multiple heads to produce diverse predictions.These methods have several advantages: first, the common elements of theprediction are calculated only once and then shared among members; second,the model is trained entirely at one time.Test-Time Augmentation Test-time augmentation (TTA) is a tech-nique in machine learning and computer vision that is used to enhance theperformance of models during inference. It involves applying transformationssuch as rotations, flips, and color adjustments to the input data to createmultiple augmented versions. The model predicts on each augmented input,and the final prediction is aggregated from these individual predictions, typi-cally by averaging or voting. TTA improves model robustness and accuracy,especially when the distribution of test data differs from the training data[92, 164]. However, care must be taken to ensure that augmentations donot corrupt the original data, as inconsistent augmentations can introducenoise and distort the data distribution. Balancing augmentation techniquesto avoid distribution shifts is challenging, and defining metrics to measureaugmentation effects is difficult. In intuitive domains like images or audio,humans can easily assess the similarity between original and augmented data,but this is harder with less intuitive data such as time series. Despite this,262.5. UNCERTAINTY AS DIVERSITYcounterintuitive augmentations have shown good results in experiments. Ran-dAugment randomly applies augmentation operations to images to improvemodel robustness and generalization by creating a diverse training dataset[45]. CutMix combines two training images by pasting a patch from oneonto another and mixing their labels proportionally, helping to regularize themodel and improve its generalization ability [206].Monte-Carlo Dropout and the Gaussian Process Dropout is com-monly employed in numerous deep learning models to prevent overfitting, andit is posited that dropout essentially averages the modelsâ weights. Multipleinferences on the same input image with random dropout at various layersin the network can create diverse predictions that capture uncertainty [24].Dropout and its variations in neural networks can be seen as a Bayesianapproximation to a well-established probabilistic model, namely the Gaussianprocess (GP) [66]. A GP provides predictive distributions at test pointsXâ1, . . . ,Xân given a set of training data {(X1,Y1), . . . , (Xn,Yn)}. Thesemodels assume that the ground truth labels are noisy, as shown in (2.1), butadd the assumption that the noise is normally-distributed, ϵi â¼ N (0, Ï2Ïµ ). Itis also commonly assumed that the mapped values Y are zero mean. A GPthen models the covariance matrix between samples using a kernel whichmeasures the similarity between two input points,Kxx = K(Xi,Xj |Î), (2.29)where Î are hyperparameters of the kernel function. The Kernel definesall a priori functions from which samples are drawn to create the posteriordistribution. Since these functions need to be able to model the data, it isessential that these kernel functions are composed with knowledge of the data;for example, whether the data are periodic or stationary determines whetherthe kernel must be periodic or stationary. The underlying assumption of aGaussian process is that Y and YË are a multivariate normal distribution[Yfâ]â¼ N([00],[Kxx + Ï2Ïµ I KxxâKxâx Kxâxâ]). (2.30)As can be seen, the kernel function is used to create a covariance matrix;the posterior distribution of fâ|Xâ then has mean Kxâx(Kxx + Ï2Ïµ I)â1 andvariance Kxâxâ âKxâx(Kxx + Ï2Ïµ I)â1Kxx. This provides both an expectedvalue and a measure of uncertainty about the prediction. One of the mainadvantages of Gaussian processes is that, given a suitable kernel, the bestparameters Î can be discoverable through optimization. However, in practice,GPs are limited by the calculation of the inverse matrix, which takes O(n3)272.6. UNCERTAINTY AS ROBUSTNESS AND SENSITIVITYtime and is prohibitively expensive in large dimensions. GPs also require akernel that fits the problem, which may be difficult to discover or realize inhigher-dimensional data. Finally, a GP must take into account all trainingdata each time it makes a prediction, which can be problematic if the trainingdata are regulated or sensitive and computationally expensive for a largenumber of samples.Bayesian Neural Networks Bayesian Neural Networks (BNNs) rep-resent a class of artificial neural networks that integrate Bayesian inferenceto quantify the uncertainty in their predictions. In contrast to conventionalneural networks, which yield point estimates, BNNs provide a probabilisticinterpretation of the model parameters, thereby facilitating more robustdecision making under uncertainty. By incorporating prior knowledge anditeratively updating beliefs with observed data, BNNs improve generalizationand attenuate overfitting [20, 134]. A notable challenge is the computationalcomplexity inherent in the training of BNNs, as they require sampling tech-niques such as Markov Chain Monte Carlo (MCMC) or variational inference,which are often computationally intensive and time-consuming [134].Variational Bayesian methods are a family of techniques in Bayesianinference that approximate probability densities through optimization. Thesemethods aim to find a simpler distribution that is close to the true posteriordistribution by minimizing the Kullback-Leibler (KL) divergence betweenthem (see Equation (2.25)). By minimizing this divergence, variationalBayesian methods provide a computationally efficient way to perform infer-ence in complex models. While recent advancements in variational inferenceand Monte Carlo methods have rendered the training of BNNs more compu-tationally viable, scaling BNNs to large datasets and intricate architecturesremains problematic due to the escalated computational demands [20]. Addi-tionally, selecting appropriate prior distributions poses a significant challenge,as it can profoundly influence the modelâs performance and convergence.Finally, the interpretability of BNNs is often problematic, given that theprobabilistic nature of the model introduces an additional layer of complexityin comprehending the learned representations and decision-making processes[7].2.6 Uncertainty as Robustness and SensitivityRobustness and sensitivity are critical aspects of neural networks thatdetermine their performance and reliability. Robustness refers to the abilityof a neural network to maintain its performance when subjected to various282.6. UNCERTAINTY AS ROBUSTNESS AND SENSITIVITYtypes of noise, perturbations, or adversarial attacks. A robust neural networkcan handle unexpected inputs and still produce accurate and reliable outputs.On the other hand, sensitivity measures how the output of a neural networkchanges in response to small variations in the input. High sensitivity canindicate that the neural network is overfitting to the training data, making itless generalizable to new, unseen data. Balancing robustness and sensitivityis essential for developing neural networks that are both accurate and resilientin real-world applications.Sensitivity The sensitivity of a model is generally defined as the changein output relative to the change in input; a model that generates largechanges in output for small changes to input is said to be highly sensitive.Mathematically, this concept is captured in the Lipschitz constant. Givena function f : Rn â Rm, f is Lipschitz continuous if its maximum rate ofchange is bounded by some non-negative constant; specifically, on the setÏ â Rn, the function must satisfy||f(x)â f(y)|| â¤ L||xâ y||, âx, y â Ï, (2.31)where L is a nonnegative constant; the smallest possible L is called theLipschitz constant; such a function f can be called L-Lipschitz, denoted asL(f). Intuitively, the Lipschitz constant gives the maximum change in theoutput given a fixed change in the input.The Lipschitz constant can be seen as a measure of robustness of thefunction f ; when the Lipschitz constant is small, large changes in the outputare possible only given large changes in the input. For simple, known functions,this bound can be calculated analytically. For more complex functions, suchas neural networks, it is common to seek an approximation or upper boundof L [61, 74].From [74], a worst-case upper bound on L can be found through theproduct of the Lipschitz constants of the layers from Equation (2.7)L(X) â¤lâi=1L(f(i)). (2.32)Notably, this worst-case upper bound can be very pessimistic, even ifthe Lipschitz constants of the underlying layers are the best possible. Still,an understanding of the Lipschitz constants of the underlying layers can beuseful in informing the sensitivity of the network. For fully connected layersof the formf(X) =WX+ b (2.33)292.6. UNCERTAINTY AS ROBUSTNESS AND SENSITIVITYthe Lipschitz constant can be derived from Equation (2.31), yielding theoperator norm of WL(f) = supX1âX2 Ì¸=0â¥W(X1 âX2)â¥pâ¥X1 âX2â¥p . (2.34)Common norm values p include the 1-norm (sum of parameter absolutevalue), 2-norm (Euclidean distance), and â (maximum value); while techni-cally not satisfying the properties of a norm, a value of 0 is also commonlyused in machine learning to denote the count of non-zero parameters. Theauthors continue with similar analysis for convolutional layers. They furthershow that L for pooling, ReLU activation, and softmax layers is 1.Adversarial Robustness The ability to produce good predictions fromadversarial examples is a concept now known as robustness. Szegedy et al.used box-constrained L-BFGS with line search to find the smallest L2 normperturbation that caused incorrect classification of ImageNet examples [176].Goodfellow et al. proposed the Fast Gradient Sign Method (FGSM), a stepgradient update approach to quickly find the optimal perturbation valuefor a non-targeted adversarial attack [73]. Deepfool defines the robustnessof a binary classifier as the distance to the decision boundary. For affinebinary classifiers, this boundary is a hyperplane, and a closed-form solutionexists to find the minimum disturbance required for a prediction to crossthis hyperplane. This approach is extended to multiclass and nonlinearfunctions, with the disturbance calculated using an iterative approach on alinear approximation of the function around the test point [130].Adversarial attacks can be classified based on the frequency of the attack,falsification method (increasing false positives or false negatives), prior knowl-edge of the network architecture and parameters, and whether the disturbanceis applied to change the network output to a specific or random prediction.Comprehensive categorizations are provided in [3, 205]. The robustness of anetwork can be defined by two parameters, the size of some perturbation tothe input δ (defined as â¥xâ xââ¥) and the change in the classification outputϵ. A classifier can be robust to perturbations up to a size if the resultingclassification is not changed,â¥xâ xââ¥â â¤ ϵ, f(x) = f(xâ). (2.35)Pointwise robustness is defined as the smallest perturbation for which alltest examples within the perturbation distance still result in correct label,minÎ´âRd{|Î´| : f(x+ δ) Ì¸= f(x)}. (2.36)302.6. UNCERTAINTY AS ROBUSTNESS AND SENSITIVITYAdversarial frequency quantifies the count of adversarial examples withina specified distance from a test sample that possess a different label thanthe test sample. Essentially, if f exhibits high adversarial frequency, it lackspointwise robustness for numerous example points. Adversarial severity isdefined as the maximum alteration in predicted output given a maximumperturbation. Frequency and severity reflect distinct robustness character-istics. A neural network might exhibit high adversarial frequency but lowadversarial severity, suggesting that most adversarial examples are approxi-mately a certain distance from the original point. On the other hand, a neuralnetwork might display low adversarial frequency but high adversarial severity,indicating general robustness with occasional severe robustness failures [11].There are several common defenses to adversarial attacks that can beimplemented at training time. Lipschitz margin training enlarges the guardedarea around data points in the data distribution where the network is provablyrobust. The guarded area of every data point is defined by a hyperspherearound the point, the radius of which is enlarged by maximizing the numberof robust sample points during the training process [183]. Gouk et al. describea method for constraining the Lipschitz constant. Their proposed methodis founded on projected Stochastic Gradient Descent (SGD), where at eachstep they project the weight matrices of each layer back to the closestvalue which maintains the required Lipschitz constant. They further showthat, when using batch normalization, the γ parameter present in the lineartransformation portion must also be constrained. Excitingly, they empiricallyconfirm that the introduction of the inductive bias of the Lipschitz constraintspeeds up training [74].Defensive distillation is the retraining of a network based on the softoutputs of a trained network. More specifically, a network is first trainedusing hard labels of the output classes. The output probability vectors ofthe trained network are then used to train another network with the samestructure. This explicit relative knowledge increases the generalizability ofthe newly trained network [140]. Distillation is also an important tool whendealing with partially or poorly labeled data. It involves the transfer ofknowledge from a large model to a small model, which requires teaching thelatter without compromising its validity. If both models are trained on thesame data, the small model may not have the capacity to learn a conciseknowledge representation with the same computational resources and dataas the large model. However, the function learned from the complex datacan provide information about a concise representation of knowledge. Thedistribution of values among the outputs of a record can indicate how the largemodel represents knowledge. Therefore, the goal of economical deployment312.7. CONFORMAL PREDICTIONof a valid model can be achieved by training only the large model on thedata and taking advantage of its better ability to learn concise knowledgerepresentations; this simpler function is then distilled into the smaller model,by training it to learn the soft output of the large model [86, 139].2.7 Conformal PredictionAll of the measures of uncertainty presented until now are heuristic innature. There is some metric (distance, similarity, probability, etc.) that canbe interpreted as more or less uncertain depending on its value. However,none of these metrics provide any guarantees regarding the identificationof uncertain samples. Conformal prediction (ConP) is able to provide thisnotion of rigorous uncertainty through a theoretically sound calibrationmethod, which generates prediction sets which guarantee marginal coverage.Conformal Prediction can employ any heuristic notion of uncertainty, and isparticularly useful when detecting covariate shift (see Figure 2.2).Figure 2.2: Conformal Prediction, applied on top of heuristic notions ofuncertainty to produce rigorous marginal coverage.ConP begins with a fitted predictive model and creates prediction setsusing additional calibration data. The predictive model outputs estimatedprobabilities for each class and a set of unseen images are used as calibration322.7. CONFORMAL PREDICTIONdata. A prediction set of possible labels is constructed, where the probabilitythat the set contains the correct label is almost exactly 1 minus a user-chosenerror rate. This property is called marginal coverage. The construction of theprediction set involves a calibration step that uses a defined non-conformityscoring function; this non-conformity score should be high when the modelis wrong. This method guarantees prediction sets that satisfy the validitycondition, regardless of the model or data distribution.ConP involves using a pre-trained model, such as a neural network clas-sifier, and generating prediction sets using some calibration data. ConP isapplicable in various domains, including computer vision and natural lan-guage processing, converting heuristic uncertainties into statistically rigorousmeasures.Conformal Prediction constructs a prediction set P where [4]1â α â¤ P (YË â P)) â¤ 1â α+ 1n+ 1. (2.37)ConP is applied in three steps [4]:1. Define a heuristic non-conformity score function s(x, y) â R, wherehigher scores represent higher uncertainty that x is y.2. Compute qË as the (n+1)(1âÎ±)n quantile of the calibration scores s1 =s(X1, Y1), ..., sn = s(Xn, Yn).3. Use this quantile to form the prediction sets for new examples: C(Xtest) ={y : s(Xtest, y) â¤ qË}.Common non-conformity scores into the Least-Ambiguous set-valuedClassifier (LAC), taken as 1 â YËc, where YËc is the logistic output of thecorrect label. LAC is simple and cheap to calculate, but still provides rigorousguarantees on marginal covarage [158]. The Adaptive Prediction Set (APS)method sums all logistic outputs which are higher than the ground truthclass,s(X,Y) =kâj=1f(Xi)Ïj , (2.38)where YËy = Ïk and k is the ground truth correct class. Quantiles arecalculated in the same way as step 2 above. At inference, the logistic outputis again sorted from largest to smallest, and classes are included until thecumulative sum of the logistic outputs meets the target quantile [154]. UsingLAC, it is possible at inference that none of the logistic outputs are greater332.8. UNCERTAINTY AND FAIRNESSthan the target quantile; in such a case, the prediction set would be empty.APS alleviates this concern, always producing prediction sets with at leastone item.Recent studies have leveraged the computational efficiency and theoreticalassurances of ConP [188] during the training phase. Notably, [54] introducesa differentiable approximation of the cumulative distribution function (CDF)for softmax scores. This approximation is subsequently utilized to computea loss function that integrates uncertainty from the calibration set duringtraining. The researchers demonstrate that this approach can result inmore compact conformal prediction sets with improved conditional coverage,following precise calibration using reserved data.2.8 Uncertainty and FairnessThe increasing integration of AI into critical domains like healthcare andself-driving cars holds immense promise for societal advancement. However,it also brings to the crucial challenge of ensuring fairness and mitigatingbias within these systems. Fairness in AI is a multifaceted concept, oftenfocusing on ensuring that predictions and decisions do not unfairly disadvan-tage particular groups defined by sensitive attributes such as gender, age,socioeconomic status, or other characteristics.Bias in AI can originate from various sources. One primary source is databias [148], where the data used to train AI models is not representative ofthe real-world population or reflects historical discrimination. For instance,in healthcare, an AI model trained to predict the risk of a certain diseaseprimarily on data from a male population may exhibit reduced accuracywhen applied to female patients. Similarly, in self-driving cars, a pedestriandetection system trained on datasets with a disproportionately low numberof images of individuals using wheelchairs may have a lower detection ratefor these groups, increasing their risk of accidents. Algorithmic bias [40] canbe introduced through the design and structure of the algorithm itself, evenwith balanced data. In self-driving cars, for example, an algorithm mightassign different values to the safety of different road users.Subgroup stratification is a crucial method for identifying and understand-ing fairness issues. This involves identifying sensitive attributes, dividingthe dataset by these attributes, and evaluating the modelâs performanceand fairness metrics within and across subgroups. If stratification shows ahealthcare AI for disease diagnosis has lower accuracy for a group, it revealsthe modelâs reduced effectiveness for this group. This finding can lead to mit-342.8. UNCERTAINTY AND FAIRNESSigation like collecting more diverse data, re-sampling techniques, developingfairness-aware algorithms, or post-processing to adjust predictions for theaffected subgroup. Fairness can be viewed through intergroup and intragroupperspectives.Intergroup fairness ensures different predefined groups receive similaror equitable outcomes. Metrics like disparate impact and equalized oddsprimarily measure intergroup fairness [187]. Disparate impact assesses if theselection rate (proportion receiving a positive outcome) differs significantlybetween groups. In healthcare, an AI system approving patients for a limitedtreatment shows disparate impact if the approval rate for female patientsis significantly lower than for male patients. In self-driving cars, disparateimpact in pedestrian detection could be the ratio of detection rates for adultversus child pedestrians. Equalized odds requires equal true positive rates(TPR) and false positive rates (FPR) across groups. TPR is the correctidentification rate of positive cases, and FPR is the incorrect identificationrate of negative cases as positive. A healthcare AI predicting disease riskshould have similar TPRs and FPRs for different patient groups. If it correctlyidentifies a condition in fewer female than males, or incorrectly predicts it inmore healthy female than male patients, it violates equalized odds.Intragroup fairness ensures individuals within the same group are treatedsimilarly, especially when similar in relevant non-sensitive traits, addressingdisparities even if intergroup fairness exists [187]. For a self-driving carâspedestrian detection with intergroup fairness across age groups, intragroupunfairness arises if, within adult pedestrians, detection is significantly lowerfor those wearing reflective clothing at night, a trait unrelated to age.Fairness in AI aims to ensure that automated systems do not discriminateagainst individuals or groups based on sensitive attributes, striving for equi-table outcomes. Uncertainty quantification, on the other hand, is the processof identifying, quantifying, and reducing uncertainties in model predictionsand data. The two are deeply related because the presence of uncertainty cansignificantly impact fairness. High uncertainty in predictions, particularly ifit disproportionately affects certain demographic groups, can lead to unfairoutcomes even in models that appear fair based on average performancemetrics. Quantifying and understanding the sources of this uncertainty iscrucial for identifying potential biases and developing more robust and fairAI systems, whether the uncertainty stems from noisy data, limitations inthe model, or lack of data for specific subgroups.35Chapter 3Epistemic Uncertainty in BreastCancer ScreeningMammogramsIn this chapter, we document external validation of mammography imagebased artificial intelligence (AI) risk prediction models. This work wasdone in co-ordination with BC Cancerâs organized screening program, wherevalidation of AI is essential before clinical implementation. We analyze digitalscreening mammograms for 136,700 women who underwent breast screeningin British Columbia, Canada during 1 February 2019 - January 31 2020;the data includes mammographic findings and imaging features, as well aspatient demographics and risk factors. The overall performance of the AIalgorithm measured with AUC was found to be in keeping with previousstudies of different populations at 0.938 (0.927-0.949). Fairness is evaluatedfor subgroups in 12 different categories, by comparing the AUC on a 95%confidence interval. The tested commercial AI algorithm is generalizablefor a large external cohort from Canada; it is especially promising whendetecting very early cancers, where it outperforms radiologists on a three-yearfollow-up.3.1 IntroductionBreast cancer is the second leading cause of cancer-related fatalities amongwomen, accounting for 13% of cancer deaths [32, 175]. In breast screening,radiologists analyze mammographic images for signs of cancer. Each screeninghas four standard views. Cranialcaudal, sometimes also referred to as CranialCaudal or CC, is taken directly from above. The mediolateral oblique viewor MLO, is taken from above, but angled. Each of these views are takenon both the left and right breast. An example of these 4 views is shown inFigure 3.2. Radiologists report on properties of the image, as shown in Figure3.3, and if any unusual findings such as asymmetry, architectural distortion,363.1. INTRODUCTIONFigure 3.1: Example features reported by radiologists at mammographicscreening.mass, and calcifications are observed (see Figure 3.1), the patient is recalledfor follow-up.Mammography screening has notable drawbacks, including a high false-positive rate [21] and occurrences of missed cancers [88], which are 8.4%and 9.9% at BC Cancer, respectively [14]. Double reading of mammogramsby independent radiologists can improve cancer detection compared to thetraditional single-reader assessment [18, 43, 81]. Consensus discussion orarbitration of discordant readings between radiologists can also contributeto lowering the recall rate for further assessment while maintaining highlevel of accuracy in cancer detection [84, 87]. However, such methods arecostly, and Canada employs single reading for screening mammograms asopposed to the double reading approach common in the UK and Europe.Recent studies from other countries have shown that performance of artificialintelligence systems is comparable to that of radiologists in breast cancerdetection [91, 103, 118, 124].In the Canadian Province of British Columbia, the BC Cancer BreastScreening Program has long met the national cancer detection rate benchmark;however, reducing recall rate has been identified as a priority by the CanadianStrategy for Cancer Control [33]. From a clinical perspective, the reductionof false-positive screening mammograms is important to improving screeningretention [33, 165]. BC Cancer reports a recall rate of about 20% for initialscreenings and 8% for subsequent screenings, with an interval cancer rate ofroughly 0.7 per 1000 women screened [14]. False positives cause considerableemotional stress, decreasing the likelihood of women attending subsequentscreenings [21]. In British Columbia, from 2016-2018, the overall return rateafter 36 months for women aged 50-69 with abnormal results was 60%, whichis 21% lower than those with a normal screen [14].Data from several countries shows that false-positive mammogram signifi-cantly increases the relative risk of detecting cancer in subsequent screenings373.1. INTRODUCTIONFigure 3.2: Example Mammogram. CC and MLO views are taken of boththe left and right breast for the radiologist to review.Figure 3.3: Radiologist Report. Radiologists examine mammograms toidentify imaging features asymmetry, architectural distortion, mass, andcalcifications, as well as other information such as breast density. If any ofthe 4 imaging features are observed, the patient is recalled for follow-up.383.2. MATERIALS AND METHODS[38, 123, 149]. A 30-year analysis of the BC Cancer Breast Screening Pro-gram Registry Data revealed that an abnormal mammogram increased therisk of a future cancer diagnosis by a factor of 1.73 [149], perhaps due tomisclassification of malignancies at the time of the screening [38]. Recentworks indicate that AI may be able to identify developing cancer 4 to 6 yearsbefore eventual detection [68].The purpose of this study was to evaluate the performance of commercialAI algorithm for breast cancer detection on a digital mammography datasetfrom the BC Cancer Breast Screening Program. The dataset analyzed includesrich demographic information, allowing stratified analysis over features suchas patient ethnicity, family history, breast density, BMI, and other knownrisk factors for breast cancer. we also investigate the potential of AI andradiologist double reading at a 1-year follow-up to reduce high recall ratesand radiologist workload. Limited analyses at two and three-year follow-upwere also performed.3.2 Materials and MethodsThis retrospective study evaluates the performance of a commercialartificial intelligence system for screening breast cancer detection using acohort of 136,700 digital mammograms from the BC Cancer Breast ScreeningProgram performed between February 2019 and January 2020.Radiologists report breast density using the Breast Imaging Reportingand Data System (BI-RADS) lexicon [168]. The AI algorithm can optionallyreport on breast density using the same lexicon. Breast density is a knownrisk factor for breast cancer [25], and was evaluated here.This study was approved by BC Cancer â UBC Research Ethics Board(REB H20-03683). The requirement for informed consent was waived due touse of de-identified data in this study.3.3 Study SettingThe BC Cancer Breast Screening program was launched in 1988, and hasprovided over 6,575,977 screens through 2020, while detecting 29,292 cancers[14]. Annually, the program performs approximately 250,000 studies at 36fixed centers and 3 mobile units, which visit over 170 smaller communitiesacross British Columbia.Women aged 40 years or older who are at average risk for breast cancermay screen every two years, while women at higher-than-average risk, such393.4. DATASET COMPOSITIONthose with a family history of breast cancer, are recommended to screenannually. Those with a personal history of breast cancer are not eligible toscreen within the program. A screening mammogram consists of standard4-view mammography.The screening mammograms are reported by program qualified radiolo-gists who must report a minimum of 2,500 screens annually. Single readerreporting is performed, and studies are either reported as normal or abnormal.Individuals diagnosed with cancer, including interval cancers, are ineligiblefor program screening and must arrange for diagnostic surveillance.3.4 Dataset Composition150,301 mammography studies were collected, with imaging performedbetween February 2019 and January 2020. Since digital mammographyimages were only available for women who underwent screening at 23 of the39 provincial sites, this represents a subset of the population. The AI systemgenerated predictions for 149,884 (99.7%) mammograms; the remaining didnot have the required standard views. 5,893 studies were excluded becausethey could not be matched to the outcomes data; since images were extractedfrom sites that are performing both screening and diagnostic mammography,these were likely diagnostic imaging studies.7,657 studies contained more than the four-standard view (left and rightcraniocaudal and left and right mediolateral oblique) images. Because the AIis not able to process these views, these studies were excluded from evaluation.Four studies were excluded as they contained only a single view.After these exclusions, 136,700 studies (91.0%) with associated outcomesdata and AI predictions remained for evaluation. Included in the studywere 841 (0.61%) positive cases of cancer at 1-year follow-up, and 1,039(0.76%) at 3-year follow-up. Screen-detected and interval cancers are based onpathologically confirmed diagnosis and are not distinguished in our analysis.The BC Cancer Screening Program Registry Data, including pathologicaldiagnosis of both screen-detected and interval cancers, was extracted onAugust 21, 2023. 247 cancers were screen-detected after January 2020 butbefore outcomes were extracted. The full data flow diagram is shown inFigure 3.4. Data processing and cleaning was done using Python 3.8.16 andPandas 1.5.2.403.5. AI ALGORITHM3.5 AI AlgorithmA pre-trained commercial AI algorithm Lunit MMG version 1.1.2.0 wasused in this study. None of the mammograms included in the evaluation hadbeen seen by the AI before this study. The images were de-identified and fedto the algorithm running on a GeForce RTX 2080 GPU computer with 11GB RAM.Figure 3.4: Data Flow Diagram. 150,301 screens and 150,254 outcome datarecords were found for our study date range. After matching these two sourcesand filtering on screen date and 4 standard views, 136,700 screens are usedin this study.413.6. STATISTICAL ANALYSIS AND COMPARISON WITH RADIOLOGIST PERFORMANCE3.6 Statistical Analysis and Comparison withRadiologist PerformanceRadiologists report confidence in abnormality as low, medium, or high.For the purposes of this analysis, any abnormal report was taken as a positiveidentification regardless of confidence. Abnormal assignments are furthercharacterized by lesion location and feature type. Feature type is categorizedinto the standard mammographic findings of Calcifications, ArchitecturalDistortion, Mass and/or Asymmetry. Analysis of mammographic featuresis based on the human reader assignment. In Canada, an interval breastcancer is evaluated as a post-screen cancer found after a normal screeningepisode within 0 to < 12 and, separately, 12-24 months of the screen date[14, 29, 34, 143]. Canadian guidelines for monitoring breast cancer screeningprogram performance recommend reporting only on post-screen cancersdiagnosed within 12 months to allow for better comparison between annualand biennial screening recommendations, which vary by province and, in BC,self-reported family history [143]. Therefore, our primary analysis focuses ona 1-year follow-up duration.The AI generates exam-level AI scores ranging from 0 â 1. Radiologistperformance was evaluated using sensitivity and specificity; AI performancewas also evaluated on these 2 measures, using a manufacturer-recommendedthreshold of 0.10 for binary classification, where scores above that thresholdindicates an abnormal examination.Post-hoc subgroup analysis of the AI was done using Area Under theCurve (AUC) of Receiver Operating Characteristic (ROC) methodology on a95% confidence interval, with the null hypothesis that the AI performs equallywell on all subgroups in a stratification. Additional details of the statisticalanalysis are available in supplementary material. For all comparisons, a p-value < 0.05 indicated a statistically significant difference. Statistical analysiswas performed with R 4.1.1.Calculations are done using pROC [153], which implements DeLongâsmethod [47, 172] with 2,000 bootstrap samples for generating confidenceintervals, where the bootstrap replicates contain exactly the same number ofcontrols and cases as the original sample. The implementation can compareunpaired ROC curves, which we use throughout this study, where the p-valueis computed as an unpaired t-test with unequal sample size and unequalvariance withD =V r (Îr)â V s (Îs)âSr + Ss. (3.1)423.7. RESULTSSince recent work has identified that AI and radiologists find differentcancers, we consider hypothetical combinations of AI and Radiologist; wereport specificity, sensitivity, and number of recalled screenings when bothreaders must agree, and where readers can independently identify a caseas abnormal. Moreover, since recent works have indicated that relativeperformance between AI and Radiologist depends on follow-up duration, weanalyze sensitivity using follow-up durations from 90 days to 3 years. Thisanalysis includes 2-year follow-up, which is the screening interval for averagerisk women and is also clinically relevant. Analysis of 3-year follow-up ismotivated by recent works which indicate that AI may be able to identifydeveloping cancer 4 to 6 years before eventual detection [68]; longer follow-upsalso allow a fairer comparison between AI and radiologists, where abnormalfindings by the radiologist were able to impact time to diagnosis whereas AIfindings were not. However, since 3-year follow-up is not clinically relevant,limited subpopulation analysis is provided as an online supplement.I calculate standard error for sensitivity and specificity for both Radiolo-gists and AI as:SE (sens) =âsens · (1â sens)TP + FN;SE (spec) =âspec · (1â spec)FP + TN. (3.2)The 95% confidence interval presented in Table 3.1 is then calculated as1.96 · SE.I compare Radiologist and AI sensitivity and specificity using pooledvariance as:z =pÌ1 â pÌ2âpË (1â pË) 2n, (3.3)Where p1 = TP1, p2 = TP2, pË =(TP 1+TP 2)2n , and n is the number of groundtruth positive cancers. A one-sided p-value is then taken using the pnormfunction in R.3.7 ResultsAt 1-year follow-up, the overall AUC of the AI algorithm was 0.93 (95% CI:0.92 â 0.94). Prediction residuals were centered near 0; the residuals less than 0indicate the relatively small proportion of true-positive examples in the dataset(841 of 136,700, 0.61%) (Figure 3.5). However, when binary classificationwas performed using the 0.10 manufacturer-recommended threshold, AI had433.7. RESULTSFigure 3.5: Overall Results. Circle indicates radiologist operating point anddiamond indicates AI operating point at manufacturerâs specified threshold(0.10).lower sensitivity (89.4%) compared with radiologists (92.6%; p=.01). AI alsohad lower specificity (78.5%) compared with radiologists (91.8%; p < .001).Dataset Characteristics The dataset consists of 136,700 patients (age:µ=58.8, Ï=9.4, M=59.0, IQR=14.0) who underwent breast cancer screeningat BC Cancer between February 2019 and January 2020. These patientswere taken from a cohort of 150,301 mammographic studies during thattimeframe, with exclusions made for studies that had no recorded screeningdate or more or fewer views than standard 4-view mammography. The fulldata flow diagram is shown in Figure 3.4. Subgroup analysis is performedover 13 stratifications including: 7 patient features such as age, ethnicity,and self-reported family history of cancer; 4 mammographic imaging featuresof calcification, asymmetry, architectural distortion, and mass. The final 2stratifications are manufacturer of the mammographic imaging equipmentand whether the radiologist had access to the previous mammogram forthe patient at reading. These stratifications, including counts and positiveinstances of cancer in each subgroup, are shown in Table 3.1.Family History The AI achieved a higher AUC in women with no familyhistory of breast cancer (0.94 (95% CI: 0.93-0.96)) versus in women withfamily history of breast cancer (0.90 (95% CI: 0.88-0.93); p = 0.003).Breast Density Radiologist and AI performance in breast density classifi-cation is shown in Table 3.1, and AI performance based on radiologist-assignedbreast density classifications is further illustrated in Figure 3.6. In 97,687443.7. RESULTSTable 3.1: Results of Commercial AI For Mammography Screening at 1-yearFollow-UpArtificial Intelligence RadiologistStratification n pos ROC AUC Sens. (%) Spec. (%) Sens. (%) Spec. (%)Overall 136,700 841 0.93[0.92-0.94] 89.4± 1.1 78.5± 0.1 92.6± 0.9 91.8± 0.1BMI1Underweight 3,459 19 0.82[0.67-0.96] 78.9± 9.3 80.2± 0.7 84.2± 8.3 90.2± 0.5Normal 40,442 223 0.93[0.91-0.96] 88.8± 2.1 80.1± 0.2 91.5± 1.9 92.0± 0.1Overweight 28,295 182 0.94[0.92-0.96] 91.2± 2.1 75.4± 0.3 94.5± 1.7 91.8± 0.2Obese 17,365 120 0.95[0.92-0.97] 92.5± 2.4 75.2± 0.3 94.2± 2.1 92.4± 0.2Unknown 47,139 297 0.90[0.85-0.96] 88.2± 1.9 80.2± 0.2 92.3± 1.6 91.4± 0.1Age(y)1<40 167 0 - - - - -40-49 25,500 103 0.91[0.88-0.95] 86.4± 3.3 81.4± 0.2 90.3± 2.9 87.3± 0.250-59 45,154 240 0.93[0.91-0.95] 88.3± 2.1 80.5± 0.2 92.1± 1.7 92.0± 0.160-69 45,675 311 0.93[0.91-0.95] 89.7± 1.7 77.6± 0.2 95.5± 1.2 93.2± 0.170-75 17,427 143 0.99[0.91-0.96] 91.6± 2.3 73.2± 0.3 89.5± 2.6 93.9± 0.275+ 2,777 44 0.90[0.85-0.96] 93.2± 3.8 68.5± 0.9 90.9± 4.3 92.5± 0.5Prev. Mammo3Compared 120,463 714 0.93[0.92-0.94] 89.1± 1.2 78.3± 0.1 92.4± 1.0 93.2± 0.1Not Compared 15,369 123 0.95[0.93-0.98] 93.5± 2.2 80.5± 0.3 95.9± 1.8 80.7± 0.36Unknown 868 4 0.53[0.17-0.89] 25.0± 21.7 78.7± 1.4 20.5± 21.7 86.0± 1.1Family History2Yes 31,116 207 0.90[0.88-0.93] 85.5± 2.4 76.0± 0.2 87.0± 2.3 93.1± 0.1No 94,039 571 0.94[0.93-0.96] 91.2± 1.2 79.3± 0.1 95.3± 0.9 91.8± 0.01Unknown 11,545 63 0.90[0.85-0.96] 85.7± 4.4 78.9± 0.4 87.3± 4.2 87.7± 0.3Menop.2Pre- 26,932 145 0.94[0.92-0.96] 88.3± 2.7 80.6± 0.2 89.7± 2.5 88.0± 0.2Post- 95,027 620 0.93[0.92-0.94] 90.0± 1.2 78.0± 0.1 93.7± 1.0 92.9± 0.1Unknown 14,741 76 0.94[0.85-0.96] 86.8± 3.9 78.4± 0.3 89.5± 3.5 91.6± 0.2Breast Density3A 16,815 59 0.96[0.94-0.99] 93.2± 3.3 86.7± 0.3 96.6± 2.4 95.3± 0.1B 61,007 376 0.94[0.92-0.95] 90.4± 1.5 77.3± 0.2 94.7± 1.2 92.0± 0.1C 50,656 359 0.93[0.91-0.95] 89.7± 1.6 76.9± 0.2 91.4± 1.5 90.6± 0.1D 8,220 47 0.84[0.76-0.91] 74.5± 6.4 80.9± 0.4 80.9± 5.7 90.1± 0.3Unknown 2 0 - - - - -Sympt.2Yes 4,596 83 0.97[0.94-0.99] 97.6± 1.7 76.3± 0.6 92.8± 2.8 87.8± 0.5No 131,626 758 0.93[0.92-0.94] 88.5± 1.2 78.6± 0.1 92.6± 1.0 91.9± 0.1Unknown 478 0 - - - - -Calc.3 Yes 2,321 328 0.87[0.85-0.90] 91.8± 1.5 51.4± 1.1 -* -*No 134,379 513 0.92[0.91-0.94] 87.9± 1.4 78.9± 0.1 87.9± 1.4 93.1± 0.1Asy.3 Yes 7,191 289 0.92[0.90-0.94] 92.5± 1.5 66.6± 0.6 -* -*No 129,509 518 0.94[0.92-0.95] 87.8± 1.4 79.2± 0.1 88.7± 1.4 96.7± 0.1Arch Dist.3 Yes 1,317 187 0.96[0.94-0.98] 97.9± 1.1 58.4± 1.5 -* -*No 135,383 654 0.92[0.90-0.93] 87.0± 1.3 78.7± 0.1 90.5± 1.1 92.5± 0.1Mass3 Yes 2,643 266 0.94[0.92-0.96] 94.4± 1.4 64.8± 1.0 -* -*No 134,057 575 0.92[0.90-0.93] 87.1± 1.4 78.8± 0.1 89.2± 1.3 93.4± 0.1Manuf.1GE 13,146 76 0.93[0.89-0.96] 78.9± 4.7 93.8± 0.1 96.1± 2.2 90.7± 0.2Hologic 117,736 725 0.94[0.92-0.95] 90.2± 1.1 77.7± 0.2 92.0± 1.0 91.7± 0.1Siemens 5,818 40 0.97[0.94-0.99+] 95.0± 3.4 60.5± 0.6 97.5± 2.5 94.7± 0.3Note.â Sensitivity and specificity data are presented as mean Â± standard error. *Strati-fication on imaging features was done on radiologistsâ assessments. Since all abnormalfeatures are sent for follow-up, radiologistsâ sensitivity is always 1.000 and specificity isundefined. Abbreviations are as follows: pos indicates the number of diagnostic confirmedcancer cases; AUC = area under the receiver operating characteristic curve; BMI indicatesBody Mass Index; Prev. Mammogram indicates the radiologist was able to comparethe previous mammogram with the current screening; Menop. indicates whether thepatient was pre- or post-menopause; Sympt. indicates whether or not the patient wassymptomatic at the time of screening; Calc. indicates radiologist identified calcificationsin the screening; Asy. indicates radiologist identified asymmetry in the screening; Arch.indicates radiologist identified architectural distortion in the screening; Mass indicatesradiologist identified mass in the screening; Manuf. Identifies the manufacturer of themammography equipment. 1 â Recorded at patient intake. 2 â Self-reported by the patient.3 â Recorded by the Radiologist. 453.7. RESULTSFigure 3.6: Stratified AUC - Patient Risk Factors. Circle indicates radiologistoperating point and diamond indicates AI operating point at manufacturerâsspecified threshold (0.10).cases (71.5%) the AI and the radiologist agreed in their determination ofbreast density; in a further 38,801 cases (28.4%) the disagreement consistedof adjacent classifications (A and B, B and C, or C and D). With eitherdetermination, AI detection performance on breast density D significantlyunderperformed; this is consistent with the radiologist performance. AI AUCfor patients stratified by radiologist-determined BI-RADS breast densitycategories were A: 0.96 (95% CI: 0.94-0.91); B: 0.94 (95% CI: 0.92-0.95); C:0.93 (95% CI: 0.91-0.95) and D: 0.84 (95% CI: 0.76-0.91) (AAUC > DAUC , p= 0.002; BAUC > DAUC , p = 0.009; CAUC > DAUC , p = 0.02).Mammographic Features The mass, no mass, asymmetry, and noasymmetry ROC curves are shown in Figure 3.7. The algorithm showedno evidence of a difference for cancers with associated mass (0.92-0.96) vs.no mass (0.90-0.93) (Ha Y esAUC Ì¸= NoAUC , p = 0.09); nor for asymmetry(0.90-0.94) vs. no asymmetry (0.92-0.95) (Ha Y esAUC Ì¸= NoAUC , p =0.18). The algorithm performance for cancers with associated architecturaldistortion was 0.96 (0.94-0.98) vs 0.92 (0.90 - 0.93) when this feature wasnot present (Ha Y esAUC > NoAUC , p < .001); for cancers with and withoutassociated calcifications, the AUCs were 0.87 (0.85-0.90) and 0.92 (0.91 -0.94), respectively(Ha Y esAUC > NoAUC , p = .003).463.8. THE IMPACT OF FOLLOW-UP DURATIONFigure 3.7: Stratified AUC - Mammographic Findings3.8 The Impact of Follow-Up DurationWith a 1-year follow-up, radiologists outperformed the commercial AI inboth sensitivity (92.6±1.0% vs. 89.4±1.1%; Ha RadSens > AISens, p=.011)and specificity (91.8±0.1% vs. 78.5±1.1%; Ha RadSpec > AISpec, p<.001)respectively. Because the dominant term in specificity is the total number ofstudies (136,700), specificity remains largely constant over a 90-day to 3-yearfollow up for both radiologists (91.7±0.1% to 91.8±0.1%) and the AI system(78.4±0.1% to 78.6±0.1%). Sensitivity, however, varies greatly over the sameinterval. Performance of the AI shows no evidence of a difference comparedwith radiologists at 2-year follow-up (Ha RadSens > AISens, p=.69). Overallperformance at a 3-year follow-up is shown in Figure 3.8. The impact offollow-up on Sensitivity for these combined use cases are also included in473.8. THE IMPACT OF FOLLOW-UP DURATIONFigure 3.8: Results at 3-year Follow Up. Circle indicates radiologist operatingpoint and diamond indicates AI operating point at manufacturerâs specifiedthreshold (0.10). Effect of follow-up Length on sensitivity for AI, radiologist,and hypothetical double reading scenarios.Figure 3.8.Three categories (calcifications, breast density, and family history) wereidentified as having statistically different performance at 1-year-follow-up ona 95% confidence interval. These same categories are shown in Figure 3.9 witha 3-year follow-up. Notably, breast density D still underperforms. However,both the calcification and family history ROC curves show no statisticallydifferent performance. For calcifications, performance on the No category hasnow dropped to that of the Yes category, showing that Yes category cancerstend to be diagnosed early. we believe that this is because calcificationsare relatively easy to identify in imaging and are investigated when found.For family history, the AI algorithm still underperforms slightly on the Yescategory.Hypothetical Double Reading Scenarios Table 3.2 presents theconfusion matrix for comparisons between the AI and Radiologist, showingthey agree in 76% of readings. Table 3.2 presents 1-year follow-up classificationmetrics, radiologist workload, and patient recalls for the AI, radiologist, andhypothetical double reading scenarios. Notably, only 3% of the time didradiologists and the AI agree that a screening was abnormal. In 73% ofcases, the Radiologist and AI agree that the screening is normal. In thehypothetical scenario where both AI and radiologists read all screens and apatient is recalled if either reader indicates abnormal, a combined sensitivity483.8. THE IMPACT OF FOLLOW-UP DURATIONFigure 3.9: Categories identified with differing subgroup performance (3-yearfollow-up). Circle indicates radiologist operating point and diamond indicatesAI operating point at manufacturerâs specified threshold (0.10).of 96.4% is achieved; however, this would have produced 37,180 abnormalscreens in our cohort (an increase of 25,215, representing a 211% increase inthe number of recalled screenees). There would be no reduction in radiologistworkload in such a scenario.If we instead consider a use case where a patient is recalled only ifboth readers agree, combined sensitivity drops to 85.6%, but the numberof abnormal screens in our cohort is reduced to 4,707 (a reduction of 7,258,representing a 60.7% reduction in the number of recalled screenees). In sucha scenario radiologists would only need to read screens deemed abnormal bythe AI, resulting in a 78.1% reduction in exams read. The drop in sensitivity493.9. DISCUSSIONrepresents an additional 59 missed cancers in our cohort when compared tothe radiologist reader (121 missed vs. 62).Table 3.2: Radiologist and AI Confusion MatrixAINormal AbnormalRadio. Normal 0.73 0.18Abnormal 0.05 0.03Table 3.3: 1-year overall follow-up classification metrics, radiologist workload,and patient recalls. AI values are binarized at the manufacturerâs recom-mended threshold of 0.10.Radiologist AI Either BothTP 779 752 811 720FP 11,186 29,170 36,369 3,987TN 124,673 106,689 99,490 131,872FN 62 89 30 121Radiologist Workload 136,700 - 136,700 29,922 (21.9%)Patient Recalls 11,965 29,922 (250%) 37,180 (311%) 4,707 (39.3%)3.9 DiscussionThis retrospective study evaluated the performance of Lunit MMG version1.1.2.0 on an independent dataset from a population-based breast screeningprogram which employs a single radiologist reader. At 1-year follow-up, theAI achieved AUC of 0.93 (95% CI: 0.92-0.94) but underperformed radiologistson sensitivity (92.6±1.0% vs 89.4±1.1%; p =.01). At 2-year follow-up,there was no evidence of a difference between AI and radiologist sensitivity(83.5±1.2% vs 84.3±1.2%, respectively; p = .69). AI performance differedsignificantly based on various mammographic and demographic features, withhigher performance in individuals with BI-RADS breast density categoryA versus category D (AUC: 0.96 vs 0.84; P = .002), higher performance inmammograms with versus without architectural distortion (AUC: 0.96 vs0.92; p = .003), and lower performance in mammograms with versus withoutcalcification (AUC: 0.87 vs 0.92; p < .001).The algorithm performs comparably to the results identified by Kim et al.,who also analyzed Lunit MMG AI. we found ROC AUC 95% CI performanceof 0.92 â 0.94, whereas they found ROC AUC 95% CI performance of 0.938 â503.9. DISCUSSION0.968 in the USA and 0.918 â 0.958 in the UK. However, their South Koreanpopulation achieved a significantly higher ROC AUC 95% CI of 0.963 â 0.978[91]. This suggests that further work should be done to identify the lowerperformance relative to the South Korean population.Dembrower et al. (2023) explored the combination of Lunit InsightMMG AI and two radiologist readers through a fully blinded triple-readerpaired study of 55,581 women [48]. They found that, on a 95% confidenceinterval, single reading by AI, double reading by AI and radiologist, andtriple reading by AI and two radiologists were all non-inferior to doublereading by two radiologists [48]. While most evidence is retrospective [63],a 2023 randomized, controlled trial from Sweden found that AI supportedscreening met safety limits while reducing workload by 44.3% [118]. Larsenet al. found that 86.8% of the screen-detected cancers were assigned highestrisk by Transpara version 1.7.0, as were 44.9% of interval cancers. Theyalso found a high rate of AI true negatives, indicating the AI could be usedto reduce workload [103]; this finding agrees with those of [49, 104, 160].Our hypothetical analysis of double reading by AI and Radiologist agreeswith this body of evidence, with a possible workload reduction of 78.1%.Moreover, in such a combined reading scenario, the abnormal call rate woulddrop by 60.7%. However, this results in an additional 59 missed cancers anda sensitivity drop from 92.6% to 85.6%. From a clinical perspective, thereduction of false-positive screening mammograms is important to improvingscreening retention [33, 165]. However, neither of the hypothetical doublereading scenarios offer clear clinical advantage over a single radiologist reading.Interesting future work could include exploring different thresholds for theAI to balance sensitivity with workload and recall rate.We found a statistically significant difference depending on BI-RADSbreast density; this difference was most suboptimal for radiologist assignedbreast density category D, indicating that the radiologistâs estimate of breastdensity is indeed accounting for the potential of a mammographically occultcancer in extremely dense breasts (Ha AAUC > DAUC , p = 0.0016; BAUC >DAUC , p = 0.0088; CAUC > DAUC , p = 0.016). This is contrary to the resultsfrom [104], which found that AI-based screening performed consistently acrossbreast densities.We found statistically significant difference in cases of screens with humanidentified calcifications 0.87 (0.85-0.90) with vs. 0.92 (0.91 - 0.94) without(Ha Y esAUC > NoAUC , p = .003). we believe this could be explained byimage resizing during pre-processing. An early evaluation of this possibilitywas presented at RSNA 2022, where it was found that AI downsampling ofthe images likely impedes performance on small cancers manifesting as calcifi-513.9. DISCUSSIONcations; AI algorithm prediction value increased for 17 of the 19 cancer caseswith only associated calcifications (7 of them originally AI false negatives)when cropped image sections with the calcifications were analyzed withoutdownsampling [150]. we suspect it is likely that most of these cancers areDCIS.The BC Cancer Breast Screening program permits women with self-declared family history of breast cancer to attend annually. Therefore, itis possible that the cancers detected in such individuals are smaller thanthose of their counterparts, due to the yearly screening interval. Our resultsindicate statistically significant difference in the AI detecting these smaller,early cancers in the study cohort on a 95% confidence interval (0.90 (95% CI:0.88-0.93) vs. 0.94 (95% CI: 0.93-0.96); p = 0.003). we expect that furtheranalysis with additional data would help to clarify potential performancedifference between these two groups.Comparison with the results from Kühl et al. is difficult because theyreport results at two different thresholds, whereas our analysis uses theAI manufacturer-specified threshold of 0.10. However, our conclusions areconsistent: the AI underperforms on small cancers, found here throughcalcifications and family history subgroups. Our results further agree thatthe AI underperforms radiologists on short follow-ups, but overperforms onfollow up durations greater than two years [101]. It is unclear whether thisindicates the AI is better at detecting small cancers or is better at identifyingfeatures in mammographic imaging that predicts a patient might developcancer; this would be an interesting direction for future work.This study had some limitations. First, the study is the limited bydata available for some subgroups. For example, almost half of the cohortare of unknown ethnicity. However, screening program attendance followsroughly the same proportions of ethnic groups as is present in the provinceof British Columbia [56]. As such, we expect that the missing ethnicitieswould be distributed across the ethnic groups in a similar proportion. Second,screens with more than the four standard views were excluded to promotefair comparison between radiologists, who can consider such views, and theAI which cannot. The non-standard view cases had a cancer rate of 0.91%,whereas standard views had a rate of 0.61%; this is possibly motivated bya technologist suspected abnormality or to address a technical limitation.This discrepancy would be an interesting avenue for future work. Third,this work is retrospective, and action has been taken on Radiologist findings.This put the AI is at a disadvantage, where the correctness of its predictionsmay take time to manifest; conversely, radiologists have directly impactedtime to diagnosis in our data. we believe this is a possible cause of the523.10. CALCIFICATIONS AND EPISTEMIC UNCERTAINTYAI underperformance on short follow-up durations and overperformanceon durations longer than 2 years. Finally, our analysis is caseâbased, andinsensitive to breast side (left vs right) and location of the cancer; this is dueto exam-level scoring by the AI. However, it is possible that, if multiple lesionsare identified, the radiologist might act on the wrong one after a true-positiveabnormal identification by the AI. Discrepancies in AI performance may bemore accurately assessed by a region-of-interestâbased analysis.At 1-year follow-up, the AI achieved AUC of 0.93 (95% CI: 0.92-0.94) butunderperformed radiologists on sensitivity (92.6±1.0% vs 89.4±1.1%; p =.01).At 2-year follow-up, there was no evidence of a difference between AI andradiologist sensitivity (83.5±1.2% vs 84.3±1.2%, respectively; p = .69). Theseresults are in keeping with previous studies from other countries, suggestingthat the algorithm is robust to domain shift between different populations.AI performance differed significantly based on various mammographic anddemographic features, with higher performance in individuals with BI-RADSbreast density category A versus category D (AUC: 0.96 vs 0.84; P = .002),higher performance in mammograms with versus without architectural distor-tion (AUC: 0.96 vs 0.92; p = .003), and lower performance in mammogramswith versus without calcification (AUC: 0.87 vs 0.92; p < .001). Futurework could explore the underperformance on these stratifications, particularlycalcifications, which we believe could be related to downsampling of imagesbefore AI is applied.3.10 Calcifications and Epistemic UncertaintyThe commercial AI underperformed on images where calcification wasobserved by the radiologist. Upon investigation into what could have causedthis, it was found that various manufacturers produce mammographics imagesof different sizes, ranging from 2294x1914 to 4096x3328 pixels. The first stageof the AI is to resize this image to 1920x1920, which can distort small featuressuch as calcifications. To test the impact of this resizing, an experimentwas performed with a small number of images (n=19) to crop the imagesbefore sending them to the AI. AI confidence was observed before croppingand after cropping, grouped by images with calcifications, images with otherfeatures (asymmetry, architectural distortion, or mass) and images withno cancer. These results are presented in Figure 3.10. A one-sided t-testshowed that there is a statistically significant difference in AI confidence aftercropping (p=.003), whereas there was none for other features (p=.80) nornon-cancerous images (p=.55). While the sample size was limited because of533.10. CALCIFICATIONS AND EPISTEMIC UNCERTAINTYthe expertise required to crop these images while maintaining the featuresof interest, this is strong evidence that the resizing was contributing to theAIâs ability to identify cancer in images with calcifications. An example of amammogram which would not have been recalled for follow-up with resizing,but would be recalled without resizing, is presented in Figure 3.11.Figure 3.10: Confidence of the commercial AI tested in Chapter 3 before andafter cropping to the expected AI size for images with radiologists observedcalcification, other features, and no cancer.The difficulty in the AI detecting cancer in images with calcifications canbe seen as uncertainty. In this case, this epistemic uncertainty is introducedin the modeling stage.543.11. CONCLUSIONFigure 3.11: AI Confidence for a single mammogram before cropping (left)and after cropping (right), for an image with calcifications present.3.11 ConclusionThis chapter used a retrospective study to identify several subgroupswhich underperformed in a commercial AI applied to breast cancer screeningusing mammography. Most concerning is the resizing of images introducesepistemic uncertainty, as detailed in Chapter 1, which leads to lower perfor-mance on images with calcifications present. However, this is an area whereradiologists are already well prepared to identify cancers, as all calcificationsare recommended for follow-up. Rather than focusing on these areas whereradiologists excel, perhaps AI in healthcare should focus on areas which areinherently difficult for radiologists such as dense breasts of detection of cancerbefore it is obviously visible in the mammogram [125].Fairness in AI applied to healthcare, while a critical goal, faces substantiallimitations in clinical implementation, particularly concerning subgroup anal-ysis and the nature of validation studies. Defining and achieving fairness iscomplex; what constitutes a âfairâ outcome can vary, and attempts to mitigatebias for one subgroup might inadvertently disadvantage another or reduceoverall model performance. Subgroup analysis, intended to ensure an AI553.11. CONCLUSIONmodel performs equitably across different demographics, is often hampered byinsufficient data for certain groups, leading to statistically underpowered orunreliable findings. This is a significant concern because AI models trained onbiased datasets can perpetuate or even amplify existing health disparities. Forinstance, an algorithm predominantly trained on data from one demographicmay show lower accuracy or make erroneous predictions for underrepresentedgroups, leading to inequitable care if implemented clinically. The challenge iscompounded by the difficulty in prospectively validating fairness across allconceivable subgroups before widespread clinical use. Retrospective studies,such as the one presented here, form the bulk of initial AI development andvalidation; however, they may not capture the full spectrum of biases orhow an AI tool will perform in real-world, dynamic clinical settings. Movingtowards robust prospective trials is essential to evaluate not just accuracybut also the real-world clinical utility and equity impacts of AI interventionsbefore they become standard practice.56Chapter 4Measuring AleatoricUncertainty with WhiteboxPerturbationsIn this chapter, a novel method for generating and injecting adversarialperturbations into 3D point clouds is proposed and evaluated, utilizing apretrained model for car detection on the KITTI dataset within the mmde-tection3d framework. The perturbations are crafted by an adversarial Con-volutional Neural Network (ConvNet) that is integrated into the detectionpipeline. This ConvNet is trained to maximize the baseline loss while en-suring that the perturbations remain subtle and balanced across the pointcloudâs dimensions and directions. Empirical results, derived from a test setof 3,769 instances, indicate that even minimal perturbations can significantlydegrade model performance. This highlights the vulnerability of the baselinemodel to adversarial attacks, underlining the necessity for rigorous robustnessevaluation of 3D point-cloud detection models, especially in safety-criticalapplications such as autonomous driving. These perturbations are found tobe within the sensing error of the Lidar device, providing an estimate of thealeatoric uncertainty inherent in the dataset.4.1 IntroductionThere is an increase in popularity of 3D Point clouds for different appli-cations such as self-driving cars, model reconstruction, drone applications,and quality inspection for construction and manufacturing. In most of theseapplications, trained deep neural networks are employed for 3D object classi-fication and segmentation. Some of these applications have privacy, safety,or security concerns which must be satisfied. Face recognition, autonomousdriving, voice command recognition, and fraud detection are examples ofsecurity-critical applications. Recent works have demonstrated that networkinputs can be intentionally manipulated to create incorrect predictions [3, 205].574.1. INTRODUCTIONMore concerning, such incorrect predictions often have high certainty levels.In computer vision tasks in particular, small perturbations can result in shiftsin predictions even for state-of-the-art algorithms [73]. These perturbationsare often imperceptible to humans.Deep neural networks have been proven to be vulnerable to adversarialattacks carefully designed to fool the system. An early exploration of adversar-ial attacks led to questions on neural networksâ ability to generalize; Szegedyet al. [177] and Goodfellow et al. [73] showed that virtually imperceptiblechanges to the input image can lead to incorrect predictions in contemporarymodels. The Jacobian-based Saliency Map Attack (JSMA) [194] shows thatlarge output variations can be achieved even when the perturbations areapplied to a small portion of input features.To ensure that safety-critical applications have a highly efficient algorithmto detect object correctly, it is important to understand how the adversarialattacks can affect the neural networks employed in such algorithms. Theunderstanding of adversarial attacks can also provide directions to designdefensive systems [75, 140, 183]. Although there has been rigorous researchon 2D point clouds, the interest in 3D point clouds is relatively recent [112,167, 174, 189]. In this paper we propose and evaluate a novel method ofgenerating and injecting adversarial perturbations to 3D point clouds. Themethod is applied to a pretrained model used for car detection in the KITTIdataset taken from the mmdetection3d model zoo.This method differentiates itself by employing an adversarial ConvolutionalNeural Network (ConvNet) to introduce perturbations into the 3D pointcloud; this ConvNet can be used gradient-free during inference to approximaterobustness. Training the ConvNet is done using a whitebox method. Figure 4.1shows the outline of our proposed method. We balance the perturbationsacross both point cloud dimension and perturbation direction by introducingthese terms to the training loss function. We evaluate the method usingAverage Precision (AP) and Intersection-over-Union (IoU) for four commonmetrics, and stratify results over easy, medium, and hard examples.In summary, the contributions of this section are as follows:1. presents a novel whitebox approach in adversarial attack. While therehas been some previous research on adversarial attack on 3D pointcloud detection systems, none of those approaches used the structure ofthe detection system to their advantage. The proposed method employsan adversary with knowledge about the detection system to designtargeted attacks.2. shows the effectiveness of the method by testing it on the SECOND584.2. RELATED WORKFigure 4.1: Overview of the perturbation model.detection system, and evaluating how the efficiency of the systemsignificantly falls due to the attacks.3. measures the impact of the model using IoU of: the bounding box, theBirdâs Eye View footprint, and the 3D bounding box. Average orien-tation similarity (AOS) is also measured. These metrics are providedboth for the baseline and adversarial models, and are broken down byeasy, medium, and hard examples based on visibility and point density,which are influenced by the objectâs distance from the sensor, occlusion,and truncation.The rest of the chapter is organized as follows. Section 4.2 gives an overviewof related work in this field. Section 4.3 defines the proposed method andmaterials used for evaluation, and Section 4.4 contains results and relateddiscussion.4.2 Related WorkAdversarial attacks can be classified based on the frequency of the at-tack, falsification method (increasing false positives or false negatives), priorknowledge of the network architecture and parameters, and whether thedisturbance is applied to change the network output to a specific or randomprediction. Comprehensive categorizations are provided in [3, 205].594.2. RELATED WORKDeepfool defines the robustness of a binary classifier as the distance to thedecision boundary. For affine binary classifiers, this boundary is a hyperplane,and a closed-form solution exists to find the minimum disturbance requiredfor a prediction to cross this hyperplane. This approach is extended tomulti-class and non-linear functions, with the disturbance calculated usingan iterative approach on a linear approximation of the function around thetest point [131].Early work on applying deep learning to point clouds resulted in VoxelNet,which replaced hand-crafted feature representations with deep-learning basedfeature extraction [211]. Similarly, PointNet++ pioneered deep learningon non-uniformly sampled point clouds, applied to both segmentation andclassification tasks [147]. PointPillars extended PointNet with a 2D CNNbackbone and a detection head, achieving state-of-the art results on theKITTI dataset [102]. Sparsley embedded convolutional detection (SECOND)provided faster training and inference for deep learning methods on pointclouds through sparse convolution; SECOND also introduced directional lossand improved data augmentation techniques [199].Combining multiple predictors is a viable path to improve deep learning-based point cloud methods. MVX-Net fused both point cloud and camera datafor 3D object detection. VoteNet, on the other hand, combined predictionsgenerated from only point clouds, without relying on RGB images; predictionswere combined using Hough Voting [146]. Imvotenet combined these twoideas, using information from 2D RGB images to narrow the 3D search spacefrom which the predictors were generated [145].Efficiency is also a concern in processing 3D point clouds; this is paramountin edge applications with limited or no connectivity to cloud-based compu-tational power. 3DSSD removed feature propagation layers and predictionrefinement; instead, they introduced a novel sampling strategy which retainedpositive points for localization and negative points for classification [202].PointRCNN simplifies detection of complex objects by first detecting intra-object parts; these parts are aggregated to score bounding boxes and refinelocalisation. However, this method relies on domain knowledge regarding thecomposition of objects [166].Because of strong commercial interest in self-driving cars, commondatasets using 3D point clouds include street scenes with the intent offacilitating autonomous navigation; example datasets include KITTI [67],Waymo [57, 171], nuScenes [31], and Lyft [121].604.3. METHODFigure 4.2: Pre-Trained Model Structure and Perturbation Method.4.3 MethodThis chapter introduces adversarial perturbations into a neural networkused for car detection in 3D point clouds. Evaluation is done on a pre-trained model based on SECOND [199] applied to the KITTI dataset [67],as implemented in mmdetection3d [128]. Perturbations are inserted intoextracted voxel features by way of an adversarial ConvNet. The adversarialConvNet is trained and validated on the same data used to train the baselinemodel. The ConvNet seeks to maximize prediction loss while limiting theperturbations introduced.The [211] pre-trained model selected from mmdetection3d consists of thefollowing stages:1. Voxelization2. Voxel Encoder (HardSimpleVFE)3. Encoder (SparseEncoder)4. Backbone (SECOND)5. Neck (SECONDFPN)6. Head (Anchor3D)For the whitebox model to be trainable, gradients must be available forbackpropagation. Voxelization in the baseline model is a non-differentiable614.3. METHODprocess, so the perturbations are applied between steps 2 and 3. There, aConvNet is introduced which calculates the perturbation to be applied toeach point, independently; this perturbation is then summed with the outputof step 2 and passed to step 3. This process is displayed in Figure 4.2.Figure 4.3: The Con-vNet to Generate Per-turbations.The Adversarial ConvNet is an overcompleteencoder-decoder architecture with 9 convolutionallayers, as shown in Figure 4.3. Instance normaliza-tion is applied at the start of the network; batchnormalization and the ReLU activation functionare applied between convolutional layers. Input tothe first convolutional layer is 4 features, definedby the number of dimensions in the point cloudreadings; the number of features is doubled at eachconvolutional layer, to a maximum of 32 features;dimensionality reduction is done in the second halfof the network, producing 4 output features whichcomprise the perturbation. All convolution layersare 1-dimensional with a kernel size of 1, meaningthat the network operates on each point in isola-tion of other points; this also allows the network toaccept point clouds of varying length.The loss function is a composition of the base-line loss and measures to control the properties ofthe perturbations. Baseline loss is the sum of 3components:Lbaseline = Lclassification + 0.2 â Ldirectional + 2 â Lbounding_box. (4.1)The adversarial loss function controls the intensity, as well as bias andimbalance of the perturbations. Experimental evaluation with random searchwas used to find component weightings which still converge while balancingperturbation size and errors. Intensity loss is calculated as the mean L2 Normover all perturbations in a sample:Lintensity =âpâP â¥pâ¥2|P | (4.2)where P is the set of perturbations to be applied, whose length is equal tothe number of voxel features.Controlling bias is needed to prevent the model from learning to shiftall points in a single direction, moving them outside of the ground truth624.4. RESULTS AND DISCUSSIONannotations. This is calculated as the L2 norm of the mean value of eachperturbation dimensionLbias = â¥âpâP pd|P | ,âd â Dâ¥2 (4.3)where D is the number of dimensions in the point cloud readings. In thecase of the velodyne point cloud used in the KITTI dataset, there are 4dimensions.Controlling imbalance causes the model to prefer small changes in alldirections over large changes in a single dimension. This is defined as thestandard deviation of the mean in each of the point cloud dimensions:Limbalance = Ï[âpâP pd|P | ,âd â D](4.4)where Ï is the standard deviation function.These losses are weighted and summed with the negation of baseline losswhile training the adversarial network:L = âLmodel + 3 · Lintensity + 10 · Lbias + 10 · Limbalance. (4.5)Optimization is conducted over 20 epochs with 3,712 training samples,utilizing a learning rate of 0.0018 and a weight decay of 0.01. The trainingemploys a batch size of 6, tailored for the available computational resourcesof dual NVIDIA RTX A4500 GPUs. AdamW [116] is used as the optimizer,providing enhanced handling of weight decay over the standard Adam [93].The implementation leverages and modifies the training configurations of themmdetection3d framework [129].4.4 Results and DiscussionThe evaluation of the adversarial model compared to the baseline isconducted on a test set comprising 3,769 examples. These results highlightthe differential impact on the Average Precision (AP) metrics across variousconfigurations. AP is a widely utilized metric in object detection thatmeasures the precision of a model across varying levels of recall, offeringa holistic assessment of model performance. It calculates the area underthe precision-recall curve and is a standard benchmark in prominent objectdetection evaluations such as the PASCAL VOC challenge [58]. AP can becomputed under different configurations, notably AP11 and AP40, whichrefer to the number of points interpolated along the precision-recall curve.634.4. RESULTS AND DISCUSSIONAP11, an older method used in challenges like PASCAL VOC, samples elevenpoints [58], providing a broader overview of model performance. In contrast,AP40 offers a finer evaluation with forty interpolated points, capturing moredetailed variations in precision across recall levels, thus providing a morerigorous assessment [111].Intersection over Union (IoU) is a measure used to evaluate the accuracyof an object detector on a particular dataset. It calculates the ratio of theintersection area between the predicted bounding box and the ground truthbox to their union area [58]. Different IoU thresholds, such as 0.70 and0.50, reflect varying levels of strictness in the overlap criterion, with higherthresholds requiring more precise overlap, suitable for tasks demanding highlocalization accuracy. For instance, an IoU of 0.70 might be required inapplications like autonomous driving, where accurate localization is crucialfor obstacle avoidance and navigation. On the other hand, a lower thresholdlike 0.50 could be sufficient for applications such as surveillance systems,where the exact boundary precision is less critical, and the primary goal is todetect the presence and general location of persons or objects.In object detection, especially in domains such as autonomous driving,a variety of metrics are employed to evaluate the effectiveness of detectionmodels [67]:â Bounding Box (bbox): This metric assesses the accuracy of 2Dbounding boxes predicted by the model, using the IoU to check howwell the predicted boxes align with the ground truth. It is fundamentalfor evaluating the modelâs ability to localize objects in two dimensions.â Birdâs Eye View (bev): The BEV metric is crucial in 3D objectdetection, evaluating the modelâs performance from a top-down perspec-tive. It measures how accurately the model predicts the 2D footprint ofobjects on the ground, which is important for navigation and positioningtasks in autonomous driving.â 3D: Extending to three dimensions, this metric considers the accuracyof the predicted 3D bounding boxes against the actual objects. It isessential for applications requiring depth perception, providing insightsinto the spatial arrangement and size of objects.â Average Orientation Similarity (aos): AOS evaluates the orien-tation accuracy of the detected objects by comparing the predictedand actual angles of the bounding boxes. This metric is significantwhere the objectâs orientation is critical, such as in scenarios involvingobstacle navigation.644.4. RESULTS AND DISCUSSIONIn 3D point cloud object detection, difficulty levels â easy, medium,and hard â classify detected objects based on visibility and point density,which are influenced by the objectâs distance from the sensor, its occlusion,and truncation. âEasyâ objects are typically close to the sensor with highpoint density and minimal occlusion. âMediumâ objects may be fartheraway, have moderate point density, or be partially occluded. âHardâ objectsare often distant, sparsely represented in the point cloud, heavily occluded,or significantly truncated, presenting substantial challenges for accuratedetection. These categories are essential for evaluating a modelâs ability toperform reliably under varied operational conditions typically encountered inreal-world 3D environments.These metrics collectively offer comprehensive insights into the modelâsperformance, aiding in the refinement of detection systems for specific ap-plications. This study utilizes these metrics to compare the performance ofthe baseline model and the adversarial model, evaluating the robustness ofthe system under adversarial attack. This comparison will help to illustratehow well the model can maintain its detection capabilities when faced withintentionally crafted perturbations.Tables 4.1, 4.2, 4.3, and 4.4 illustrate the comparative performance ofbaseline and adversarial models across different IoU thresholds and APmetrics. Under stricter IoU thresholds (0.70), both AP11 and AP40 metricsshow that the baseline model outperforms the adversarial model across alldifficulty levels, highlighting the adversarial perturbationsâ significant impacton detection accuracy. This shows that the method is capable of producingadversarial examples that can fool the original model. Conversely, undermore lenient IoU thresholds (0.50), the drop in performance is less severe,offering insights into the modelsâ effectiveness in scenarios that require lessprecise object localization and reflecting practical application contexts whereexact overlap is less critical.A consistent trend observed across the evaluation metrics; bbox, bev, 3d,and aos, shows a more pronounced drop in accuracy for hard and mediumcases compared to easy cases, respectively. This pattern indicates that theadversarial perturbations are particularly effective at disrupting the modelâsperformance in scenarios where object detection is inherently more challenging.Additionally, the differences in performance drop between IoU thresholdsof 0.70 and 0.50 highlight the impact of adversarial perturbations. Themore significant decline at a stricter IoU threshold of 0.70 compared to 0.50underscores the sensitivity of the detection models to adversarial attacks,especially in applications demanding high precision, such as autonomousdriving. These observations underline the vulnerability of the models in654.4. RESULTS AND DISCUSSIONTable 4.1: Comparison of Model Performances for AP11 with IoU 0.70Metric Difficulty Baseline (AP11) Adversarial (AP11) Drop in Accuracy (%)Easy 95.2 90.4 5.0bbox Medium 89.6 79.0 11.9Hard 88.0 77.4 12.0Easy 90.0 89.3 0.7bev Medium 87.3 78.7 9.8Hard 84.3 76.8 8.9Easy 88.4 77.0 12.83d Medium 78.2 61.3 21.7Hard 76.0 57.3 25.0Easy 94.9 90.2 5.0aos Medium 89.1 78.6 11.8Hard 87.2 76.6 12.1Table 4.2: Comparison of Model Performances for AP40 with IoU 0.70Metric Difficulty Baseline (AP40) Adversarial (AP40) Drop in Accuracy (%)Easy 97.4 95.0 2.4bbox Medium 92.4 80.3 13.0Hard 89.2 77.8 12.7Easy 92.6 91.7 1.0bev Medium 88.4 81.2 7.2Hard 85.2 77.1 8.1Easy 90.4 80.7 10.73d Medium 81.3 61.5 19.8Hard 76.2 57.7 18.5Easy 97.2 94.7 2.6aos Medium 91.8 79.8 12.0Hard 88.4 77.0 11.4Table 4.3: Comparison of Model Performances for AP11 with IoU 0.50Metric Difficulty Baseline (AP11) Adversarial (AP11) Drop in Accuracy (%)Easy 95.2 90.4 5.0bbox Medium 89.6 79.0 11.9Hard 88.0 77.4 12.0Easy 95.3 90.8 4.7bev Medium 89.9 85.8 4.4Hard 88.7 84.5 4.2Easy 95.2 90.8 4.43d Medium 89.8 84.2 5.6Hard 88.5 79.9 8.6Easy 94.9 90.2 4.7aos Medium 89.1 78.6 10.5Hard 87.2 76.6 10.6664.4. RESULTS AND DISCUSSIONTable 4.4: Comparison of Model Performances for AP40 with IoU 0.50Metric Difficulty Baseline (AP40) Adversarial (AP40) Drop in Accuracy (%)Easy 97.4 95.0 2.4bbox Medium 92.4 80.3 12.1Hard 89.2 77.8 11.4Easy 97.5 95.7 1.8bev Medium 94.7 87.8 6.9Hard 91.7 85.8 5.9Easy 97.4 95.8 1.63d Medium 94.6 85.7 8.9Hard 91.5 83.4 8.1Easy 97.2 94.7 2.5aos Medium 91.8 79.8 12.0Hard 88.4 77.0 11.4Figure 4.4: Histogram of L2 Norm Magnitude (left) and Percentages of theOriginal Vectorâs L2 Norm (right)safety-critical scenarios, reinforcing the need for robust adversarial defensemechanisms.The statistical analysis of L2 norms expressed as a percentage of theoriginal vectorâs L2 norm underscores the subtlety and precision of theadversarial perturbations applied to the point cloud data. As can be seenin Figure 4.4, the mean L2 percentage stands at 0.08%; this suggests that,on average, the perturbations are relatively minor in magnitude. However,the standard deviation of 0.09% points to variability in the perturbationintensity across different instances within the dataset. This variability iscrucial as it indicates that while some perturbations remain minimal, othersapproach the upper range of 0.60%, potentially pushing the boundaries of674.5. CONCLUSIONtypical sensor error tolerances. For context, the sensor used to generate theKITTI point clouds, a Velodyne HDL-64E [67], has a reported sensing error inthe range of -0.2 to 0.2m [69], indicating that the perturbations are generallywithin the sensing error. This comparison highlights the effectiveness of theperturbations in challenging the robustness of the detection model underadversarial conditions, without exceeding the normal operational variancesexpected from the sensor.Resulting perturbations, while subtle, are effective at manipulating thedetection process. The adversarial adjustments are designed to be impercep-tible to humans, thus showcasing how these models can subtly alter detectioninputs. This is particularly significant in contexts where detection accuracyis paramount, such as autonomous driving. The adversarial approach leadsto crucial detection failures including missed detections and false positives.These types of errors are concerning in safety-critical applications, it wouldbe perilous to miss a car in a self-driving application. This emphasizes thenecessity for robust defenses against adversarial attacks.4.5 ConclusionThis chapter introduced a novel method of generating whitebox per-turbations, and controlling their amplitude through regularization. Theseperturbations are able to significantly degrade model performance, evenwhen controlled to within sensing error, which can be seen as an approx-imation of the aleatoric uncertainty within the data. In mission-criticalAI applications like self-driving cars and healthcare diagnostics, accuratelymeasuring aleatoric uncertainty is paramount for ensuring robustness andtrustworthiness. Aleatoric uncertainty, which arises from irreducible noiseand randomness inherent in input data, such as variations in sensor read-ings due to environmental conditions, directly impacts the reliability of AIpredictions. By generating perturbations that mimic the expected rangeand characteristics of sensing errors, we can simulate realistic noisy datainputs. Analyzing the variation and confidence of the AIâs output under theseperturbed conditions allows for the quantification of how much the inherentdata noise influences the final decision. This understanding of aleatoricuncertainty, specifically through the lens of sensing error perturbations, iscrucial for developing AI systems that can acknowledge the limits of theirperception and provide appropriately calibrated confidence estimates, therebyenhancing safety and preventing potentially catastrophic errors in real-worlddeployments.68Chapter 5Predicting and explainingperformance and diversity ofneural network architecture forsemantic segmentationChapters 3 and 4 explored use cases where uncertainty in AI led tounfair or potentially unsafe applications. This uncertainty can be the resultof data bias and modeling, as in the case of the breast cancer screeningapplication, which is an example of epistemic uncertainty. Sensing limitationscan also lead to aleatoric uncertainty, as in the case of the self-driving carapplication. However, these chapters required careful analysis to measurethis uncertainty, either by subgroup stratification or tuning hyperparametersto ensure perturbations were within sensing error.This chapter seeks a more general way to measure uncertainty by leverag-ing diversity. By training many models, and measuring the diversity acrosspredictions, we can get a heuristic notion of uncertainty for any application.However, training many models is computationally expensive and many meth-ods rely on identifying the best combination of models post-hoc. We proposesearching for network architectures that achieve similar performance whilepromoting diversity, in order to facilitate ensembling. We hypothesize thatrelatively small changes in network depth and width, along with a variedactivation function, can achieve this goal.5.1 IntroductionMuch of the previous work on ensembling considers the classification ofsimpler datasets such as FMNist [196] and CIFAR10 [97]. Many methods relyon producing a pool of models a priori and growing ensembles iteratively bygreedily selecting the model that best improves performance [37, 192]. Thisis possible on simpler tasks, but becomes prohibitively expensive on harder695.1. INTRODUCTIONtasks such as Semantic Segmentation of the CityScapes dataset [44]; notably,many state-of-the-art segmentation models can take hours to train [182].Activation functions, an integral part of a neural network, are appliedto the output of each perceptron. They were initially modeled off biologicalorganisms and represented the action potential of a cell [156]. Today, manyvarieties of activation functions have been explored, which all introducedifferent properties into the network. The authors of [209] show that alocal approximation of an equilibrium point xo in the network can be foundthrough the Jacobian. The system is unstable if the roots of the characteristicequation of this differential equation are negative and at critical stability whenequal to zero. The roots for the sigmoidal activation are found to includezero, which indicates saddle points and gives a mathematical explanation forthe vanishing/exploding gradient problem that plagues deep neural networks.The hyperbolic tangent has strictly positive roots; this suggests that it isbetter suited for neural networks.ReLU is widely used in neural networks, as the function promotes sparsityin the network [71]. However, ReLU also has a zero root, suggesting thatit is likely to produce vanishing gradients. In an attempt to resolve suchissues, many approximations and variations in ReLU have been exploredin the literature, most notably Leaky ReLU and Swish [151]. In particular,Swish has been shown to have strictly positive roots [209].[151] explored many activation functions and found that Swish achievedthe best performance; however, they also found that sin(x)x +x and cos(x)âxperformed well, noting âthe use of periodic functions in activation functionshas only been briefly explored in prior workâ and that such functions were âafruitful route for further research.â As a guide, the authors suggested that theperiodic function should be combined with the raw preactivation by addingor subtracting [151].The authors of [132] and [120] show that many activation functionsexhibit similar performance and that none performed best across all datasetsin various tasks. Both works explored varied activation functions withinensembles. The authors of [132] stochastically replace activation layers withdifferent functions, and show that these perform similarly to homogeneousactivation function networks. [120] compare an ensemble of ReLU networks toan ensemble of varied activation functions, and show that the latter performsbetter for all ensemble sizes. Together, these papers strongly suggest thatvarying the activation function is a viable method for introducing diversitythat improves the ensembling performance.To understand the performance and diversity of varied network archi-705.2. FORMULATIONtecture parameters, as applied to semantic segmentation of the CityScapesdataset, we produce 144 U-Net [155] models using a grid search of networkparameters such as depth, width, and activation function. Our goal is to ex-tensively explore the effect of these network parameters to promote informedproduction of ensemble members.When evaluating diversity, we found that the combinatoric nature ofnumerous large examples in the semantic segmentation task was prohibitivelyexpensive. To alleviate this, we split diversity into two parts, allowing removalof uninteresting parts of the image to simplify the more expensive task ofcalculating correlation.We hypothesize that performance and diversity can be predicted fromarchitecture. This would provide a better understanding of the impact ofnetwork architecture on performance and diversity; the explanations couldalso be used in place of naïve search methods or a model pool to informarchitecture selection when growing an ensemble.5.2 FormulationFor semantic segmentation, performance is often calculated using Intersection-Over-UnionM(YË,Y) = YË â©YYË âªY . (5.1)For this work, diversity is calculated in two parts: agreement DA and corre-lation DC . Agreement is the fraction of the image in which the predictionsmatch exactly,DA(YË1, YË2) = |YË1 == YË2||YË1|. (5.2)The correlation is calculated as the Pearsonâs correlation coefficient in theportions of the images where the predictions are not the same. Thus, the cor-relation score is similar to the commonly used Image Correlation Coefficient,but with the uninteresting pixels removed for efficiency. In particular, IoUand DA are in the range [0,1], whereas DC is in the range [-1,1].5.3 MethodThis chapter hypothesizes that it is possible to predict the diversity of twolearned models F1 and F2 given their architectures. To simplify the evaluationof this hypothesis, it is tested for a fixed task on a fixed dataset D; parametersÎ are learned using a defined optimization method and hyperparameters.715.3. METHODThis work evaluates the performance and diversity of 144 similar modelsproduced through a network architecture grid search, applied to semanticsegmentation of the CityScapes dataset [44]. Each model is trained indepen-dently using PyTorch Lightning [59] with 2 P100 12GB GPUs.For each model, performance is measured as mean intersection-over-unionover all classes (mIoU). Diversity is measured on each pair of models for eachclass on all predictions. When calculating diversity, large images combinedwith the combinatoric nature of pairing models make the task computationallyexpensive; to reduce this burden, diversity is split into two parts: Agreement,the percentage of the image where both predictions match exactly; andCorrelation, the Pearson correlation coefficient calculated only on the portionof the images where the predictions differ.A Tukey test is used to compare the mean performance in pairs, usinga confidence interval 95% (α = 0.05). Pairs without significant differencein their mean are identified. InterpretML is employed with ExplainableBoosting Machines [136] to produce fits for Performance, Agreement, andCorrelation using underlying parameters (e.g. network depth, width, andactivation function). Diversity predictions also include the performancedifference between the underlying models as a feature.5.3.1 Model OverviewAn exhaustive grid search over parameters shown in Table 5.1 was usedto produce 144 models. All models use the same dataset, data augmentation,loss function, optimizer and related hyperparameters.Table 5.1: Parameters Explored. A grid search is performed over all possiblecombinations, with one parameter selected from each set, for a total of 144models.Parameter Possible ValuesActivation {ReLU, Leaky ReLU, Swish, Cos, TanH, Piecewise Linear}Depth {4, 6, 8}Residual Units {2, 4}Width {30, 50}Width Ratio {1.2, 1.4}A configurable U-Net [155] is employed for all models, as implementedby [8]. An initial 3-channel to 3-channel convolution with a 1x1 kernel,followed by 2D instance normalization, is employed before the model to allow725.3. METHODlearning of pre-processing. The final logistic function is Sigmoid; the modelhas a 19-channel one-hot coded logistic output. Random weight initializationfollows the Kaiming Uniform method [82]; bias initialization uses the methodpresented in [106].The grid search includes the ubiquitous activation functions Swish, ReLU,and Leaky ReLU. Hyperbolic Tangent is also explored, as [209] suggests;moreover, a piecewise linear function is included, consisting of three parts,where Ïl(x) from Equation (2.4) is defined asÏl(x) :=ï£±ï£´ï£²ï£´ï£³Î±1(xâ x1)â 1, if x < x1,xx2, if x1 â¤ x â¤ x2,Î±2(xâ x2) + 1, if x > x2,(5.3)where x1 = âw2 , x2 = w2 ; so âx/x1 = x/x2 making this piecewise-linearfunction continuous. For the purposes of this dissertation, this function wasparameterized with w = 0.5. This function is plotted in Figure 5.1a. TheCos activation function, suggested by [151] and [77] (see Figure 5.1b) definesÏl(x) := xâ cos(x).(a) Piecewise Linear (b) xâ cos(x)Figure 5.1: Activation Functions735.3. METHODTable 5.2: Class and example countsClass Train Val Testpole 2,631 292 515building 2,624 291 507road 2,617 289 511vegetation 2,580 283 511car 2,533 280 495traffic sign 2,514 278 489sidewalk 2,497 283 497sky 2,401 265 462person 2,104 236 405bicycle 1,520 178 293traffic light 1,496 166 287terrain 1,435 156 304fence 1,141 119 225rider 981 104 191wall 900 107 165motorcycle 468 50 88truck 337 39 63bus 267 31 51train 125 14 25total 2,659 299 519Dataset Preparation Since thetest dataset provided by CityScapeshas no annotations, only the trainand validation datasets are used.The fine annotations are used; coarseannotations are excluded. In total,3,477 images of size 2048x1024 RGBimages are provided with annota-tions. Class labels are mapped tothe 20-class labelling method recom-mended in the CityScapes documen-tation [44].Dataset splitting is done by sam-pling each class in order, randomlyreserving 15% of images in whichthat class is represented for testing.The least frequently observed classesare split first. This is repeated forthe non-test images, reserving 10% ofthe remaining images for validation.The counts of examples by class anddata split are shown in Table 5.2.The same splits are used for all mod-els trained.Images with classes which appearinfrequently (e.g. train) are repeated to promote learning during the trainingphase. The validation and testing datasets are not repeated. Since manyclasses are present in a single image, it is impossible to repeat only theinfrequent classes and so it is generally impossible to perfectly balance thedataset. The dataset is considered balanced when the least frequently seenclass is seen in at least 20% of the examples. If this condition is not met,all examples from the least frequently seen class are repeated by the integerfloor of the reciprocal of the percentage of examples in which it is seen. Thisresults in repeating examples for train 21 times, truck 10 times, bus 9 times,and motorcycle 5 times. At this point, the algorithm ends as the dataset isconsidered balanced. This increases the number of training examples from2,659 to 13,397; final counts are detailed in Table 5.3.745.3. METHODTable 5.3: Training Example CountsClass # %road 13,275 99.1pole 13,269 99.0building 13,247 98.9vegetation 13,114 97.9traffic sign 12,850 95.9car 12,826 95.7sky 12,555 93.7sidewalk 12,530 93.5person 11,095 82.8traffic light 9,043 67.5bicycle 8,092 60.4terrain 7,290 54.4fence 5,658 42.2rider 5,611 41.9truck 4,529 33.8motorcycle 4,344 32.4wall 4,274 31.9bus 3,645 27.2train 3,045 22.7Data Augmentation Patchingto 512x512 is employed during train-ing due to limited GPU memoryavailable. After the patch is ap-plied, online data augmentation isperformed to increase diversity in thetraining set. The Albumentations li-brary created by [30] is employed,with a random scaling from 0.9 to1.1 of image size, a rotation of theimage from -10 to 10 degrees, andrandom flipping across the horizon-tal axis. Noise is introduced throughrandom image compression, blurring,or Gaussian noise. Intensity is ran-domly modified through brightness,contrast, or both with a probabilityof 25%. The patch is then applieda second time if the image size haschanged, and pixel values are clippedto the range of 0 to 1.Training Parameters Trainingis performed with a batch size of 10,which allows training on 12GB GPUsfor all models. The Adam optimizer described by [93] is employed; Adam ispreferred for evaluating the method, as faster convergence is needed over thebetter results obtained using Stochastic Gradient Descent [23, 93, 203]. NoL1-Regularization was employed, as previous research shows that it can leadto model bias due to sensitivity to outliers [60]. Similarly, no weight decay isemployed. An initial learning rate of 5Ã10â3 is used, with a reduction throughdivision by 10 which occurs every 3 epochs of non-improving validation results;the minimum learning rate is limited to 5Ã 10â4. Training is stopped whenvalidation results do not improve for 6 epochs. The full implementation isdone using PyTorch Lightning [59], and all random variables are seeded to afixed value before training begins to promote reproducibility.Loss Function Binary Focal Loss (BFL) with a focusing factor of 2 isemployed, as it is designed to detect sparse classes in unbalanced datasets[110]. Tversky Loss [159] is also employed, as the overlap function is shown towork well in highly unbalanced segmentation tasks [170]. For this application,both alpha and beta parameters of the Tversky Loss function are set to 0.5; a755.4. RESULTS AND DISCUSSIONfocusing factor of 2 is also used, similar to BFL. These two losses are summedinto the final loss function. Loss is calculated only for classes represented inthe image; loss is ignored for unrepresented classes.5.3.2 Explaining Performance and DiversityPredictions are produced for all models on both the test and validationsets. For each model, there are 519 predictions in the test set for each ofthe 19 classes. For explaining performance of the 144 models, this givesn=1,419,984 samples. Diversity is calculated pairwise on 144 models, giving144 choose 2 = 10,296 pairs. For each model, all predictions for a class areloaded, flattened to a 1D vector, and concatenated; diversity is calculatedbetween these combined vectors for all predictions. Repeated for 19 classes,this gives n=195,624 samples for diversity.Explainable Boosting Machines (EBM) are used to produce interpretablefits for IoU, DA and DC . The data is split evenly into 50% trainingand 50% testing examples. Fit quality is evaluated on testing examplesonly. Default parameters are used for all other cases, as implementedby [136]. EBMs are a bagging, gradient-boosted tree method which pro-duce interpretable Generalized Additive Models with pairwise interactions.The model takes input features X â Rn and fit a function of the form:g(E[y]) = β0 +âni=0 fi(xi) +âni=0,j>i fi,j(xi, xj). The importance of eachfeature and interaction can be found by inspecting fi and fi,j , respectively[117].Diversity can be achieved trivially through one good prediction and onepoor; as such, model performance is, intuitively, a very good predictor ofdiversity. Since model performance can be reasonably predicted, fits fordiversity include the performance difference between models. Includingperformance difference as an independent feature is intended to reduce itsimpact as a confounding factor with other parameters.5.4 Results and DiscussionThe 144 models produced by grid search of parameters shown in Table 5.1are first evaluated for generalizability. They show similar performance on boththe validation and test datasets across all classes (see Figure 5.2). A Tukey testbetween models finds no significant difference between means of many pairswithin a 95% confidence interval (see Figure 5.4). The best performing model,for example, is found to have no significant difference in mean performancecompared to the next 15 best-performing models; performance for these765.4. RESULTS AND DISCUSSIONmodels is shown in Figure 5.3. These top 16 models display variety among allnetwork parameters except for the number of Residual Units (see Table 5.4).This shows that many different network architectures achieve comparableperformance. Many top models on popular benchmarks may be no betterwhen generalizing than other comparable models. This conclusion is inkeeping with [162]; they found that HARK-ing, p-hacking, and inconsistentbaselines often lead to hyperparameter optimization overfitting the test set.Excellent fits for IoU, agreement, and correlation are achieved when classis used as a feature. To improve generalizability of results, these explanationsare then stratified over class to remove it as a feature; this produces acceptablefits (see Table 5.6 and Figure 5.6). Notably, explanations of performancelargely agree with the accepted wisdom of the machine learning community.This gives confidence that the explanations are finding information of value,and that applying the method to understand diversity is a promising avenueof research.5.4.1 Model PerformancePerformance for all 144 models is shown in Figure 5.2. The models showgood generalizability. It is apparent that some classes are much easier toidentify than others, with significant performance difference between theworst performing and best performing classes. Further results presented inthis paper are generated using the test set only.Figure 5.8a shows that depth and residual units are the most importantfactors in network performance; the explanations also show that deeperis, generally, better. Activation is the third most important factor, withimportance approximately half that of the depth parameters. The widely-used semi-bounded activation functions Swish, ReLU, and Leaky ReLUoutperform the others; Hyperbolic Tangent is found to produce the worstperformance. The explanations also show that wider networks producebetter performance. There are moderate interactions between network sizeparameters.A Tukey test is used to identify models with performance that is notsignificantly different within 95% confidence intervals (see Figure 5.4); forexample, the best-performing group includes 16 models. The Tukey testresults for these models are shown in Table 5.5; parameters which producedthese top-performing models are shown in Table 5.4. This evaluation confirmsthat varied activation functions can produce similar results, with 4 differentfunctions being represented. Moreover, models with depth of 6 and 8 are bothrepresented, as are widths of 30 and 50 and width ratios 1.2 and 1.4. Only the775.4. RESULTS AND DISCUSSIONnumber of residual units are homogeneous across all of the best-performingmodels. This evaluation confirms that similar performance can be achievedwith varied network architectures.Figure 5.2: mIoU by ClassFigure 5.3: Performance of the top 16 Models.785.4. RESULTS AND DISCUSSIONFigure 5.4: Count of Similar Models by RankTable 5.4: Architecture Parameters for Best Performing Models.Test Rank Val Rank mIoU Activation Depth Residual Units Width Width Ratio1 1 0.628 Leaky ReLU 8 4 50 1.42 3 0.625 Swish 8 4 50 1.43 2 0.624 ReLU 8 4 50 1.24 6 0.623 Leaky ReLU 8 4 50 1.25 4 0.622 Leaky ReLU 6 4 50 1.46 7 0.619 ReLU 8 4 50 1.47 5 0.618 Piecewise Linear 8 4 50 1.48 8 0.618 Swish 6 4 50 1.49 9 0.615 Leaky ReLU 6 4 50 1.210 10 0.614 Swish 8 4 50 1.211 11 0.614 ReLU 6 4 50 1.412 14 0.613 Swish 6 4 50 1.213 15 0.610 Leaky ReLU 6 4 30 1.414 12 0.609 ReLU 8 4 30 1.415 13 0.608 Swish 6 4 30 1.216 18 0.606 ReLU 6 4 50 1.2795.4. RESULTS AND DISCUSSIONTable 5.5: Tukey Test Values for Best Model PairsRank diff lwr upr p_adj2 0.004 -0.020 0.027 1.0003 0.005 -0.019 0.028 1.0004 0.006 -0.018 0.029 1.0005 0.006 -0.017 0.029 1.0006 0.009 -0.014 0.032 1.0007 0.010 -0.013 0.033 1.0001 8 0.011 -0.013 0.034 1.0009 0.013 -0.010 0.036 1.00010 0.014 -0.009 0.037 0.99911 0.014 -0.009 0.038 0.99812 0.015 -0.008 0.038 0.99113 0.018 -0.005 0.042 0.65714 0.019 -0.004 0.042 0.58015 0.020 -0.003 0.043 0.39516 0.022 -0.001 0.045 0.1223 0.001 -0.022 0.024 1.0004 0.002 -0.021 0.025 1.0005 0.003 -0.021 0.026 1.0006 0.005 -0.018 0.029 1.0007 0.006 -0.017 0.030 1.0008 0.007 -0.016 0.030 1.0002 9 0.010 -0.014 0.033 1.00010 0.011 -0.013 0.034 1.00011 0.011 -0.012 0.034 1.00012 0.012 -0.012 0.035 1.00013 0.015 -0.008 0.038 0.99314 0.015 -0.008 0.039 0.98615 0.016 -0.007 0.040 0.94216 0.018 -0.005 0.042 0.6514 0.001 -0.022 0.024 1.0005 0.002 -0.022 0.025 1.0006 0.004 -0.019 0.028 1.0007 0.005 -0.018 0.029 1.0008 0.006 -0.017 0.029 1.0009 0.009 -0.015 0.032 1.0003 10 0.010 -0.014 0.033 1.00011 0.010 -0.013 0.033 1.00012 0.011 -0.013 0.034 1.00013 0.014 -0.009 0.037 0.99914 0.014 -0.009 0.038 0.99815 0.015 -0.008 0.039 0.98716 0.017 -0.006 0.041 0.822Rank diff lwr upr p_adj5 0.000 -0.023 0.024 1.0006 0.003 -0.020 0.026 1.0007 0.004 -0.019 0.028 1.0008 0.005 -0.018 0.028 1.0009 0.008 -0.016 0.031 1.0004 10 0.008 -0.015 0.032 1.00011 0.009 -0.015 0.032 1.00012 0.009 -0.014 0.033 1.00013 0.013 -0.010 0.036 1.00014 0.013 -0.010 0.036 1.00015 0.014 -0.009 0.037 0.99916 0.016 -0.007 0.040 0.9456 0.003 -0.020 0.026 1.0007 0.004 -0.019 0.027 1.0008 0.005 -0.019 0.028 1.0009 0.007 -0.016 0.030 1.00010 0.008 -0.015 0.031 1.0005 11 0.008 -0.015 0.032 1.00012 0.009 -0.014 0.032 1.00013 0.012 -0.011 0.036 1.00014 0.013 -0.011 0.036 1.00015 0.014 -0.010 0.037 1.00016 0.016 -0.007 0.039 0.9707 0.001 -0.022 0.024 1.0008 0.002 -0.022 0.025 1.0009 0.004 -0.019 0.028 1.00010 0.005 -0.018 0.028 1.0006 11 0.006 -0.018 0.029 1.00012 0.006 -0.017 0.029 1.00013 0.010 -0.014 0.033 1.00014 0.010 -0.013 0.033 1.00015 0.011 -0.012 0.034 1.00016 0.013 -0.010 0.036 1.0008 0.001 -0.023 0.024 1.0009 0.003 -0.020 0.026 1.00010 0.004 -0.019 0.027 1.00011 0.004 -0.019 0.028 1.0007 12 0.005 -0.018 0.028 1.00013 0.008 -0.015 0.032 1.00014 0.009 -0.014 0.032 1.00015 0.010 -0.013 0.033 1.00016 0.012 -0.011 0.035 1.000Rank diff lwr upr p_adj9 0.003 -0.021 0.026 1.00010 0.003 -0.020 0.027 1.00011 0.004 -0.019 0.027 1.0008 12 0.004 -0.019 0.028 1.00013 0.008 -0.015 0.031 1.00014 0.008 -0.015 0.031 1.00015 0.009 -0.014 0.032 1.00016 0.011 -0.012 0.035 1.00010 0.001 -0.022 0.024 1.00011 0.001 -0.022 0.024 1.00012 0.002 -0.021 0.025 1.0009 13 0.005 -0.018 0.028 1.00014 0.006 -0.018 0.029 1.00015 0.007 -0.017 0.030 1.00016 0.009 -0.014 0.032 1.00011 0.000 -0.023 0.024 1.00012 0.001 -0.022 0.024 1.00010 13 0.004 -0.019 0.028 1.00014 0.005 -0.019 0.028 1.00015 0.006 -0.018 0.029 1.00016 0.008 -0.015 0.031 1.00012 0.001 -0.023 0.024 1.00013 0.004 -0.019 0.027 1.00011 14 0.004 -0.019 0.028 1.00015 0.005 -0.018 0.029 1.00016 0.008 -0.016 0.031 1.00013 0.003 -0.020 0.027 1.00012 14 0.004 -0.020 0.027 1.00015 0.005 -0.019 0.028 1.00016 0.007 -0.016 0.030 1.00014 0.000 -0.023 0.024 1.00013 15 0.001 -0.022 0.025 1.00016 0.004 -0.020 0.027 1.00014 15 0.001 -0.022 0.024 1.00016 0.003 -0.020 0.026 1.00015 16 0.002 -0.021 0.025 1.000805.4. RESULTS AND DISCUSSION5.4.2 Model DiversityAgreement between models is strongly class dependent. Classes that takeup more of the image have generally lower agreement than smaller classes.This shows that the method is identifying areas where predictions are easierand of little interest, such as background pixels. Removing these pixels is aproductive method to increase performance of the correlation calculation.Correlation is also class dependent; in this case, harder classes to identifyhave lower correlation than easier classes. In fact, the building, road, andvegetation classes are all very strongly correlated across all models. Thisindicates that ensembling is likely of less value in these easier-to-predictclasses.There is positive correlation in almost all cases. Only 5 of the 19 classesshow any negative correlation in any pairs; moreover, in those 5 cases, negativecorrelation is infrequent. In no case is the 25th percentile below 0.05. This isa strong indication that the models are finding similar functions, especiallyin the better performing classes.Together with performance, these metrics present an informative methodto identify classes which are a good candidate for ensembling.(a) Agreement (b) CorrelationFigure 5.5: Diversity by Class815.4. RESULTS AND DISCUSSION5.4.3 Explaining Performance and DiversityTable 5.6: Explanationquality (R2). Aggregatemetrics exclude overall fit.class IoU Agreement Correlationoverall 0.96 0.95 0.96truck 0.88 0.89 0.62sidewalk 0.87 0.88 0.83bus 0.87 0.87 0.59car 0.86 0.88 0.81rider 0.85 0.80 0.57building 0.85 0.85 0.80person 0.81 0.83 0.78fence 0.78 0.84 0.66wall 0.78 0.80 0.60pole 0.77 0.74 0.81traffic sign 0.75 0.61 0.65motorcycle 0.72 0.68 0.64traffic light 0.70 0.50 0.49road 0.69 0.82 0.77train 0.69 0.89 0.58bicycle 0.64 0.79 0.81vegetation 0.57 0.71 0.74terrain 0.57 0.48 0.62sky 0.52 0.48 0.50mean 0.74 0.75 0.68median 0.77 0.80 0.65std 0.11 0.14 0.11Explainable Boosting Machines [136]achieve excellent fits when class is included asa feature, as can be seen in Table 5.6 (identi-fied in the overall row on top). However, classis the most significant feature in predictingboth diversity and performance. While thismakes sense intuitively and produces goodfits, it reduces the generalizability of the ex-planations. To accommodate for this, the ex-planations are stratified by class, producing19 distinct explanations shown in Table 5.6and Figure 5.6. The stratified fits are not asgood, with mean R2 values for IoU, Agree-ment, and Correlation of 0.74, 0.75, and 0.68,respectively. This shows that predicting cor-relation between models is the most difficulttask. It is also apparent that for some classes,such as Sky, it is much more difficult to ex-plain the desired properties of the models.Overall, the fits are of reasonable quality, in-dicating that the method has value and theexplanations are worth exploring.Diversity between models is best achieved with varied network size. Forexample, from Figure 5.11b, depth of 4 shows relatively low correlation withdepths of both 6 and 8. Notably, homogeneous depth pairs show positivecorrelation, and depth of 6 shows positive correlation with a depth of 8.Figure 5.6: Explanation Quality, stratified over 19 classes in CityScapesdataset. Note that the y-axis is scaled to allow more accurate viewing.825.4. RESULTS AND DISCUSSION(a) Performance (b) DiversityFigure 5.7: Overall Importance of Parameters(a) Performance (b) DiversityFigure 5.8: Stratified Importance of ParametersThis indicates that larger models generally converge to similar solutions. Itis also possible that performance is a confounding factor in these values,despite introducing performance difference as an independent feature. Infact, performance difference is the strongest predictor of diversity, both foragreement and correlation metrics. From Figure 5.9, both agreement andcorrelation decrease as performance difference increases; this indicates anincrease in diversity between the models.Activation function has moderate importance for both agreement andcorrelation. However, from Figure 5.10b it is unclear exactly how to exploitthis importance. None of the activation function pairs stand out as promoting835.4. RESULTS AND DISCUSSIONgreater diversity than the others. Even when the same activation function isused in both models, a negative correlation is possible, as is seen for Swish.This is a stark contrast from the activation functionâs impact on performance;thus, it is preferable to select better-performing activation functions thanattempting to vary across models. However, selecting similarly performing,varied activations such as ReLU and Swish does not seem to reduce diversity.(a) Agreement (b) CorrelationFigure 5.9: Overall Performance Difference Effect on Diversity(a) Performance (b) DiversityFigure 5.10: Stratified Activation Score845.5. LIMITATIONS AND FUTURE WORK(a) Performance (b) DiversityFigure 5.11: Stratified Depth Score(a) Performance (b) DiversityFigure 5.12: Stratified Residual Units Score(a) Performance (b) DiversityFigure 5.13: Stratified Width Score855.5. LIMITATIONS AND FUTURE WORK(a) Performance (b) DiversityFigure 5.14: Stratified Width Ratio Score5.5 Limitations and Future WorkCityScapes was selected for evaluation as it is one of the most difficultsemantic segmentation benchmarks available. It is an imbalanced datasetboth in number of class examples and example size within images. Thisallows evaluation of the method on both small, sparse components, andlarge components; it also include the effect of balancing one class againstanother. However, it is unclear how this method would perform on simplerdatasets which are largely learnable by a single neural network of moderatesize. The method may provide no benefit on such data, where architecturaldifferences may not lead to strong differences between models. It would beparticularly interesting to see this evaluation applied to additional tasks suchas classification, where Top-1 Accuracy above 90% is now common on popularbenchmarks.The ultimate goal of this method is to facilitate efficient constructionof performant ensembles. Rather than grid search an iterative algorithmcould be used to learn and select the next model to train and test, basedon previously trained models. This could replace naïve search methods ora model pool to inform architecture selection when growing an ensemble,negating the need for an a-priori model pool entirely.The diversity explanations presented here use performance of the underly-ing networks as an input feature. This was done to control for trivial diversitybetween one good and one poor model. It is well-known, and confirmedhere, that performance is predictable from network architecture. However,it is unclear if this diversity can be predicted before training either model,using only predicted performance as an input. Such a finding would facilitatepredicting pairwise performance (Equation 2.28) and selecting models to865.6. CONCLUSIONtrain which are likely to improve this metric.This chapter shows that, similar to performance, diversity is predictablefrom network architecture. While it is widely understood both theoreticallyand experimentally that diversity is a benefit when averaging models toreduce variance, it remains to be seen how efficient and effective this methodis compared to other state-of-the-art methods of producing models suchas the diversified ensemble [210] or batch ensembles [191]. However, thesecomparable methods are not yet tested on semantic segmentation tasks. Acomprehensive comparison of these methods applied to semantic segmentationwould be valuable.5.6 ConclusionThis chapter seeks to understand and explain the performance and di-versity of varied network parameters applied to the CityScapes SemanticSegmentation Dataset. A grid search produces 144 models with varied networkarchitecture parameters, which are then evaluated on performance (mIoU)and pairwise diversity. Explainable Boosting Machines (EBM) achieve excel-lent fits for both performance and diversity when class is used as a feature.To improve generalizability of results, these explanations are then stratifiedover class to remove it as a feature; this produces acceptable fits. The ex-planations for performance largely agree with the accepted wisdom of themachine learning community, which shows that the method is extractinginformation of value.A Tukey test identifies no significant difference between means of manymodel pairs (α = 0.05). Specifically, using a 95% confidence interval, thetop 16 models are all found to have no significant difference in mean IoUcompared to the best model. These 16 models display large architecturediversity, showing that similar performance can be achieved with manydifferent network architectures.The explanations show that diversity between models is best achieved withvaried network size. Homogeneous network size parameters generally showpositive correlation, and larger models tend to converge to similar solutions.Activation function has a clear impact on performance, with semi-boundedfunctions such as Swish, ReLU, and Leaky ReLU achieving the best results.Activation function also has moderate importance for both agreement andcorrelation, though it is unclear which combinations exploit this in all cases.Finally, this work shows that both performance and pairwise diversity islearnable from network architecture; future work could focus on informing875.6. CONCLUSIONnetwork selection for ensembling using this information.Vision Transformers (ViTs) [52] have rapidly emerged as the state-of-the-art for semantic segmentation tasks, demonstrating remarkable performanceon challenging benchmarks like the CityScapes dataset. Indeed, severalViT-based architectures now exceed a mIoU of 0.8. Given the findings inthis chapter, which successfully utilized Explainable Boosting Machines andstatistical analyses to understand the impact of varied network parameterson the performance and diversity of 144 different model architectures, futurework should explore the applicability of this analytical methodology to ViTs.Such an investigation would be valuable in determining if similar relation-ships between network architecture choices (like network size and attentionmechanisms specific to ViTs) and outcomes like performance, pairwise di-versity, and even class-specific performance imbalances, hold true for thesetransformer-based models. This could yield significant insights into optimiz-ing ViT ensembles and further probing the limits of model-centric AI withinthis new architectural paradigm, potentially leading to more computationallyefficient and robust segmentation solutions.This chapter served two purposes for this dissertation: first, it showed thelimitations of model-centric AI and hyperparameter tuning; second, it showedthat diversity, an important property of network ensembles, can be predictedand promoted by varying network architecture. The first goal is arguablythe more important, and showing that many models can achieve statisticallynon-inferior performance despite requiring different computational resourcesdemonstrates the diminishing returns of model-centric AI.A final interesting observation from this analysis is the highly imbalancedperformance across classes in the CityScapes dataset, an example of modelbias towards the majority classes indicating a lack of fairness. Some classesperform exceptionally well, such as sky and road. Other classes perform verypoorly, such as truck, motorcycle, wall, rider, and bus. Interestingly, thereappears to be different reasons for poor performance. Bicycle, for example,has a reasonable number of examples but very small components in the image;train and bus, on the other hand, are very large components in an image butseen very rarely. Moreover, the balancing method used, oversampling andaugmenting, we used in this chapter does not appear very effective, giventhe disparity in final model performance across classes. In the final chapter,we explore this class imbalance and try to device an efficient way to focuson examples with high uncertainty, whether that uncertainty is caused bydifficult features or under-representation in the dataset.88Chapter 6Conformal-in-the-LoopThis chapter seeks to improve upon the CityScapes Semantic Segmentationdataset [44] results from Chapter 5. There, we showed that model-centric AIhad diminishing returns, with many architectures of different sizes producingsimilar results. For this chapter, we make one notable change; we turn theproblem from multi-label classification (where one pixel can take on manyvalues) to multi-class classification, where the network is forced to select asingle class for each pixel. To improve performance, we seek a data-centricapproach which can improve upon the main difficulty of the CityScapesdataset: some classes are significantly harder than others. Class imbalance isa known issue in deep learning, and a major source of unfairness and biasin models. Usually, the best course to resolving this imbalance is to collectmore data, such that the model has more information to learn from. However,collecting and labelling these examples is a time-consuming task, so we turninstead to data-centric AI as an alternative.As shown in Chapters 1 and 2, three common ways to employ data-centricAI are: to correct or prune noisy labels; to select examples in an order thatimproves learning; or, to automatically label additional examples. These arecalled, respectively, Confident Learning, Curriculum Learning, and ActiveLearning. Here, since we are unable to replicate the data collection processused in the CityScapes dataset, we cannot use additional data, even unlabeleddata, without expanding the domain of the model. However, CityScapesdoes provide coarsely annotated examples, which could possibly be refined.Interestingly, the first two approaches generally agree that harder examplesshould be weighted higher. This idea is also employed in boosting, with themost notable algorithm AdaBoost where subsequent learners weight examplesbased on previous error [65]. However, this weighting is only useful to a point.If examples are hard due to mislabelling, these examples should be ignoredentirely.896.1. INTRODUCTION6.1 IntroductionClass imbalance [2, 46] arises when certain classes, often referred to asâheadâ classes, are represented by a large number of samples, while others, orâtailâ classes, are underrepresented. Datasets with such imbalance tend to biasmodels toward the majority classes, leading to underrepresentation and under-learning of minority classes. Imbalance-related biases are well-documented:models trained on such data generally optimize for majority-class accuracy,resulting in reduced performance on minority categories. Common approachesto address class imbalance include resampling and reweighting, which adjusteither the sample distributions or the class-specific weights to encouragelearning from minority classes.In many real-world scenarios, noisy labels and class imbalance coexist,presenting a multifaceted problem that single-focus methods struggle tosolve. However, techniques aimed at mitigating label noise typically assumea balanced dataset, ignoring the complexities that arise in imbalanced data.Similarly, methods addressing class imbalance generally presume that labelsare accurate, failing to account for noise in the underrepresented classes. Insuch cases, label noise can exacerbate learning bias: when minority classescontain mislabeled samples, models may treat hard-to-learn but clean samplesas noise, neglecting genuine examples from the tail classes. This overlapcomplicates the modelâs ability to distinguish between true minority samplesand noisy samples from majority classes.Class imbalance is a pervasive issue in many machine learning tasks,where the majority class is overrepresented in training data, often leadingmodels to underperform on minority classes. This imbalance causes modelsto overpredict the majority class, resulting in frequent misclassification ofminority instances, as noted in several studies [12, 13]. Solutions for addressingclass imbalance generally fall into two main categories: data-centric andalgorithmic approaches [96].Data-centric methods focus on adjusting the training data distribution.Undersampling the majority class, though effective, risks discarding criticalfeatures, which can lead to a loss of valuable information. Clustering-basedundersampling has been proposed to mitigate this by removing redundantor noisy samples while retaining informative examples [10, 100, 122, 200].Oversampling the minority class can also help [79, 173], but it can increase therisk of overfitting and prolong training. Synthetic data generation techniques,such as SMOTE [53, 135], and data augmentation [41, 207] are commonlyused to introduce new information, effectively balancing the data withoutmerely duplicating samples.906.1. INTRODUCTIONOn the other hand, algorithmic approaches often modify the loss functionto place greater emphasis on learning from minority samples. One prominentmethod is Focal Loss, which incorporates a focusing parameter, γ, to pri-oritize difficult examples, and a per-class weighting factor, α, to counteractimbalance [110]. While α is usually determined by the class distribution, itcan overlook the variability in sample quality. The focusing parameter isfrequently set to 2 but can be fine-tuned to optimize model performance.Recent advances have introduced additional loss functions tailored for im-balanced datasets. For instance, Label-Distribution-Aware Margin (LDAM)loss encourages larger margins for minority classes, aiming for better gen-eralization in imbalanced settings [35]. Another effective approach is theDistribution-Balanced Loss, which has shown significant improvements inmulti-label text classification by re-weighting the contributions of positiveinstances during training [195].This chapter begins with the observation that the issues of noisy labelsand class imbalance often conflict. Class imbalance necessitates that thelearning algorithm focuses on hard examples, whereas noisy labels can resultin these hard examples being mislabeled. we aim to tackle the challenges oftraining with noisy, imbalanced data by identifying hard examples for higherweighting while excluding really hard examples that are possibly mislabeled.To achieve this, we apply ConP to enable online data curation in a singletraining run with minimal overhead. While previous research has leverageduncertainty quantification to enhance semi-supervised learningâgrowingtraining sets through online labeling of unknown examples or pruning out-of-sample instancesâthese methods often incur high computational costs,typically requiring multiple training runs and cross-validation of models. Incontrast, our approach utilizes softmax outputs to generate minimal predictionsets, providing a practical measure of uncertainty. These prediction sets caneffectively exclude potentially mislabeled data by yielding empty predictions,streamlining the training process while addressing both class imbalance andnoisy labels.These interconnected challenges highlight the need for a more robustapproach to training DNNs on imbalanced and noisy data. To addressthis, we introduce Conformal-in-the-Loop (CitL), a novel uncertainty-awaretraining method that dynamically adjusts sample weights and prunes datain response to uncertainty levels calculated through conformal prediction.Specifically, CitL applies conformal prediction to assess sample uncertainty,which is then used to strategically modify training in two primary ways:(1) Sample Weighting â assigning higher weights to samples with moderateuncertainty that are likely to be informative, thereby enhancing learning in916.2. CONFORMAL PREDICTIONminority classes, and (2) Sample Pruning â identifying and removing sampleswith high uncertainty that are likely mislabeled, reducing the noise that wouldotherwise degrade model performance. By leveraging these two mechanisms,CitL selectively refines the training data, ensuring the model focuses onreliable, informative samples. Through this targeted curation of data, CitLaddresses the dual challenges of imbalance and noise while maintainingcomputational efficiency. Extensive experimental evaluations demonstratethat CitL significantly improves model performance across common datachallenges of imbalanced, noisy labels. To summarize, our contributions are:â Conformal prediction is an effective tool for balancing easy and chal-lenging examples during training.â A weighting and pruning mechanism that performs well under labelnoise, showing particular effectiveness at lower noise levels commonlyencountered in practice.â Highly uncertain examples are pruned at training time by leveragingthe Least-Ambiguous set-valued Classifier (LAC)âs ability to produceempty prediction sets.â Dynamic weighting to examples at different stages of training to addressclass imbalance, with higher weights early on to condition the networkon difficult features.â Minimal computational overhead, increasing training time per step byonly 11% for classification and 4% for segmentation tasks.6.2 Conformal PredictionConformal prediction (ConP) is a statistical method that leverages apre-trained model, such as a neural network classifier, to construct predictionsets with statistically valid uncertainty estimates. The model is first usedto generate class probabilities, and a subset of unseen data, designated ascalibration data, is used to calibrate the prediction sets. ConP provides a pre-diction set for each instance such that the probability of containing the correctlabel is approximately 1â α, where α is a user-defined error tolerance. Thiscalibration step involves a non-conformity score that quantifies the modelâsconfidence in its predictions, with higher scores indicating less confidence.This framework ensures statistically valid prediction sets regardless of theunderlying model or data distribution.926.2. CONFORMAL PREDICTIONConP is widely applicable across fields like computer vision and natu-ral language processing, converting heuristic uncertainties into statisticallygrounded measures. For a prediction set P, ConP constructs this set tosatisfy:1â α â¤ P (YË â P) â¤ 1â α+ 1n+ 1, (6.1)where n is the number of calibration samples. This bound provides a proba-bilistic guarantee on the prediction setâs accuracy.The ConP process involves three primary steps:1. Non-Conformity Scoring: Define a heuristic-based non-conformityscore function s(x, y) â R, where higher scores denote higher uncertaintythat sample x belongs to class y.2. Quantile Calculation: Calculate the quantile qË as the (n+1)(1âÎ±)nquantile of the calibration scores (s(X1, Y1), ..., s(Xn, Yn)), which ad-justs for the desired confidence level.3. Prediction Set Construction: Formulate the prediction set for newinstances as C(Xtest) = {y : s(Xtest, y) â¤ qË}.Two common choices for non-conformity scoring are the Least AmbiguousClassifier (LAC) and the Adaptive Prediction Set (APS). LAC, representedby 1â YËc, where YËc is the modelâs output probability for the correct label,is computationally efficient and ensures marginal coverage guarantees [158].APS, in contrast, sums the model outputs that are higher than the score of thetrue class. At inference, APS iteratively includes classes in the prediction setuntil the cumulative probability meets the target quantile [154]. While LACmay result in empty prediction sets if no scores exceed the target quantile,APS always produces a non-empty set.Recent research highlights ConPâs computational efficiency and theoreti-cal validity for training-phase applications. For instance, [54] introduces adifferentiable approximation of the cumulative distribution function (CDF)of softmax outputs, which informs a novel loss function that incorporatescalibration uncertainty into the training process. This method enables morecompact prediction sets and enhances conditional coverage through precisecalibration, supported by a hold-out calibration dataset.6.2.1 Proposed CitL MethodFig. 6.1 provides an overview of the proposed Conformal-in-the-Loop(CitL) framework. At a high level, CitL integrates conformal prediction936.2. CONFORMAL PREDICTIONFigure 6.1: An overview of the proposed Conformal-in-the-Loop (CitL)framework. conformal prediction is used to generate prediction sets during self-supervised machine learning. Training examples are weighted by uncertainty;validation examples are used to calibrate the uncertainty model.(ConP) into the training process to adjust the weighting of examples, therebyenhancing overall network performance. Formally, given a set of examples{X1, . . . ,Xn} and ground truth labels {Y1, . . . , Yn}, we use a neural networkto learn an approximate classification function f : Xâ YË . The classifier isaccurate when the function produces the correct ground truth class, YË = Y .However, using ConP, we allow YË to take on a set of class labels from possibleclasses {C1, . . . , Cm}, rather than a single label, which we call the predictionset P. we begin with the notion that the size of the prediction set is anestimate of the practical uncertainty of a model. Given the possibility ofempty prediction sets, both pruning and weighting are achieved by multiplyingthe loss by the prediction set size:Lâ²x = |Px| Lx. (6.2)Notably, the weighting of the loss is not required to be differentiable,removing the greatest limitation to applying additional non-conformity scores,as experienced in [54]. This work evaluates the Least-Ambiguous set-valuedClassifier (LAC) [158] implementation of conformal prediction, relying on itsability to produce empty prediction sets in order to prune highly uncertainexamples. However, it would be interesting to consider APS [154], and thispossibility is discussed in future work.Since ConP can be achieved using only the softmax outputs of the classifier,which are required for training already, we can cache the non-conformity scoresfrom each batch and pass them to the conformal classifier. we summarize ourepoch training process in Algorithm 1. Note that this is a slightly modified946.2. CONFORMAL PREDICTIONAlgorithm 1: CitL Training EpochRequire: A parameterized function f â²(X,W)Require: A fit conformal classifier CRequire: A loss function L(YË , Y )1: for Each Minibatch in Training Set do2: β â Random Sample from {(X0, Y0), ...(Xn, Yn)}3: Î²YË â f â²(Î²x,W)4: βP â C(Î²yË)5: WâW â Î·â(|Î²P|L(Î²YË , βY ))6: end forstandard iterative mini-batch optimization algorithm, with line 4 added tocalculate the conformal prediction sets and line 5 modified to weight theloss by prediction set size. The validation epoch is also modified to fit theconformal classifier on the first portion of the dataset, which then evaluatesthe uncertainty on the remaining data, as summarized in Algorithm 2.Fitting the conformal classifier introduces memory and computationalcomplexity proportional to the size of the calibration set, making it advanta-geous to keep this set as small as possible while still ensuring representativesampling of the data. During training, and once the conformal classifier iscalibrated in the validation step, computational dependency shifts to themini-batch size, which is typically manageable in most setups. This designallows efficient scaling, as the memory demands in subsequent steps remainrelatively low.6.2.2 Evaluation BenchmarksFor our evaluation benchmarks, we focus on two tasks: multiclass classi-fication and semantic segmentation. For multiclass classification, we selectCIFAR-10 [97] since it is known to be well-balanced; we then syntheticallyintroduce label noise and class imbalance. For semantic segmentation, weselect the CityScapes dataset [44], which is known to be imbalanced.Multiclass Classification. we utilize the CIFAR-10 dataset, which consistsof 32x32 color images. Given the small dimensions of these images, weresize them to 224x224x3 to provide adequate training information for thenetwork. The dataset includes 10 classes, each with 6000 images, totaling60,000 images. The validation set is created by randomly sampling 10% ofthe training examples.Semantic Segmentation. we use the CityScapes dataset, which contains956.2. CONFORMAL PREDICTIONAlgorithm 2: CitL Validation EpochRequire: A parameterized function f â²(X,W)Require: A loss function L(YË , Y )Require: Calibration Set Size |A|1: Vx â []2: VYË â []3: VY â []4: while Validation Data is not exhausted do5: β â Random Sample from {(X0, Y0), . . . , (Xn, Yn)}6: Î²YË â f â²(Î²x,W)7: if idx < |A| then8: Vx.append(βx)9: VYË .append(Î²YË )10: VY .append(βY )11: else if idx = |A| then12: Fit C from Vx, VYË , and VY .13: else14: βP â C(Î²YË )15: end if16: end while2048x1024 RGB images of urban street scenes. Training images are patchedto 512x1024x3, effectively reducing the total number of pixels by a factor offour to allow a batch size of 6. Validation and testing are conducted at fullresolution with a batch size of 1. Class labels are mapped to the 20-classlabelling method recommended in the CityScapes documentation [44], andthe background class is ignored when calculating loss or evaluation metrics.Since the test dataset provided by CityScapes has no public annotations, wereserve 20% of the 3,477 image train set for validation; the 500 image val setis reserved for our test set.6.2.3 Experimental Setup and Implementation DetailsMulticlass Classification with CIFAR-10. To examine the methodâscapability to identify and mitigate class imbalance, we randomly downsamplethe training and validation sets for the Airplane and Automobile classes to20% of the total available examples for each class, with this downsamplingperformed prior to the train-validation split and not applied to the testset. Label noise is synthetically added to the remaining training examples;966.2. CONFORMAL PREDICTIONimportantly, no label noise is introduced into the validation or test sets tomaintain accurate labels for this small subset. For calibration, we allocate20% of the validation images from CIFAR-10 as the calibration set, enablingus to utilize a larger portion of the validation set for calibration.Semantic Segmentation with CityScapes. we allocate 10% of thevalidation images as the calibration set. Given the large number of pixelsinvolved in semantic segmentation, quantile estimation can be computationallyintensive; thus, we uniformly sample 10% of the pixel examples from thecalibration set, resulting in a total of 1% of the validation pixels being usedfor fitting the quantile.Other Details. we implement our method, ConP, using the Least AmbiguousSet-Valued Classifier (LAC) [157] in PyTorch, leveraging GPU accelerationfor enhanced computational efficiency. Data augmentation is performedusing the Albumentations library [30], primarily aimed at preventing modeloverfitting. For CIFAR-10, we apply augmentations such as Horizontal Flip(50%), Crop and Pad (ranging from 0 to 56 pixels), and Coarse Dropout(covering a 56x56 area). For the CityScapes dataset, augmentation beginswith a random scaling of the image size by up to ±10%, followed by a potentialrotation of up to 10 degrees occurring 50% of the time. The image is thenrandomly cropped to a fixed size of 512x1024 pixels, with a 50% probabilityof horizontal flipping, and there is a 25% chance of applying one of severaltransformations, including reducing image quality to 90%, applying a blur,or adding Gaussian noise, along with brightness and contrast adjustmentsperformed in 25% of cases.6.2.4 Training SetupThe training environment includes a devcontainer with Python v3.12,PyTorch Lightning v2.4.0 [144], PyTorch v2.4 [6], and CUDA 12.1.0. Clas-sification tasks are performed using MNasNet-Small [179] from the timmlibrary [193], while the segmentation task employs EfficientNet-b0 [180] in aU-Net [155] configuration from Segmentation Models PyTorch [198]. Clas-sification models are trained from scratch, whereas segmentation modelsuse transfer learning from pretrained ImageNet weights. The best modelto evaluate against the test set is selecting using minimum validation lossfor baseline models. When using the method, the dynamic loss reweightingmeans that loss values cannot be effectively compared against epochs, so weselect the best model by maximum validation accuracy for classification andmIoU for segmentation.The learning rate is fixed at 5Ã 10â4 and is reduced by a factor of 0.2976.2. CONFORMAL PREDICTIONupon plateau, with a patience of 10 epochs. The minimum learning rate isset to 10â6, and weight decay is configured at 10% of the initial learningrate. Early stopping [142] is applied based on validation loss with a patienceof 20 epochs. The ADAM optimizer [93] is used with default parametersβ = (0.9, 0.999). All experiments are conducted on a single A6000 GPU with256GB of system RAM.For segmentation, weighting is unnormalized, allowing examples to beweighted anywhere from 0 to the total number of classes, which is 19 inthe case of CityScapes. Although this was found to work well with thesmooth gradient provided by the large number of examples in segmentation,classification has a much smaller number of examples per batch, leading tolarge variations in gradient magnitude. To mitigate the effect of this variationon learning rate, the weighting of each batch is normalized by the meanweight value in the batch.We conduct a grid search to identify the optimal α values, exploring arange of 0.10 to 0.19 for classification and 0.01 to 0.10 for segmentation tasks.6.2.5 Evaluation MetricsGiven N as the number of classes, TPi as the number of true positives forclass i, TNi as the number of true negatives for class i, FPi as the number offalse positives for class i, and FNi as the number of false negatives for classi, we compute the macro accuracy for classification tasks as follows:Accuracy =1NNâi=1TPi + TNiTPi + TNi + FPi + FNi. (6.3)This metric provides an average accuracy across all classes, allowing for abalanced assessment of model performance, particularly in scenarios involvingclass imbalance. For segmentation tasks, we evaluate performance using themean Intersection over Union (mIoU), defined as:mIoU =1NNâi=1TPiTPi + FPi + FNi. (6.4)To address potential issues with undefined mIoU valuesâespecially whena class is not represented and the union is zero, we concatenate all examplesfor a given class in the test set before performing the calculation.986.3. RESULTS AND DISCUSSION6.2.6 Reproducibilityhttps://github.com/brandongk-ubco/conformal-in-the-loop hosts the com-plete implementation while https://app.neptune.ai/o/conformal-in-the-loop/org/citl/runsstores records of training runs and ONNX models.6.3 Results and DiscussionWe compare the proposed CitL method against a baseline model trainedwith either conventional cross-entropy loss or focal loss. Unless specifiedotherwise, the term âbaselineâ refers to the model trained using cross-entropyloss. Furthurmore, we conduct six distinct experiments for classification, eachvarying the noise level from 0% to 50%. For segmentation, we perform asingle experiment at 0% noise.6.3.1 Preliminary Experiment: Selecting the Alpha ValueFrom the grid search, we identify the optimal α values for each experiment.Table 6.1 summarizes the best performance metricsâaccuracy for CIFAR-10and mIoU for CityScapesâachieved by our CitL method with these optimalα values, as well as the performance of the baseline model. These identified αvalues are consistently utilized in subsequent analyses across all correspondingexperiments.Table 6.1: Performance metrics (accuracy for CIFAR-10 and mIoU forCityScapes) for the CitL method with optimal α values compared to thebaseline model across different datasets. â represents the difference in per-formance between CitL and the baseline.Noise % α Baseline Method âImbalancedCIFAR-100 0.18 83.0 85.5 2.510 0.11 75.5 79.5 4.020 0.14 74.0 77.1 3.130 0.13 69.4 72.5 3.140 0.11 67.4 70.8 3.450 0.17 63.3 69.4 6.1CityScapes - 0.02 66.1 71.1 5.0996.3. RESULTS AND DISCUSSIONFigure 6.2: Segmentation mIoU on the validation set for various values of α.The baseline performance is indicated by the dotted blue line.Figure 6.3: Segmentation mIoU for values of α [0.01, 0.10]. The best mIoUof 71.1 is achieved at α = 0.02. Baseline mIoU (66.1) is represented by thedotted blue line.6.3.2 Classification ResultsFig. 6.4 illustrates the accuracy of our method on the CIFAR-10 datasetacross varying levels of synthetic noise added to the training set. Notably,our method performs comparably to the cross-entropy baseline when no noiseis introduced and surpasses baseline performance at all noise levels above 0%.Table 6.2 provides a breakdown of the methodâs performance using theoptimal α (see Table 6.1) across different noise levels, categorized by class.As expected, the method demonstrates significant improvements in the down-sampled classes of airplane and automobile. At every noise level, the airplaneclass exhibits the lowest performance; therefore, enhancing performance inthis class contributes to achieving a more balanced model overall.1006.3. RESULTS AND DISCUSSIONFigure 6.4: Classification accuracy on imbalanced CIFAR-10 using our CitLmethod and the baseline with cross-entropy loss (blue dotted line) and focalloss (orange dotted line). The box plots show 10 α values ranging from 0.10to 0.19.6.3.3 Segmentation ResultsFig. 6.3 illustrates the segmentation results of our method on the CityScapesdataset using various values of α, comparing them to the baseline model.At lower values of α, our method significantly outperforms the baseline,demonstrating a clear advantage in segmentation accuracy. However, as αincreases, we observe a noticeable decline in performance.Fig. 6.2 presents the validation mean Intersection over Union (mIoU)across training epochs. The results indicate that improvements in mIoU dueto weighting become evident early in the training process, with significantgains observed after just a few epochs. The decline in performance withincreasing α is discussed in more detail in Section 6.3.6.Table 6.3 details the performance of our method with α = 0.02 by class.As expected, the method shows notable improvements in challenging classessuch as motorcycle, truck, bus, rider, train, fence, traffic light, traffic sign,bicycle, person, terrain, pole, and sidewalk, with fmy of these classes achievingmIoU improvements exceeding 10. Additionally, even some of the relativelyeasier classes, including building, vegetation, road, and sky, exhibit minorperformance gains when our method is applied. Only the wall class showed amodest decrease in mIoU of 0.5.1016.3. RESULTS AND DISCUSSIONTable 6.2: Classwise Accuracy for the imbalanced CIFAR-10 dataset acrossvarious levels of synthetic noise. The table shows the baseline performance,performance with the method applied, and the resulting improvement foreach class. Light red represents the worst-performing class (airplane in allcases) whose accuracy is improved by the method at all noise levels. Lightgreen represents the best-performing class (ship or truck).Noise Level 0% 10% 20% 30% 40% 50%baseline method â baseline method â baseline method â baseline method â baseline method â baseline method âAirplane 64.4 73.6 9.2 57.4 62.0 4.6 45.3 62.4 17.1 42.1 48.2 6.1 45.0 49.1 4.1 34.4 52.7 18.3Automobile 83.8 82.0 -1.8 74.4 80.4 6.0 72.4 83.3 10.9 59.2 74.3 15.1 51.4 66.8 15.4 57.4 72.7 15.3Bird 81.9 83.5 1.6 70.3 77.4 7.1 74.6 72.8 -1.8 74.9 68.2 -6.7 61.7 66.8 5.1 62.0 63.1 1.1Cat 73.9 76.3 2.4 64.2 66.7 2.5 60.5 66.1 5.6 39.6 51.6 12.0 53.3 61.5 8.2 47.4 48.5 1.1Deer 83.8 88.9 5.1 78.0 82.5 4.5 70.4 77.2 6.8 63.8 69.6 5.8 59.2 69.9 10.7 67.5 73.0 5.5Dog 76.2 81.3 5.1 68.2 73.1 4.9 69.9 72.7 2.8 74.6 75.0 0.4 72.1 65.9 -6.2 62.5 65.1 2.6Frog 89.2 89.0 -0.2 83.2 84.4 1.2 86.0 80.7 -5.3 83.0 81.4 -1.6 82.2 79.0 -3.2 71.9 73.2 1.3Horse 90.0 92.5 2.5 79.7 88.7 9.0 81.9 82.0 0.1 81.3 83.3 2.0 78.2 76.3 -1.9 70.1 78.2 8.1Ship 93.5 94.0 0.5 90.2 90.6 0.4 88.2 88.2 0.0 91.6 87.3 -4.3 85.8 87.7 1.9 79.0 86.1 7.1Truck 93.7 93.8 0.1 89.6 89.6 0.0 90.5 85.5 -5.0 84.1 85.6 1.5 85.1 84.9 -0.2 81.2 81.7 0.5Best - Worst 29.3 20.4 -8.9 32.8 28.6 -4.2 45.2 25.8 -19.4 49.5 39.1 -10.4 40.8 38.6 -2.2 46.8 33.4 -13.4Overall 83.0 85.5 2.4 75.5 79.5 4.0 74.0 77.1 3.1 69.4 72.5 3.0 67.4 70.8 3.4 63.3 69.4 6.1Table 6.3: Classwise IoU for the CityScapes dataset with the best trainingrun using α = 0.02. The table shows the baseline performance, performancewith the method applied, and the resulting improvement for each class.Baseline Method âRoad 97.3 97.5 0.2Sidewalk 80.3 81.3 1.0Building 90.1 90.8 0.7Wall 45.6 45.1 -0.5Fence 47.5 51.3 3.8Pole 58.1 60.9 2.8Traffic light 57.9 64.3 6.3Traffic sign 72.0 74.3 2.3Vegetation 91.3 91.8 0.5Terrain 56.9 58.5 1.6Sky 93.2 94.1 0.9Person 76.7 78.8 2.2Rider 50.9 55.0 4.1Car 92.8 93.5 0.7Truck 49.0 66.7 17.7Bus 48.0 66.0 18.1Train 39.6 59.6 20.1Motorcycle 36.7 47.8 11.1Bicycle 71.9 73.8 1.9Overall 66.1 71.1 5.01026.3. RESULTS AND DISCUSSION6.3.4 Computational OverheadFig. 6.5 and Table 6.4 present an analysis of the overhead associated withour method across various tasks. Notably, classification tasks demonstratea higher overhead compared to segmentation tasks, likely because baselinetraining steps are faster. However, the overall increase in overhead remainssmall, with a mean increase of 11.2% in training time per step for classificationand 4.0% for segmentation. These findings are encouraging and suggest thatour method is likely to scale effectively to larger tasks and datasets.Figure 6.5: Average per training-step time of CitL over baseline. MultipleCitL values represent different α hyperparameters. (CIFAR-10 11.2 ± 0.8%,CityScapes 4.0 ± 0.4%)Table 6.4: Percentage overhead of our CitL relative to the baseline, showingmean, standard deviation (Std), minimum (Min), and maximum (Max) valuesacross datasets.dataset mean std min maxCIFAR-10 11.2% 0.8 9.6 13.2CityScapes 4.0% 0.4 3.0 4.46.3.5 Example WeightingThis section presents an analysis of the weighting generated by our methodon the imbalanced CIFAR-10 dataset and the CityScapes dataset. Theprimary objective is to demonstrate the methodâs effectiveness in addressingdataset imbalances by emphasizing challenging examples.1036.3. RESULTS AND DISCUSSION(a) Weight Range, showing the dif-ference in mean weights between thehighest-weighted and lowest-weightedclasses on the imbalanced CIFAR-10dataset, stratified by noise level. Onlythe best performing α value is shownfor each noise level.(b) Uncertainty by noise level for theimbalanced CIFAR-10 dataset. Un-certainty is defined as the percent-age of examples whose prediction setcontains more than one possible class.With no noise, the model is able toreduce uncertainty to 0; noise makesthis reduction to 0 impossible.Figure 6.6: Weight range and uncertainty by training step and noise level.For the classification task, Fig. 6.6a illustrates the variation in mean weightbetween the highest- and lowest-weighted classes across different trainingsteps. Notably, weighting is highest during the early training epochs anddecreases as training progresses, reflecting the modelâs evolving uncertainty,as shown in Fig. 6.6b. In the absence of noise, the model successfully reducesuncertainty to zero; however, this complete reduction is unattainable in thepresence of noise. Similarly, noise inhibits the methodâs ability to diminishthe weighting effect to zero.Fig. 6.7 illustrates the breakdown of weighting for CIFAR-10 by classthroughout the training epochs. In Epoch 1, the network demonstrates a lackof learning about the dataset, assigning roughly equal weights to all classes.By Epoch 2, the model begins to produce lower weights for the ship and truckclasses. Between Epochs 10 and 20, the method identifies the downsampledclass of airplane as having a high level of uncertainty, thus assigning it a higherweight. In contrast, the other downsampled class, automobile, is not weightedas heavily, likely due to its shared features with other classes, allowing it tomaintain good performance despite downsampling. Notably, the cat and dogclasses receive the highest weights, likely due to their similarity and difficultyin distinction. As training progresses, the weighting for airplane, cat, anddog declines, reflecting the networkâs increasing certainty in its predictions,1046.3. RESULTS AND DISCUSSION(a) Epoch 1 (b) Epoch 2(c) Epoch 10 (d) Epoch 20(e) Epoch 40 (f) Epoch 100Figure 6.7: Mean weight by class across training epochs for the imbalancedCIFAR-10 dataset with no synthetic label noise. Each subfigure shows thedistribution of mean class weights at different training epochs, illustratinghow the algorithm adjusts class weights over time.1056.3. RESULTS AND DISCUSSION(a) Epoch 1 (b) Epoch 2(c) Epoch 20 (d) Epoch 60(e) Epoch 100 (f) Epoch 140Figure 6.8: Mean weight by class across training epochs for the CityScapesdataset for best value of α = 0.02. Each subfigure shows the distribution ofmean class weights at different training epochs, illustrating how the algorithmadjusts class weights over time.1066.3. RESULTS AND DISCUSSION(a) Imbalanced CIFAR-10 at noiselevel 0%(b) Imbalanced CIFAR-10 at noiselevel 10%(c) CityScapesFigure 6.9: Amount Pruned by Training Step and Alpha.culminating in small weight adjustments by Epoch 40. By Epoch 100, themodel shows negligible differences in class weighting.Fig. 6.8 presents the breakdown of weighting for CityScapes by class.Initially, the model exhibits near-uniform uncertainty across classes. ByEpoch 2, easier classes such as road, vegetation, sky, and car attaining lowerweights, while more challenging classes like bicycle receive higher weights. Thistrend continues as the model refines its weights over time; by Epoch 20, theaverage weight for all classes falls below 3. Notably, by Epoch 30, the weightfor the motorcycle class rises to over 3.5, indicating the modelâs increasedfocus on harder-to-predict classes, despite overall improved performance. InEpochs 40 and 60, the model demonstrates reduced re-weighting, with onlythe most challenging classes (e.g., traffic light, traffic sign, rider, motorcycle,and bicycle) weighted near or exceeding 2.1076.3. RESULTS AND DISCUSSION(a) Imbalanced CIFAR-10 at noiselevel 0%(b) Imbalanced CIFAR-10 at noiselevel 10%(c) CityScapesFigure 6.10: Quantile by Training Step and Alpha.6.3.6 Example PruningPruning is controlled by α, with higher values leading to more pruning.From Fig. 6.9, we can see that many of the lower values of α perform nopruning at all, relying entirely on weighting to balance classes. Fig. 6.9a showsthe percentage of examples pruned for the imbalanced CIFAR-10 datasetwith no noise. No pruning is done until α â¥ 0.16, and at the best α = 0.18approximately 2% of the dataset is pruned. Similarly for CityScapes, at thebest α = 0.02, approximately 1% of the data is pruned.High values of α result in poor performance due to the preferential pruningof highly uncertain examples, which are usually from minority classes. Thiseffect is shown in Fig. 6.12 for CityScapes at the worst α = 0.09. At epoch 2,the relatively easy classes of sidewalk and terrain are weighted very highly.By epoch 20, the hard classes are heavily pruned; by epoch 60 the classes ofwall, traffic light, rider, truck, bus, train, motorcycle and bicycle are nearlypruned completely.The introduction of noise adds more uncertainty to the network which1086.3. RESULTS AND DISCUSSIONreduces the amount of pruning overall. Again, no pruning is done untilα â¥ 0.16; moreover, the amount of pruning is reduced when compared to the0% noise experiment. Pruning to the amount of noise (10%) is not achieveduntil approximately α = 0.24. Fig. 6.11 shows that the method performs welluntil α â¥ 0.21, at which point it starts to degrade in the same fashion asCityScapes experiment explored in Fig. 6.3. Still, for all values of α until 0.30,the method outperforms both the cross entropy loss and focal loss baselines.At α = 0.20, the last before the method begins to degrade, the method prunesapproximately 2.5% of the training examples.Figure 6.11: Imbalanced CIFAR-10 classification performance at 10% noiselevel. The blue dotted line represents the baseline using Cross Entropy Loss;the red dotted line represents the baseline using Focal Loss. The methodoutperforms both baselines at all tested values of α. At α = 0.20, the methodprunes â 2.5% of examples. Performance declines when α â¥ 0.21.6.3.7 Considerations for Selecting AlphaIn the proposed method, the parameter α serves as a crucial balancingfactor between the weighting of uncertain examples and the pruning of ex-tremely challenging examples. Lower values of α favor increased weightingand reduced pruning, whereas higher values promote greater pruning anddiminished weighting. The significance of the 0.50 quantile cannot be over-stated, as it represents the threshold at which the maximum prediction setsize is limited to 1. At this threshold, the network can influence relative classweights solely through pruning.In the context of the CityScapes dataset, this critical threshold is reachedwith α values of 0.6 and above (see Fig. 6.10c). Notably, this is the point1096.3. RESULTS AND DISCUSSION(a) Epoch 1 (b) Epoch 2(c) Epoch 20 (d) Epoch 60Figure 6.12: Mean weight by class across training epochs for the CityScapesdataset at worst value for α = 0.09. Each subfigure shows the distribution ofmean class weights at different training epochs, illustrating how the algorithmadjusts class weights over time.where the method begins to underperform compared to the baseline (seeFig. 6.3), indicating that α should be chosen to maintain the quantile belowthis level when dealing with well-labeled, large datasets.Low values of α can lead the algorithm to overemphasize hard examples,which may decrease the modelâs certainty over time (see Fig. 6.6b). Thisphenomenon is reflected in the quantile value, which one would expect toincrease as the network undergoes training. In Fig. 6.10a, for α values between0.10 and 0.15, the quantile appears to plateau; moreover, for α values below0.10, the quantile declines towards the end of training. This decline is likelya result of the network overfitting to uncertain examples. The impact of thiseffect becomes more pronounced with the introduction of noise, necessitatingeven higher values of α to sustain an increasing quantile (see Fig. 6.10b).For the CityScapes dataset, the quantile increases across all values of α.we attribute this to the high quality of labeling in this finely labeled dataset,1106.4. CONCLUSIONScoupled with the significant number of examples present in the segmentationtask. These factors contribute to a smoother loss landscape. Consequently,our method may be better suited for datasets with a large number of examplesper batch, suggesting its greater efficacy in segmentation tasks compared toclassification.6.3.8 Model CalibrationThe calibration of the model is explored on the CityScapes dataset. Themodel shows a higher probability of predicting Bus, whose classwise IoUincreased by 18.1. This increase is largely the result of resolving uncertainexamples, where confidence ranged from 0.4 to 0.8. The overall calibrationof the model is not largely effected, with the reliability curve very closeto that of the baseline model (see Figure 6.13a). Similarly for the classMotorcycle, whose classwise IoU increased by 11.1, the model shows a muchhigher probability of identifying this class. However, in this example, theincreased performance is coming from the confidence range 0.1 to 0.6 in thebaseline model. Again, the reliability curve of both the CitL and baselinemodels are very close, showing negligible impact on model calibration. Forwall, which is the only class which saw a decrease in performance (-0.5), weagain see a much higher rate of predicting this class. However, the reliabilitycurve in this case shows a general reduction of performance, especially athigher confidence values; this shows that the method is focusing on thesehard examples, but is likely overfitting either because of a limited numberof examples or limited diversity across examples. This could potentially beresolved by increasing data augmentation or introducing other regularizationtechniques.6.4 ConclusionsThis chapter proposes a novel method, Conformal-in-the-Loop (CitL),which integrates conformal prediction (ConP) directly into the training pro-cess to create an uncertainty-aware approach for weighting and pruningtraining examples with minimal computational overhead. Our approachshows improved accuracy in noisy, imbalanced multiclass classification on theCIFAR-10 dataset and achieves superior mIoU on the CityScapes dataset.CitL is particularly effective in addressing small, underrepresented classesthat are often overshadowed by dominant, easily segmented classes. wealso present an in-depth analysis of CitLâs mechanisms, demonstrating howit prioritizes uncertain examples early in training and increasingly weights1116.4. CONCLUSIONS(a) Bus (+18.1) (b) Motorcycle (+11.1)(c) Wall (-0.5)1126.4. CONCLUSIONSmarginal examples later, highlighting its adaptability. Notably, this is accom-plished using a single hyperparameter, α, without requiring prior knowledgeof the dataset structure.The main limitation of this work is the challenge of distinguishing mis-labeled examples from unlearned examples in very noisy datasets (greaterthan 10%). This difficulty can result in preferential pruning of the minorityclass. Confident Learning addresses this issue by utilizing class-conditionalnoise rates [137]. An intriguing extension of our work would be to applyclass-conditional conformal prediction [50]. Exploration of noisy data wasalso limited to the classifcation task; segmentation of CityScapes was donewith only the finely labelled data. Initial work, not presented here, attemptedto explore the ability to resolve labeling issues with the coarse dataset, thiswork produce inadequete results with mIoU of only 0.21. A possible futureablation study could explore mixing of the coarse and fine data, and measur-ing mIoU as the percentage of coarse data was increased, comparing CitL tothe baseline under these conditions.The weighting of minority and majority classes is governed by the hy-perparameter α, which adjusts the size of the prediction set. However, thesize of this prediction set depends on the number of classes in the dataset.Consequently, for smaller tasks, such as binary classification, the maximumweighting is also relatively low. While this is intuitively reasonableâsincefewer classes can simplify the problemâit would be worthwhile to explorealternative weighting schemes, such as a non-linear function of the predictionset size.Using LAC to manage both weighting and pruning introduces a discon-tinuity for highly uncertain examples when they transition from maximumweighting to being pruned. Although this behavior is comparable to otherloss weighting methods [89], our findings indicate that the primary advantageof our method stems from the weighting process. It may be worthwhileto explore alternative conformal prediction methods, such as APS [154], toenable independent weighting and pruning.The simultaneous control of pruning and weighting creates sensitivity tothe hyperparameter α, which we addressed through a grid search. Futurework could explore dynamically determining α using heuristic methods. Thisexploration could build on the insights from Section 6.3.7, which suggestsmaintaining the quantile below 0.5 while avoiding quantile collapse due toexcessive focus on uncertain examples.While CitL demonstrates improvements in IoU, particularly for underrep-resented classes, it faces limitations regarding fairness, especially concerningthe pruning of minority classes and the re-weighting of examples. A key1136.4. CONCLUSIONSissue is the difficulty in distinguishing mislabeled examples from unlearnedones in very noisy datasets. The re-weighting mechanism, controlled by thehyperparameter α, is also sensitive; its dependence on the total number ofclasses means that for tasks with fewer classes, such as binary classification,the maximum weighting applied can low potentially limiting the support givento minority classes. Moreover, focusing solely on IoU is not the most robustapproach to evaluate fairness. To achieve a more nuanced understandingof fairness, it is crucial to explore and promote alternative metrics. Suchan approach aligns with the principles of Mondrian Conformal Prediction(MCP) [22], which ensures predictive validity conditional on specific groups.Adopting MCP would facilitate the use of fairness metrics that assess perfor-mance and error rates on a per-group basis, offering a clearer view of howthe model impacts different subgroups.In conclusion, our results indicate that CitL consistently improves modelperformance, achieving up to a 6.1% increase in classification accuracy anda 5.0 mIoU improvement in segmentation. Moreover, the computationaloverhead is minimal, with training time per step increasing by only 11% forclassification tasks and 4% for segmentation tasks.114Chapter 7Conclusion and Future WorkThis dissertation investigated the critical role of uncertainty quantifica-tion (UQ) in enhancing the robustness and fairness of artificial intelligencesystems. Chapter 3 explored a practical application in healthcare, specificallythe external testing of a commercial AI for breast cancer screening. Thisstudy highlighted a critical fairness concern: the AIâs underperformance onimages containing calcifications, a key indicator of potential malignancy. Thisissue was attributed to model uncertainty introduced by the AI when resiz-ing images. Furthermore, the AIâs confidence scores proved uninformativein distinguishing between images with and without these crucial features.Chapter 4 examined the impact of data uncertainty on AI for autonomoussystems. By analyzing a popular car detection algorithm using the KITTIdataset, we demonstrated how sensor limitations and inherent data variabilitycan degrade model performance, underscoring the need for robust UQ insafety-critical applications such as self-driving cars.Chapter 5 introduced a novel model-centric approach to measure andmitigate this uncertainty. We proposed promoting diversity by varyingmodel architectures within ensembles, rather than solely relying on data orhyperparameter variations. This chapter demonstrated that ensembles ofarchitecturally diverse models can achieve strong performance and that bothmodel performance and diversity can be predicted. This offers a path tomore efficient ensemble training. However, this exploration also underscoredclass imbalance as a significant challenge. For instance, there was difficultyidentifying underrepresented classes like bicycles or buses in urban scenes.Notably, Chapter 3 also shows significant class imbalance, where calcificationsrepresent 1.7% of the screening examples but contain 39.0% of the positivecancer examples.Motivated by the identified class imbalance issues, Conformal-in-the-Loop(CitL) was presented in Chapter 6. CitL integrates conformal predictioninto the training loop, enabling uncertainty-aware weighting and pruning ofexamples to address uncertainty at training time. This approach demonstratedsignificant improvements in classification accuracy and segmentation mIoUon CIFAR-10 and CityScapes, with minimal computational overhead (11%115for classification, 4% for segmentation). CitL proved particularly effectivefor improving performance on small, underrepresented classes, a crucial steptowards fairer AI systems.The presented research offers several promising avenues for future work,aimed at further advancing robust and fair AI through uncertainty quantifi-cation. Regarding the breast cancer screening AI, future investigations couldinclude incorporating nonstandard mammography views, which exhibited ahigher cancer rate and whose exclusion may introduce bias. Further workcould also explore methodologies to account for the retrospective nature ofthe study, where radiologist interventions may have influenced outcomes.Region-of-interest based analysis could also offer precise evaluation of AIperformance, especially when multiple lesions are present.Chapter 5 successfully employed Explainable Boosting Machines andstatistical analyses to explore relationships between varied network param-eters and the resulting performance and diversity. Future work could beto investigate the applicability of this framework to Vision Transformers(ViTs). Specifically, an investigation could determine if similar predictiverelationships exist between ViT-specific architectural choices, such as networksize, patch embedding strategies, and variations in attention mechanisms.CitL showed the potential for inadvertently pruning minority classes, andfuture work could involve extending CitL with class-conditional conformalprediction. Further, the control of minority and majority class weightingvia the hyperparameter α could be refined by exploring non-linear functionsof prediction set size, particularly for tasks with a small number of classes.Perhaps α could also be controlled dynamically, varying pruning and weightingover time. Investigating alternative conformal prediction methods, such asAPS, might allow for the decoupling of example weighting and pruning,potentially resolving discontinuities observed with the current LAC approach.Another possible practical application of this work lies in the refinement ofsegmentation labels, particularly in segmentation where labeling is costly.This could be explored by evaluating model performance when trained onmixed datasets of coarse and fine labels.In conclusion, this dissertation demonstrated that a deep understandingand active mitigation of model and data uncertainty are fundamental tobuilding AI systems that are robust and fair. Through studies in breastcancer screening and autonomous vehicle perception, and the developmentof novel methods like architectural ensemble diversity and Conformal-in-the-Loop training, we have shown pathways to address challenges such asdemographic underperformance, sensor limitations, and class imbalance.116Bibliography[1] Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya,I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman,S., Anadkat, S., et al. Gpt-4 technical report. arXiv preprintarXiv:2303.08774 (2023). â pages 11[2] Aguiar, G., Krawczyk, B., and Cano, A. A survey on learningfrom imbalanced data streams: taxonomy, challenges, empirical study,and reproducible experimental framework. Machine learning 113, 7(2024), 4165â4243. â pages 15, 90[3] Akhtar, N., and Mian, A. Threat of adversarial attacks on deeplearning in computer vision: A survey. IEEE Access 6 (2018), 14410â14430. â pages 30, 57, 59[4] Angelopoulos, A. N., and Bates, S. A gentle introduction toconformal prediction and distribution-free uncertainty quantification.arXiv preprint arXiv:2107.07511 (2021). â pages 33[5] Angluin, D., and Laird, P. Learning from noisy examples. MachineLearning 2, 4 (apr 1988), 343â370. â pages 13[6] Ansel, J., Yang, E., He, H., Gimelshein, N., Jain, A., Voz-nesensky, M., Bao, B., Bell, P., Berard, D., Burovski, E.,Chauhan, G., Chourdia, A., Constable, W., Desmaison, A.,DeVito, Z., Ellison, E., Feng, W., Gong, J., Gschwind, M.,Hirsh, B., Huang, S., Kalambarkar, K., Kirsch, L., Lazos, M.,Lezcano, M., Liang, Y., Liang, J., Lu, Y., Luk, C., Maher, B.,Pan, Y., Puhrsch, C., Reso, M., Saroufim, M., Siraichi, M. Y.,Suk, H., Suo, M., Tillet, P., Wang, E., Wang, X., Wen, W.,Zhang, S., Zhao, X., Zhou, K., Zou, R., Mathews, A., Chanan,G., Wu, P., and Chintala, S. PyTorch 2: Faster Machine Learn-ing Through Dynamic Python Bytecode Transformation and GraphCompilation. In 29th ACM International Conference on Architectural117BibliographySupport for Programming Languages and Operating Systems, Volume 2(ASPLOS â24) (Apr. 2024), ACM. â pages 97[7] Arbel, J., Pitas, K., Vladimirova, M., and Fortuin, V. Aprimer on bayesian neural networks: Review and debates, 2023. âpages 28[8] Aylward, S., Ourselin, S., Maier-Hein, K., Kalpathy-Cramer,J., Cardoso, J., Rubit, D., Zhou, K., Navab, N., and Feng, A.Medical open network for ai. https://monai.io/, 2020. â pages 72[9] Badirli, S., Liu, X., Xing, Z., Bhowmik, A., and Keerthi,S. S. Gradient boosting neural networks: Grownet. arXiv preprintarXiv:2002.07971 (2020). â pages 10[10] Barandela, R., Valdovinos, R. M., Sánchez, J. S., and Ferri,F. J. The imbalanced training sample problem: Under or over sampling?In Structural, Syntactic, and Statistical Pattern Recognition: JointIAPR International Workshops, SSPR 2004 and SPR 2004, Lisbon,Portugal, August 18-20, 2004. Proceedings (2004), Springer, pp. 806â814.â pages 16, 90[11] Bastani, O., Ioannou, Y., Lampropoulos, L., Vytiniotis, D.,Nori, A., and Criminisi, A. Measuring neural net robustness withconstraints. In Advances in Neural Information Processing Systems(2016), D. Lee, M. Sugiyama, U. Luxburg, I. Guyon, and R. Garnett,Eds., vol. 29, Curran Associates, Inc. â pages 31[12] Bauder, R. A., and Khoshgoftaar, T. M. The effects of varyingclass distribution on learner behavior for medicare fraud detection withimbalanced big data. Health information science and systems 6 (2018),1â14. â pages 16, 90[13] Bauder, R. A., Khoshgoftaar, T. M., and Hasanin, T. Anempirical study on class rarity in big data. In 2018 17th IEEE inter-national conference on machine learning and applications (ICMLA)(2018), IEEE, pp. 785â790. â pages 16, 90[14] BC Cancer. Bc cancer breast screening 2019 programresults. ttp://www.bccancer.bc.ca/screening/Documents/Breast-Screening-Program-Report-2019.pdfOnline:, Sept. 2020. â pages 37,39, 42118Bibliography[15] Bengio, Y., Louradour, J., Collobert, R., and Weston, J.Curriculum learning. In Proceedings of the 26th Annual InternationalConference on Machine Learning (New York, NY, USA, 2009), ICMLâ09, Association for Computing Machinery, p. 41â48. â pages 12[16] Bengio, Y., Simard, P., and Frasconi, P. Learning long-termdependencies with gradient descent is difficult. IEEE transactions onneural networks 5, 2 (1994), 157â166. â pages 7[17] Bergstra, J., and Bengio, Y. Random search for hyper-parameteroptimization. J. Mach. Learn. Res. 13, 1 (feb 2012), 281â305. â pages26[18] Blanks, R. G., Wallis, M. G., and Moss, S. M. A comparison ofcancer detection rates achieved by breast cancer screening programmesby number of readers, for one and two view mammography: results fromthe uk national health service breast screening programme. Journal ofMedical Screening 5, 4 (1998), 195â201. â pages 37[19] Blum, A., and Mitchell, T. Combining labeled and unlabeled datawith co-training. In Proceedings of the eleventh annual conference onComputational learning theory (1998), pp. 92â100. â pages 14[20] Blundell, C., Cornebise, J., Kavukcuoglu, K., and Wierstra,D. Weight uncertainty in neural networks. In Proceedings of the32nd International Conference on International Conference on MachineLearning - Volume 37 (2015), ICMLâ15, JMLR.org, p. 1613â1622. âpages 28[21] Bond, M., Pavey, T., Welch, K., Cooper, C., Garside, R.,Dean, S., and Hyde, C. Systematic review of the psychological con-sequences of false-positive screening mammograms. Health TechnologyAssessment 17, 13 (Mar. 2013), 1â170, vâvi. â pages 37[22] Boström, H., Johansson, U., and Löfström, T. Mondrian con-formal predictive distributions. In Proceedings of the Tenth Symposiumon Conformal and Probabilistic Prediction and Applications (08â10 Sep2021), L. Carlsson, Z. Luo, G. Cherubin, and K. An Nguyen, Eds.,vol. 152 of Proceedings of Machine Learning Research, PMLR, pp. 24â38.â pages 114[23] Bottou, L. Stochastic learning. In Advanced Lectures on MachineLearning, O. Bousquet and U. von Luxburg, Eds., Lecture Notes in119BibliographyArtificial Intelligence, LNAI 3176. Springer Verlag, Berlin, 2004, pp. 146â168. â pages 75[24] Bouthillier, X., Konda, K., Vincent, P., and Memisevic, R.Dropout as data augmentation, 2016. â pages 27[25] Boyd, N., Martin, L., Bronskill, M., Yaffe, M., Duric, N.,and Minkin, S. Breast tissue composition and susceptibility to breastcancer. JNCI Journal of the National Cancer Institute 102, 16 (7 2010),1224â1237. â pages 39[26] Breiman, L. Bagging predictors. Machine learning 24, 2 (1996),123â140. â pages 25[27] Browne, M. W. Cross-validation methods. Journal of MathematicalPsychology 44, 1 (2000), 108â132. â pages 25[28] Buolamwini, J., and Gebru, T. Gender shades: Intersectionalaccuracy disparities in commercial gender classification. In Proceedingsof the 1st Conference on Fairness, Accountability and Transparency (23â24 Feb 2018), S. A. Friedler and C. Wilson, Eds., vol. 81 of Proceedingsof Machine Learning Research, PMLR, pp. 77â91. â pages 20[29] Burhenne, H., Burhenne, L., Goldberg, F., Hislop, T., Worth,A., Rebbeck, P., and Kan, L. Interval breast cancers in the screeningmammography program of british columbia: analysis and classification.American Journal of Roentgenology 162, 5 (5 1994), 1067â1071. âpages 42[30] Buslaev, A., Iglovikov, V. I., Khvedchenya, E., Parinov, A.,Druzhinin, M., and Kalinin, A. A. Albumentations: Fast andflexible image augmentations. Information 11, 2 (2020). â pages 75,97[31] Caesar, H., Bankiti, V., Lang, A. H., Vora, S., Liong, V. E.,Xu, Q., Krishnan, A., Pan, Y., Baldan, G., and Beijbom, O.nuScenes: A multimodal dataset for autonomous driving. In 2020IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR) (2020), pp. 11618â11628. â pages 60[32] Canadian Cancer Statistics Advisory Committee incollaboration with the Canadian Cancer Society,Statistics Canada and the Public Health Agency of120BibliographyCanada. Canadian Cancer Statistics 2021. http://cancer.ca/Canadian-Cancer-Statistics-2021-EN, Nov. 2021. Accessed: [6-Nov-2024]. â pages 36[33] Canadian Partnership Against Cancer. Pan-canadian frame-work for action to address abnormal call rates in breast cancerscreening. https://s22457.pcdn.co/wp-content/uploads/2020/07/Framework-for-action-ACR-breast-screening-EN.pdf, 2020. [Ac-cessed 06-12-2024]. â pages 37, 51[34] Cancer Care Ontario. The ontario can-cer screening performance report 2020. Online:https://www.cancercareontario.ca/sites/ccocancercare/files/assets/OntarioCancerScreeningReport2020.pdf,Feb. 2021. â pages 42[35] Cao, K., Wei, C., Gaidon, A., Arechiga, N., and Ma, T.Learning imbalanced datasets with label-distribution-aware margin loss.Advances in neural information processing systems 32 (2019). â pages16, 91[36] Caruana, R., Lawrence, S., and Giles, C. L. Overfitting inneural nets: Backpropagation, conjugate gradient, and early stopping.In Advances in neural information processing systems (2001), pp. 402â408. â pages 5[37] Caruana, R., Niculescu-Mizil, A., Crew, G., and Ksikes,A. Ensemble selection from libraries of models. In Proceedings of theTwenty-First International Conference on Machine Learning (New York,NY, USA, 2004), ICML â04, Association for Computing Machinery,p. 18. â pages 26, 69[38] Castells, X., Román, M., Romero, A., Blanch, J., Zu-bizarreta, R., Ascunce, N., Salas, D., Burón, A., and Sala,M. Breast cancer detection risk in screening mammography after afalse-positive result. Cancer Epidemiology 37, 1 (2013), 85â90. â pages39[39] Chen, P., Liao, B. B., Chen, G., and Zhang, S. Understanding andutilizing deep neural networks trained with noisy labels. In Proceedingsof the 36th International Conference on Machine Learning (09â15 Jun2019), K. Chaudhuri and R. Salakhutdinov, Eds., vol. 97 of Proceedingsof Machine Learning Research, PMLR, pp. 1062â1070. â pages 13121Bibliography[40] Chen, R. J., Wang, J. J., Williamson, D. F., Chen, T. Y.,Lipkova, J., Lu, M. Y., Sahai, S., and Mahmood, F. Algorithmicfairness in artificial intelligence for medicine and healthcare. NatureBiomedical Engineering 7, 6 (2023), 719â742. â pages 34[41] Chen, X., Zhou, Y., Wu, D., Zhang, W., Zhou, Y., Li, B., andWang, W. Imagine by reasoning: A reasoning-based implicit semanticdata augmentation for long-tailed classification. In Proceedings of theAAAI conference on artificial intelligence (2022), vol. 36, pp. 356â364.â pages 16, 90[42] Cheng, B., Collins, M. D., Zhu, Y., Liu, T., Huang, T. S.,Adam, H., and Chen, L.-C. Panoptic-deeplab: A simple, strong, andfast baseline for bottom-up panoptic segmentation. In Proceedings of theIEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR) (June 2020). â pages 11[43] Ciatto, S., Ambrogetti, D., Bonardi, R., Catarzi, S., Risso,G., Rosselli Del Turco, M., and Mantellini, P. Second read-ing of screening mammograms increases cancer detection and recallrates. results in the florence screening programme. Journal of MedicalScreening 12, 2 (2005), 103â106. â pages 37[44] Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler,M., Benenson, R., Franke, U., Roth, S., and Schiele, B. Thecityscapes dataset for semantic urban scene understanding. In Proc.of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR) (2016). â pages 11, 70, 72, 74, 89, 95, 96[45] Cubuk, E. D., Zoph, B., Shlens, J., and Le, Q. V. Randaugment:Practical automated data augmentation with a reduced search space.In Proceedings of the IEEE/CVF Conference on Computer Vision andPattern Recognition (CVPR) Workshops (June 2020). â pages 27[46] Dablain, D., Jacobson, K. N., Bellinger, C., Roberts, M.,and Chawla, N. V. Understanding cnn fragility when learning withimbalanced data. Machine Learning 113, 7 (2024), 4785â4810. â pages15, 90[47] DeLong, E. R., DeLong, D. M., and Clarke-Pearson, D. L.Comparing the areas under two or more correlated receiver operatingcharacteristic curves: a nonparametric approach. Biometrics 44, 3(Sept. 1988), 837â845. â pages 42122Bibliography[48] Dembrower, K., Crippa, A., Colón, E., Eklund, M., andStrand, F. Artificial intelligence for breast cancer detection in screen-ing mammography in sweden: a prospective, population-based, paired-reader, non-inferiority study. The Lancet Digital Health 5, 10 (10 2023),e703âe711. â pages 51[49] Dembrower, K., Wåhlin, E., Liu, Y., Salim, M., Smith, K.,Lindholm, P., Eklund, M., and Strand, F. Effect of artificialintelligence-based triaging of breast cancer screening mammograms oncancer detection and radiologist workload: a retrospective simulationstudy. The Lancet Digital Health 2, 9 (Sep 2020), e468âe474. â pages51[50] Ding, T., Angelopoulos, A., Bates, S., Jordan, M., and Tib-shirani, R. J. Class-conditional conformal prediction with manyclasses. In Advances in Neural Information Processing Systems (2023),A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine,Eds., vol. 36, Curran Associates, Inc., pp. 64555â64576. â pages 113[51] Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn,D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer,M., Heigold, G., Gelly, S., et al. An image is worth 16x16words: Transformers for image recognition at scale. arXiv preprintarXiv:2010.11929 (2020). â pages 10[52] Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn,D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M.,Heigold, G., Gelly, S., Uszkoreit, J., and Houlsby, N. Animage is worth 16x16 words: Transformers for image recognition atscale, 2021. â pages 88[53] Douzas, G., Bacao, F., and Last, F. Improving imbalancedlearning through a heuristic oversampling method based on k-meansand smote. Information sciences 465 (2018), 1â20. â pages 16, 90[54] Einbinder, B.-S., Romano, Y., Sesia, M., and Zhou, Y. Traininguncertainty-aware classifiers with conformalized deep learning, 2022.â pages 34, 93, 94[55] Elsken, T., Metzen, J. H., and Hutter, F. Efficient multi-objective neural architecture search via lamarckian evolution, 2019. âpages 8, 9123Bibliography[56] Epp, J., and Rajapakshe, R. Breast cancer risk predictions by birthcohort and ethnicity in a population-based screening mammographyprogram. British Journal of Radiology 95, 1136 (Aug. 2022), 20211388.Epub 2022 Jul 7. â pages 52[57] Ettinger, S., Cheng, S., Caine, B., Liu, C., Zhao, H., Pradhan,S., Chai, Y., Sapp, B., Qi, C., Zhou, Y., Yang, Z., Chouard,A., Sun, P., Ngiam, J., Vasudevan, V., McCauley, A., Shlens,J., and Anguelov, D. Large scale interactive motion forecasting forautonomous driving : The waymo open motion dataset, 2021. â pages60[58] Everingham, M., Eslami, S. A., Van Gool, L., Williams, C. K.,Winn, J., and Zisserman, A. The pascal visual object classeschallenge: A retrospective. International journal of computer vision111, 1 (2015), 98â136. â pages 63, 64[59] Falcon, W., and The PyTorch Lightning team. PyTorchLightning, Mar. 2019. â pages 72, 75[60] Fan, J., and Li, R. Variable selection via nonconcave penalizedlikelihood and its oracle properties. Journal of the American statisticalAssociation 96, 456 (2001), 1348â1360. â pages 75[61] Fazlyab, M., Robey, A., Hassani, H., Morari, M., and Pappas,G. Efficient and accurate estimation of lipschitz constants for deepneural networks. In Advances in Neural Information Processing Systems(2019), H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc,E. Fox, and R. Garnett, Eds., vol. 32, Curran Associates, Inc. â pages29[62] Fornacon-Wood, I., Mistry, H., Johnson-Hart, C., Faivre-Finn, C., OâConnor, J. P., and Price, G. J. Understanding thedifferences between bayesian and frequentist statistics. Internationaljournal of radiation oncology, biology, physics 112, 5 (2022), 1076â1082.â pages 19, 20[63] Freeman, K., Geppert, J., Stinton, C., Todkill, D., John-son, S., Clarke, A., and Taylor-Phillips, S. Use of artificialintelligence for image analysis in breast cancer screening programmes:systematic review of test accuracy. BMJ 374 (9 2021), n1872. â pages51124Bibliography[64] Freund, Y., Schapire, R., and Abe, N. A short introductionto boosting. Journal-Japanese Society For Artificial Intelligence 14,771-780 (1999), 1612. â pages 10[65] Freund, Y., and Schapire, R. E. A desicion-theoretic generalizationof on-line learning and an application to boosting. In ComputationalLearning Theory (Berlin, Heidelberg, 1995), P. Vitányi, Ed., SpringerBerlin Heidelberg, pp. 23â37. â pages 10, 89[66] Gal, Y., and Ghahramani, Z. Dropout as a bayesian approximation:Representing model uncertainty in deep learning. In Proceedings ofThe 33rd International Conference on Machine Learning (New York,New York, USA, 20â22 Jun 2016), M. F. Balcan and K. Q. Weinberger,Eds., vol. 48 of Proceedings of Machine Learning Research, PMLR,pp. 1050â1059. â pages 27[67] Geiger, A., Lenz, P., and Urtasun, R. Are we ready for au-tonomous driving? the kitti vision benchmark suite. In 2012 IEEE Con-ference on Computer Vision and Pattern Recognition (2012), pp. 3354â3361. â pages 60, 61, 64, 68[68] Gjesvik, J., Moshina, N., Lee, C. I., Miglioretti, D. L., andHofvind, S. Artificial Intelligence Algorithm for Subclinical BreastCancer Detection. JAMA Network Open 7, 10 (10 2024), e2437402âe2437402. â pages 39, 43[69] Glennie, C., and Lichti, D. D. Static calibration and analysis ofthe Velodyne HDL-64E S2 for high accuracy mobile scanning. RemoteSensing 2, 6 (2010), 1610â1624. â pages 68[70] Glorot, X., and Bengio, Y. Understanding the difficulty of trainingdeep feedforward neural networks. In Proceedings of the thirteenthinternational conference on artificial intelligence and statistics (2010),pp. 249â256. â pages 7[71] Glorot, X., Bordes, A., and Bengio, Y. Deep Sparse RectifierNeural Networks. In 14th International Conference on Artificial In-telligence and Statistics (Fort Lauderdale, United States, Apr. 2011),vol. 15, pp. 315â323. â pages 70[72] Goodfellow, I., Bengio, Y., and Courville, A. Deep Learning.MIT Press, 2016. http://www.deeplearningbook.org. â pages 8, 9125Bibliography[73] Goodfellow, I. J., Shlens, J., and Szegedy, C. Explaining andharnessing adversarial examples, 2015. â pages 30, 58[74] Gouk, H., Frank, E., Pfahringer, B., and Cree, M. J. Regular-isation of neural networks by enforcing lipschitz continuity. MachineLearning 110, 2 (2021), 393â416. â pages 29, 31[75] Gouk, H., Frank, E., Pfahringer, B., and Cree, M. J. Regular-isation of neural networks by enforcing Lipschitz continuity. MachineLearning 110, 2 (2021), 393â416. â pages 58[76] Graham-Knight, J. B. Efficient 2d image segmentation. Masterâsthesis, University of British Columbia, 2020. â pages 9, 10, 25[77] Graham-Knight, J. B., Djavadifar, A., Najjaran, H., andLasserre, P. Boosted dense segmentation networks for constraineddistributed systems. In 2021 IEEE International Conference on Systems,Man, and Cybernetics (SMC) (2021), pp. 3101â3107. â pages 25, 73[78] Guo, C., Pleiss, G., Sun, Y., and Weinberger, K. Q. Oncalibration of modern neural networks. In Proceedings of the 34thInternational Conference on Machine Learning - Volume 70 (2017),ICMLâ17, JMLR.org, p. 1321â1330. â pages 20[79] Han, M., Guo, H., Li, J., and Wang, W. Global-local informationbased oversampling for multi-class imbalanced data. InternationalJournal of Machine Learning and Cybernetics 14, 6 (2023), 2071â2086.â pages 16, 90[80] Hansen, L., and Salamon, P. Neural network ensembles. IEEETransactions on Pattern Analysis and Machine Intelligence 12, 10(1990), 993â1001. â pages 8, 9[81] Harvey, S. C., Geller, B., Oppenheimer, R. G., Pinet, M.,Riddell, L., and Garra, B. Increase in cancer detection and recallrates with independent double interpretation of screening mammog-raphy. AJR American Journal of Roentgenology 180, 5 (May 2003),1461â1467. â pages 37[82] He, K., Zhang, X., Ren, S., and Sun, J. Delving deep into rectifiers:Surpassing human-level performance on ImageNet classification. In2015 IEEE International Conference on Computer Vision (ICCV) (dec2015), IEEE, pp. 1026â1034. â pages 73126Bibliography[83] He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learningfor image recognition. In The IEEE Conference on Computer Visionand Pattern Recognition (CVPR) (June 2016). â pages 7[84] Healy, N. A., OâBrien, A., Knox, M., Hargaden, G., Smith,C., Fenlon, H., McNicholas, M., Phelan, N., and Flanagan,F. Consensus review of discordant imaging findings after the introduc-tion of digital screening mammography: Irish national breast cancerscreening program experience. Radiology 295, 1 (2020), 35â41. PMID:32043946. â pages 37[85] Hein, M., Andriushchenko, M., and Bitterwolf, J. Why relunetworks yield high-confidence predictions far away from the trainingdata and how to mitigate the problem. In 2019 IEEE/CVF Conferenceon Computer Vision and Pattern Recognition (CVPR) (2019), pp. 41â50.â pages 21[86] Hinton, G., Vinyals, O., and Dean, J. Distilling the knowledge ina neural network, 2015. â pages 32[87] Hofvind, S., and Lee, C. I. Consensus reads: The more sets of eyesinterpreting a mammogram, the better for women. Radiology 295, 1(Apr. 2020), 42â43. Epub 2020 Feb 11. â pages 37[88] Hovda, T., Hoff, S. R., Larsen, M., Romundstad, L., Sahlberg,K. K., and Hofvind, S. True and missed interval cancer in organizedmammographic screening: A retrospective review study of diagnosticand prior screening mammograms. Academic Radiology 29, Suppl 1(Jan. 2022), S180âS191. Epub 2021 Apr 26. â pages 37[89] Jiang, L., Zhou, Z., Leung, T., Li, L.-J., and Fei-Fei, L. Men-tornet: Learning data-driven curriculum for very deep neural networkson corrupted labels. In ICML (2018). â pages 12, 13, 113[90] Jones, C. R., and Bergen, B. K. People cannot distinguish gpt-4from a human in a turing test, 2024. â pages 11[91] Kim, H.-E., Kim, H. H., Han, B.-K., Kim, K. H., Han, K., Nam,H., Lee, E. H., and Kim, E.-K. Changes in cancer detection andfalse-positive recall in mammography using artificial intelligence: aretrospective, multireader study. The Lancet Digital Health 2, 3 (2020),e138âe148. â pages 37, 51127Bibliography[92] Kimura, M. Understanding test-time augmentation. In InternationalConference on Neural Information Processing (2021), Springer, pp. 558â569. â pages 26[93] Kingma, D. P., and Ba, J. Adam: A method for stochastic opti-mization, 2017. â pages 63, 75, 98[94] Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C.,Gustafson, L., Xiao, T., Whitehead, S., Berg, A. C., Lo,W.-Y., et al. Segment anything. In Proceedings of the IEEE/CVFInternational Conference on Computer Vision (2023), pp. 4015â4026.â pages 11[95] Kramer, O. K-Nearest Neighbors. Springer Berlin Heidelberg, Berlin,Heidelberg, 2013, pp. 13â23. â pages 22[96] Krawczyk, B. Learning from imbalanced data: open challenges andfuture directions. Progress in artificial intelligence 5, 4 (2016), 221â232.â pages 16, 90[97] Krizhevsky, A., Hinton, G., et al. Learning multiple layers offeatures from tiny images. â pages 69, 95[98] Krizhevsky, A., Sutskever, I., and Hinton, G. E. Imagenetclassification with deep convolutional neural networks. In Advances inNeural Information Processing Systems (2012), F. Pereira, C. Burges,L. Bottou, and K. Weinberger, Eds., vol. 25, Curran Associates, Inc.â pages 4[99] Krogel, M.-A., and Scheffer, T. Multi-relational learning, textmining, and semi-supervised learning for functional genomics. MachineLearning 57 (2004), 61â81. â pages 14[100] Kubat, M., Matwin, S., et al. Addressing the curse of imbalancedtraining sets: one-sided selection. In Icml (1997), vol. 97, Citeseer,p. 179. â pages 16, 90[101] Kühl, J., Elhakim, M., Stougaard, S., Rasmussen, B., Nielsen,M., Gerke, O., Larsen, L., and Graumann, O. Population-wide evaluation of artificial intelligence and radiologist assessment ofscreening mammograms. European Radiology 34, 6 (11 2023), 3935â3946.â pages 52128Bibliography[102] Lang, A. H., Vora, S., Caesar, H., Zhou, L., Yang, J., andBeijbom, O. Pointpillars: Fast encoders for object detection frompoint clouds. In 2019 IEEE/CVF Conference on Computer Vision andPattern Recognition (CVPR) (2019), pp. 12689â12697. â pages 60[103] Larsen, M., Aglen, C., Lee, C., Hoff, S., Lund-Hanssen, H.,Lång, K., Nygård, J., Ursin, G., and Hofvind, S. Artificialintelligence evaluation of 122 969 mammography examinations froma population-based screening program. Radiology 303, 3 (3 2022),502â511. â pages 37, 51[104] Lauritzen, A., Rodríguez-Ruiz, A., Von Euler-Chelpin, M.,Lynge, E., Vejborg, I., Nielsen, M., Karssemeijer, N., andLillholm, M. An artificial intelligenceâbased mammography screeningprotocol for breast cancer: Outcome and radiologist workload. Radiology304, 1 (7 2022), 41â49. â pages 51[105] LeCun, Y., Bottou, L., Bengio, Y., and Haffner, P. Gradient-based learning applied to document recognition. Proceedings of theIEEE 86, 11 (1998), 2278â2324. â pages 11[106] LeCun, Y. A., Bottou, L., Orr, G. B., and Müller, K.-R.Efficient BackProp. Springer Berlin Heidelberg, Berlin, Heidelberg,2012, pp. 9â48. â pages 73[107] Lee, K., Lee, K., Lee, H., and Shin, J. A simple unified frameworkfor detecting out-of-distribution samples and adversarial attacks. InProceedings of the 32nd International Conference on Neural InformationProcessing Systems (Red Hook, NY, USA, 2018), NIPSâ18, CurranAssociates Inc., p. 7167â7177. â pages 22[108] Li, Y., Chen, C. L. P., and Zhang, T. A survey on siamese network:Methodologies, applications, and opportunities. IEEE Transactions onArtificial Intelligence 3, 6 (2022), 994â1014. â pages 23, 24[109] Li, Y., Chen, C. L. P., and Zhang, T. A survey on siamese network:Methodologies, applications, and opportunities. IEEE Transactions onArtificial Intelligence 3, 6 (2022), 994â1014. â pages 23[110] Lin, T.-Y., Goyal, P., Girshick, R., He, K., and Dollar, P.Focal loss for dense object detection. In Proceedings of the IEEEInternational Conference on Computer Vision (ICCV) (Oct 2017). âpages 16, 20, 75, 91129Bibliography[111] Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P.,Ramanan, D., Dollár, P., and Zitnick, C. L. Microsoft COCO:Common objects in context. In Computer VisionâECCV 2014: 13thEuropean Conference, Zurich, Switzerland, September 6-12, 2014, Pro-ceedings, Part V 13 (2014), Springer, pp. 740â755. â pages 64[112] Liu, D., Yu, R., and Su, H. Extending adversarial attacks anddefenses to deep 3d point cloud classifiers. In 2019 IEEE InternationalConference on Image Processing (ICIP) (2019), pp. 2279â2283. âpages 58[113] Liu, H., Li, C., Li, Y., and Lee, Y. J. Improved baselines withvisual instruction tuning, 2024. â pages 11[114] Liu, H., Simonyan, K., and Yang, Y. Darts: Differentiable ar-chitecture search. arXiv preprint arXiv:1806.09055 (2018). â pages9[115] Liu, P., Wang, L., Ranjan, R., He, G., and Zhao, L. A survey onactive deep learning: From model driven to data driven. ACM Comput.Surv. 54, 10s (sep 2022). â pages 15[116] Loshchilov, I., and Hutter, F. Decoupled weight decay regular-ization. arXiv preprint arXiv:1711.05101 (2017). â pages 63[117] Lou, Y., Caruana, R., Gehrke, J., and Hooker, G. Accurateintelligible models with pairwise interactions. In Proceedings of the 19thACM SIGKDD International Conference on Knowledge Discovery andData Mining (New York, NY, USA, 2013), KDD â13, Association forComputing Machinery, p. 623â631. â pages 76[118] Lång, K., Josefsson, V., Larsson, A.-M., Larsson, S., Hög-berg, C., Sartor, H., Hofvind, S., Andersson, I., and Rosso,A. Artificial intelligence-supported screen reading versus standard dou-ble reading in the mammography screening with artificial intelligencetrial (masai): a clinical safety analysis of a randomised, controlled,non-inferiority, single-blinded, screening accuracy study. The LancetOncology 24, 8 (8 2023), 936â944. â pages 37, 51[119] Ma, C., Jiang, Y., Wen, X., Yuan, Z., and Qi, X. Codet: Co-occurrence guided region-word alignment for open-vocabulary objectdetection. Advances in Neural Information Processing Systems 36(2024). â pages 11130Bibliography[120] Maguolo, G., Nanni, L., and Ghidoni, S. Ensemble of convo-lutional neural networks trained with different activation functions.Expert Systems with Applications 166 (mar 2021), 114048. â pages 25,70[121] Mandal, S., Biswas, S., Balas, V. E., Shaw, R. N., and Ghosh,A. Lyft 3D object detection for autonomous vehicles. In ArtificialIntelligence for Future Generation Robotics, R. N. Shaw, A. Ghosh,V. E. Balas, and M. Bianchini, Eds. Elsevier, 2021, pp. 119â136. âpages 60[122] Mani, I., and Zhang, I. knn approach to unbalanced data distribu-tions: a case study involving information extraction. In Proceedings ofworkshop on learning from imbalanced datasets (2003), vol. 126, ICML,pp. 1â7. â pages 16, 90[123] McCann, J., Stockton, D., and Godward, S. Impact of false-positive mammography on subsequent screening attendance and risk ofcancer. Breast Cancer Research 4, 5 (2002), R11. Epub 2002 Jul 17.â pages 39[124] McKinney, S. M., Sieniek, M., Godbole, V., Godwin, J.,Antropova, N., Ashrafian, H., Back, T., Chesus, M., Cor-rado, G. S., Darzi, A., Etemadi, M., Garcia-Vicente, F.,Gilbert, F. J., Halling-Brown, M., Hassabis, D., Jansen, S.,Karthikesalingam, A., Kelly, C. J., King, D., Ledsam, J. R.,Melnick, D., Mostofi, H., Peng, L., Reicher, J. J., Romera-Paredes, B., Sidebottom, R., Suleyman, M., Tse, D., Young,K. C., Fauw, J. D., and Shetty, S. International evaluation of anai system for breast cancer screening. Nature 577, 7788 (Jan. 2020),89â94. Epub 2020 Jan 1. Erratum in: Nature. 2020 Oct;586(7829):E19.doi: 10.1038/s41586-020-2679-9. â pages 37[125] Milch, H. S., and Lee, C. I. Establishing the evidence needed forai-driven mammography screening. Radiology: Artificial Intelligence 7,3 (2025), e250152. PMID: 40172323. â pages 55[126] Minderer, M., Gritsenko, A., and Houlsby, N. Scaling open-vocabulary object detection. Advances in Neural Information ProcessingSystems 36 (2024). â pages 11[127] Ming, Y., Sun, Y., Dia, O., and Li, Y. How to exploit hyperspher-ical embeddings for out-of-distribution detection? In The Eleventh131BibliographyInternational Conference on Learning Representations (2023). â pages22[128] MMDetection3D Contributors. MMDetection3D: OpenMMLabnext-generation platform for general 3D object detection, 2020. âpages 61[129] MMDetection3D Contributors. MMDetection3D: OpenMMLabnext-generation platform for general 3D object detection, 2020. âpages 63[130] Moosavi-Dezfooli, S.-M., Fawzi, A., and Frossard, P. Deepfool:A simple and accurate method to fool deep neural networks. In 2016IEEE Conference on Computer Vision and Pattern Recognition (CVPR)(2016), pp. 2574â2582. â pages 30[131] Moosavi-Dezfooli, S.-M., Fawzi, A., and Frossard, P. Deepfool:A simple and accurate method to fool deep neural networks. In 2016IEEE Conference on Computer Vision and Pattern Recognition (CVPR)(2016), pp. 2574â2582. â pages 60[132] Nanni, L., Lumini, A., Ghidoni, S., and Maguolo, G. Stochasticselection of activation layers for convolutional neural networks. Sensors20, 6 (2020). â pages 25, 70[133] Natarajan, N., Dhillon, I. S., Ravikumar, P., and Tewari, A.Cost-sensitive learning with noisy labels. Journal of Machine LearningResearch 18, 155 (2018), 1â33. â pages 12[134] Neal, R. M. Bayesian learning for neural networks, vol. 118. SpringerScience & Business Media, 2012. â pages 28[135] Nitesh, V. C. Smote: synthetic minority over-sampling technique. JArtif Intell Res 16, 1 (2002), 321. â pages 16, 90[136] Nori, H., Jenkins, S., Koch, P., and Caruana, R. Interpretml: Aunified framework for machine learning interpretability. arXiv preprintarXiv:1909.09223 (2019). â pages 72, 76, 82[137] Northcutt, C., Jiang, L., and Chuang, I. Confident learning: Es-timating uncertainty in dataset labels. Journal of Artificial IntelligenceResearch 70 (2021), 1373â1411. â pages 13, 113132Bibliography[138] Northcutt, C. G., Wu, T., and Chuang, I. L. Learning withconfident examples: Rank pruning for robust classification with noisylabels. In Proceedings of the Thirty-Third Conference on Uncertainty inArtificial Intelligence (Sydney, Australia, 2017), UAIâ17, AUAI Press.â pages 13[139] Oquab, M., Darcet, T., Moutakanni, T., Vo, H., Szafraniec,M., Khalidov, V., Fernandez, P., Haziza, D., Massa, F., El-Nouby, A., et al. Dinov2: Learning robust visual features withoutsupervision. arXiv preprint arXiv:2304.07193 (2023). â pages 32[140] Papernot, N., McDaniel, P., Wu, X., Jha, S., and Swami,A. Distillation as a defense to adversarial perturbations against deepneural networks. In 2016 IEEE Symposium on Security and Privacy(SP) (2016), pp. 582â597. â pages 31, 58[141] Pham, H., Guan, M. Y., Zoph, B., Le, Q. V., and Dean, J.Efficient neural architecture search via parameter sharing, 2018. âpages 8[142] Prechelt, L. Automatic early stopping using cross validation: quan-tifying the criteria. Neural Networks 11, 4 (1998), 761â767. â pages98[143] Public Health Agency of Canada. Report from the evaluationindicators working group: Guidelines for monitoring breast cancerscreening program performance, 11 2019. â pages 42[144] PyTorch Lightning Contributors. PyTorch Lightning: Theultimate PyTorch research framework., 2019. â pages 97[145] Qi, C. R., Chen, X., Litany, O., and Guibas, L. J. ImVoteNet:Boosting 3D object detection in point clouds with image votes. In 2020IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR) (2020), pp. 4403â4412. â pages 60[146] Qi, C. R., Litany, O., He, K., and Guibas, L. Deep houghvoting for 3D object detection in point clouds. In 2019 IEEE/CVFInternational Conference on Computer Vision (ICCV) (2019), pp. 9276â9285. â pages 60[147] Qi, C. R., Yi, L., Su, H., and Guibas, L. J. PointNet++: Deephierarchical feature learning on point sets in a metric space. In Advances133Bibliographyin Neural Information Processing Systems (2017), I. Guyon, U. V.Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, andR. Garnett, Eds., vol. 30, Curran Associates, Inc. â pages 60[148] Quy, T. L., Roy, A., Iosifidis, V., Zhang, W., and Ntoutsi, E.A survey on datasets for fairness-aware machine learning. WIREs DataMining and Knowledge Discovery 12, 3 (mar 2022). â pages 34[149] Rajapakshe, R., Miao, H., Sam, J., Farnquist, B., and Hart-man, M. Risk of breast cancer after a false-positive screening mammo-gram in relation to mammographic abnormality: A population-basedstudy in british columbia. Cancer Research 78, 4 Supplement (Feb.2018), PD2â16. â pages 39[150] Rajapakshe, R., Yang, L., Liang, P., Graham-Knight, J.,Shen, H., Mar, C., Wright, Q., and Kim, K. Limitationsof downsampling of mammography images in microcalcificationdetection by ai algorithms; an initial clinical experience. Online:https://eppro02.ativ.me/web/page.php?page=Session&project=RSNA22&id=P5125,Dec. 2022. â pages 52[151] Ramachandran, P., Zoph, B., and Le, Q. V. Searching foractivation functions. â pages 70, 73[152] Ren, M., Zeng, W., Yang, B., and Urtasun, R. Learning toreweight examples for robust deep learning. In ICML (2018). â pages12[153] Robin, X., Turck, N., Hainard, A., Tiberti, N., Lisacek, F.,Sanchez, J.-C., and Müller, M. proc: an open-source package forr and s+ to analyze and compare roc curves. BMC Bioinformatics 12,77 (2011). â pages 42[154] Romano, Y., Sesia, M., and Candes, E. Classification with validand adaptive coverage. Advances in Neural Information ProcessingSystems 33 (2020), 3581â3591. â pages 33, 93, 94, 113[155] Ronneberger, O., Fischer, P., and Brox, T. U-net: Convolu-tional networks for biomedical image segmentation. In Medical Im-age Computing and Computer-Assisted Intervention â MICCAI 2015(Cham, 2015), N. Navab, J. Hornegger, W. M. Wells, and A. F. Frangi,Eds., Springer International Publishing, pp. 234â241. â pages 71, 72,97134Bibliography[156] Rosenblatt, F. The perceptron: a probabilistic model for informationstorage and organization in the brain. Psychological review 65, 6 (1958),386. â pages 70[157] Sadinle, M., Lei, J., and Wasserman, L. Least ambiguous set-valued classifiers with bounded error levels. Journal of the AmericanStatistical Association 114, 525 (June 2018), 223â234. â pages 97[158] Sadinle, M., Lei, J., and Wasserman, L. Least ambiguous set-valued classifiers with bounded error levels. Journal of the AmericanStatistical Association 114, 525 (2019), 223â234. â pages 18, 33, 93, 94[159] Salehi, S. S. M., Erdogmus, D., and Gholipour, A. Tverskyloss function for image segmentation using 3d fully convolutional deepnetworks. In Machine Learning in Medical Imaging (Cham, 2017),Q. Wang, Y. Shi, H.-I. Suk, and K. Suzuki, Eds., Springer InternationalPublishing, pp. 379â387. â pages 75[160] Schaffter, T., Buist, D., Lee, C., Nikulin, Y., Ribli, D., Guan,Y., Lotter, W., Jie, Z., Du, H., Wang, S., Feng, J., Feng,M., Kim, H.-E., Albiol, F., Albiol, A., Morrell, S., Wojna,Z., Ahsen, M., Asif, U., Jimeno Yepes, A., Yohanandan, S.,Rabinovici-Cohen, S., Yi, D., Hoff, B., Yu, T., Chaibub Neto,E., Rubin, D., Lindholm, P., Margolies, L., Mcbride, R.,Rothstein, J., Sieh, W., Ben-Ari, R., Harrer, S., Trister, A.,Friend, S., Norman, T., Sahiner, B., Strand, F., Guinney, J.,Stolovitzky, G., Mackey, L., Cahoon, J., Shen, L., Sohn, J.,Trivedi, H., Shen, Y., Buturovic, L., Pereira, J., Cardoso,J., Castro, E., Kalleberg, K., Pelka, O., Nedjar, I., Geras,K., Nensa, F., Goan, E., Koitka, S., Caballero, L., Cox,D., Krishnaswamy, P., Pandey, G., Friedrich, C., Perrin,D., Fookes, C., Shi, B., Cardoso Negrie, G., Kawczynski,M., Cho, K., Khoo, C., Lo, J., Sorensen, A., and Jung, H.Evaluation of combined artificial intelligence and radiologist assessmentto interpret screening mammograms. JAMA Network Open 3, 3 (32020), e200265. â pages 51[161] Schwenk, H., and Bengio, Y. Training methods for adaptive boost-ing of neural networks. In Advances in neural information processingsystems (1998), pp. 647â653. â pages 10135Bibliography[162] Sculley, D., Snoek, J., Wiltschko, A., and Rahimi, A. Winnerâscurse? on pace, progress, and empirical rigor, 2018. â pages 77[163] Sehwag, V., Chiang, M., and Mittal, P. Ssd: A unified frameworkfor self-supervised outlier detection, 2021. â pages 22[164] Shanmugam, D., Blalock, D., Balakrishnan, G., and Guttag,J. Better aggregation in test-time augmentation. In Proceedings ofthe IEEE/CVF international conference on computer vision (2021),pp. 1214â1223. â pages 26[165] Shen, Y., Winget, M., and Yuan, Y. The impact of false positivebreast cancer screening mammograms on screening retention: a retro-spective population cohort study in alberta, canada. Canadian Journalof Public Health 108 (2017), e539âe545. â pages 37, 51[166] Shi, S., Wang, Z., Shi, J., Wang, X., and Li, H. From pointsto parts: 3D object detection from point cloud with part-aware andpart-aggregation network. IEEE Transactions on Pattern Analysis andMachine Intelligence 43, 8 (2021), 2647â2664. â pages 60[167] Shi, Z., Chen, Z., Xu, Z., Yang, W., Yu, Z., and Huang, L.Shape prior guided attack: Sparser perturbations on 3d point clouds.Proceedings of the AAAI Conference on Artificial Intelligence 36, 8(Jun. 2022), 8277â8285. â pages 58[168] Sickles, E. A., DâOrsi, C. J., Bassett, L. W., Appleton, C. M.,Berg, W. A., Burnside, E. S., et al. Acr bi-rads® mammography.ACR BI-RADS® atlas, breast imaging reporting and data system 5(2013), 2013. â pages 39[169] Soviany, P., Ionescu, R. T., Rota, P., and Sebe, N. Curriculumlearning: A survey. International Journal of Computer Vision 130, 6(2022), 1526â1565. â pages 12[170] Sudre, C. H., Li, W., Vercauteren, T., Ourselin, S., andCardoso, M. J. Generalised dice overlap as a deep learning lossfunction for highly unbalanced segmentations. In Deep learning inmedical image analysis and multimodal learning for clinical decisionsupport. Springer, 2017, pp. 240â248. â pages 75[171] Sun, P., Kretzschmar, H., Dotiwalla, X., Chouard, A., Pat-naik, V., Tsui, P., Guo, J., Zhou, Y., Chai, Y., Caine, B.,136BibliographyVasudevan, V., Han, W., Ngiam, J., Zhao, H., Timofeev, A.,Ettinger, S., Krivokon, M., Gao, A., Joshi, A., Zhao, S.,Cheng, S., Zhang, Y., Shlens, J., Chen, Z., and Anguelov, D.Scalability in perception for autonomous driving: Waymo open dataset,2020. â pages 60[172] Sun, X., and Xu, W. Fast implementation of delongâs algorithm forcomparing the areas under correlated receiver operating characteristiccurves. IEEE Signal Processing Letters 21, 11 (2014), 1389â1393. âpages 42[173] Sun, Y., Cai, L., Liao, B., Zhu, W., and Xu, J. A robustoversampling approach for class imbalance problem with small disjuncts.IEEE Transactions on Knowledge and Data Engineering 35, 6 (2022),5550â5562. â pages 16, 90[174] Sun, Y., Chen, F., Chen, Z., and Wang, M. Local aggressiveadversarial attacks on 3d point cloud. In Asian Conference on MachineLearning, ACML 2021, 17-19 November 2021, Virtual Event (2021),V. N. Balasubramanian and I. W. Tsang, Eds., vol. 157 of Proceedingsof Machine Learning Research, PMLR, pp. 65â80. â pages 58[175] Sung, H., Ferlay, J., Siegel, R. L., Laversanne, M., Soerjo-mataram, I., and Jemal, A. Global cancer statistics 2020: Globocanestimates of incidence and mortality worldwide for 36 cancers in 185countries. CA: A Cancer Journal for Clinicians 71 (May 2021), 209â249.â pages 36[176] Szegedy, C., Inc, G., Zaremba, W., Sutskever, I., Inc, G.,Bruna, J., Erhan, D., Inc, G., Goodfellow, I., and Fergus, R.Intriguing properties of neural networks. In ICLR (2014). â pages 30[177] Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan,D., Goodfellow, I., and Fergus, R. Intriguing properties of neuralnetworks, 2014. â pages 58[178] Tan, M., Chen, B., Pang, R., Vasudevan, V., Sandler, M.,Howard, A., and Le, Q. V. Mnasnet: Platform-aware neuralarchitecture search for mobile. In 2019 IEEE/CVF Conference onComputer Vision and Pattern Recognition (CVPR) (2019), pp. 2815â2823. â pages 8137Bibliography[179] Tan, M., Chen, B., Pang, R., Vasudevan, V., Sandler, M.,Howard, A., and Le, Q. V. Mnasnet: Platform-aware neuralarchitecture search for mobile. In The IEEE Conference on ComputerVision and Pattern Recognition (CVPR) (June 2019). â pages 97[180] Tan, M., and Le, Q. EfficientNet: Rethinking model scaling forconvolutional neural networks. In Proceedings of the 36th InternationalConference on Machine Learning (09â15 Jun 2019), K. Chaudhuri andR. Salakhutdinov, Eds., vol. 97 of Proceedings of Machine LearningResearch, PMLR, pp. 6105â6114. â pages 7, 97[181] Tan, M., and Le, Q. EfficientNet: Rethinking model scaling forconvolutional neural networks. In Proceedings of the 36th InternationalConference on Machine Learning (09â15 Jun 2019), K. Chaudhuri andR. Salakhutdinov, Eds., vol. 97 of Proceedings of Machine LearningResearch, PMLR, pp. 6105â6114. â pages 8[182] Tao, A., Sapra, K., and Catanzaro, B. Hierarchical multi-scaleattention for semantic segmentation. â pages 10, 11, 70[183] Tsuzuku, Y., Sato, I., and Sugiyama, M. Lipschitz-margin train-ing: Scalable certification of perturbation invariance for deep neuralnetworks. In Advances in Neural Information Processing Systems (2018),S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi,and R. Garnett, Eds., vol. 31, Curran Associates, Inc. â pages 31, 58[184] Van Amersfoort, J., Smith, L., Teh, Y. W., and Gal, Y. Uncer-tainty estimation using a single deep deterministic neural network. InInternational conference on machine learning (2020), PMLR, pp. 9690â9700. â pages 24[185] Van der Maaten, L., and Hinton, G. Visualizing data using t-sne.Journal of machine learning research 9, 11 (2008). â pages 22[186] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones,L., Gomez, A. N., Kaiser, L. u., and Polosukhin, I. Attentionis all you need. In Advances in Neural Information Processing Systems(2017), I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus,S. Vishwanathan, and R. Garnett, Eds., vol. 30, Curran Associates, Inc.â pages 10138Bibliography[187] Verma, S., and Rubin, J. Fairness definitions explained. In Proceed-ings of the international workshop on software fairness (2018), pp. 1â7.â pages 35[188] Vovk, V., Gammerman, A., and Shafer, G. Algorithmic learningin a random world, vol. 29. Springer, 2005. â pages 34[189] Wang, X., Cai, M., Sohel, F., Sang, N., and Chang, Z. Adversar-ial point cloud perturbations against 3d object detection in autonomousdriving systems. Neurocomputing 466 (2021), 27â36. â pages 58[190] Wen, Y., Tran, D., and Ba, J. Batchensemble: an alternativeapproach to efficient ensemble and lifelong learning. In InternationalConference on Learning Representations (2020). â pages 26[191] Wen, Y., Tran, D., and Ba, J. Batchensemble: an alternativeapproach to efficient ensemble and lifelong learning. In InternationalConference on Learning Representations (2020). â pages 87[192] Wenzel, F., Snoek, J., Tran, D., and Jenatton, R. Hyperpa-rameter ensembles for robustness and uncertainty quantification. In Ad-vances in Neural Information Processing Systems (2020), H. Larochelle,M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin, Eds., vol. 33, CurranAssociates, Inc., pp. 6514â6527. â pages 26, 69[193] Wightman, R. Pytorch image models. https://github.com/rwightman/pytorch-image-models, 2019. â pages 97[194] Wiyatno, R., and Xu, A. Maximal Jacobian-based saliency mapattack, 2018. â pages 58[195] Wu, T., Huang, Q., Liu, Z., Wang, Y., and Lin, D. Distribution-balanced loss for multi-label classification in long-tailed datasets. InComputer VisionâECCV 2020: 16th European Conference, Glasgow,UK, August 23â28, 2020, Proceedings, Part IV 16 (2020), Springer,pp. 162â178. â pages 16, 91[196] Xiao, H., Rasul, K., and Vollgraf, R. Fashion-mnist: a novelimage dataset for benchmarking machine learning algorithms. â pages69[197] Xu, H., Xie, S., Tan, X. E., Huang, P.-Y., Howes, R., Sharma,V., Li, S.-W., Ghosh, G., Zettlemoyer, L., and Feichtenhofer,C. Demystifying clip data. â pages 11139Bibliography[198] Yakubovskiy, P. Segmentation models pytorch. https://github.com/qubvel/segmentation_models.pytorch, 2020. â pages 97[199] Yan, Y., Mao, Y., and Li, B. SECOND: Sparsely embedded convo-lutional detection. Sensors (Basel) (2018). â pages 60, 61[200] Yan, Y., Zhu, Y., Liu, R., Zhang, Y., Zhang, Y., and Zhang,L. Spatial distribution-based imbalanced undersampling. IEEE Trans-actions on Knowledge and Data Engineering 35, 6 (2022), 6376â6391.â pages 16, 90[201] Yang, J., Zhou, K., Li, Y., and Liu, Z. Generalized out-of-distribution detection: A survey, 2024. â pages 17[202] Yang, Z., Sun, Y., Liu, S., and Jia, J. 3DSSD: Point-based 3d singlestage object detector. In 2020 IEEE/CVF Conference on ComputerVision and Pattern Recognition (CVPR) (2020), pp. 11037â11045. âpages 60[203] Yazan, E., and Talu, M. F. Comparison of the stochastic gradientdescent based optimization techniques. In 2017 International ArtificialIntelligence and Data Processing Symposium (IDAP) (sep 2017), IEEE,pp. 1â5. â pages 75[204] Yu, K., Albro, S., DeSalvo, G., Kothawade, S., Rashwan,A., Tavakkol, S., Batmanghelich, K., and Yin, X. Two-stepactive learning for instance segmentation with uncertainty and diversitysampling. arXiv preprint arXiv:2309.16139 (2023). â pages 15[205] Yuan, X., He, P., Zhu, Q., and Li, X. Adversarial examples:Attacks and defenses for deep learning. IEEE Transactions on NeuralNetworks and Learning Systems 30, 9 (Sep. 2019), 2805â2824. â pages30, 57, 59[206] Yun, S., Han, D., Oh, S. J., Chun, S., Choe, J., and Yoo,Y. Cutmix: Regularization strategy to train strong classifiers withlocalizable features. In Proceedings of the IEEE/CVF internationalconference on computer vision (2019), pp. 6023â6032. â pages 20, 27[207] Zang, Y., Huang, C., and Loy, C. C. FASA: Feature augmentationand sampling adaptation for long-tailed instance segmentation. InProceedings of the IEEE/CVF international conference on computervision (2021), pp. 3457â3466. â pages 16, 90140Bibliography[208] Zha, D., Bhat, Z. P., Lai, K.-H., Yang, F., Jiang, Z., Zhong,S., and Hu, X. Data-centric artificial intelligence: A survey, 2023. âpages 11[209] Zhang, G., Li, G., Shen, W., and Zhang, W. The expressivityand training of deep neural networks: Toward the edge of chaos?Neurocomputing (Amsterdam) 386 (2020), 8â17. â pages 70, 73[210] Zhang, S., Liu, M., and Yan, J. The diversified ensemble neuralnetwork. In Advances in Neural Information Processing Systems (2020),H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, Eds.,vol. 33, Curran Associates, Inc., pp. 16001â16011. â pages 26, 87[211] Zhou, Y., and Tuzel, O. VoxelNet: End-to-end learning for pointcloud based 3D object detection. In 2018 IEEE/CVF Conference onComputer Vision and Pattern Recognition (2018), pp. 4490â4499. âpages 60, 61[212] Zoph, B., and Le, Q. V. Neural architecture search with reinforce-ment learning, 2017. â pages 8[213] Zoph, B., Vasudevan, V., Shlens, J., and Le, Q. V. Learningtransferable architectures for scalable image recognition, 2018. â pages8141
Thesis/Dissertation
eng
Computer Science
Vancouver : University of British Columbia Library
Attribution-NonCommercial-ShareAlike 4.0 International
Graduate
Lasserre, P.
Najjaran, Homayoun
Efficient online uncertainty management : Conformal-in-the-Loop for measuring and actioning uncertainty at training time
Text