- Library Home /
- Search Collections /
- Open Collections /
- Browse Collections /
- UBC Theses and Dissertations /
- Uncertainty estimation of weakly supervised predictive...
Open Collections
UBC Theses and Dissertations
UBC Theses and Dissertations
Uncertainty estimation of weakly supervised predictive models for out-of-distribution detection in digital pathology Abolfath Beygi Dezfouli, Parisa
Abstract
The successful integration of deep learning in medical imaging relies upon the reliability and predictiveness of the models. It is important that these models provide accurate predictions for known classes while also delivering well-calibrated uncertainty estimates, especially for unseen classes and anomalies that are regarded as the out-of-distribution (OOD) data, distinct from the data used for model training. Accurate uncertainty estimates can potentially reduce the adverse effects of OOD regions on the target classification task in a clinical workflow. This, in turn, can prevent models from silently failing when confronted with unfamiliar diseases or abnormalities. Our work introduces two distinct approaches, namely M-Branch (Multi-Branch) and VPS (Virtual Patch Synthesis), for training multi-instance learning (MIL) models in histopathology, endowing them with the capability to effectively estimate predictive uncertainty. We conduct a comprehensive performance evaluation by comparing our proposed models to a state-of-the-art MIL model in whole-slide image (WSI) classification, equipped with temperature scaling for enhanced calibration, referred to as CLAM-T, focusing on the task of OOD detection. In our study, we consider the classification of Non-Small Cell Lung Cancer (NSCLC) subtypes, primarily distinguishing between LUAD (lung adenocarcinoma) and LUSC (lung squamous cell carcinoma) as in-distribution classes, while also differentiating NSCLC as in-distribution from Lower Grade Glioma (LGG) as out-of-distribution. Our top-performing model, M-Branch, efficiently estimates predictive uncertainty through the deployment of multiple branches of attention-based networks, complemented by a diversity-promoting loss. We employ two key evaluation metrics, FPR95 and AUC, to assess OOD detection performance. M-Branch excels in this regard, achieving an FPR95 of 38.39 and an AUC of 84.86, outperforming both VPS (FPR95: 49.76, AUC: 81.95) and CLAM-T (FPR95: 49.29, AUC: 83.00). Moreover, we demonstrate that the incorporation of a meta-loss function within M-Branch significantly enhances OOD detection, as evident from the improvements in FPR95 and AUC. Our research makes a substantial contribution to the field of medical image analysis by equipping MIL models with the ability to estimate predictive uncertainty effectively. These advancements have promising implications for enhancing the reliability and performance of deep learning models in medical imaging and digital pathology, particularly in real-world healthcare applications.
Item Metadata
Title |
Uncertainty estimation of weakly supervised predictive models for out-of-distribution detection in digital pathology
|
Creator | |
Supervisor | |
Publisher |
University of British Columbia
|
Date Issued |
2024
|
Description |
The successful integration of deep learning in medical imaging relies upon the reliability and predictiveness of the models. It is important that these models provide accurate predictions for known classes while also delivering well-calibrated uncertainty estimates, especially for unseen classes and anomalies that are regarded as the out-of-distribution (OOD) data, distinct from the data used for model training. Accurate uncertainty estimates can potentially reduce the adverse effects of OOD regions on the target classification task in a clinical workflow. This, in turn, can prevent models from silently failing when confronted with unfamiliar diseases or abnormalities. Our work introduces two distinct approaches, namely M-Branch (Multi-Branch) and VPS (Virtual Patch Synthesis), for training multi-instance learning (MIL) models in histopathology, endowing them with the capability to effectively estimate predictive uncertainty. We conduct a comprehensive performance evaluation by comparing our proposed models to a state-of-the-art MIL model in whole-slide image (WSI) classification, equipped with temperature scaling for enhanced calibration, referred to as CLAM-T, focusing on the task of OOD detection.
In our study, we consider the classification of Non-Small Cell Lung Cancer (NSCLC) subtypes, primarily distinguishing between LUAD (lung adenocarcinoma) and LUSC (lung squamous cell carcinoma) as in-distribution classes, while also differentiating NSCLC as in-distribution from Lower Grade Glioma (LGG) as out-of-distribution. Our top-performing model, M-Branch, efficiently estimates predictive uncertainty through the deployment of multiple branches of attention-based networks, complemented by a diversity-promoting loss. We employ two key evaluation metrics, FPR95 and AUC, to assess OOD detection performance. M-Branch excels in this regard, achieving an FPR95 of 38.39 and an AUC of 84.86, outperforming both VPS (FPR95: 49.76, AUC: 81.95) and CLAM-T (FPR95: 49.29, AUC: 83.00). Moreover, we demonstrate that the incorporation of a meta-loss function within M-Branch significantly enhances OOD detection, as evident from the improvements in FPR95 and AUC.
Our research makes a substantial contribution to the field of medical image analysis by equipping MIL models with the ability to estimate predictive uncertainty effectively. These advancements have promising implications for enhancing the reliability and performance of deep learning models in medical imaging and digital pathology, particularly in real-world healthcare applications.
|
Genre | |
Type | |
Language |
eng
|
Date Available |
2024-01-23
|
Provider |
Vancouver : University of British Columbia Library
|
Rights |
Attribution-NonCommercial-NoDerivatives 4.0 International
|
DOI |
10.14288/1.0438766
|
URI | |
Degree | |
Program | |
Affiliation | |
Degree Grantor |
University of British Columbia
|
Graduation Date |
2024-05
|
Campus | |
Scholarly Level |
Graduate
|
Rights URI | |
Aggregated Source Repository |
DSpace
|
Item Media
Item Citations and Data
Rights
Attribution-NonCommercial-NoDerivatives 4.0 International