Open Collections

Lensink, Keegan

University of British Columbia

This thesis explores the design of deep neural network architectures that inherit and leverage the properties of differential equations and numerical methods to address challenges that arise when applying deep learning to problems with high-dimensional inputs and outputs. We show through numerical experiments that not only are these properties inherited by the resulting networks, but that they are beneficial for various applications in scientific computing. The first network introduced, IMEXNet, is a novel deep neural network architecture that is motivated by implicit-explicit methods for solving differential equations. IMEXNet introduces an implicit step that connects all pixels in each channel of the image and therefore addresses the field of view problem while still being comparable to standard convolutions in terms of the number of parameters and computational complexity. Compared to similar explicit networks, such as residual networks, the network has improved forward stability, which has recently shown to reduce the sensitivity to small changes in the input features and improve generalization. The second network, HyperNet, is a fully reversible neural network architecture inspired by reversible hyperbolic differential equations and the leapfrog discretization method. Addressing memory constraints associated with conventional neural networks, and even partially reversible networks, HyperNet introduces a reversible coarsening operation utilizing a learnable form of the Discrete Wavelet Transform. This enables the design of networks with constant memory requirements, irrespective of the number of layers or blocks. We also extend the use of the networks to Variational Auto Encoders, where optimization begins from an exact recovery and we discover the level of compression through minimizing a regularized objective. Its properties are applied to the real-world task of segmenting pulmonary opacification in high-resolution 3D chest CT scans of COVID-19 patients, accompanied by a soft-labelling methodology to address the inter-observer variability inherent in the tasks, further demonstrating its feasibility for large-scale applications with modest hardware.

Text

2025-03-19

Attribution-NonCommercial-NoDerivatives 4.0 International

http://hdl.handle.net/2429/90511

Geophysics

University of British Columbia

UBCV

http://creativecommons.org/licenses/by-nc-nd/4.0/