Open Collections

Irwin, Brian

University of British Columbia

The goal of this thesis is to develop new numerical techniques for the solution of noisy optimization problems. Noisy optimization problems occur in diverse fields, such as engineering design, machine learning, operations research, and geophysics. Thus, effective numerical techniques for solving noisy optimization problems are of practical interest in a variety of industrial applications. This thesis presents three numerical techniques for solving noisy optimization problems: Secant penalized Broyden-Fletcher-Goldfarb-Shanno (SP-BFGS), Neural Network Accelerated Implicit Filtering (NNAIF), and Linear Model Based Gradient Estimation (LMBGE). SP-BFGS is a new variant of the Broyden-Fletcher-Goldfarb-Shanno (BFGS) quasi-Newton method designed to perform well when gradient measurements are corrupted by noise. SP-BFGS provides a means of incrementally updating the new inverse Hessian approximation with a controlled amount of bias towards the previous inverse Hessian approximation, which allows one to replace the overwriting nature of the original BFGS update with an averaging nature that resists the destructive effects of noise and can cope with negative curvature measurements. I study the theoretical and empirical behaviour of SP-BFGS in this thesis. NNAIF intelligently combines the established literature on implicit filtering (IF) optimization methods with a neural network surrogate model of the objective function, resulting in accelerated derivative free methods for unconstrained optimization problems. I show that NNAIF directly inherits the convergence properties of IF optimization methods, and study the performance of NNAIF via numerical experiments with noisy problems from the Constrained and Unconstrained Testing Environment with safe threads (CUTEst) optimization benchmark set. LMBGE leverages techniques from linear inverse problems to estimate the gradient of a noisy objective function, and can be viewed as a noise robust extension of standard finite difference based approaches for gradient estimation. I demonstrate the use of LMBGE in the solution of the coverage directed generation (CDG) problem, an industrial problem encountered in the verification subfield of electrical and computer engineering.

Text

2023-10-05

Attribution-NonCommercial-NoDerivatives 4.0 International

http://hdl.handle.net/2429/86077

Geophysics

University of British Columbia

UBCV

http://creativecommons.org/licenses/by-nc-nd/4.0/