Open Collections

Deep Learning: A Bayesian Perspective

Banff International Research Station for Mathematical Innovation and Discovery

Deep learning is a form of machine learning for nonlinear high dimensional pattern matching and prediction. We present a Bayesian probabilistic perspective, and provide a number of insights, for example, more efficient algorithms for optimization and hyper-parameter tuning, and an explanation of finding good predictors. Traditional high-dimensional data reduction techniques, such as principal component analysis (PCA), partial least squares (PLS), reduced rank regression (RRR), projection pursuit regression (PPR) are all shown to be shallow learners. Their deep learning counterparts exploit multiple deep layers of data reduction which provide performance gains. We discuss stochastic gradient descent (SGD) training optimisation, and Dropout (DO) that provide estimation and variable selection, as well as Bayesian regularization, which is central to finding weights and connections in networks to optimize the bias-variance trade-off. To illustrate our methodology, we provide an analysis of spatio-temporal data. Finally, we conclude with directions for future research.

Mathematics; Statistics; Computer science; Data mining

video/mp4

Author affiliation: George Mason University

2018-04-30

Attribution-NonCommercial-NoDerivatives 4.0 International

http://hdl.handle.net/2429/65701

Unreviewed

http://creativecommons.org/licenses/by-nc-nd/4.0/