Open Collections

A blob method for diffusion and applications to sampling and two layer neural networks.

Banff International Research Station for Mathematical Innovation and Discovery

Given a desired target distribution and an initial guess of that distribution, composed of finitely many samples, what is the best way to evolve the locations of the samples so that they more accurately represent the desired distribution A classical solution to this problem is to allow the samples to evolve according to Langevin dynamics, the stochastic particle method corresponding to the Fokker-Planck equation. In todayâ s talk, I will contrast this classical approach with a deterministic particle method corresponding to the porous medium equation. This method corresponds exactly to the mean-field dynamics of training a two layer neural network for a radial basis function activation function. We prove that, as the number of samples increases and the variance of the radial basis function goes to zero, the particle method converges to a bounded entropy solution of the porous medium equation. As a consequence, we obtain both a novel method for sampling probability distributions as well as insight into the training dynamics of two layer neural networks in the mean field regime. This is joint work with Karthik Elamvazhuthi (UCLA), Matt Haberland (Cal Poly), and Olga Turanova (Michigan State).

Mathematics; Partial Differential Equations; Probability Theory And Stochastic Processes

video/mp4

Author affiliation: University of California Santa Barbara

2023-10-24

Attribution-NonCommercial-NoDerivatives 4.0 International

http://hdl.handle.net/2429/86304

Unreviewed

http://creativecommons.org/licenses/by-nc-nd/4.0/