Open Collections

Explanation of variability through optimal transport

Banff International Research Station for Mathematical Innovation and Discovery

An optimal-transport based methodology is proposed for the explanation of variability in data. The central idea is to estimate and simulate conditional distributions $\rho(x|z)$ by mapping them optimally to their Wasserstein barycenter $\mu(y)$, thus removing all $z$-dependence. The barycenter problem needs to be formulated and solved in a data-driven format, as the distributions are only known through samples. A particular implementation is developed, ``attributable components’’, in which the maps are restricted to $z$-dependent, non-parametric rigid translations. It is shown that this proposal encompasses standard methodologies, such as least-square regression, k-means clustering and principal components, and extends them broadly to explain accurately and robustly variability driven by complex sets of explicit and latent covariates in a computationally effective way. Applications are shown to climate science, medicine, risk estimation and variations of the Netflix problem.

Mathematics; Calculus of variations and optimal control; optimization; Statistics; Scientific computing

video/mp4

Author affiliation: Courant Institute

2017-10-29

Attribution-NonCommercial-NoDerivatives 4.0 International

http://hdl.handle.net/2429/63471

Unreviewed

http://creativecommons.org/licenses/by-nc-nd/4.0/