Open Collections

Post hoc inference for multiple testing

Banff International Research Station for Mathematical Innovation and Discovery

When testing a large number of hypotheses simultaneously, a common practice (eg in genomic applications) is to (i) select a subset of candidate hypotheses and (ii) refine this selection using domain-based knowledge. Unfortunately, it is generally not possible to provide a statistical guarantee (e.g. controlled False Discovery Rate) for the resulting set of candidates. This gap between statistical theory and applications has motivated the development of post hoc procedures, for which the candidate sets can be selected "after having seen the data". Goeman and Solari (Stat. Science, 2011) have proposed a construction of post hoc procedures based on "closed testing". Their main procedure is sharp when the hypotheses are independent, but may be conservative under positive dependence. We introduce an alternative framework for post hoc inference, based on the control of a multiple testing risk called the joint Family-Wise Error Rate (JFWER). We propose JFWER-controlling procedures tailored to the case where the joint distribution of the test statistics under the null hypothesis is known, or can be sampled from. We discuss their performance and their link to the procedures proposed by Goeman and Solari. This is joint work with Gilles Blanchard and Etienne Roquain.

Mathematics; Statistics; Biology and other natural sciences

video/mp4

Author affiliation: CNRS

2017-09-24

Attribution-NonCommercial-NoDerivatives 4.0 International

http://hdl.handle.net/2429/63096

Unreviewed

http://creativecommons.org/licenses/by-nc-nd/4.0/