BIRS Workshop Lecture Videos

Banff International Research Station Logo

BIRS Workshop Lecture Videos

Post hoc inference for multiple testing Neuvial, Pierre


When testing a large number of hypotheses simultaneously, a common practice (eg in genomic applications) is to (i) select a subset of candidate hypotheses and (ii) refine this selection using domain-based knowledge. Unfortunately, it is generally not possible to provide a statistical guarantee (e.g. controlled False Discovery Rate) for the resulting set of candidates. This gap between statistical theory and applications has motivated the development of post hoc procedures, for which the candidate sets can be selected "after having seen the data". Goeman and Solari (Stat. Science, 2011) have proposed a construction of post hoc procedures based on "closed testing". Their main procedure is sharp when the hypotheses are independent, but may be conservative under positive dependence. We introduce an alternative framework for post hoc inference, based on the control of a multiple testing risk called the joint Family-Wise Error Rate (JFWER). We propose JFWER-controlling procedures tailored to the case where the joint distribution of the test statistics under the null hypothesis is known, or can be sampled from. We discuss their performance and their link to the procedures proposed by Goeman and Solari. This is joint work with Gilles Blanchard and Etienne Roquain.

Item Media

Item Citations and Data


Attribution-NonCommercial-NoDerivatives 4.0 International