Improved Statistical Inference to Avoid Spurious Scientific Discoveries       

Mathematics Accomplishment | 2015

IBM researchers: Vitaly Feldman, Moritz Hardt

Where the work was doneIBM Almaden Research Center

What we accomplished: From abstract of STOC paper: "A great deal of effort has been devoted to reducing the risk of spurious scientific discoveries, from the use of sophisticated validation techniques, to deep statistical methods for controlling the false discovery rate in multiple hypothesis testing. There is, however, a fundamental disconnect between the theoretical results and the practice of data analysis: The theory of statistical inference assumes a fixed collection of hypotheses to be tested, or learning algorithms to be applied, selected non-adaptively before the data are gathered, whereas in practice data are shared and reused with hypotheses and new analyses being generated on the basis of data exploration and the outcomes of previous analyses. In this work we initiate a principled study of how to guarantee the validity of statistical inference in adaptive data analysis."

Related links: