Prof. Amichai Painsky (Tel Aviv University)
Inferring the Unseen
Abstract: Consider a finite sample from an unknown distribution over a countable alphabet. Unobserved events are alphabet symbols which do not appear in the sample. Estimating the probabilities of unobserved events is a basic problem in statistics and related fields, which was extensively studied in the context of point estimation. In this work we introduce a novel interval estimation scheme for unobserved events. Our proposed framework applies selective inference, as we construct confidence intervals (CIs) for the desired set of parameters. Interestingly, we show that the obtained CIs are dimension-free, as they do not grow with the alphabet size. Further, we show that our CIs are (almost) tight, in the sense that they cannot be further improved without violating the prescribed coverage rate. We utilize our proposed scheme for large alphabet modeling. We introduce a novel inference framework for large alphabet distributions which outperforms currently known methods while maintaining the desired confidence level. Our proposed method is robust, easy to apply and provides favorable performance guarantees.
Bio: Dr. Amichai Painsky is an Assistant Professor at Tel Aviv University. Amichai received his B.Sc. in EE from Tel Aviv University (2007), his Masters in EE from Princeton University (2009) and his Ph.D. in Statistics from Tel Aviv University (2016). Following his graduation, Amichai spent two years as a Post Doctoral Fellow, jointly affiliated with the Israeli Center of Research Excellence (I-CORE) and the Signals, Information and Algorithms Lab at MIT. His research interests include statistical inference and learning, with an emphasis on high-dimensional and large scale problems.