IDeAS
Fall 2017
IDeAS Seminar: Towards demystification of deep learning: function space analysis of the representation layersSpeaker: Shai Dekel, Tel Aviv University Date: Sep 19 2017  2:30pm Event type: IDeAS Room: McDonnell Hall, Room 102A Abstract: We propose a function space approach to Representation Learning [1] and the analysis of the representation layers in deep learning architectures. We show how to compute a `weaktype' Besov smoothness index that quantifies the geometry of the clustering in the feature space. This approach was already applied successfully to improve the performance of machine learning algorithms such as the Random Forest [2] and treebased Gradient Boosting [3]. Our experiments demonstrate that in wellknown and wellperforming trained networks, the Besov smoothness of the training set, measured in the corresponding hidden layer feature map representation, increases from layer to layer which relates to the `unfolding' of the clustering in the feature space. We also contribute to the understanding of generalization [4] by showing how the Besov smoothness of the representations, decreases as we add more mislabeling to the training data. We hope this approach will contribute to the demystification of some aspects of deep learning. References: [1] Y. Bengio , A. Courville and P. Vincenty, Representation Learning: A Review and New Perspectives, IEEE Transactions on Pattern Analysis and Machine Intelligence 8 (2013), 17981828. [2] O. Elisha and S. Dekel , Wavelet decompositions of Random Forests  smoothness analysis,sparse approximation and applications, Journal of machine learning research 17 (2016), 138. [3] S. Dekel, O. Elisha and O. Morgan, Wavelet decomposition of Gradient Boosting, preprint. [4] C. Zhang, S. Bengio, M. Hardt, B. Recht and O. Vinyals, Understanding deep learning requires rethinking generalization, In ICLR 2017 conference proceedings. Shai serves as Head of AI at WIX and is a visiting associate professor at the school of mathematical sciences at TelAviv University. For further information see: https://www.shaidekel.com/ 
IDeAS Seminar: Stability of some superresolution problemsSpeaker: Dmitry Batenkov, MIT Date: Sep 27 2017  2:30pm Event type: IDeAS Room: 224 Fine Hall Abstract: The problem of computational superresolution asks to recover an object from its noisy and limited spectrum. In this talk, we consider two inverse problems of this flavor, mainly from the point of view of stability estimates. In the first problem, we assume that the object's spectrum is a finite sum of exponentials modulated by polynomials (extending the wellresearched case where the polynomials are constants). We derive upper bounds on the problem condition number and show that the attainable resolution exhibits Höldertype continuity with respect to the noise level [1,3]. As an application we consider the approximation of a piecewisesmooth function from its Fourier coefficients. We can show that the asymptotic accuracy of our approach is only dictated by the smoothness of the function between the jumps, even if the jump locations are not known [2]. The second problem is concerned with ongoing work on the weighted extrapolation problem on the real line for functions of finite exponential type where we abandon the sparsity assumption. It turns out that the extrapolation range scales logarithmically with the noise level, while the pointwise extrapolation error exhibits again a Höldertype continuity. References: [1] A. Akinshin, D. Batenkov, and Y. Yomdin, “Accuracy of spiketrain Fourier reconstruction for colliding nodes,” in 2015 International Conference on Sampling Theory and Applications (SampTA), 2015, pp. 617–621. [2] D. Batenkov, “Complete algebraic reconstruction of piecewisesmooth functions from Fourier data,” Math. Comp., vol. 84, no. 295, pp. 2329–2350, 2015 [3] D. Batenkov, “Stability and superresolution of generalized spike recovery,” Applied and Computational Harmonic Analysis, http://dx.doi.org/10.1016/j.acha.2016.09.004. Dmitry Batenkov is a postdoctoral researcher at the Massachusetts Institute of Technology. His research interests include applied harmonic analysis, approximation theory, numerical analysis, sparse representations, sampling theory and inverse problems. http://dimabatenkov.info

IDeAS SeminarSpeaker: Veit Elser, Cornell University Date: Oct 11 2017  2:30pm Event type: IDeAS Room: 224 Fine Hall Abstract: TBA 
IDeAS Seminar: Iron Age Hebrew Epigraphy in the Silicon Age  An Algorithmic Approach To Study PaleoHebrew InscriptionsSpeaker: Barak Sober, Tel Aviv University Date: Oct 18 2017  2:30pm Event type: IDeAS Room: 224 Fine Hall Abstract: Handwriting comparison and identification, e.g. in the setting of forensics, has been widely addressed over the years. However, even in the case of modern documents, the proposed computerized solutions are quite unsatisfactory. For historical documents, such problems are worsened, due to the inscriptions’ preservation conditions. In the following lecture, we will present an attempt at addressing such a problem in the setting of First Temple Period inscriptions, stemming from the isolated military outpost of Arad (ca. 600 BCE). The solution we propose comprises: (A) Acquiring better imagery of the inscriptions using multispectral techniques; (B) Restoring characters via approximation of their composing strokes, represented as a splinebased structure, and estimated by an optimization procedure; (C) Feature extraction and distance calculation  creation of feature vectors describing various aspects of a specific character based upon its deviation from all other characters; (D) Conducting an experiment to test a null hypothesis that two given inscriptions were written by the same author. Applying this approach to the Arad corpus of inscriptions resulted in: (i) The discovery of a brand new inscription on the back side of a well known inscription (half a century after being unearthed); (ii) Empirical evidence regarding the literacy rates in the Kingdom of Judah on the eve of its destruction by Nebuchadnezzar the Babylonian king. Barak Sober received his B.Sc. degree in Mathematics and Philosophy from Tel Aviv University, Israel, in 2006. He received his M.Sc. in Applied Mathematics (summa cum laude) on the topic of “Handwritten Character Stroke Reconstruction” in 2013. He is currently pursuing his PhD in Applied Mathematics on the topic of “Approximation of Manifolds from Scattered Data”. His interests include approximation theory, machine learning, image processing and archaeology. 
IDeAS SeminarSpeaker: Nicolas Garcia Trillos, Brown University Date: Oct 25 2017  2:30pm Event type: IDeAS Room: 224 Fine Hall Abstract: TBA 
IDeAS SeminarSpeaker: Nadav Dym, Weizmann Institute Date: Nov 8 2017  2:30pm Event type: IDeAS Room: 224 Fine Hall Abstract: TBA 
IDeAS SeminarSpeaker: Joe Kileel, Princeton University Date: Nov 15 2017  2:30pm Event type: IDeAS Room: 224 Fine Hall Abstract: TBA 
IDeAS SeminarSpeaker: Gal Mishne, Yale University Date: Nov 29 2017  2:30pm Event type: IDeAS Room: 224 Fine Hall Abstract: TBA 
IDeAS SeminarSpeaker: TiYen Lan, Cornell University Date: Dec 6 2017  2:30pm Event type: IDeAS Room: 224 Fine Hall Abstract: TBA 