Consider an n by n linear system Ax=b. If the right-hand side of the system is known up to a certain error, then in process of the solution, this error gets amplified by the condition number of the matrix A, i.e. by the ratio of its largest and smallest singular values. This observation led von Neumann and his collaborators to consider the condition number of a random matrix and conjecture that it should be of order n. This conjecture was…

Henry L. Rietz, the first president of IMS, published his book “Mathematical Statistics” in 1927. One review wrote in 1928: “Professor Rietz has developed this theory so skillfully that the ’workers in other fields’, provided only that they have a passing familiarity with the grammar of mathematics, can secure a satisfactory understanding of the points involved.” In this lecture, I would like to promote the good tradition of mathematical statistics as expressed in Rietzs book in order to gain insight…

In this talk, I will discuss the prediction properties of techniques commonly used to scale up kernel methods and Gaussian processes. In particular, I will focus on data dependent and independent sub-sampling methods, namely Nystrom and random features, and study their generalization properties within a statistical learning theory framework. On the one hand I will show that these methods can achieve optimal learning errors while being computational efficient. On the other hand, I will show that subsampling can be seen…