Hilbert Space Methods for Statistics and Probability

Last update: 21 Apr 2025 21:17
First version: 30 January 2016

---

Yet Another Inadequate Placeholder

For representing possible regression functions (very natural); for representing possible probability distributions in density estimation, independence tests, and two-sample tests (less natural but important).

See also:
• Basis Selection in Function Decomposition
• Kernel Methods in Statistic and Machine Learning
• Math I Ought to Learn

Recommended, big picture:
• Bharath K. Sriperumbudur, Arthur Gretton, Kenji Fukumizu, Bernhard Schölkopf, Gert R.G. Lanckriet, "Hilbert Space Embeddings and Metrics on Probability Measures", Journal of Machine Learning Research 11 (2010): 1517--1561
• Grace Wahba, Spline Models for Observational Data

Recommended, close ups:
• Eduardo Corona, Terran Lane, Curtis Storlie, Joshua Neil, "Using Laplacian Methods, RKHS Smoothing Splines and Bayesian Estimation as a framework for Regression on Graph and Graph Related Domains" [Technical report, University of New Mexico Computer Science, 2008-06, PDF]
• Kenji Fukumizu, Le Song, Arthur Gretton, "Kernel Bayes' rule", arxiv:1009.5736

To read:
• Nancy Heckman, "The theory and application of penalized methods or Reproducing Kernel Hilbert Spaces made easy", arxiv:1111.1915
• Alexander Jung, Sebastian Schmutzhard, Franz Hlawatsch, "The RKHS Approach to Minimum Variance Estimation Revisited: Variance Bounds, Sufficient Statistics, and Exponential Families", arxiv:1210.6516
• Vern I. Paulsen and Mrinal Raghupathi, An Introduction to the Theory of Reproducing Kernel Hilbert Spaces `
• James Robins and Aad van der Vaart, "Adaptive nonparametric confidence sets", Annals of Statistics 34 (2006): 229--253, arxiv:math/0605473 ["We construct honest confidence regions for a Hilbert space-valued parameter in various statistical models. The confidence sets can be centered at arbitrary adaptive estimators, and have diameter which adapts optimally to a given selection of models."]
• Christopher G. Small and D. L. McLeish, Hilbert Space Methods in Probability and Statistical Inference
• Bharath K. Sriperumbudur, Kenji Fukumizu, Gert R. G. Lanckriet, "Universality, Characteristic Kernels and RKHS Embedding of Measures", arxiv:1003.0887