Bootstrapping, and Other Resampling Methods

30 Nov 2023 09:14

Bootstrapping is a way of figuring out the properties of statistical estimators (and other procedures, like hypothesis tests) by simulation. What we would really like to know his how different our answers could have been, if we re-ran our experiment. We can't actually do this, but we can fit a model to our data and simulate from it, and see what answer we'd get from the simulations. We can even do this from exceedingly general non-parametric estimates, like re-sampling the original data. This is a brilliant idea, and my default way of handling the uncertainty of estimation in complex models or with complex systems. But having written 3500 words on this for a magazine, plus a textbook chapter, I feel absolutely no inclination to explain myself further.

I most interested in resampling techniques for dependent data, and would be ecstatic if I could figure out a non-parametric bootstrap for networks. (Update: see my paper with Alden Green; the ecstasy was real but inevitably fleeting.) --- Presumably universal prediction algorithms could be used for this purpose?

See also: Cross-Validation; Confidence Sets, Confidence Intervals; Nonparametric Confidence Sets for Functions; Statistics