r/statistics•
qria

[Question] Techniques of resampling does not make sense to me.

Question

I am studying about a bunch of resampling methods such as jackknife, bootstraping, etc.

What I don't understand is how resampling makes sense in a high level perspective.

From my understanding, since resampling uses a subset of the sample, it essentially throws out some of the information, therefore any interesting statistical measures estimated through this method should have its quality degraded.

I'm sure there are explanations with calculations but I would like to know if there are any intuition level explanations on it.

Posted Oct 15, 2020, 12:44 AM UTC

32
eatthepieguy•

The analogy is that population is to sample as sample is to subsample. If the DGP is such that your parameter estimate is biased in a certain way, you expect the same type of bias to show up when you resample. So we can use the latter relationship -- which we fully observe -- to learn about the former.

You're not tossing out information in that you resample many times. In a sense, resampling uses more information: you're using the empirical CDF to approximate the true CDF, rather than just looking at some set of parameters.

2
funkentelchy•

since resampling uses a subset of the sample, it essentially throws out some of the information

We gain that information back at the end.

With resampling, the subsets are usually formulated such that every datapoint finds its way into at least one subsample. We compute a statistic on every subsample, then combine those statistics together at the end. This means every datapoint influences the end result via the subsample(s) it appears in, even though it may be excluded from many of the others.

For intuition, you can think of standard deviation in these terms. The "subsamples" are individual datapoints, and the "statistic" is squared difference with the mean. Squared difference is itself an estimator of variance, but it's a lousy one because we've discarded all but one of the datapoints, losing information. However, by computing it for all subsets and taking the sum, we gain back that lost information and arrive at an excellent estimator of variance.

1