Continuous Function

Machine Learning

Bootstrap, Jackknife, and Resampling

Bootstrap and jackknife resampling turn one dataset into a local uncertainty probe, while making the assumptions and limits of that probe visible.

status: publishedimportance: importantdifficulty 3/5math: undergraduateread: 15mlive demo
Back to Machine LearningNext: Evaluation Pipelines

Concept Structure

Bootstrap, Jackknife, and Resampling

01Intuition

Start with the picture, metaphor, or geometric mechanism.

02Math

Make the objects explicit and connect them with notation.

03Code

Mirror the equations with runnable implementation details.

04Interactive Demo

Manipulate the mechanism and watch the idea respond.

2prerequisites
1next concepts
3related links

Learner Contract

What this page should let you do.

You are here becauseBootstrap and jackknife resampling turn one dataset into a local uncertainty probe, while making the assumptions and limits of that probe visible.

This Machine Learning concept is the current object: keep the same invariant visible across Intuition, Math, Code, Interactive Demo.

By the end4/4 sections ready | code witness expected | live demo

Explain the mechanism, trace the main notation, and test one prediction in the live demo.

Do this firstIntuition

Read the intuition before the notation; the math should name a mechanism you already felt.

Then go nextEvaluation Pipelines

Follow this edge after making one prediction here; the next page should reuse the result, not restart the route.

Test the linkManipulate one control and predict the visible change.Then continue to Evaluation Pipelines

Claim/source review status

Substantive review recorded

1/1 claims have bounded review metadata; still check caveats and source scope.Metadata-derived; review may be AI-assisted. Not a human certification.
Claims1/1 reviewed
Sources3 cited
Codeattached
Demolive
Reviewed2026-06-30
Updatedpage 2026-06-30

Object flow

4/4 sections readyAsk about thisResearch room
ConceptBootstrap, Jackknife, and ResamplingMachine LearningEquationBootstrap, Jackknife, and Resampling equation 1Exact equation objectCodeBootstrap, Jackknife, and Resampling code witness 1Exact code witnessDemoBootstrap, Jackknife, and Resampling interactive demoVisualization objectClaimFor a fixed statistic or fixed procedure, bootstrap resamples approxi...Exact claim checkSourceBootstrap Methods: Another Look at the JackknifeExact source object
ConceptBootstrap, Jackknife, and ResamplingMachine Learning
3 sources attachedLocal snapshot ready
concept:machine-learning/bootstrap-jackknife-resampling
Codewitness nearbyPredictbefore revealRoomobject handoff
01IntuitionStart with the picture, metaphor, or geometric mechanism.02MathMake the objects explicit and connect them with notation.03CodeMirror the equations with runnable implementation details.04Interactive DemoManipulate the mechanism and watch the idea respond.
01

01

Intuition

Build the mental picture first so the rest of the page has something to attach to.

Section prompt

You have one dataset and one statistic. How much should that number wobble if the data collection had landed a little differently?

Bootstrap and jackknife resampling are ways to probe that wobble without pretending you collected new evidence. The bootstrap asks: if the observed sample is our best stand-in for the population, what happens when we repeatedly draw new samples from it with replacement and recompute the same statistic or procedure? The jackknife asks a sharper sensitivity question: what happens to the statistic when each observation is left out once?

The invariant is: resampling estimates uncertainty with respect to the data-generating draw you model. It does not fix biased sampling, leakage, dependence, broken labels, post-hoc model selection, or a contaminated test set.

That makes resampling useful near model selection and evaluation. Cross-validation helps estimate performance across train/test splits. Bootstrap and jackknife help ask how sensitive a statistic, metric, or fixed fitted procedure is to the observed sample. But they only inherit the trustworthiness of the sampling story. If rows are dependent, selected after peeking, or not representative of the deployment question, resampling the same rows more times does not make the evidence clean.

There is one important model-selection scope warning. If the target is uncertainty about a whole procedure, including hyperparameter search, then the resampling design has to represent that procedure. If you resample only after choosing the model, the uncertainty statement is conditional on the already-selected model.

Use the bootstrap when you want a direct simulation of sampling variation for a statistic and can tolerate Monte Carlo approximation. Use the jackknife when leave-one-out sensitivity or a simple bias/standard-error approximation is the right diagnostic. For smooth statistics like a mean, the jackknife is often easy to read. For nonsmooth statistics like a maximum or some medians, it can hide failure modes or become unstable.

This page is deliberately scoped. It teaches percentile bootstrap intervals and the classical jackknife leave-one-out formula. It does not teach BCa intervals, permutation tests, Bayesian bootstrap, block bootstrap for time series, clustered bootstrap, post-selection inference, or production-grade uncertainty reporting.

02

02

Math

Translate the story into symbols, assumptions, and a derivation you can inspect.

Section prompt
Equation 1x1:n=(x1,…,xn),θ^=s(x1:n),x_{1:n}=(x_1,\ldots,x_n), \qquad \hat\theta=s(x_{1:n}),Equation 2F^n=1n∑i=1nδxi.\hat F_n = \frac{1}{n}\sum_{i=1}^n \delta_{x_i}.

Let the observed sample be

x1:n=(x1,…,xn),θ^=s(x1:n),x_{1:n}=(x_1,\ldots,x_n), \qquad \hat\theta=s(x_{1:n}),

where ss is a statistic such as a mean, median, error rate, or fitted-model score.

The empirical distribution places mass 1/n1/n on each observed point:

F^n=1n∑i=1nδxi.\hat F_n = \frac{1}{n}\sum_{i=1}^n \delta_{x_i}.

A bootstrap sample is drawn from this empirical distribution:

x1:n∗(b)∼F^n,θ^∗(b)=s(x1:n∗(b)),b=1,…,B.x^{*(b)}_{1:n}\sim \hat F_n, \qquad \hat\theta^{*(b)}=s(x^{*(b)}_{1:n}), \qquad b=1,\ldots,B.

The bootstrap standard error is the sample standard deviation of the bootstrap statistics:

se^boot=1B−1∑b=1B(θ^∗(b)−θˉ∗)2.\widehat{\mathrm{se}}_{\mathrm{boot}} = \sqrt{\frac{1}{B-1}\sum_{b=1}^B \left(\hat\theta^{*(b)}-\bar\theta^*\right)^2}.

A percentile interval uses quantiles of the bootstrap statistics, for example the 2.5 percent and 97.5 percent quantiles. That interval is easy to teach and often useful, but it is not magic coverage. Coverage depends on the statistic, sample size, sampling assumptions, and interval method.

The jackknife forms leave-one-out statistics

θ^(i)=s(x1,…,xi−1,xi+1,…,xn),θˉ(⋅)=1n∑i=1nθ^(i).\hat\theta_{(i)} = s(x_1,\ldots,x_{i-1},x_{i+1},\ldots,x_n), \qquad \bar\theta_{(\cdot)}=\frac{1}{n}\sum_{i=1}^n \hat\theta_{(i)}.

The classical jackknife bias and standard-error estimates are

bias^jack=(n−1)(θˉ(⋅)−θ^),se^jack=n−1n∑i=1n(θ^(i)−θˉ(⋅))2.\widehat{\mathrm{bias}}_{\mathrm{jack}} =(n-1)(\bar\theta_{(\cdot)}-\hat\theta), \qquad \widehat{\mathrm{se}}_{\mathrm{jack}} = \sqrt{\frac{n-1}{n} \sum_{i=1}^n \left(\hat\theta_{(i)}-\bar\theta_{(\cdot)}\right)^2 }.

The formulas are not saying the deleted observations are independent new experiments. They are a local sensitivity probe: if one row has a large leave-one-out effect, your statistic is leaning hard on that row.

03

03

Code

Keep the implementation aligned with the notation so the algorithm is legible.

Section prompt
Code witness 1import random, statistics, math x = [3.2, 3.5, 3.7, 4.1, 4.2, 4.4, 4.6, 4.8, 8.8] B = 4000 rn...python

This witness estimates uncertainty for the sample mean and reports jackknife leave-one-out influence. It is intentionally small and IID-scoped.

import random, statistics, math

x = [3.2, 3.5, 3.7, 4.1, 4.2, 4.4, 4.6, 4.8, 8.8]
B = 4000
rng = random.Random(7)

def mean(xs):
    return sum(xs) / len(xs)

theta = mean(x)
boots = []
for _ in range(B):
    sample = [rng.choice(x) for _ in x]
    boots.append(mean(sample))

boots.sort()
lo = boots[int(0.025 * B)]
hi = boots[int(0.975 * B)]
boot_se = statistics.stdev(boots)

leave_one = []
for i in range(len(x)):
    loo = x[:i] + x[i + 1:]
    leave_one.append(mean(loo))

loo_bar = mean(leave_one)
jack_bias = (len(x) - 1) * (loo_bar - theta)
jack_se = math.sqrt((len(x) - 1) / len(x) * sum((v - loo_bar) ** 2 for v in leave_one))
influence = [(xi, (len(x) - 1) * (theta - loo)) for xi, loo in zip(x, leave_one)]

print("mean", round(theta, 3))
print("bootstrap percentile CI", (round(lo, 3), round(hi, 3)), "se", round(boot_se, 3))
print("jackknife bias", round(jack_bias, 3), "se", round(jack_se, 3))
print("largest influence", max(influence, key=lambda pair: abs(pair[1])))

The high value 8.8 is not "bad" by definition. The point is that resampling makes its leverage visible. If 8.8 is a real representative draw, the interval should remember it. If it is a data error or a different population, resampling cannot decide that for you.

04

04

Interactive Demo

Use direct manipulation to connect the explanation to a moving system.

Section prompt

Prediction check: choose a dataset and a statistic, then commit to where the largest leave-one-out effect should appear.

Before reveal, keep three questions separate:

  • What statistic are we resampling?
  • What sampling story lets the observed rows stand in for future draws?
  • Which row should the jackknife mark as most influential: an edge row, a center row, or no single row?

After reveal, compare the bootstrap percentile interval, the bootstrap histogram, and the jackknife influence bars. The demo is intentionally small, but the invariant is large: resampling changes row weights under a sampling assumption. It does not create new evidence, repair leakage, or decide whether an unusual row belongs to the target population.

Live Concept Demo

Explore Bootstrap, Jackknife, and Resampling

The stage is code-native and interactive. Use it to test the explanation against the mechanism.

difficulty 3/5undergraduatecode-aligned
Demo Prediction Checkpoint

Manipulate one control and predict the visible change.

Commit to what Bootstrap, Jackknife, and Resampling should make visible before reading the result.

After The First Pass

Turn the concept into an inspected object.

Once the invariant is visible in the intuition, math, code, and demo, use these panels to inspect the mechanism visually, check source support, practice the idea, and attach a grounded research question.

Mechanism Storyboard

See the idea move before the page explains it

Bootstrap and jackknife resampling turn one dataset into a local uncertainty probe, while making the assumptions and limits of that probe visible.

Prediction open01 / Intuition
Prediction lens

Start with the picture, metaphor, or geometric mechanism.

Commit first

Before reading further, choose the kind of change Bootstrap, Jackknife, and Resampling should make visible.

Visual Inquiry

Make the image answer a mathematical question

Bootstrap and jackknife resampling turn one dataset into a local uncertainty probe, while making the assumptions and limits of that probe visible.

4/4 stages readyLive demo connected
Prediction

Which visible object should carry the first intuition?

Commit first

Pick the cue that should make Bootstrap, Jackknife, and Resampling easier to reason about before the page gives the answer.

Source Grounding

Canonical references for the mechanism on this page.

paper · 1979Bootstrap Methods: Another Look at the JackknifeEfron

Canonical source introducing the bootstrap as resampling from the empirical distribution and comparing it with the jackknife.

Open source
book · 1993An Introduction to the BootstrapEfron and Tibshirani

Canonical book source for bootstrap standard errors, confidence intervals, jackknife ideas, and practical limitations.

Open source
book · 2023An Introduction to Statistical Learning: Resampling MethodsJames, Witten, Hastie, Tibshirani, and Taylor

Curriculum source for resampling methods as tools for estimating model assessment quantities and uncertainty.

Open source

Claim Review

Bootstrap and jackknife resampling turn one dataset into a local uncertainty probe, while making the assumptions and limits of that probe visible.

Status1 substantive review recorded

Claims without a substantive review badge still need exact source-support review.

Sources3 references

efron-1979-bootstrap-jackknife, efron-tibshirani-bootstrap, islr-resampling-methods

Witnesses4 local objects

Use equation, code, and demo objects to check whether the source support is operational.

Substantively reviewedFor a fixed statistic or fixed procedure, bootstrap resamples approximate sampling variation by drawing from the empirical distribution, while jackknife leave-one-out values estimate sensitivity; neither creates new evidence or repairs a biased evaluation contract.Claim metadata: source checked

Efron supports empirical-distribution bootstrap draws and jackknife comparison; Efron-Tibshirani supports bootstrap standard-error/interval practice; ISLR2 supports resampling as estimator or method uncertainty, with model assessment kept separate.

Sources: Bootstrap Methods: Another Look at the Jackknife, An Introduction to the Bootstrap, An Introduction to Statistical Learning: Resampling MethodsThe final no-new-evidence/evaluation-repair sentence is a reviewed teaching synthesis. The page teaches IID toy percentile/bootstrap and jackknife estimates, not BCa, dependent bootstrap, valid post-selection inference, or production uncertainty reporting.A bounded review summary is present; still check caveats and exact source scope.

Reviewed Efron 1979 for empirical-distribution bootstrap and jackknife comparison, Efron-Tibshirani for bootstrap standard errors and interval scope, and ISLR2 for resampling as estimator/method uncertainty. The no-new-evidence/evaluation-repair sentence is a caveated teaching synthesis. Evidence: responses/bootstrap-jackknife-source-support-review-20260630.md.

Reviewer: codex-primary-source-audit+gpt-5.5-subagents; reviewed 2026-06-30
source-span-efron-1979-bootstrap-jackknifesource-span-efron-tibshirani-bootstrapsource-span-islr-resampling-methodsmath-object-1math-object-2code-witness-1interactive-demo

Source support candidates

paper 1979Bootstrap Methods: Another Look at the Jackknife

Canonical source introducing the bootstrap as resampling from the empirical distribution and comparing it with the jackknife.

book 1993An Introduction to the Bootstrap

Canonical book source for bootstrap standard errors, confidence intervals, jackknife ideas, and practical limitations.

book 2023An Introduction to Statistical Learning: Resampling Methods

Curriculum source for resampling methods as tools for estimating model assessment quantities and uncertainty.

Mechanism witnesses

Equation 1
x1:n=(x1,…,xn),θ^=s(x1:n),x_{1:n}=(x_1,\ldots,x_n), \qquad \hat\theta=s(x_{1:n}),
Equation 2
F^n=1n∑i=1nδxi.\hat F_n = \frac{1}{n}\sum_{i=1}^n \delta_{x_i}.
Code witness 1import random, statistics, math x = [3.2, 3.5, 3.7, 4.1, 4.2, 4.4, 4.6, 4.8, 8.8] B = 4000 rn...Demo stateLive mechanism probe

Practice Loop

Try the idea before it explains itself

Bootstrap and jackknife resampling turn one dataset into a local uncertainty probe, while making the assumptions and limits of that probe visible.

Readiness0/3 checks ready
Predict

Before touching the demo, predict one visible change that should happen in Bootstrap, Jackknife, and Resampling.

Hint 1

Reveal when your model needs a nudge.

Hint 2

Reveal when your model needs a nudge.

Hint 3

Reveal when your model needs a nudge.

Object research drawerClose
ConceptBootstrap, Jackknife, and ResamplingMachine Learning
Code witness comparisonBootstrap, Jackknife, and Resampling code witness 1x = [3.2, 3.5, 3.7, 4.1, 4.2, 4.4, 4.6, 4.8, 8.8]Prediction before revealBootstrap, Jackknife, and Resampling interactive demoManipulate one control and predict the visible change.
Grounded room questionWhat is the smallest example that makes Bootstrap, Jackknife, and Resampling click without losing the math?Local snapshot ready

Research Room

Attach the question to an exact object

Pick the concept, equation, source, code witness, claim, misconception, or demo state before asking for help. The handoff stays grounded to that object.
Next local actionNo local draft saved yet

Open the draft below to save one note and next action in this browser.

conceptMachine Learning

Bootstrap, Jackknife, and Resampling

Anchored question

What is the smallest example that makes Bootstrap, Jackknife, and Resampling click without losing the math?

Local action draftNo local draft saved yetExpand only when ready to capture one local next action
Local action draft

This draft stays locally in this browser for concept:machine-learning/bootstrap-jackknife-resampling.

No local draft saved.
Evidence to inspect
  • Source ids to inspect: efron-1979-bootstrap-jackknife, efron-tibshirani-bootstrap, islr-resampling-methods
  • Definition, prerequisite, and contrast concept links
  • The equation or code witness that makes the concept operational
  • One demo state that shows the invariant instead of a slogan
What would resolve this
  • The learner can state the mechanism in their own words
  • The learner can name the prerequisite that would repair confusion
  • The learner can predict how the mechanism changes under one perturbation
Grounded AI handoff

I am working in Continuous Function's research reading room. Object: concept - Bootstrap, Jackknife, and Resampling Object key: concept:machine-learning/bootstrap-jackknife-resampling Context: Machine Learning Anchor id: concept/concept-notebook/machine-learning/bootstrap-jackknife-resampling Open question: What is the smallest example that makes Bootstrap, Jackknife, and Resampling click without losing the math? Evidence to inspect: - Source ids to inspect: efron-1979-bootstrap-jackknife, efron-tibshirani-bootstrap, islr-resampling-methods - Definition, prerequisite, and contrast concept links - The equation or code witness that makes the concept operational - One demo state that shows the invariant instead of a slogan What would resolve this: - The learner can state the mechanism in their own words - The learner can name the prerequisite that would repair confusion - The learner can predict how the mechanism changes under one perturbation Answer as a careful research tutor: stay source-grounded, separate verified evidence from assumptions, name the relevant math objects, and end with one next action.

Open source object
concept/concept-notebook/machine-learning/bootstrap-jackknife-resampling concept:machine-learning/bootstrap-jackknife-resampling