A test can be wrong in exactly two ways: it can cry wolf when nothing is happening, or it can sleep through something real. You choose the first error rate outright. You never choose the second one at all — it falls out of the cutoff you set, the size of the effect that is really out there, and how many observations you were willing to collect. Drag the three dials and watch the two errors refuse to move together.
Two worlds, one cutoff
Resting heart rate, σ = 9 bpm. H₀: μ = 74 against H₁: μ ≠ 74, two-tailed. The left bell is the sampling distribution of x̄ if H₀ is true; the right bell is the one if the true mean is really 74 + your effect. The dashed lines are the cutoffs — the same cutoffs for both worlds, because you never know which world you are in.
α — Type Iβ — Type IIpowercorrect retain
The power curve
Set the studythen read the two error rates
The three dials
Sample size n = 36
True effect — how far μ really sits from 74 +3.0 bpm
Significance level α = 0.050
α · Type I
—
β · Type II
—
Power
—
Effect in SEs
—
The error matrix — tap a cell
Reality: H₀ is true
Reality: H₀ is false
Decision: fail to reject H₀
Decision: reject H₀
200 studies, actually run
Pick the world these studies are drawn from
The 200 studies stay put while you drag the dials, so you can watch the same studies flip from miss to detection as n or the effect grows.