A p‑value answers one narrow question, and it is not the question most people think. It is not the probability that the null hypothesis is true. It is not the probability that your result happened by chance. It is this: if the null hypothesis were true, how often would studies like this one produce a result at least as extreme as the one you actually got? That is a fact about data in a hypothetical world — a shaded area under a curve you drew before you collected anything. Below you can drag your result along the axis and watch the tail shade and the number recompute. Then, because a probability like this is really just a long‑run frequency, you can build it by hand: run thousands of studies in a world where the null is exactly true, and count the extreme ones yourself.
The tail, shaded
The curve is the sampling distribution of your statistic in the world the null hypothesis describes — centred on the null value, with a spread of one standard error. The dot is what you actually observed. The shaded region is every result at least as extreme as yours, and the p‑value is simply the area of that region. Drag the dot along the axis, or use the sliders on the right.
the curve if H0 were trueyour observed resultat least this extreme — the p‑value
Count it yourself
A probability is a long‑run frequency, so the p‑value can be built without any curve at all. Each click below runs whole studies — drawing every single observation from a world where the null hypothesis is exactly true — and drops each study's statistic into the histogram. Studies that landed at least as far out as yours are shaded. The fraction of them is the p‑value, and it settles onto the area of the curve above.
Studies run0
At least this extreme0
Simulated fraction—
Area under the curve—
Why p is not the probability that H0 is true
The p‑value is computed inside the null world; it never looks up and asks how likely that world was to begin with. To ask “given a small p‑value, how often is the null true anyway?” you need two extra facts the p‑value does not contain: how often the hypotheses people test are null, and how good the studies are at detecting real effects. Here are 1,000 hypotheses tested at the usual 5% cutoff. Move the sliders and watch the answer swing between almost nothing and almost everything — while every rejected study still reported p < 0.05.
50%
50%
Four studiespick one and move its result
The study on the table
Your result, and its tail
1 · What did this sample produce?
—
2 · How much data was behind it?
—
Observed statistic—
Std. error under H0—
Test statistic z—
p‑value (the area)—
Read it out loud
Seven sentences people say about a p‑value of 0.04. Two of them are right. Tap each one.
The definition, in one lineThe p‑value is P(data at least this extreme | H0 true). Everything that goes wrong with p‑values goes wrong by flipping what is on which side of that bar.
How to think about it every time
Say the definition in the order the arithmetic actually happens, and the misreadings become impossible. First, assume. You do not test whether H0 is true; you grant it, temporarily and completely, and step into that world. Second, describe. In that world, the statistic has a known distribution — the curve above — because you know what H0 centres it on and the standard error tells you how wide it is. Third, count. Ask what share of that world's studies would land at least as far from the null value as yours did. That share is the p‑value: a small one means your data would be a rare event if H0 held, which is evidence against H0; a large one means your data is unremarkable in that world, which is not evidence for anything. Notice what never appeared in those three steps: any statement about how probable H0 was, and any statement about how big the effect is.
Why the shaded region is a tail and not a point. The p‑value counts results at least as extreme as yours, not results equal to yours. For a continuous statistic the probability of landing on any exact value is zero, so “the probability of getting this mean” is not a usable quantity at all — it is 0 for every possible outcome, including the ones nobody would find surprising. Extremeness has to be measured as a region, and the region runs outward from your result and away from the null value. That is also why a two‑sided alternative shades both tails: if a departure in either direction would have counted as evidence, then both directions belong in “at least this extreme,” and the same data honestly yields double the area.
The p‑value is not an effect size. Drag n on a study with a fixed gap between the observed result and the null value, and watch the p‑value collapse toward zero while the gap on the axis never changes. Nothing about the world moved; only the sharpness of your instrument did. A tiny p‑value from an enormous sample can accompany a difference too small to matter, and a large p‑value from a small sample can hide a difference that matters enormously. p answers “could this plausibly be noise?” and nothing else. “How big is it, and is that big enough to care?” is a separate question, answered with an estimate and an interval, not with p.
One more consequence worth sitting with: because the p‑value is a probability computed under H0, and because it is a continuous statistic, the p‑value itself is uniformly distributed between 0 and 1 when H0 is true. Every value is equally likely. A p of 0.03 is exactly as probable as a p of 0.83 in a world where nothing is going on — which is another way of saying that 5% of perfectly null studies will always clear a 5% bar, no matter how careful everyone is.