What Statistics Is

Statistics is not a branch of mathematics that exists to punish you. It is a set of honest habits for handling one stubborn fact about the world: things vary. Two people on the same diet lose different amounts of weight, two identical machines produce slightly different parts, two polls of the same election disagree by three points. Statistics is the discipline of describing that variation clearly, and of deciding what a limited pile of data does — and does not — allow you to conclude about the much larger world it came from. This chapter builds the vocabulary that every later chapter leans on, and it does so with arithmetic no harder than what you already do to split a restaurant bill.

Section 1Why statistics — turning data into decisions

Variation is the reason this subject exists

If every patient responded to a drug identically, you would need one patient to test it. If every light bulb failed at exactly 1,200 hours, you would need one bulb to write the warranty. Nothing behaves that way. Measurements of the same thing differ, individuals differ from one another, and repeating a study gives slightly different answers each time. Statistics is the science of collecting, organizing, summarizing, and drawing conclusions from data in the presence of that variation.

The practical payoff is decisions. A hospital deciding whether to change a protocol, a company deciding which of two web pages to keep, a city deciding where to put a bus route — all of them face the same problem. They have some data, the data are noisy, the data cover only part of the situation, and a choice still has to be made. Statistics is how you make that choice defensible rather than merely confident.

The field splits cleanly in two, and knowing which half you are in is most of the battle. Descriptive statistics summarizes the data you actually have. Inferential statistics uses those data to make a claim about a larger group you did not measure, and attaches an honest statement of uncertainty to that claim.

Descriptive statistics: making a pile of numbers speak

Suppose you record your commute, in minutes, on nine workdays:

22, 25, 19, 31, 24, 27, 21, 88, 23

Nine numbers is already too many to hold in your head. Descriptive statistics compresses them. Start with the mean (the arithmetic average), written x̄ and read “x-bar”:

x̄ = Σx ÷ n

The symbol Σ (capital sigma) just says “add them all up,” and n is how many values you have. Adding: 22 + 25 + 19 + 31 + 24 + 27 + 21 + 88 + 23 = 280, and n = 9, so

x̄ = 280 ÷ 9 = 31.1 minutes

Now the median, the middle value once the data are sorted. Sorted, the nine values are 19, 21, 22, 23, 24, 25, 27, 31, 88. With n = 9 the middle position is the 5th, so the median is 24 minutes. And the range, the crudest measure of spread, is largest minus smallest: 88 − 19 = 69 minutes.

Look at what those three numbers reveal. The mean says 31.1 minutes; the median says 24. They disagree because of a single day — the 88 — when something went badly wrong. Delete that one day and the remaining eight sum to 192, giving a mean of 192 ÷ 8 = 24.0 minutes, exactly matching the median. One unusual value dragged the average up by seven minutes. If you budgeted 31 minutes for your commute you would be early almost every single day.

That is the first real lesson of the subject: a summary is a choice, and different summaries answer different questions. The median answers “what is a typical day?” The mean answers “what happens on average over many days?” — which is the right question if you are totalling up hours lost to commuting over a year.

Inferential statistics: from the part to the whole

Descriptive work stops at the data in front of you. The moment you say “and therefore the population is like this,” you have crossed into inference. If 640 of 1,000 surveyed voters favor a measure, the descriptive fact is that 64% of those 1,000 favored it. The inferential claim — “about 64% of all voters favor it, give or take three points” — is a leap, and inference is the machinery that makes the leap safely and quantifies how far it might land off target.

Two tools do most of that work, and both are routinely misdescribed, so meet them correctly now even though the details come later. A confidence interval is a range built from your sample by a procedure with a known long-run success rate; “95% confidence” describes the procedure, meaning that if the study were repeated over and over, about 95% of the intervals produced that way would contain the true value. It does not mean there is a 95% chance that the one interval in front of you contains it. A p-value is the probability of getting data at least as extreme as yours if a specified “nothing is going on” hypothesis were true. It is emphatically not the probability that the hypothesis is true.

Both tools carry conditions, and the conditions are the part most often skipped. Their advertised performance — the 95%, the p-value — holds only if the sample was actually drawn at random from the population being described, if the observations are independent of one another, and if the model behind the calculation is a reasonable fit to the data. Feed either procedure a self-selected sample and it will still return a tidy interval and a crisp p-value; those numbers simply will not mean what they claim to. Assumptions are not fine print appended to a result. They are the load-bearing wall the result stands on, and every later chapter will state them before it states a conclusion.

Section 2Population and sample

The whole group and the piece you can afford

A population is the entire collection of individuals, objects, or measurements you actually want to know about. A sample is the subset of that population you manage to observe. Notice that “population” does not have to mean people: it can be every weld on a bridge, every transaction last quarter, or every possible roll of a die. What defines a population is the question you are asking. If you want to know about licensed electricians in Ohio, that is your population; a study of the electricians employed by one Ohio contractor is a sample from it, and possibly a poor one.

Measuring every member of a population is a census. It is the ideal, and it is usually impossible. Populations are too big, too expensive to reach, too spread out, still being created (next year's customers do not exist yet), or destroyed by measurement — you cannot crash-test every car you intend to sell, or taste-test every jar of sauce. A well-chosen sample of 1,200 people can describe a nation of 300 million to within a few percentage points, for the price of a few days of phone calls. That efficiency is the whole reason sampling exists.

Parameter versus statistic

This distinction carries the rest of the book, so learn it once and properly. A parameter is a number that describes a population. A statistic is a number that describes a sample. The parameter is what you want; the statistic is what you can get. Both get their own symbols, and mixing them up is the single most common notation error in a first course.

A memory hook that actually works: Population goes with Parameter and with Greek letters; Sample goes with Statistic and with ordinary Roman letters. Greek for the truth you cannot see, Roman for the estimate you can compute.

Worked example: one parameter, many statistics

Take a tiny population of five employees whose years of service are 2, 4, 6, 8, and 10. Because this population is small enough to measure completely, we can compute the parameter directly:

μ = (2 + 4 + 6 + 8 + 10) ÷ 5 = 30 ÷ 5 = 6 years

The population standard deviation uses σ = √[ Σ(x − μ)² ÷ N ]. The deviations from 6 are −4, −2, 0, 2, 4; squaring gives 16, 4, 0, 4, 16, which sum to 40. So σ² = 40 ÷ 5 = 8 and σ = √8 ≈ 2.83 years.

Now pretend we could only afford to look at three employees. Different samples of three give different answers:

The parameter μ never moved: it is 6, permanently. The statistic x̄ came out 4.0, 6.0, and 8.0. That spread of answers is sampling error — the unavoidable difference between a sample statistic and the population parameter that arises purely because you looked at a part instead of the whole. Sampling error is not a mistake and it cannot be eliminated, only measured and shrunk (larger samples make it smaller).

Notice also the n − 1 in the sample variance. Deviations are measured from x̄, which is itself pulled toward the sample's own values, so the squared deviations come out slightly too small on average. Dividing by n − 1 instead of n corrects that tilt, so that s² neither systematically over- nor under-estimates σ². Population variance divides by N; sample variance divides by n − 1.

Sampling bias is a different animal entirely. Bias is a systematic tilt in how the sample was chosen that pushes the statistic away from the parameter in a consistent direction — surveying only people who answer landlines, only customers who bothered to leave a review, only patients healthy enough to return for follow-up. The defense against sampling error is a bigger sample. The defense against bias is a properly randomized one, in which every member of the population has a known, non-zero chance of selection.

Section 3Types of data

Qualitative versus quantitative

A variable is any characteristic that can differ from one individual to the next; the recorded values are the data. The first fork in the road is whether the variable names a quality or measures a quantity.

Qualitative (also called categorical) variables place each individual into a category: blood type, brand of phone, county of residence, whether a part passed inspection. The values are labels. You can count how many fall in each category and compute percentages, but the labels themselves cannot be added, averaged, or subtracted.

Quantitative (also called numerical) variables record an amount or a count: height, salary, number of siblings, reaction time in milliseconds. Their values are genuine numbers on which arithmetic means something. A useful test: if the answer to “how much” or “how many” is a number that you would meaningfully add up, the variable is quantitative.

Discrete versus continuous

Quantitative variables split again. A discrete variable can take only separated values, typically from counting — you can list the possibilities, and there are gaps between them. The number of children in a household is 0, 1, 2, 3, …; no household contains 2.4 children. A continuous variable can take any value in an interval, typically from measuring — between any two possible values there is always another. A person's weight might be 68 kg, 68.4 kg, or 68.4173 kg; the only limit is the precision of the scale.

The rule of thumb: counted things are discrete, measured things are continuous. Money is the classic gray case — strictly discrete because it comes in cents, but with so many closely spaced possible values that we treat it as continuous in practice.

Numbers that are not quantities

Plenty of data look numeric and are not. ZIP codes, jersey numbers, patient ID numbers, and phone numbers are made of digits, but the digits function as names. The average ZIP code of your customers is a meaningless number that a spreadsheet will cheerfully compute for you. The same trap appears when categories are coded for a computer — entering 1 = single, 2 = married, 3 = divorced does not make marital status quantitative, and the “mean marital status of 1.8” is nonsense. Always ask what the number stands for, not what it looks like.

A worked point on the discrete side that trips people up: an average of a discrete variable need not be a possible value. Take five households with 1, 2, 0, 3, and 2 children. The sum is 1 + 2 + 0 + 3 + 2 = 8, and n = 5, so

x̄ = 8 ÷ 5 = 1.6 children per household

No household has 1.6 children, and that is fine. The mean is a summary of the group, not a description of any individual member. “The average family has 1.6 children” is a correct statement about families collectively and an absurd one about any family in particular.

Section 4Levels of measurement

Four levels, each adding one property

Classifying data as qualitative or quantitative is a good first cut, but it is too coarse to tell you which calculations are legitimate. The levels of measurement refine it into four rungs. Each rung keeps everything the rung below it has and adds exactly one new capability, so the ladder is easy to hold in memory.

Nominal is the bottom rung: categories with no natural order. Blood type, religion, marital status, department. You can count and compare frequencies, and that is all. The only measure of center available is the mode — the most common category.

Ordinal adds order. Categories can be ranked, but the distances between ranks are unknown or unequal. Class rank, military rank, letter grades, survey answers from “strongly disagree” to “strongly agree”, cancer stage I–IV. First, second, and third place tell you who beat whom but not by how much: the winner may have finished a tenth of a second ahead of second place and second place five minutes ahead of third. Mode and median are available; the mean is questionable.

Interval adds equal, meaningful distances. The gap between 20 and 30 is the same size as the gap between 70 and 80, so subtraction and addition are legitimate. What interval scales lack is a true zero — zero is a chosen landmark rather than the absence of the thing. Temperature in Celsius or Fahrenheit is the standard example; so are calendar years and IQ scores. Mode, median, and mean are all fair game, but ratios are not.

Ratio adds the true zero, where 0 genuinely means “none of it.” Height, weight, age, income, distance, reaction time, counts of anything. Because zero is real, ratios finally mean what they say: 40 kg is twice 20 kg, and a 60-minute task takes three times as long as a 20-minute task. Every summary statistic is permitted.

The zero test, worked out in numbers

The interval-versus-ratio line is the one students actually get wrong, and a single calculation settles it. Is 20°C twice as hot as 10°C? Naively 20 ÷ 10 = 2, so it looks like it. Test that claim by re-expressing the same two temperatures on a different scale and recomputing the ratio.

Convert to Fahrenheit with °F = (9 ÷ 5) × °C + 32:

10°C → (9 ÷ 5)(10) + 32 = 18 + 32 = 50°F
20°C → (9 ÷ 5)(20) + 32 = 36 + 32 = 68°F
Ratio: 68 ÷ 50 = 1.36

Convert to kelvin, which does have a true zero, with K = °C + 273.15:

10°C → 283.15 K
20°C → 293.15 K
Ratio: 293.15 ÷ 283.15 = 1.035

The same two physical states produce ratios of 2, 1.36, and 1.035 depending on which scale you happen to write them in. A quantity whose “twice as much” changes when you change units is not really a ratio at all. The kelvin figure is the honest one, because 0 K is the actual absence of thermal energy, while 0°C is just the freezing point of water — a convenient landmark someone chose. Differences, however, survive the translation perfectly: 20°C − 10°C = 10 Celsius degrees, and 293.15 K − 283.15 K = 10 kelvins. That is exactly what “interval level” means: differences are trustworthy, ratios are not.

Why the level dictates the arithmetic

Consider four patients who rate their pain on a 0–10 scale as 2, 3, 3, and 8. The mean is (2 + 3 + 3 + 8) ÷ 4 = 16 ÷ 4 = 4.0, and the median is the average of the two middle values, (3 + 3) ÷ 2 = 3.0. Which should you report? A self-reported pain rating is ordinal: nothing guarantees that the step from 2 to 3 hurts as much as the step from 7 to 8, and one patient's 8 is not another's 8. The mean of 4.0 quietly assumes equal spacing that the scale does not provide, and it has been dragged upward by the single 8. The median of 3.0 assumes only that the ratings can be ranked, which is true, so it is the safer summary.

In practice, researchers do sometimes average ordinal scales — averaging Likert responses is nearly universal — and it is often defensible when the categories are many and roughly evenly spaced. But it is a modeling assumption, not a mathematical entitlement, and it should be made knowingly. The general rule holds: statistics computed at a level higher than your data support will produce numbers that look precise and mean nothing.

Section 5How statistics mislead

Correlation is not causation

Two variables are correlated when they move together. That fact alone permits four very different explanations, and the arithmetic cannot tell them apart. A may cause B. B may cause A (hospitals are full of sick people; hospitals do not cause sickness). Some third factor — a confounding variable — may cause both. Or it may be coincidence, which is far more common than intuition suggests: sift enough series against enough other series and you will find pairs that track each other beautifully and have nothing whatever to do with one another.

The classic confounding example: ice cream sales and drownings rise and fall together across the year. Ice cream does not drown anyone. Hot weather independently drives both. The way to break a confound is a randomized experiment, in which the researcher assigns the treatment by chance so that all other factors — known and unknown — are balanced across groups on average. Observational data can suggest causes; randomization is what lets you claim one.

The standard repertoire of tricks

Most misleading statistics are not fabricated. They are true numbers presented so that a reasonable reader draws a false conclusion. A short catalogue covers most of what you will meet.

The relative-risk trick deserves the arithmetic, because it is the one that changes real decisions. Suppose a condition affects 2 in 1,000 untreated people, and a new drug cuts that to 1 in 1,000. Two headlines are available, both true:

Relative risk reduction = (0.002 − 0.001) ÷ 0.002 = 0.001 ÷ 0.002 = 0.50 = 50% — “the drug halves your risk.”

Absolute risk reduction = 0.002 − 0.001 = 0.001 = 0.1 percentage points — one fewer case per thousand people treated.

From the absolute figure you get the number needed to treat: NNT = 1 ÷ 0.001 = 1,000. One thousand people must take the drug for one to benefit. “Cuts your risk in half” and “999 out of 1,000 people get no benefit” describe the identical study. Marketing prints the first; a fair report prints both. Whenever you see a percentage change in a risk, find the baseline before you react.

Reading a claim critically

You do not need advanced technique to catch most bad statistics. You need a short list of questions asked stubbornly.

• What exactly was measured, and on whom? Definitions do enormous work. “Unemployment,” “homelessness,” and “engagement” all have several defensible definitions that give different numbers.
• Who was in the sample, and how were they chosen? Volunteers, subscribers, and people who answered the phone are not random samples of anyone.
• What is n, and what is the margin of error? A result with no stated uncertainty is a result you cannot evaluate.
• Compared with what? A number without a comparison group is not evidence. “70% improved” means nothing if 70% improve without treatment.
• Is this an experiment or an observation? Only randomized assignment supports a causal claim.
• Who is missing? Dropouts, non-responders, and failed businesses are invisible in the data and often the most informative cases.
• Who paid, and what result did they want? Not disqualifying, but it tells you where to look hardest.
• Does the conclusion actually follow? Statistical significance is not practical importance; a tiny, useless effect can be “significant” in a large enough study.

Two interpretations are worth nailing down permanently, because they are misstated constantly — including in published work. A p-value of 0.03 means: if there were truly no effect, data at least as extreme as ours would occur about 3% of the time. It does not mean there is a 3% chance the effect is absent, a 97% chance the finding is real, or that the effect is large. The mirror-image error is equally common and equally damaging: a large p-value — 0.40, say — does not show that there is no effect. Failing to find convincing evidence against a hypothesis is not the same as establishing that it is true, and a small or noisy study fails to detect real effects routinely. “We did not reject it” means only that these data are compatible with it; absence of evidence is not evidence of absence. And a 95% confidence interval of 2 to 8 points describes a procedure that, used repeatedly across many studies, captures the true value about 95% of the time. For the single interval printed in front of you, the true value is either inside it or it is not; the 95% is a property of the method, not a probability attached to this one result.

ReferenceWords worth knowing

A quick reference for the terms introduced in this chapter. Skim it now to see the shape of the vocabulary, then come back whenever a word in a later chapter refuses to sit still.

Bias (sampling bias)

A systematic tendency for a sampling method to produce statistics that miss the parameter in a consistent direction. Unlike sampling error, it is not reduced by taking a larger sample.

Categorical (qualitative) variable

A variable whose values are labels placing each individual into a group, such as blood type or county. Counts and percentages are meaningful; arithmetic on the labels is not.

Census

A study in which every member of the population is measured, producing parameters directly rather than estimating them.

Confidence interval

A range of plausible values for a parameter, produced by a procedure with a stated long-run success rate. “95% confidence” describes how often the procedure captures the truth across many repetitions, not the chance that one particular interval does.

Confounding variable

An outside factor related to both variables under study, capable of producing a correlation between them even when neither causes the other.

Continuous variable

A quantitative variable that can take any value within an interval, limited only by measurement precision — height, time, temperature.

Correlation

A tendency for two variables to move together. A correlation is consistent with four different situations — one variable causing the other, the reverse, a confounding variable driving both, or coincidence — and the number itself cannot distinguish among them. Only a randomized experiment licenses a causal claim.

Descriptive statistics

Methods for organizing and summarizing the data actually collected, with no claim made about anyone not measured.

Discrete variable

A quantitative variable whose possible values are separated, usually arising from counting — number of children, cars sold per day.

Inferential statistics

Methods for drawing conclusions about a population from a sample, always accompanied by a statement of uncertainty.

Interval level

A level of measurement with ordered categories and equal, meaningful distances, but no true zero. Differences are meaningful; ratios are not. Celsius temperature and calendar years are examples.

Mean

The arithmetic average, Σx ÷ n for a sample and Σx ÷ N for a population. It uses every value, which is why a single extreme observation can pull it well away from the bulk of the data.

Median

The middle value of a sorted data set — the average of the two middle values when the count is even. Unaffected by how extreme the extreme values are, so it is the more honest summary of a skewed distribution.

Mode

The most frequently occurring value or category. The only measure of center available for nominal data.

Nominal level

The lowest level of measurement: unordered categories. Only counts, proportions, and the mode are meaningful.

Ordinal level

A level of measurement whose categories can be ranked, but with unknown or unequal distances between ranks. The median is appropriate; the mean requires an added assumption.

Parameter

A numerical characteristic of a population, such as μ or σ. Fixed in value and, in practice, unknown.

Population

The complete collection of individuals, objects, or measurements that a question is about.

P-value

The probability of observing data at least as extreme as those obtained, computed assuming a specified null hypothesis is true. It is not the probability that the hypothesis is true.

Quantitative (numerical) variable

A variable recording an amount or a count, on which arithmetic is meaningful.

Random sample

A sample chosen by a chance mechanism in which every member of the population has a known, non-zero probability of selection — the principal defense against bias.

Range

The largest value minus the smallest — the simplest measure of spread, and the most fragile, since it is determined entirely by the two most extreme observations.

Ratio level

The highest level of measurement: ordered, equally spaced, and possessing a true zero meaning “none.” Ratios such as “twice as much” are meaningful. Weight, income, and elapsed time are examples.

Sample

The subset of a population actually observed and measured.

Sampling error

The natural difference between a sample statistic and the population parameter, caused solely by observing a part rather than the whole. It shrinks as sample size grows.

Statistic

A numerical characteristic of a sample, such as x̄ or s. Computable from data, and different for every sample drawn.

Variable

Any characteristic that can differ from one individual to another; the recorded values of a variable are the data.

Sampling & Data Collection

Almost every number you will ever meet in a report, a news story, or a research paper came from a sample — a few people standing in for many. That fact should make you nervous until you understand why it works, and it works far better than most people expect: a well-chosen sample of a thousand can describe a nation of three hundred million to within a few percentage points. The catch is the word well-chosen. A sample earns its authority from how it was selected, never from how big it is, and the most spectacular statistical failures in history were enormous samples collected badly. This chapter shows you where good data comes from, how it goes wrong, and how to tell the difference between a study that can prove a cause and one that can only spot a pattern.

Section 1Why sampling works

Population, sample, parameter, statistic

The population is every individual you want to say something about — every registered voter, every bag of chips off the line, every patient with a certain diagnosis. The sample is the subset you actually measure. A number that describes the population is a parameter; a number computed from your sample is a statistic. Parameters are fixed and (almost always) unknown. Statistics are known and, because a different sample would have given different numbers, they wobble.

Keeping the two straight is the single most useful habit in this book, because the entire subject is one long answer to one question: how close is my statistic to the parameter I care about? Notation helps — Greek letters for the truth, Roman letters for the estimate.

A census measures the whole population, so it reports parameters directly. We sample instead of taking a census for four reasons: cost, time, inaccessibility (you cannot find every homeless person in a city), and destruction (testing whether a fuse blows destroys the fuse — a census of fuses would leave none to sell).

Why a random sample looks like its population

Representativeness is not something you can eyeball. A sample "looks representative" to a human only after the human has decided which features matter, and the features that matter are usually the ones nobody thought of. Random selection solves this without knowing anything about the population: if every individual has the same chance of being drawn, then every characteristic — age, income, opinion, blood type, the ones you measured and the ones you never imagined — is represented in the sample in roughly its population proportion, on average. Randomness does not guarantee a good sample every time. It guarantees that the misses are unbiased and, crucially, that their size is calculable.

That last point is what makes sampling a science rather than a hope. The wobble from sample to sample has a name, sampling variability, and a measurable size, the standard error. For a sample mean drawn from a population with standard deviation σ,

SE(x̄) = σ ÷ √n

That formula comes with conditions attached, and it is worth naming them now because every technique later in this book inherits them. It requires the observations to be independent — in practice, that the individuals were drawn at random and that the sample is no more than roughly 10% of the population, so that removing one unit barely changes the odds for the next. Violate either condition and the formula misreports the true wobble: sample a large slice of a small population and the real standard error is smaller than σ/√n; use a cluster design, where neighbors resemble neighbors, and it is larger. A standard error computed from a convenience sample is not conservative or approximate — it is meaningless, because the probabilities it is built from do not exist.

A worked example: how precision grows

Suppose commute times in a large city have a standard deviation of σ = 12 minutes, and we want to estimate the mean commute μ.

• n = 36: SE = 12 ÷ √36 = 12 ÷ 6 = 2.0 minutes
• n = 144: SE = 12 ÷ √144 = 12 ÷ 12 = 1.0 minute
• n = 576: SE = 12 ÷ √576 = 12 ÷ 24 = 0.5 minutes

Read the pattern: to cut the standard error in half you must quadruple the sample, because n sits under a square root. Going from 36 to 144 people (four times the work) buys you twice the precision, not four times. This is the law of diminishing returns that governs every survey budget ever written.

It also tells you how far off a typical sample lands — provided one more assumption holds: that the sampling distribution of x̄ is approximately normal. That is exact when the population itself is normal, and approximately true for a large enough n whatever the population shape, by the Central Limit Theorem. Granting it, if the true mean commute is μ = 24 minutes and n = 36, then roughly 95% of samples produce an x̄ within about 2 standard errors of the truth — that is, between 24 − 2(2.0) = 20 minutes and 24 + 2(2.0) = 28 minutes. Notice what is random here: the sample, not μ. The population mean sits still; our estimate is what moves. The "95%" describes how often the procedure lands close, across all the samples we might have drawn — it is never a probability statement about μ, which has no probability distribution at all.

Now look hard at that formula. N, the population size, does not appear in it. As long as the sample is a small fraction of the population, a sample of 1,000 is essentially as precise for a country of 300 million as it is for a town of 30,000. This is the most counter-intuitive true fact in introductory statistics, and it is why national polls of about 1,000 people are not absurd. What you need is a big enough spoonful, not a spoonful proportional to the size of the pot — provided the soup is stirred.

Section 2Sampling methods

The sampling frame comes first

Before you can select anybody you need a sampling frame: the actual list of units you will draw from. The frame is where good intentions quietly die. You may want "all adults in the county," but your frame is "all landline numbers in the county," and those are not the same population. Every method below is only as good as the frame it runs on, so write down your frame explicitly and ask who is missing from it.

The four probability methods

A probability sample is one in which every individual has a known, nonzero chance of selection. Only probability samples support the inference machinery in the rest of this book.

A simple random sample (SRS) of size n is one chosen so that every possible group of n individuals is equally likely to be the sample. In practice you number the frame 1 to N and let a computer draw n distinct numbers. Note the definition carefully: it is stronger than "everyone has an equal chance." Systematic sampling gives everyone an equal chance too, yet most groups of size n can never be selected by it, so it is not an SRS.

Stratified sampling splits the frame into strata — groups whose members resemble each other, such as class year, region, or age band — and then takes a separate random sample inside every stratum. When the strata really do differ from one another, this beats an SRS of the same size, because you have removed between-group variation from your estimate by force instead of leaving it to luck. It also guarantees you will not accidentally draw zero seniors.

Cluster sampling also divides the frame into groups, but the logic is reversed. Clusters are naturally occurring bundles — city blocks, clinics, classrooms — each of which ideally resembles the whole population in miniature. You randomly select whole clusters and then measure everyone inside the chosen ones. This is a cost method: visiting 30 blocks and interviewing every household is far cheaper than chasing 600 addresses scattered across a city. The price is precision. Neighbors resemble neighbors, so 600 people from 30 blocks carry less independent information than 600 people drawn at random.

Systematic sampling takes every k-th unit after a random start, with k = N/n. It is fast and needs no numbered list — you can do it at a factory exit or a store door. Its one real danger is periodicity: if the list repeats with a cycle that matches k, the sample locks onto one phase of the cycle. Sampling every 7th day of hospital admissions, for instance, means every observation falls on the same weekday.

A worked example: proportional allocation and a systematic plan

A student body of N = 12,000 breaks down as 4,200 first-years, 3,300 sophomores, 2,700 juniors, and 1,800 seniors (4,200 + 3,300 + 2,700 + 1,800 = 12,000 ✓). We want n = 400 students, allocated proportionally, so each stratum contributes n_h = n × (N_h / N). Since 400/12,000 = 1/30, we simply divide each stratum by 30:

• First-years: 4,200 ÷ 30 = 140
• Sophomores: 3,300 ÷ 30 = 110
• Juniors: 2,700 ÷ 30 = 90
• Seniors: 1,800 ÷ 30 = 60

Check the total: 140 + 110 + 90 + 60 = 400 ✓. Every student's selection probability is 140/4,200 = 1/30 for a first-year, 60/1,800 = 1/30 for a senior — identical, which is exactly the point of proportional allocation.

The same 400 by systematic sampling instead: k = 12,000 ÷ 400 = 30. Draw a random start between 1 and 30 — say 17 — then take students 17, 47, 77, 107, and so on, each one 30 further down the list. The j-th pick is 17 + 30(j − 1), so the 400th is 17 + 30(399) = 17 + 11,970 = 11,987, comfortably inside the list ✓. Fast, tidy, and perfectly fine here — unless the enrollment list happens to cycle every 30 names.

Convenience sampling — surveying the people already in front of you — and its cousin voluntary response sampling, in which people select themselves by calling in or clicking a link, are not probability methods at all. Nobody's chance of selection is known, so no standard error can be computed and no margin of error is meaningful. They are useful for generating hypotheses and for pilot-testing a questionnaire, and for nothing else. Real surveys often chain methods together into a multistage design: cluster on counties, then on blocks, then take an SRS of households, then randomly pick one adult inside each household.

Section 3Bias and error

Two completely different ways to be wrong

Every estimate misses the truth, but there are two unrelated reasons why, and confusing them is the most expensive mistake in applied statistics. Sampling error is the ordinary, unavoidable difference between a sample statistic and the population parameter that arises simply because you measured some individuals rather than all of them. It is random, it is centered on zero, and it shrinks predictably as n grows — that is the σ/√n from Section 1.

Bias is systematic: a tendency for your method to miss in the same direction sample after sample. Bias does not shrink with n. Collect ten times as much data with the same flawed method and you get the same wrong answer with a tighter, more persuasive error bar around it. Think of it as target shooting: random error is a wide shot group, bias is a tight shot group centered off the bullseye. More ammunition tightens the group; only fixing the sights moves it.

The named biases, and where they come from

Undercoverage occurs when part of the population cannot appear in the frame at all — a landline-only poll in a cell-phone world, or a mall survey that misses everyone who does not shop at malls. Voluntary response bias arises because people who take the trouble to respond hold stronger opinions, usually negative, than those who do not. Nonresponse bias hits after selection: your sample was chosen properly, but a large chunk did not answer, and the ones who answered differ from the ones who did not. Response bias is about the measurement itself — question wording ("Do you support wasteful spending on X?"), social desirability (people under-report drinking and over-report voting and flossing), interviewer effects, and faulty recall.

The last five rows are nonsampling error — every source of error that has nothing to do with which individuals you picked. Nonsampling error is the reason a census can still be wrong: the U.S. census reaches for everybody and still misses people and miscodes answers. In most real surveys, nonsampling error is larger than sampling error, and it is the part nobody reports.

A worked example: bias dwarfs the margin of error

A researcher randomly selects 1,000 adults and asks whether they exercise at least three times a week. Only 400 respond, and 260 of those say yes:

p̂ = 260 ÷ 400 = 0.65, or 65%

The reported 95% margin of error for a proportion is about 1.96 × √(p̂(1 − p̂) / n). Before using it, check what it assumes: a random sample of independent individuals, and enough of both outcomes for the normal approximation to be trustworthy — the usual rule of thumb is at least 10 successes and 10 failures. Here we have 260 yes and 140 no ✓, so the arithmetic is legitimate. Watch what happens next anyway. Step by step:

• p̂(1 − p̂) = 0.65 × 0.35 = 0.2275
• 0.2275 ÷ 400 = 0.00056875
• √0.00056875 ≈ 0.02385
• 1.96 × 0.02385 ≈ 0.0467 → about ±4.7 percentage points

Now suppose a determined follow-up reaches the 600 nonrespondents and finds that only 25% of them exercise three times a week — 0.25 × 600 = 150 people. The truth across all 1,000 selected adults is

(260 + 150) ÷ 1,000 = 410 ÷ 1,000 = 0.41, or 41%

The estimate was off by 0.65 − 0.41 = 0.24, twenty-four percentage points, while the advertised uncertainty was under five. The bias is five times the entire margin of error. And here is the sting: repeat the study with 10,000 selected adults at the same 40% response rate and the estimate stays near 65% while the margin of error falls to about ±1.5 points. You will have bought yourself a far more precise version of the same wrong number.

Section 4Observational studies versus experiments

One question separates them

The dividing line is embarrassingly simple: who decided who got what? In an observational study the researcher measures individuals as they already are, recording an exposure that the individuals (or life) chose for them. In an experiment the researcher assigns the explanatory variable — deciding, by a chance mechanism, which subjects receive which treatment — and then compares the outcomes.

Observational studies come in familiar shapes. A cross-sectional study takes a snapshot at one point in time. A prospective cohort study identifies people now and follows them forward, waiting for outcomes to appear. A retrospective case-control study starts with people who already have the outcome, matches them to people who do not, and looks backward for differences in exposure. All three observe; none assigns.

Why observation cannot prove cause

A confounding variable (also called a lurking variable) is a third factor associated with the explanatory variable and related to the response, so that its effect cannot be untangled from the effect you care about. Since people sort themselves into the exposed and unexposed groups, and they sort by all sorts of hidden reasons, observational groups differ in a thousand ways besides the exposure.

Watch how badly this can go, with exact numbers. Take 1,000 coffee drinkers, of whom 600 smoke, and 1,000 non-coffee-drinkers, of whom 200 smoke. Let lung cancer strike 10% of smokers and 1% of nonsmokers — and let coffee itself do absolutely nothing.

• Coffee drinkers: (600 × 0.10) + (400 × 0.01) = 60 + 4 = 64 cases → 64/1,000 = 6.4%
• Non-coffee-drinkers: (200 × 0.10) + (800 × 0.01) = 20 + 8 = 28 cases → 28/1,000 = 2.8%

Coffee drinkers get lung cancer at 6.4% versus 2.8% — a rate ratio of 6.4 ÷ 2.8 ≈ 2.3, so a headline could truthfully announce that coffee drinkers are more than twice as likely to develop lung cancer. Every number above is real and correctly computed; the causal story is pure fiction manufactured by smoking. That is confounding, and no amount of extra data removes it, because more data means more of the same tangle.

So when you see an association between X and Y, five explanations compete: X causes Y; Y causes X (reverse causation — do sneakers cause fitness, or do the fit buy sneakers?); a confounder causes both; the association is a fluke of sampling; or the study was biased. Observation alone cannot referee among them.

What each design licenses you to say

Two different randomizations do two different jobs, and students mix them up constantly. Random sampling — how subjects got into the study — buys generalization to the population. Random assignment — how subjects got sorted once inside — buys causal conclusions. They are independent choices, so there are four cases.

Observational studies are not second-class citizens; they are often the only ethical or feasible option. You cannot randomly assign people to smoke for thirty years. What tips the balance toward causation without an experiment is an accumulated pattern — a large effect, a dose-response gradient, consistency across many populations and designs, correct time order, and a plausible mechanism — which is exactly how the case against tobacco was built.

Section 5Designing an experiment

The vocabulary, then the four principles

The individuals in an experiment are experimental units (called subjects when they are people). A factor is an explanatory variable you control; its settings are levels; a treatment is one combination of levels. The response is what you measure. A study of two drug doses at two delivery times has two factors, each at two levels, and therefore 2 × 2 = 4 treatments.

A good experiment rests on four ideas. Control means holding other conditions constant across groups and, above all, including a control group — a comparison arm receiving no treatment, a placebo, or the current standard of care. Without a comparison group you cannot separate the treatment's effect from the passage of time, from the illness resolving on its own, or from patients regressing toward their usual state. Randomization assigns units to treatments by chance. Replication means enough units per treatment that ordinary variation averages out — not merely repeating one measurement. Blocking groups similar units together and randomizes within each block, the experimental cousin of stratified sampling.

Randomization deserves its own sentence, because it is the whole reason experiments can prove things. It is not there to be fair. It is there to make the treatment groups equivalent, on average, in every respect — age, severity, genetics, diet, optimism, and the hundred variables nobody thought to record — so that when the outcomes differ, the only systematic difference left standing is the treatment. Matching on known variables cannot do this, because you can only match on what you thought of.

Placebo and blinding

A placebo is an inert imitation of the treatment — a sugar pill, a saline injection, a sham procedure. It exists because the placebo effect is real and often large: people improve because they believe they are being treated. If the control group gets nothing at all, the treatment group's improvement mixes the drug's effect with the belief's effect, and you cannot pull them apart. A placebo control makes the belief equal in both arms so the difference isolates the drug.

Blinding keeps people from knowing who got what. In a single-blind study the subjects do not know; in a double-blind study neither the subjects nor the people who interact with them and assess the outcome know. Double-blinding matters because expectation leaks: an enthusiastic assessor who knows a patient got the new drug rates that patient's "moderate" improvement as "marked." Whoever measures the response should be blind, always.

A worked example: building the design

We want to compare a new headache drug against the standard drug and against a placebo, with 180 volunteers.

Completely randomized design. Number the volunteers 1 to 180 and use a computer to shuffle them into three arms of 180 ÷ 3 = 60 each. Every subject gets an identical-looking capsule, so nobody — subject, nurse, or assessor — can tell the arms apart: double-blind. Measure minutes to meaningful relief. Suppose the group means come back as placebo 42 minutes, standard drug 30, new drug 26. The comparisons are 42 − 30 = 12 minutes faster for the standard drug than placebo, and 30 − 26 = 4 minutes faster for the new drug than the standard. Because assignment was random, no pre-existing difference between the groups can explain those gaps in any systematic way; the remaining question is whether a gap this size could plausibly arise from chance alone, which is what the inference tools later in this book are built to answer.

Now block it. Suppose baseline headache severity strongly affects recovery time. Split the 180 into blocks first — say 108 with moderate headaches and 72 with severe ones — and randomize within each block: 108 ÷ 3 = 36 moderate per arm and 72 ÷ 3 = 24 severe per arm, and 36 + 24 = 60 per arm as before, 180 in total ✓. Every arm now has an identical severity mix by construction rather than by luck, so severity can no longer muddy the comparison, and the leftover variation the treatment effect must stand out from is smaller.

Two design cautions worth carrying forward. Keep lurking differences out of the procedure: if the new drug is given in a quiet private room and the placebo in a crowded hallway, the room is confounded with the drug. And beware attrition — subjects who drop out are usually the ones doing worst, so dropping them from the analysis quietly re-creates the very selection bias randomization was meant to destroy.

ReferenceWords worth knowing

A quick reference for the terms in this chapter — skim it now, or circle back whenever a word trips you up.

Bias

A systematic tendency for a method to miss the truth in the same direction; unlike random error, it does not shrink as the sample grows.

Blinding

Withholding treatment assignment from participants (single-blind) or from participants and the researchers assessing them (double-blind).

Blocking

Grouping similar experimental units together and randomizing treatments within each group, so a known nuisance variable cannot distort the comparison.

Census

An attempt to measure every member of a population rather than a sample; still vulnerable to nonsampling error.

Cluster sample

A sample obtained by randomly choosing whole naturally occurring groups and measuring everyone inside them.

Confounding variable

A third variable associated with the explanatory variable and related to the response, so that their effects cannot be separated.

Control group

The comparison arm of an experiment, receiving no treatment, a placebo, or the current standard of care.

Convenience sample

A sample of whoever is easiest to reach; selection probabilities are unknown, so no valid inference is possible.

Experiment

A study in which the researcher assigns the explanatory variable to units, ideally at random, and then compares responses.

Margin of error

The half-width of an interval estimate, quantifying chance variation from sampling only — never bias or nonsampling error.

Nonresponse bias

Distortion arising when selected individuals do not participate and differ systematically from those who do.

Nonsampling error

Any error unrelated to which individuals were chosen — coverage gaps, nonresponse, inaccurate answers, measurement and processing mistakes.

Observational study

A study that records an exposure the subjects already have rather than assigning it; establishes association, not causation.

Parameter

A fixed numerical summary of a population, such as μ, σ, or p; almost always unknown.

Placebo

An inert imitation of a treatment, used so that the control group's belief in being treated matches the treatment group's.

Population

Every individual about whom you want to draw a conclusion.

Random assignment

Using chance to allocate subjects to treatments; it balances known and unknown variables and is what permits causal conclusions.

Replication

Assigning enough units to each treatment that ordinary variation averages out; not the same as repeating a single measurement.

Response bias

Distortion in the answers themselves — from question wording, social desirability, interviewer effects, or faulty recall.

Sample

The subset of the population actually measured.

Sampling error

The random difference between a sample statistic and the population parameter caused by measuring only part of the population.

Sampling frame

The concrete list of units from which a sample is actually drawn; anyone absent from it cannot be selected.

Simple random sample (SRS)

A sample chosen so that every possible group of n individuals is equally likely to be selected.

Standard error

The typical sample-to-sample wobble of a statistic; for a sample mean it is σ/√n, assuming independent observations.

Statistic

A numerical summary computed from a sample, such as x̄, s, or p̂; it varies from sample to sample.

Stratified sample

A sample formed by dividing the population into similar groups (strata) and drawing a random sample from within every stratum.

Systematic sample

A sample taken by choosing every k-th unit after a random start, with k = N/n; vulnerable to periodicity in the list.

Undercoverage

A coverage gap in which part of the population cannot appear in the sampling frame at all, and so can never be selected.

Voluntary response bias

Distortion arising when people select themselves into a sample, over-representing those with strong opinions.

Displaying Data

A column of two hundred numbers tells you almost nothing; the same two hundred numbers drawn as a picture tell you almost everything in about four seconds. That is the whole promise of this chapter. You will learn to organize raw data into a frequency table, to choose the right picture for the kind of variable you have — bars and pies for categories, histograms and dot plots for measurements — and to read the shape that picture reveals. You will also learn the other half of the skill, the defensive half: how a perfectly honest set of numbers can be drawn in a way that lies, and how to spot it. No formula in this chapter is harder than dividing one number by another, so if arithmetic makes you nervous, this is a friendly place to start.

Section 1Frequency distributions

From a pile of numbers to a table

Raw data arrive as a list. A frequency distribution is simply a table that says, for each possible value (or each range of values), how many times it occurred. That count is the frequency, written f. The sum of all the frequencies is the sample size, n, so the very first thing you do after building a frequency table is add the column up and confirm it equals n. If it does not, you miscounted, and every number downstream is wrong.

Two more columns turn a bare count into something you can actually reason with. The relative frequency is the fraction of the data in that row:

relative frequency = f ÷ n

Relative frequencies always sum to 1 (or to 100% if you express them as percentages), give or take a tiny rounding wobble. They matter because they let you compare groups of different sizes: 12 out of 40 and 30 out of 100 are the same story (0.300 either way), but the raw counts 12 and 30 are not obviously comparable at all.

The cumulative frequency is a running total — the number of observations at or below that row. It answers “how many were at most this much?” without any further arithmetic. Its final entry must equal n, which is a second free error check.

A worked example, start to finish

A survey asked n = 40 adults, “On how many of the past seven days did you exercise?” The answers are whole numbers from 0 to 7, so each value gets its own row. Here is the complete table, with every column computed.

Check the two totals: 6 + 8 + 10 + 7 + 5 + 2 + 1 + 1 = 40 ✓, and 0.150 + 0.200 + 0.250 + 0.175 + 0.125 + 0.050 + 0.025 + 0.025 = 1.000 ✓. Now read answers straight off the table. What proportion exercised at most 3 days? The cumulative relative entry in the “3” row: 0.775, or 77.5%. What proportion exercised more than 3 days? Take the complement: 1 − 0.775 = 0.225. Verify it the long way — 5 + 2 + 1 + 1 = 9 people, and 9 ÷ 40 = 0.225 ✓. The two routes agree, which is exactly what a well-built table should guarantee.

Grouped distributions: when values do not repeat

The table above worked because there were only eight distinct values. Measurements like commute times, weights, or incomes rarely repeat, so a row-per-value table would be as long as the data set and just as useless. The fix is a grouped frequency distribution: sort the values into a handful of non-overlapping classes and count how many fall into each. Five to ten classes is the usual sweet spot — too few and you flatten the pattern into a featureless block, too many and you shatter it into noise.

To find the class width, take the range and divide by the number of classes you want, then always round up to a convenient number:

class width ≈ (maximum − minimum) ÷ (number of classes), rounded up

Rounding up is not a suggestion. If you round down, the classes collectively span less than the data does and the largest value has nowhere to go.

Worked example. Thirty commute times, in minutes, sorted: 5, 7, 8, 11, 12, 12, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 28, 29, 31, 33, 35, 38, 40, 42, 47, 52, 58. The minimum is 5 and the maximum is 58, so the range is 58 − 5 = 53. Aiming for 6 classes: 53 ÷ 6 = 8.83, which rounds up to a width of 9. Starting at the minimum, 5, the classes are 5–13, 14–22, 23–31, 32–40, 41–49, 50–58. The midpoint (or class mark) of a class is the average of its two limits.

The cumulative column ends at 30 = n ✓. Reading it: 22 of the 30 commuters — that is 22 ÷ 30 ≈ 0.733, or about 73% — travel 31 minutes or less. Plotting cumulative frequency against the upper class limit and connecting the points gives the rising curve called an ogive, the figure at the top of this section.

Section 2Displaying qualitative data

Bar charts: the workhorse

Qualitative (categorical) data record which group something belongs to — blood type, major, mode of transport, brand preference. There is no arithmetic to do on the categories themselves; the only number in the room is how many landed in each. A bar chart displays exactly that: one bar per category, with bar height proportional to frequency or relative frequency.

Three rules make a bar chart honest. First, the bars are separated by gaps, because the horizontal axis is a list of names, not a measuring scale — there is no space “between” Bus and Bicycle to fill in. Second, all bars are the same width, so that height is the only thing carrying information. Third, the frequency axis starts at zero; we will see in Section 5 exactly how much damage is done when it does not.

Because categories have no natural order, you are free to sort them. Sorting the bars from tallest to shortest produces a Pareto chart, which makes the dominant categories jump out and is the default choice whenever the question is “what are the biggest contributors?”

Pie charts: parts of one whole

A pie chart shows the same information as proportions of a single circle. Each category gets a wedge whose central angle is its relative frequency times the full 360 degrees:

slice angle = (f ÷ n) × 360°

Worked example. Two hundred commuters (n = 200) named their primary mode of transport. Every column below is computed from the count.

Every check passes: the counts sum to 200 = n, the relative frequencies sum to 1.00, and the angles sum to 151.2 + 82.8 + 54.0 + 50.4 + 21.6 = 360.0° ✓. Notice the rows of this table are already in descending order of frequency, so charting them as listed gives you a Pareto chart for free.

When a pie chart is the wrong tool

A pie chart carries one strict requirement that people break constantly: the categories must be mutually exclusive and exhaustive — every observation in exactly one slice, all slices together making one complete whole. Data from a “check all that apply” question fail this immediately. If 200 people can each name two hobbies, the counts sum to more than 200, the percentages sum past 100%, and a pie chart of them is not merely ugly, it is a false statement about the data. Use a bar chart instead.

Even when a pie is legitimate, it is often the weaker display. Human eyes compare lengths far more accurately than angles or wedge areas, so a 14% slice and a 15% slice are effectively indistinguishable while the corresponding bars are not. Pies also collapse under more than about five or six categories, and comparing two pies side by side is close to hopeless. A reasonable habit: reach for a pie only when you have a few categories and your point is genuinely “this piece is roughly half of everything”; otherwise, reach for bars.

Section 3Displaying quantitative data

Histograms

A histogram is the picture of a grouped frequency distribution: the horizontal axis is the measured variable on a real numeric scale, each class becomes a bar, and bar height is the frequency (or relative frequency) in that class. Because the horizontal axis is continuous, the bars touch. That single visual detail is the whole difference between a histogram and a bar chart, and it encodes a real difference in the data: in a bar chart the gap says “these are separate categories,” while in a histogram the shared edge says “this class ends exactly where the next one begins.”

Worked example. Twenty-five resting pulse rates, in beats per minute: 58, 61, 62, 64, 65, 67, 68, 68, 70, 71, 72, 72, 73, 74, 75, 76, 77, 78, 80, 81, 83, 85, 88, 91, 96. Using classes of width 10:

Both checks pass: 1 + 7 + 10 + 5 + 2 = 25 = n ✓ and 0.04 + 0.28 + 0.40 + 0.20 + 0.08 = 1.00 ✓. The class boundaries in the middle column are the halfway points between one class’s upper limit and the next class’s lower limit; they are what you actually draw the bars against, which is how the bars come to touch without overlapping. The tallest bar sits over 70–79, holding 40% of the data, and the bars fall away more gradually on the right than on the left: the far-left bar holds 1 observation while the far-right bar holds 2, and the maximum (96) sits 17 beats above the peak class while the minimum (58) sits only 12 beats below it. That stretched right-hand side is mild right skew, and the arithmetic agrees — the mean of these 25 rates is 1855 ÷ 25 = 74.2 while the median (the 13th sorted value) is 73.0. Mean above median, tail to the right: a hint about shape that we develop in the next section.

One warning about histograms: the picture depends on choices you made. Change the number of classes and the same data can look smooth, spiky, or bimodal. That is not a flaw to be hidden but a reason to try two or three bin widths before you believe any feature you see.

Stem-and-leaf plots

A stem-and-leaf plot (or stemplot) gives you a histogram’s shape and keeps every original number. Split each value into a stem (the leading digits) and a leaf (the final digit), list the stems in a column, and write each leaf beside its stem in increasing order. For the pulse data, the stem is the tens digit:

Turn that plot ninety degrees counter-clockwise and you are looking at the histogram from the previous table — identical counts, 1, 7, 10, 5, 2. The advantage is that nothing was thrown away: you can still read off the minimum (58), the maximum (96), and any repeated value (72 appears twice). Every stemplot needs a key, because “6 | 4” could just as easily mean 6.4 in another data set. Stemplots are wonderful for roughly 15 to 60 values and become unmanageable beyond that.

Dot plots

A dot plot is the simplest display of all: draw a number line and stack one dot above each value for every time it occurs. For the exercise-days data from Section 1, the column above 0 is 6 dots tall, above 1 is 8 dots, above 2 is 10 dots, and so on down to a single dot above 6 and above 7. With n = 40 the whole distribution fits on one line and, like a stemplot, no information is lost.

Dot plots shine for small data sets, for discrete counts, and for spotting outliers — a lone dot far out on the line is impossible to miss, whereas a histogram might hide it inside a wide bin. They become an unreadable smear past roughly 50 observations, at which point a histogram is the better instrument.

Section 4The shape of a distribution

Symmetric, skewed right, skewed left

Once a display exists, describe it in three words: shape, center, and spread. Chapter 4 handles center and spread; shape is this section. A distribution is symmetric when you could fold the picture down the middle and the two halves would roughly match — the bell-shaped curve is the famous case, but a flat distribution is symmetric too.

When one side stretches further than the other, the distribution is skewed, and here is the naming rule that trips up nearly everyone: a distribution is named for the direction of its long tail, not for where the pile of data sits. A skewed right (positively skewed) distribution has most of its data bunched on the left with a thin tail trailing off to the right — incomes, house prices, and hospital lengths of stay all look like this. A skewed left (negatively skewed) distribution is the mirror image: a bunch on the right, a tail dragging to the left — scores on an easy exam, or age at death in a wealthy country.

Worked example: right skew. Ten annual salaries at a small firm, in thousands of dollars: 32, 34, 35, 36, 38, 40, 42, 44, 46, 253 (the last is the owner). The sum is 32 + 34 + 35 + 36 + 38 + 40 + 42 + 44 + 46 + 253 = 600, so the mean is

x̄ = Σx ÷ n = 600 ÷ 10 = 60.0 thousand

With n = 10 the median is the average of the 5th and 6th sorted values: (38 + 40) ÷ 2 = 39.0 thousand. The mean sits 21 thousand above the median because that one enormous value pulls it there, and nine of the ten employees earn less than the “average” salary. The long right tail did that.

Worked example: left skew. Ten scores on a straightforward exam: 45, 62, 78, 85, 88, 90, 91, 93, 94, 96. The sum is 822, so x̄ = 822 ÷ 10 = 82.2, while the median is (88 + 90) ÷ 2 = 89.0. Now the mean falls below the median, dragged down by the low outlier at 45 — the signature of a left tail.

Treat that third column as a reliable rule of thumb, not a theorem. For most real, single-peaked data it holds, and it is genuinely useful: if someone reports a mean far above the median, you should picture a right tail before you see the graph. But mathematicians have constructed skewed distributions where the ordering reverses, so confirm with the picture rather than trusting the rule blindly.

Uniform distributions and modality

A uniform distribution has roughly the same frequency in every class, so the histogram looks like a flat plateau. Rolling a fair die a few thousand times produces one; so does the last digit of a well-mixed set of ID numbers. Uniformity is worth naming because it is a strong claim: it says no value in the range is any more common than any other.

Modality counts the peaks. A unimodal distribution has one clear peak, a bimodal distribution has two, and anything with three or more is multimodal. Modality is the single most informative feature you can read off a histogram, because two peaks usually mean two populations have been mixed together. Waiting times at a clinic that runs both walk-in and scheduled visits will often be bimodal; so will heights in a group of mixed adults and children. When you see two peaks, do not summarize — split the data and describe each group on its own. A mean computed across a bimodal distribution can land squarely in the valley between the peaks, describing a value that almost nobody actually has.

Section 5Misleading graphs

The truncated axis

Every technique in this chapter can be turned against the reader, and the most common weapon by far is the truncated axis — a frequency or value axis that starts somewhere above zero. On a bar chart this is close to fatal, because a bar communicates by its length, and length is only meaningful when measured from zero. Cut the bottom off and the visible bar no longer represents the quantity at all; it represents the quantity minus whatever arbitrary number you chose to hide.

Worked example. A company’s quarterly sales, in millions of dollars, are Q1 = 98, Q2 = 100, Q3 = 102, Q4 = 104. The real change from Q1 to Q4 is

(104 − 98) ÷ 98 = 6 ÷ 98 = 0.0612, or about 6.1%

Drawn against an axis running from 0 to 110, the Q4 bar is 104 ÷ 98 = 1.061 times the Q1 bar — visibly a bit taller, honestly modest. Now truncate the axis so it starts at 97. The visible bar heights become 98 − 97 = 1, 100 − 97 = 3, 102 − 97 = 5, and 104 − 97 = 7 units. The Q4 bar is now seven times the height of the Q1 bar, an apparent increase of (7 − 1) ÷ 1 = 6.00, or 600%. Nothing in the data changed. One number on one axis changed, and a 6% drift across the whole year became an explosion.

Edward Tufte gave this exaggeration a name and a formula. The lie factor is the size of the effect shown in the picture divided by the size of the effect in the data:

lie factor = (apparent effect) ÷ (actual effect) = 6.00 ÷ 0.0612 ≈ 98

A truthful graph has a lie factor near 1. This one overstates the growth by roughly a hundredfold. So: bar charts and histograms must begin at zero, without exception. Line graphs of a quantity that never approaches zero — body temperature, or a stock index — may legitimately start elsewhere, because a line communicates through its slope rather than its length. Even then, the axis must be clearly labeled and the break made obvious, and you should still ask whether the slope you are being shown is dramatic or merely zoomed.

Distorted scales, distorted areas

The second family of tricks distorts the scale itself rather than its starting point. A few worth recognizing:

• Area and volume inflation. A pictogram doubles a quantity by doubling both the width and the height of an icon. But doubling both dimensions multiplies the area by 2 × 2 = 4, and a drawing that also looks three-dimensional implies a volume 2³ = 8 times larger. The data said 2; the reader perceives 4 or 8. Icons should be repeated, not resized.
• Unequal class widths. A histogram whose classes are 0–9, 10–19, and then 20–59 puts four classes’ worth of data into one bar. That bar looks enormous simply because it is wide. Keep class widths equal, or plot density (frequency ÷ class width) instead of raw frequency.
• Uneven axis spacing. A time axis that shows 1990, 2000, 2010, 2015, 2018, 2020 at equal intervals compresses early decades and stretches recent years, manufacturing a hockey-stick curve out of steady growth.
• Dual vertical axes. Two series on two different scales can be made to cross, converge, or diverge at will by rescaling one axis. Any “relationship” you see is a decision the chartmaker made.
• Logarithmic scales. These are entirely legitimate — they are the right tool for data spanning several orders of magnitude — but on a log scale a straight line means constant percentage growth, not constant growth. An unlabeled log axis will make explosive growth look tame.

ReferenceWords worth knowing

A quick reference for the terms in this chapter — skim it now, or circle back whenever a word trips you up.

Bar chart

A display of qualitative data using equal-width bars separated by gaps, with bar height proportional to frequency or relative frequency.

Bimodal

Having two distinct peaks; usually a sign that two different populations have been mixed into one data set.

Class width

The size of each interval in a grouped frequency distribution; approximately the range divided by the number of classes, always rounded up.

Cumulative frequency

A running total of frequencies — the number of observations at or below a given value or class. Its last entry equals n.

Dot plot

A display in which one dot is stacked above a number line for each observation at that value.

Frequency (f)

The number of times a particular value or class occurs in a data set.

Frequency distribution

A table pairing each value or class with the number of observations that fall in it.

Grouped frequency distribution

A frequency table built from ranges (classes) rather than individual values; used when values rarely repeat.

Histogram

A display of quantitative data in which classes sit on a continuous axis and the bars therefore touch at the class boundaries.

Lie factor

The size of the effect shown in a graphic divided by the size of the effect in the data. An honest graph has a lie factor near 1.

Ogive

A line graph of cumulative frequency plotted against the upper class limits; it rises from 0 to n.

Outlier

An observation lying far from the bulk of the data; often obvious on a dot plot or stemplot and easily hidden by wide histogram bins.

Pareto chart

A bar chart with the bars arranged from tallest to shortest, emphasizing the largest categories.

Pie chart

A circle divided into wedges whose central angles are proportional to relative frequency; valid only for mutually exclusive, exhaustive categories.

Relative frequency

The proportion of the data in a class, f ÷ n; the column sums to 1, or 100%.

Skewed left (negatively skewed)

A distribution whose long tail extends toward the low values; the mean is typically less than the median.

Skewed right (positively skewed)

A distribution whose long tail extends toward the high values; the mean is typically greater than the median.

Stem-and-leaf plot

A display splitting each value into a leading stem and a trailing leaf, preserving every original observation. Requires a key.

Symmetric distribution

A distribution whose halves are approximate mirror images about its center.

Truncated axis

A value axis that begins above zero, exaggerating apparent differences between bars.

Uniform distribution

A distribution in which every class has roughly the same frequency, producing a flat display.

Unimodal

Having a single clear peak.

Center & Spread

A pile of numbers tells you nothing until you can answer two questions about it: where is the middle? and how spread out is everything else? Those two questions — center and spread — are the entire business of this chapter, and together they are the most useful pair of ideas in all of descriptive statistics. You will learn three ways to describe the middle and four ways to describe the spread, and you will learn when each one tells the truth and when it quietly lies. None of this requires talent for math; it requires only that you add, subtract, divide, and take a square root, one careful step at a time. Every formula here is worked out numerically at least once, so you can follow the arithmetic with your finger if you want to.

Section 1Measures of center

Three different ideas of "middle"

Ordinary English has one word for the middle of a group of numbers. Statistics has three, because there are three genuinely different things you might mean. The mean is the balance point: add everything up and share it out equally. The median is the positional middle: line the values up smallest to largest and take the one standing in the center. The mode is the most popular value: whichever number shows up most often. On nicely symmetric data all three land in nearly the same place, which is why it is easy to think they are interchangeable. They are not.

Start with a small, honest data set. Seven people report the number of hours they spent on a task:

4, 8, 9, 9, 12, 13, 15

The mean — written x̄ ("x-bar") for a sample and μ ("mu") for a population — is the sum divided by the count:

x̄ = Σx / n

Here Σx = 4 + 8 + 9 + 9 + 12 + 13 + 15 = 70, and n = 7, so x̄ = 70 / 7 = 10 hours. The capital Greek sigma, Σ, is just an instruction: "add up everything that follows."

The median needs the values sorted, which they already are. With n = 7 (odd), the middle position is the (7 + 1) / 2 = 4th value. Counting in — 4, 8, 9, 9, 12, 13, 15 — the median is 9 hours. When n is even there is no single middle value, so you average the two middle ones. For 3, 7, 8, 12, 13, 14 the two middle values are 8 and 12, and the median is (8 + 12) / 2 = 10.

The mode is the value that occurs most often. Only 9 appears twice, so the mode is 9 hours. A data set can have one mode, two (bimodal), several (multimodal), or none at all if every value appears exactly once.

Why the choice matters: the outlier problem

Here is where the three measures come apart. Suppose a six-person company reports annual pay, in thousands of dollars:

38, 42, 45, 48, 52, 275

The mean is (38 + 42 + 45 + 48 + 52 + 275) / 6 = 500 / 6 = 83.3 thousand. The median, averaging the two middle values, is (45 + 48) / 2 = 46.5 thousand. Look at what just happened: the "average" salary at this company is $83,300, and five of the six employees earn less than that. One extreme value dragged the mean nearly forty thousand dollars away from anything a typical employee experiences. The median did not budge.

That difference has a name. A statistic is resistant (or robust) if extreme values barely move it. The median and the mode are resistant; the mean is not. The mean is not "wrong" here — it is a perfectly correct balance point, and if you want the total payroll, the mean times the head count gives it to you exactly. It is simply answering a different question than the one most people are asking when they say "typical."

The relationship between mean and median is also a quick shape detector. In a right-skewed (positively skewed) distribution — a long tail stretching toward high values, like income or hospital length of stay — the mean is pulled above the median. In a left-skewed distribution the mean falls below the median. When mean and median are close, the distribution is roughly symmetric.

Picking the right one

The choice is not a matter of taste; it follows from the data's shape and its measurement level. You cannot average eye color, and you should not average a data set whose whole story is in its tail.

One more practical point: reported "averages" in the news are almost always means, and almost always applied to skewed quantities — income, home prices, wait times, medical costs. Whenever you see the word average attached to money or time, the honest follow-up question is "and what was the median?"

Section 2Measures of spread

Why the center is only half the story

Consider two groups of five measurements:

Set A: 48, 49, 50, 51, 52
Set B: 30, 40, 50, 60, 70

Both sum to 250, so both have a mean of exactly 50. Report only the mean and the two sets are identical twins. Look at them and they are nothing alike: Set A is tightly packed, Set B is scattered. Spread (also called dispersion or variability) is the measurement that separates them, and in practice it is often the more important of the two. A process that averages the right answer but swings wildly is usually a worse process than one that is slightly off but consistent.

The crudest measure of spread is the range:

Range = maximum − minimum

For Set A the range is 52 − 48 = 4; for Set B it is 70 − 30 = 40. That already separates them, and the range is instantly understandable, which is why it is so widely quoted. Its weakness is fatal for serious work: it uses only two numbers and throws away everything in between, and both of those numbers are the most extreme values in the data set — exactly the ones most likely to be flukes. Add one typo to your spreadsheet and the range can change by an order of magnitude.

Variance and standard deviation, step by step

A better measure uses every value. The natural idea is to measure how far each value sits from the mean — its deviation, (x − x̄) — and then average those distances. There is one snag: the deviations always sum to exactly zero, because the mean is the balance point. The positives and negatives cancel perfectly, every time. The fix is to square each deviation first, which makes everything positive and, as a bonus, penalizes large departures more heavily than small ones.

Squaring gives the variance. There are two versions, and the difference is not cosmetic:

Population: σ² = Σ(x − μ)² / N
Sample: s² = Σ(x − x̄)² / (n − 1)

The numerator Σ(x − x̄)² is called the sum of squares, abbreviated SS. Because variance is in squared units — squared dollars, squared minutes, which nobody can picture — we usually take the square root and report the standard deviation, which returns to the original units:

σ = √(σ²)     s = √(s²)

Read the standard deviation as roughly how far a typical value sits from the mean. That is not its literal definition (it is the root-mean-square deviation, not the average absolute deviation), but it is the right mental picture and it will not mislead you.

Now the arithmetic, worked in full for Set A treated as a sample. First the mean: x̄ = 250 / 5 = 50.

Divide the sum of squares by n − 1 = 4:

s² = 10 / 4 = 2.5     s = √2.5 ≈ 1.58

Run the same machinery on Set B. The deviations are −20, −10, 0, +10, +20; their squares are 400, 100, 0, 100, 400; SS = 1000. So s² = 1000 / 4 = 250 and s = √250 ≈ 15.81. Every deviation in Set B is exactly ten times the matching deviation in Set A, so the standard deviation is exactly ten times as large — a useful sanity check, and a reminder that the standard deviation scales the same way the data do.

Had we treated Set A as an entire population, we would have divided SS by N = 5 instead: σ² = 10 / 5 = 2 and σ = √2 ≈ 1.41. Notice that the sample version is always the larger of the two.

Why n − 1, and how to compare spreads fairly

The n − 1 in the sample formula is called Bessel's correction, and the reason for it is worth understanding rather than memorizing. When you compute deviations from the sample mean, you are measuring distances from a center that was itself calculated from those same data points — the sample mean sits right in the middle of your particular sample by construction. Distances from it are therefore systematically a little too small compared with distances from the true population mean μ, which you do not know. Dividing by the smaller number n − 1 inflates the result just enough to correct that bias: s² computed this way is an unbiased estimator of σ², meaning that over many repeated samples it averages out to the right answer. The quantity n − 1 is also called the degrees of freedom: once you know the mean and any four of five values, the fifth is forced, so only four of the deviations are free to vary.

Finally, a standard deviation only means something next to its own mean. An s of 15 is enormous for adult body temperature and trivial for annual income. To compare variability across different scales or different units, use the coefficient of variation, which expresses the standard deviation as a percentage of the mean:

CV = (s / x̄) × 100%

For Set A, CV = (1.58 / 50) × 100% ≈ 3.2%. For Set B, CV = (15.81 / 50) × 100% ≈ 31.6%. Set B is about ten times as variable, relative to its own center. The CV requires a ratio-level variable with a meaningful zero and a positive mean; it is nonsense for a scale like Celsius temperature, where zero is arbitrary.

Section 3The five-number summary

Quartiles and the summary itself

The median splits a sorted data set into two halves. Split each half again and you get the quartiles: three cut points that carve the data into four equal-sized quarters. Q1, the first quartile, has 25% of the data below it; Q2 is the median with 50% below; Q3, the third quartile, has 75% below it. Quartiles are a special case of percentiles — the pth percentile is the value below which roughly p% of the data fall, so Q1 is the 25th percentile and Q3 is the 75th.

Put the two extremes on either end and you have the five-number summary: minimum, Q1, median, Q3, maximum. Five numbers, and you know the center, the spread, the shape, and the reach of the data. It is the most information-per-character description in statistics.

Work an example. Twelve response times, in minutes, already sorted:

3, 7, 8, 12, 13, 14, 18, 21, 22, 26, 31, 58

Step 1 — median. With n = 12 (even), average the 6th and 7th values: (14 + 18) / 2 = 16.

Step 2 — Q1. Take the lower half, the six values below the median position: 3, 7, 8, 12, 13, 14. Its median is (8 + 12) / 2 = 10.

Step 3 — Q3. Take the upper half: 18, 21, 22, 26, 31, 58. Its median is (22 + 26) / 2 = 24.

Step 4 — the summary. Minimum 3, Q1 10, median 16, Q3 24, maximum 58.

From these you immediately get the interquartile range:

IQR = Q3 − Q1 = 24 − 10 = 14

The IQR is the width of the middle 50% of the data. Compare it with the range, which is 58 − 3 = 55. The range says these data sprawl across 55 minutes; the IQR says the typical half of them fit inside a 14-minute window. Both are true, and the disagreement is telling you that something unusual lives out in the right tail. Like the median, the IQR is resistant — you could change that 58 to 5800 and the IQR would not move at all.

A small warning that saves a lot of confusion later: there are several legitimate conventions for computing quartiles, differing in whether the median value itself is included when splitting an odd-sized data set, and in how they interpolate. The method used above — take the median of each half — is the one taught in most introductory treatments, which exclude the median from both halves when n is odd; Tukey's original hinges instead include it in both. With an even n, as here, the two agree exactly. Different software packages may report Q1 and Q3 that differ slightly from your hand calculation. Neither is wrong; they are answering the question with different rules.

Boxplots and the 1.5 × IQR rule

A boxplot (box-and-whisker plot) draws the five-number summary. A box spans from Q1 to Q3, so its width is the IQR; a line inside marks the median. Whiskers then extend outward — but not automatically to the minimum and maximum. They stop at the most extreme values that are still considered ordinary, and anything beyond gets plotted as an individual dot.

What counts as ordinary is set by the 1.5 × IQR rule, which defines two fences:

Lower fence = Q1 − 1.5 × IQR
Upper fence = Q3 + 1.5 × IQR

Any value outside the fences is flagged as an outlier. For our response times, 1.5 × IQR = 1.5 × 14 = 21, so:

So the boxplot's left whisker reaches down to 3 (the smallest value, comfortably inside the lower fence), the right whisker stops at 31 (the largest value that is not an outlier), and 58 is drawn as a lone point off to the right. Note that no value can fall below the lower fence of −11 here, since times cannot be negative — so this rule can only ever flag high outliers in these data, which is worth remembering before you read the single flag as evidence of anything by itself. The skew is better judged from the summary as a whole: the outlier sits in the upper tail, Q3 is farther from the median (24 − 16 = 8) than Q1 is (16 − 10 = 6), and the mean of these twelve values, 233 / 12 ≈ 19.4, sits well above the median of 16. All three point the same way — the data are right-skewed.

Boxplots earn their keep when you place several side by side. Comparing three groups by boxplot lets you read differences in center (median lines), differences in spread (box lengths), differences in skew (whether the median sits off-center in its box), and the presence of stragglers — all at a glance and all without assuming anything about the shape of the distributions.

Section 4Z-scores

Standardizing a value

Two exams, two different scoring systems. You scored 82 on one and 88 on the other. Which performance was better? The raw numbers cannot answer that, because they are measured on different rulers. What you need is a way to strip away the scale of each exam and ask a scale-free question: how far above or below its own mean did each score fall, measured in standard deviations?

That is exactly what a z-score (also called a standard score) does:

z = (x − μ) / σ     or, for a sample,     z = (x − x̄) / s

Read the formula in two moves. The numerator (x − μ) says how far the value sits from the center, in the original units. Dividing by σ says in what currency to express that distance: standard deviations. The result is a pure number with no units at all, which is what makes it comparable across any two distributions you like.

Now settle the exam question. Suppose Exam A had a mean of 75 with a standard deviation of 5, and Exam B had a mean of 80 with a standard deviation of 10.

The lower raw score is the stronger performance. On Exam A you beat your classmates by 1.40 standard deviations; on Exam B, only 0.80. The 88 looked better only because Exam B was easier and more spread out. This is the whole point of standardizing: it converts "how many points" into "how unusual," and only the second one is comparable.

The sign carries meaning on its own. A positive z means above the mean, a negative z means below, and z = 0 means the value is exactly at the mean. Standardize an entire data set and the transformed values always have a mean of 0 and a standard deviation of 1. Try it on Set A from Section 2 (x̄ = 50, s ≈ 1.58): the values 48, 49, 50, 51, 52 become z-scores of about −1.26, −0.63, 0, +0.63, +1.26, which sum to zero.

Running the formula backwards, and reading the size

The formula rearranges, which is just as useful. Solving z = (x − μ) / σ for x:

x = μ + zσ

Suppose a measurement has μ = 70 and σ = 8, and you are told a particular observation has z = −1.50. Then x = 70 + (−1.50)(8) = 70 − 12 = 58. Any question of the form "what value sits two standard deviations above the mean?" is answered this way.

As for how big a z-score has to be before it is interesting, a common working convention treats |z| > 2 as unusual and |z| > 3 as very unusual. Those thresholds come from the empirical rule in the next section and therefore assume a roughly bell-shaped distribution. They are conventions, not laws.

One caution that trips up nearly everyone. Computing a z-score requires no assumption about the shape of the distribution — you can standardize any data set with a mean and a standard deviation, however lopsided. But turning a z-score into a percentile or a probability does require knowing the shape. The right-hand column of the table above is valid for bell-shaped data and can be badly wrong for skewed data. A z of +2 sits at about the 98th percentile of a bell-shaped distribution, but in a strongly right-skewed one it can land nearer the 95th — and with other shapes it can miss in the opposite direction. The only way to know is to look at the distribution.

Section 5The empirical rule

68–95–99.7 for bell-shaped data

If a distribution is bell-shaped — roughly symmetric, single-peaked, with tails thinning smoothly on both sides — then the standard deviation does something remarkable: it predicts, quite precisely, how much of the data lies where. This is the empirical rule, also called the 68–95–99.7 rule:

Work an example. A standardized assessment is bell-shaped with μ = 100 and σ = 15. Then:

• One SD: 100 − 15 = 85 and 100 + 15 = 115. About 68% of scores fall between 85 and 115.
• Two SD: 100 − 30 = 70 and 100 + 30 = 130. About 95% of scores fall between 70 and 130.
• Three SD: 100 − 45 = 55 and 100 + 45 = 145. About 99.7% of scores fall between 55 and 145.

Because the bell is symmetric, you can split the leftovers evenly and answer one-sided questions too. Outside ±2σ lies 100% − 95% = 5% of the data, half in each tail, so about 2.5% of scores exceed 130 and about 2.5% fall below 70. Outside ±1σ lies 32%, so about 16% of scores exceed 115. And since 68% sits inside ±1σ, half of that — 34% — lies between 100 and 115.

A footnote for later chapters: "95%" is a rounding. The interval containing exactly 95% of a normal distribution is μ ± 1.96σ, not μ ± 2σ. The 1.96 will reappear constantly once you reach confidence intervals, so it is worth meeting now.

Chebyshev's theorem: the guarantee that always holds

The empirical rule has a hard precondition: the data must be bell-shaped. Apply it to a strongly skewed distribution — incomes, wait times, insurance claims — and it will lie to you, often badly. So what can you say when you do not know the shape, or when you know it is not bell-shaped?

Chebyshev's theorem answers that. For any distribution whatsoever — any shape, any skew, any number of peaks, so long as it has a mean and a standard deviation to speak of — the proportion of data lying within k standard deviations of the mean is at least:

1 − 1 / k²     (for any k > 1)

Substitute a few values of k. For k = 2: 1 − 1/2² = 1 − 1/4 = 0.75, so at least 75% of any data set lies within 2 SD of its mean. For k = 3: 1 − 1/9 = 0.889, so at least 88.9% lies within 3 SD. For k = 4: 1 − 1/16 = 0.9375, at least 93.75%. The restriction k > 1 exists because at k = 1 the formula gives 1 − 1 = 0, a true but useless statement.

A worked case. A right-skewed set of processing times has μ = 200 seconds and σ = 20 seconds, and you are asked what fraction of jobs finish between 140 and 260 seconds. First find k: the interval reaches 60 seconds on either side of the mean, and 60 / 20 = 3, so k = 3. Chebyshev then guarantees at least 1 − 1/3² = 1 − 0.111 = 88.9% of jobs fall in that window. You did not need to know the shape at all — that is the point.

Read that table carefully, because the two columns are not competing estimates. Chebyshev states a floor that is never violated; the empirical rule states an approximation that holds when the bell shape holds. There is no contradiction in "at least 75%" and "about 95%" describing the same interval — 95% is comfortably at least 75%. Chebyshev buys universality at the price of vagueness. The empirical rule buys precision at the price of an assumption you must check.

How do you check it? Look at the data before you assume: plot a histogram, compare the mean with the median, sketch a boxplot and see whether the median sits centered in its box. Every tool from Sections 1 through 3 exists partly to answer this one question.

ReferenceWords worth knowing

A quick reference for the terms in this chapter — skim it now, or circle back whenever a word trips you up.

Bessel's correction

Dividing the sum of squares by n − 1 rather than n when computing a sample variance, so that s² is an unbiased estimate of the population variance σ².

Boxplot

A box-and-whisker plot: a box from Q1 to Q3 with the median marked inside, whiskers to the most extreme non-outlier values, and outliers drawn as separate points.

Chebyshev's theorem

For any distribution regardless of shape, at least 1 − 1/k² of the data lie within k standard deviations of the mean, for any k > 1.

Coefficient of variation (CV)

The standard deviation expressed as a percentage of the mean, (s / x̄) × 100%; used to compare variability across different scales or units.

Degrees of freedom

The number of values in a calculation that are free to vary; for a sample variance it is n − 1, because fixing the mean determines the last value.

Deviation

The signed distance of a single value from the mean, (x − x̄). Deviations from the mean always sum to zero.

Empirical rule

For bell-shaped distributions, roughly 68%, 95%, and 99.7% of the data lie within one, two, and three standard deviations of the mean.

Five-number summary

The set minimum, Q1, median, Q3, maximum — a compact description of center, spread, and shape.

Interquartile range (IQR)

Q3 − Q1, the width of the middle 50% of the data; a resistant measure of spread.

Mean

The arithmetic average, x̄ = Σx / n for a sample and μ = Σx / N for a population; the balance point of the data.

Median

The middle value of sorted data; with an even count, the average of the two middle values. Also the 50th percentile and the second quartile.

Mode

The most frequently occurring value; the only measure of center usable with categorical data. A data set may have one, several, or no mode.

Outlier

A value far from the bulk of the data; commonly defined as any value beyond Q1 − 1.5 × IQR or Q3 + 1.5 × IQR.

Percentile

The value below which a given percentage of the data falls; the 25th, 50th, and 75th percentiles are the three quartiles.

Quartile

One of the three cut points (Q1, Q2, Q3) dividing sorted data into four equal-sized groups.

Range

Maximum minus minimum; the simplest measure of spread and the least resistant, since it depends only on the two extreme values.

Resistant measure

A statistic that changes little when extreme values are added or altered; the median and IQR are resistant, the mean and range are not.

Skewed distribution

An asymmetric distribution with a longer tail on one side; right (positive) skew pulls the mean above the median, left (negative) skew pulls it below.

Standard deviation

The square root of the variance (σ or s); roughly, how far a typical value sits from the mean, expressed in the original units.

Sum of squares (SS)

Σ(x − x̄)², the total of the squared deviations; the numerator of every variance formula.

Variance

The mean of the squared deviations: σ² = SS / N for a population, s² = SS / (n − 1) for a sample. Its units are the original units squared.

Z-score

A standardized value, z = (x − μ) / σ, giving the number of standard deviations a value lies above or below its mean; unitless, so comparable across distributions.

Probability Basics

Everything in the second half of statistics rests on one idea: if you repeated this process forever, how often would that happen? That is all probability is — a bookkeeping system for the long run. It is also the place where a lot of otherwise confident people quietly decide that math is not for them, usually because someone showed them a formula before showing them what it counts. This chapter goes the other way: we will count first, name the rules second, and work every rule out in arithmetic you can check on your fingers. By the end you will be able to read a probability off a table, off a tree, and off a sentence — and, just as importantly, spot the three or four ways people get probability wrong in print every single day.

Section 1What probability is

Random processes, outcomes, and the sample space

A random process is any situation where you know the list of things that could happen but not which one will happen this time: a coin flip, a die roll, a customer either renewing a subscription or not, a machine part either failing this month or not. Each single thing that could happen is an outcome. The complete list of all possible outcomes — every one of them, with no overlap and nothing left out — is the sample space, written S.

Listing the sample space is the most under-rated skill in this chapter, because once you have it, most beginner problems become counting problems. Flip a coin twice and the sample space is S = {HH, HT, TH, TT} — four outcomes, not three. Students frequently write three, reasoning "two heads, two tails, or one of each." That is a mistake worth making once and never again: HT and TH are genuinely different outcomes of the process, and merging them will quietly corrupt every probability you compute afterward.

An event is any collection of outcomes you care to name — a subset of the sample space. "At least one head" is the event {HH, HT, TH}. "Both flips match" is {HH, TT}. An event can hold one outcome, many outcomes, all of them (that is S itself), or none of them (the impossible event). We name events with capital letters: A, B, and so on.

The basic probability formula

When every outcome in the sample space is equally likely — a fair coin, a fair die, a well-shuffled deck, a name drawn blindly from a hat — probability is pure counting:

P(A) = (number of outcomes in A) ÷ (total number of outcomes in S)

That word equally likely is load-bearing. The formula is not a definition of probability; it is a shortcut that works only when the outcomes really are interchangeable. A weighted die, a survey where some people are more likely to respond, a deck someone has stacked — none of these qualify, and for those we get probabilities some other way (from data, from a model, or from a longer-run record).

Three facts follow immediately and are worth memorizing, because they are the sanity checks you will run on every answer for the rest of the book:

• Every probability satisfies 0 ≤ P(A) ≤ 1. A negative probability or a probability of 1.4 is an arithmetic error, full stop.
• P(S) = 1. Something in the sample space has to happen.
• The complement rule: P(A^c) = 1 − P(A), where A^c ("A complement") is the event that A does not happen.

Worked example: rolling two fair dice

Roll a red die and a blue die. Keeping them distinguishable is the trick — each of the 6 red faces can pair with each of the 6 blue faces, so S holds 6 × 6 = 36 equally likely outcomes, written as ordered pairs like (3, 5).

What is P(sum = 7)? Count the pairs that add to 7: (1,6), (2,5), (3,4), (4,3), (5,2), (6,1). That is 6 outcomes.

P(sum = 7) = 6 ÷ 36 = 1/6 ≈ 0.167

What is P(sum ≥ 10)? Sum of 10: (4,6), (5,5), (6,4) — 3 outcomes. Sum of 11: (5,6), (6,5) — 2 outcomes. Sum of 12: (6,6) — 1 outcome. Total 3 + 2 + 1 = 6 outcomes, so P(sum ≥ 10) = 6/36 = 1/6 ≈ 0.167 as well.

What is P(sum is not 7)? Rather than counting 30 pairs, use the complement rule: P(not 7) = 1 − 1/6 = 5/6 ≈ 0.833. Whenever the event you want is messy and its opposite is tidy, flip to the complement.

Notice the last row. The probabilities of a complete, non-overlapping list of outcomes always sum to exactly 1 — that is not a coincidence, it is P(S) = 1 showing up in a table. If your column does not add to 1, you have either double-counted an outcome or lost one.

What a probability actually means

Saying P(sum = 7) = 0.167 does not promise you a seven in the next six rolls. It is a statement about the long run: if you rolled these dice ten thousand times, roughly 16.7% of the rolls would total seven, and the more you rolled the closer that percentage would sit to 16.7%. This is the law of large numbers — the observed proportion of an event converges to its true probability as the number of trials grows.

What the law of large numbers explicitly does not say is that the process "corrects" itself. Ten tails in a row does not make heads more likely on the eleventh flip; the coin has no memory, and P(heads) is still 0.5. The long-run proportion drifts back toward 0.5 not because future flips compensate, but because ten flips is a vanishing share of ten thousand — the early surplus gets diluted, not repaid.

Section 2The addition rule

"Or" means at least one of them

In everyday speech "or" is often exclusive — "soup or salad" means pick one. In statistics, "or" is always inclusive. The event "A or B" (the union, written A ∪ B) happens if A happens, if B happens, or if both happen. Meanwhile "A and B" (the intersection, A ∩ B) happens only when both occur together. Get these two words nailed down now, because every rule in this chapter is a sentence about one or the other.

The general addition rule is:

P(A or B) = P(A) + P(B) − P(A and B)

Why subtract? Draw two overlapping circles. When you add the area of circle A to the area of circle B, the lens-shaped overlap gets counted twice — once inside each circle. Subtracting P(A and B) removes the duplicate. That is the entire content of the rule: add the pieces, then pay back the double-count.

Mutually exclusive events

Two events are mutually exclusive (also called disjoint) when they cannot both happen on the same trial. In the picture, the circles do not touch. In symbols, P(A and B) = 0 — and the moment that term is zero, the correction term in the addition rule vanishes:

If A and B are mutually exclusive: P(A or B) = P(A) + P(B)

So the "simple" addition rule is not a separate law to memorize — it is the general rule with a zero in it. Learn one formula, not two. Roll a single die: "a 1" and "a 2" are mutually exclusive (a die cannot land on both at once), so P(1 or 2) = 1/6 + 1/6 = 2/6 = 1/3 ≈ 0.333.

The complement rule from Section 1 is the tidiest special case of all. A and A^c are mutually exclusive and together they cover the whole sample space, so P(A) + P(A^c) = P(S) = 1, which rearranges to P(A^c) = 1 − P(A).

Worked example: one card from a standard deck

Draw one card from a well-shuffled 52-card deck. Let H = "the card is a heart" and F = "the card is a face card" (jack, queen, or king).

• Hearts: 13 cards, so P(H) = 13/52 = 0.250.
• Face cards: 3 ranks × 4 suits = 12 cards, so P(F) = 12/52 ≈ 0.231.
• Both at once: the jack, queen, and king of hearts — 3 cards — so P(H and F) = 3/52 ≈ 0.058.

Apply the general addition rule:

P(H or F) = 13/52 + 12/52 − 3/52 = 22/52 = 11/26 ≈ 0.423

Check it by brute force: 13 hearts, plus the 9 face cards that are not hearts, equals 22 cards. It matches. Had you forgotten to subtract, you would have gotten 25/52 ≈ 0.481 — wrong by three cards, exactly the three you counted twice.

Section 3The multiplication rule

"And" means both, and both means multiply

Addition answers "or" questions. Multiplication answers "and" questions. The general multiplication rule is:

P(A and B) = P(A) × P(B|A)

Read it as a two-step story: first A happens, and then, in the world where A has already happened, B happens. The symbol P(B|A) is the conditional probability of B given A, which Section 4 unpacks in detail. For now, read the vertical bar as the word "given" or "assuming."

Independent events

Two events are independent when knowing that one occurred does not change the probability of the other — that is, when P(B|A) = P(B). Substitute that into the general rule and the conditioning drops out:

If A and B are independent: P(A and B) = P(A) × P(B)

Again, one rule with a simplification, not two rules. And this equation runs both directions: it is simultaneously the shortcut you use when events are independent and the test you use to decide whether they are. If P(A and B) equals P(A) × P(B), the events are independent; if it does not, they are dependent.

The most reliable everyday source of dependence is sampling without replacement. Draw a card, keep it, draw again: the second draw faces a 51-card deck whose composition depends on the first card. Draw a card, put it back, shuffle, draw again: the second draw faces the same 52-card deck as the first, so the draws are independent. Same deck, same question, different answer — entirely because of whether the card went back.

Worked example: two hearts in a row

Without replacement. P(first card is a heart) = 13/52. Given that it was, only 12 hearts remain among 51 cards, so P(second is a heart | first was a heart) = 12/51.

P(both hearts) = (13/52) × (12/51) = (1/4) × (4/17) = 1/17 ≈ 0.0588

With replacement. The first card goes back and the deck is reshuffled, so the second draw is a fresh 52-card deck:

P(both hearts) = (13/52) × (13/52) = (1/4) × (1/4) = 1/16 = 0.0625

Replacing the card raises the answer from 0.0588 to 0.0625 — a small gap here, but it widens fast as the sample grows relative to the population. As a working rule, when you are sampling less than about 10% of a population, the dependence is small enough that treating draws as independent is a reasonable approximation.

The "at least one" trick

Questions that begin "what is the probability of at least one…" are almost always complement problems in disguise. Counting "at least one" means counting one, two, three, … separately — tedious. Counting the opposite, "none at all," is a single multiplication. So:

P(at least one) = 1 − P(none)

Example. Roll a fair die four times. What is the probability of at least one six? On any single roll, P(not a six) = 5/6. The four rolls are independent, so P(no six in four rolls) = (5/6)⁴ = 625/1296 ≈ 0.4823. Therefore:

P(at least one six) = 1 − 625/1296 = 671/1296 ≈ 0.518

Just over a coin flip. Note that the naive guess — "1/6 per roll, four rolls, so 4/6 ≈ 0.667" — is wrong, and you can see why instantly: eight rolls by that logic would give 8/6 = 1.33, which is not a probability. Adding probabilities across repeated trials is the classic beginner error; multiplying complements is the fix.

That last row rewards a second look. On a single draw, "king" and "heart" are independent: P(king) × P(heart) = (1/13)(1/4) = 1/52, and there is in fact exactly 1 king of hearts in 52 cards, so P(king and heart) = 1/52 too. The numbers agree, so the events are independent — even though both describe the very same card. Independence is a statement about probabilities, not about physical separateness.

Section 4Conditional probability

Conditioning shrinks the sample space

A conditional probability answers a narrower question: not "how likely is A in general" but "how likely is A among the cases where B already happened." Rearranging the multiplication rule gives the definition:

P(A|B) = P(A and B) ÷ P(B),  provided P(B) > 0

The intuition is worth more than the formula. Conditioning on B throws away every outcome where B did not happen and rebuilds the sample space out of what is left. The denominator is no longer "everything" — it is "everything inside B." The numerator counts the part of that shrunken world where A also holds. Whenever you see a vertical bar, ask yourself one question: what is my denominator now?

Worked example: one roll of a fair die

Roll one fair die. Let A = "the roll is a 6" and B = "the roll is even."

Unconditionally, P(A) = 1/6 ≈ 0.167. Now condition on B. We have P(B) = 3/6 = 1/2 (the faces 2, 4, 6), and P(A and B) = 1/6 (only the face 6 is both a six and even). So:

P(A|B) = (1/6) ÷ (1/2) = (1/6) × (2/1) = 2/6 = 1/3 ≈ 0.333

Sanity-check it by counting instead: told the roll is even, the live possibilities are {2, 4, 6}, and exactly one of those three is a six. One out of three. The formula and the counting agree, as they always will.

Now reverse the bar. P(B|A) = P(A and B) ÷ P(A) = (1/6) ÷ (1/6) = 1.000. If the roll is a 6, it is certainly even. So P(A|B) = 0.333 while P(B|A) = 1.000. These are not the same number, and they are not the same question.

Dependence, independence, and the confusion of the inverse

Conditional probability gives the cleanest definition of independence there is. Events A and B are independent exactly when P(A|B) = P(A) — the condition tells you nothing, so the probability does not budge. They are dependent when P(A|B) ≠ P(A). In the die example, P(A) = 0.167 but P(A|B) = 0.333, so "is a six" and "is even" are dependent — knowing the roll is even doubles your belief that it is a six.

Compare a single card draw: let K = "king" and H = "heart." P(K) = 4/52 = 1/13 ≈ 0.0769, and P(K|H) = 1/13 as well — among the 13 hearts, exactly one is a king. The condition changed nothing, so K and H are independent. Equivalently, P(K and H) = 1/52 = (1/13)(1/4) = P(K)P(H). Both tests always give the same verdict.

The mistake to guard against for the rest of your life is the confusion of the inverse: treating P(A|B) as if it were P(B|A). It shows up everywhere. "Most people who own a private jet are wealthy" is not "most wealthy people own a private jet." "Nearly everyone in the hospital with this condition tested positive" is not "nearly everyone who tests positive has this condition." The two conditional probabilities can differ by orders of magnitude, and which one you need depends entirely on what you already know.

Section 5Two-way tables and trees

Reading probabilities off a two-way table

A two-way table (or contingency table) cross-classifies a group of individuals by two categorical variables at once. It is the single most useful object in this chapter, because every probability we have defined can be read straight off it — you never need a formula, only the right denominator.

Suppose 300 job applicants are classified by whether they hold a professional certification and whether they received an offer:

Three kinds of probability live in this table, and they differ only in what you divide by.

Joint probabilities use a single inner cell over the grand total — both things happening together:

• P(certified and offer) = 90/300 = 0.300
• P(not certified and no offer) = 105/300 = 0.350

Marginal probabilities use a row or column total over the grand total — one variable, ignoring the other. They are called marginal because they live in the margins of the table:

• P(certified) = 150/300 = 0.500
• P(offer) = 135/300 = 0.450

Conditional probabilities use a cell over a row or column total — you have restricted the world to one line of the table:

• P(offer | certified) = 90/150 = 0.600  (denominator: the 150 certified applicants)
• P(offer | not certified) = 45/150 = 0.300  (denominator: the 150 uncertified applicants)
• P(certified | offer) = 90/135 ≈ 0.667  (denominator: the 135 applicants with offers)

Look hard at the last two bullets. P(offer | certified) = 0.600 and P(certified | offer) ≈ 0.667 come from the same cell, 90 — but from different denominators, so they are different numbers answering different questions. That is the confusion of the inverse, made visible.

The table also answers the other rules. The addition rule: P(certified or offer) = 0.500 + 0.450 − 0.300 = 0.650; verify by counting cells, 90 + 60 + 45 = 195, and 195/300 = 0.650. The independence test: P(certified)P(offer) = 0.500 × 0.450 = 0.225, but the actual joint probability is 0.300. Since 0.225 ≠ 0.300, certification and receiving an offer are dependent in this group. The conditional probabilities say the same thing more vividly: 0.600 versus 0.300 — certified applicants got offers at twice the rate.

The same data as a tree

A tree diagram takes the same information and stages it in time. The first set of branches splits on the first variable; each second-stage branch is conditional on the branch above it. Two rules govern trees, and they are just the multiplication and addition rules wearing different clothes:

• Multiply along a path to get the probability of that complete path (this is P(A) × P(B|A)).
• Add across paths that produce the outcome you want (this works because different paths are mutually exclusive).

Here is our applicant data as a tree. Stage 1 branches are the marginal probabilities of certification; stage 2 branches are the conditional probabilities of an offer given that certification status.

The path probabilities sum to 1.000, which is your arithmetic check — a tree whose ends do not total 1 has an error in it.

Working the tree backwards

Adding across paths gives the law of total probability. To find the overall chance of an offer, collect every path that ends in an offer:

P(offer) = 0.300 + 0.150 = 0.450

which is exactly the marginal 135/300 we read off the table. Written out in full, this is P(offer) = P(cert)P(offer|cert) + P(no cert)P(offer|no cert) = (0.500)(0.600) + (0.500)(0.300) = 0.300 + 0.150 = 0.450.

Now the genuinely useful move: trees let you reverse the conditioning. Suppose you meet an applicant who received an offer and want the probability that they are certified — that is P(certified | offer), which is the opposite direction from the branches you were given. Use the definition of conditional probability, taking the numerator from the one path with both features and the denominator from the total you just computed:

P(certified | offer) = 0.300 ÷ 0.450 = 2/3 ≈ 0.667

which matches 90/135 from the table. This reversal is the whole engine behind Bayes' rule, and you have just done it with nothing but division. The pattern is always the same: the path you care about, divided by all the paths that end the same way.

ReferenceWords worth knowing

These are the terms that will keep appearing for the rest of the book — skim them now, and come back whenever a symbol stops making sense.

Addition rule

P(A or B) = P(A) + P(B) − P(A and B). The subtraction removes outcomes counted twice; it drops out when the events are mutually exclusive.

Complement

The event that A does not occur, written A^c. P(A^c) = 1 − P(A).

Conditional probability

The probability of A given that B has occurred: P(A|B) = P(A and B) ÷ P(B). Conditioning replaces the denominator with the given event.

Contingency table (two-way table)

A table cross-classifying individuals by two categorical variables, with row, column, and grand totals. Joint, marginal, and conditional probabilities can all be read from it.

Dependent events

Events for which P(A|B) ≠ P(A) — knowing one occurred changes the probability of the other.

Disjoint events

Another name for mutually exclusive events: they cannot both happen on the same trial, so P(A and B) = 0.

Event

Any collection of outcomes from the sample space, usually named with a capital letter.

Independent events

Events for which P(A|B) = P(A), equivalently P(A and B) = P(A)P(B). Knowing one tells you nothing about the other.

Intersection

The event that both A and B occur, written A ∩ B or "A and B."

Joint probability

The probability that two events occur together, P(A and B). On a two-way table it is a single inner cell divided by the grand total.

Law of large numbers

As the number of independent trials grows, the observed proportion of an event converges to its true probability. It does not imply that a process corrects past imbalances.

Law of total probability

Breaking an event into mutually exclusive cases and adding: P(B) = P(A)P(B|A) + P(A^c)P(B|A^c). On a tree, it is adding across all paths that end the same way.

Marginal probability

The probability of one event ignoring the other, P(A). On a two-way table it is a row or column total divided by the grand total.

Multiplication rule

P(A and B) = P(A) × P(B|A); it simplifies to P(A) × P(B) when the events are independent.

Mutually exclusive

Events that cannot occur on the same trial, so P(A and B) = 0. Not the same as independent — in fact the opposite in spirit.

Outcome

A single possible result of one run of a random process.

Probability

A number between 0 and 1 giving the long-run relative frequency with which an event occurs.

Random process

A situation whose set of possible outcomes is known but whose particular result is not determined in advance.

Replacement (with and without)

Whether a selected item is returned before the next selection. With replacement the draws are independent; without replacement they are dependent.

Sample space

The complete, non-overlapping list of all possible outcomes of a random process, written S. All its probabilities sum to 1.

Tree diagram

A staged picture of a random process where you multiply probabilities along a path and add across paths; second-stage branches are conditional on the first.

Union

The event that A occurs, B occurs, or both, written A ∪ B or "A or B." In statistics "or" is always inclusive.

Discrete Random Variables

Up to now, probability has been about single events — one card, one coin, one roll. This chapter takes the next step: instead of asking about one outcome, we describe every outcome at once, with a number attached to each and a probability attached to each number. That package is called a probability distribution, and once you have one you can answer almost anything — how likely a result is, what to expect on average, and how far from average things typically wander. Three famous distributions do an enormous amount of the work in real life, and by the end of this chapter you will be able to recognize which one fits a situation and compute with it. None of this requires clever tricks; it requires a table, a formula, and the patience to work one line at a time.

Section 1Random variables

From outcome to number

A random variable is a variable whose value is a number determined by the outcome of a chance process. That is the whole definition, and it is worth reading twice, because two ordinary words are doing precise work. Number: the result must be numeric, so if the raw outcome is "heads" or "defective," we have to convert it — count the heads, count the defects. Random: we do not know in advance which value will occur, though we may know how likely each one is.

Statisticians use a small notational habit that pays off immediately. A capital letter, usually X, names the random variable itself — the not-yet-known quantity. A lowercase letter, x, names a particular value it might take. So "X = the number of heads in three coin flips" defines the variable, and "x = 2" is one of its possible values. The expression P(X = x) then reads, in plain English, "the probability that the random variable turns out to equal this specific number." Keep the capital and lowercase straight and every formula in this chapter becomes readable.

Discrete: you can count the values

A random variable is discrete if its possible values can be listed — first value, second value, third value, and so on — with nothing squeezed in between. Counts are the classic case: the number of students absent, the number of defective units in a shipment, the number of calls arriving in an hour. You can have 3 absences or 4 absences, never 3.6.

A common misreading is that "discrete" means "only a few values." It does not. A discrete random variable may have infinitely many possible values, as long as they can be counted off in order. The number of calls arriving at a help desk in an hour could in principle be 0, 1, 2, 3, and onward forever — that list never ends, but it is still a list, so the variable is discrete. What matters is countability, not size.

Continuous: you can only measure them

A random variable is continuous if it can take any value in an interval, so that between any two possible values there is always another. Measurements behave this way: height, weight, elapsed time, temperature, blood pressure. Between 70.1 kg and 70.2 kg lies 70.15 kg, and between those lies 70.125 kg, without end.

The practical test is not "does the number have decimals?" but "was it counted or measured?" A reported time of 12.4 seconds looks like a decimal because our stopwatch stops there, but the underlying quantity is continuous — the rounding is a limitation of the instrument, not a property of the variable. Conversely, a dollar amount of $4.75 is technically discrete (money comes in cents), yet money is usually treated as continuous because the steps are so fine. Both facts can be true at once: the mathematical classification is exact, and the modeling choice is a judgment call.

Section 2Probability distributions

The two rules

A probability distribution lists every possible value of a discrete random variable together with the probability of each. It can be written as a table, as a formula, or drawn as a probability histogram whose bar heights are the probabilities. Whatever the format, a distribution is valid only if it satisfies exactly two requirements — and these two are not arbitrary rules to memorize; each one restates something you already believe about probability.

Requirement 1: ΣP(x) = 1. The probabilities of all the values must add up to exactly 1. This says the list is complete: something has to happen, and every "something" is on the list. (In practice, allow for rounding — a table of values rounded to three decimals may total 0.999 or 1.001, and that is fine.)

Requirement 2: 0 ≤ P(x) ≤ 1 for every value x. Each individual probability must be a legitimate probability — never negative, never above 1. A single 1.4 or −0.05 anywhere in the table kills it, no matter what the total comes to.

A table fails if it breaks either rule. Suppose someone offers P(0) = 0.30, P(1) = 0.45, P(2) = 0.15. The individual values are all legal, but they sum to 0.90, so 10% of the probability has gone missing — some outcome was left off the list. Now suppose P(0) = 0.60, P(1) = 0.55, P(2) = −0.15. These do sum to 1.00, but a negative probability is meaningless, so this table is invalid too.

Reading probabilities off a distribution

Let X be the number of heads in three flips of a fair coin. The eight equally likely outcomes are HHH, HHT, HTH, THH, HTT, THT, TTH, TTT. Counting heads in each gives the distribution below, and because each of the eight outcomes has probability 1/8, the probabilities are just counts out of 8.

Check the two rules: every probability sits between 0 and 1, and the total is exactly 1. Valid. Now the payoff — once a distribution is in hand, compound events are just addition, because the values are mutually exclusive.

"At most 1 head" means X = 0 or X = 1, so P(X ≤ 1) = 0.125 + 0.375 = 0.500.
"At least 2 heads" means X = 2 or X = 3, so P(X ≥ 2) = 0.375 + 0.125 = 0.500.
"At least 1 head" would mean adding 0.375 + 0.375 + 0.125 = 0.875 — but the complement is faster: P(X ≥ 1) = 1 − P(X = 0) = 1 − 0.125 = 0.875. Same answer, one subtraction.

That complement trick is worth building into your reflexes. Whenever a question says "at least one," compute 1 − P(none). With three values it saves a little effort; with a binomial of 40 trials it saves you from adding forty terms.

Filling in a missing probability

Because the probabilities must total 1, one missing entry is always recoverable. Suppose X is the number of roommates a randomly selected student has, and a survey reports P(0) = 0.15, P(1) = 0.30, P(2) = k, P(3) = 0.25, with no other values possible. Then:

0.15 + 0.30 + k + 0.25 = 1
0.70 + k = 1
k = 1 − 0.70 = 0.30

Always finish by confirming the recovered value is itself a legal probability. Here 0.30 lies between 0 and 1, so the completed table is valid. If the algebra had produced −0.10, that would not mean "probability can be negative" — it would mean the reported numbers were wrong.

Section 3Expected value

The mean of a random variable is a weighted average

The mean of a discrete random variable, written μ and also called the expected value E(X), answers the question: if this chance process were repeated over and over, what would the average result settle down to? The formula is short:

μ = E(X) = Σ [ x · P(x) ]

Read it as an instruction: multiply each value by its probability, then add up those products. It is an average weighted by likelihood — values that happen often pull the mean toward themselves; values that happen rarely barely budge it. Notice we do not divide by anything at the end. The dividing is already built in, because the weights P(x) sum to 1.

Let X be the number of pets in a randomly selected household, with the distribution P(0) = 0.40, P(1) = 0.30, P(2) = 0.20, P(3) = 0.10. (Check: 0.40 + 0.30 + 0.20 + 0.10 = 1.00, and every entry is between 0 and 1 — valid.) Then:

μ = (0)(0.40) + (1)(0.30) + (2)(0.20) + (3)(0.10)
μ = 0 + 0.30 + 0.40 + 0.30
μ = 1.0 pet

Now the interpretation, which is where students most often go wrong. The expected value is not a prediction about the next household, and it need not even be a possible value. If the distribution had come out to μ = 1.3 pets, no household would ever own 1.3 pets; the number still describes what the average approaches across many, many households. Expected value is a statement about the long run, not about any single trial — that is the law of large numbers, and it is the whole reason insurers and casinos can plan a year in advance while having no idea what happens tonight.

Variance and standard deviation

The mean tells you where the distribution sits; it says nothing about how tightly the values cluster around it. That job belongs to the variance σ² and the standard deviation σ. The definitional formula measures the average squared distance from the mean, again weighted by probability:

σ² = Σ [ (x − μ)² · P(x) ]   and   σ = √(σ²)

There is an algebraically identical shortcut that is far easier to compute, because it never requires subtracting the mean inside the sum:

σ² = Σ [ x² · P(x) ] − μ²

Both give the same answer; use whichever you find clearer. Here is the pet example carried all the way through in one table — build the columns left to right and the arithmetic takes care of itself.

By the shortcut: σ² = 2.00 − (1.00)² = 2.00 − 1.00 = 1.00. By the definition, the last column also totals 1.00. The two agree, as they must. Then σ = √1.00 = 1.0 pet.

Units matter here. The variance came out in "squared pets," which means nothing to anyone, and that is precisely why we take the square root: σ is back in the original units and can be read as "a typical distance from the mean." So households average 1.0 pet, and a typical household differs from that average by about 1.0 pet.

A rough but useful convention, the range rule of thumb, treats values more than two standard deviations from the mean as unusual. Here that boundary is μ ± 2σ = 1.0 ± 2.0, an interval from −1.0 to 3.0 pets. No possible value falls outside it — 3 sits exactly on the upper boundary — so nothing in this distribution counts as unusual — which fits: a household with 3 pets is uncommon, but not startling.

Expected value as a decision tool

Expected value earns its name in situations involving money, where it tells you what a repeated bet, policy, or purchase is worth per play. Suppose a club sells 500 raffle tickets at $3 each, with one $1,000 prize. What is the expected value of buying one ticket?

Define X as your net gain, which means the $3 you paid is subtracted in both branches. If you win, you gain 1000 − 3 = $997, and that happens with probability 1/500. If you lose, you gain −$3, with probability 499/500.

E(X) = (997)(1/500) + (−3)(499/500)
E(X) = 997/500 − 1497/500
E(X) = −500/500 = −$1.00

The interpretation is exact and slightly deflating: you cannot lose exactly $1 on this raffle — you will either be up $997 or down $3 — but if you bought a ticket in this same raffle over and over, your average result would settle near a loss of $1.00 per ticket. The club's side of the same arithmetic is the mirror image: it collects $3 per ticket and pays out $1,000 spread over 500 tickets, or $2 per ticket, so it clears an average net of $1.00 per ticket — exactly what the buyer loses. (Note that $1.00 is the club's net per ticket, not its revenue; revenue is the full $3.) That balance is how the fundraiser was planned in the first place.

Section 4The binomial distribution

The four requirements

The binomial distribution is the most heavily used discrete distribution in statistics, because an enormous number of real questions have the same skeleton: we repeat something a fixed number of times, and we count how often it works. A situation qualifies as a binomial experiment only if all four of the following hold. Miss one and the formula in the next subsection gives a wrong answer.

Two conventions clear up most confusion. First, "success" is just a label for the outcome you chose to count — it can be a defect, a failure, or a death. If you are counting defective parts, a defect is a "success," and p is the probability of a defect. Second, p is the probability of success on one trial, q = 1 − p is the probability of failure, and x is the number of successes in the n trials, so x can be any whole number from 0 to n.

The binomial formula

P(X = x) = _nC_x · p^x · q^n−x   where   _nC_x = n! / [ x! (n − x)! ]

The formula is three ideas multiplied together, and it stops looking intimidating once you see them separately. The piece p^x is the probability of getting x successes; q^n−x is the probability of the remaining n − x failures; multiplying them is legal precisely because the trials are independent (requirement 3). But that product is the probability of one specific arrangement, such as success-success-failure-failure. The binomial coefficient _nC_x counts how many arrangements produce x successes, and since each arrangement has the same probability, we multiply by that count.

Worked example. A quiz has 6 multiple-choice questions with 4 choices each. A student guesses on every question. What is the probability of exactly 2 correct?

Check the four requirements: n = 6 fixed ✓; each question is right or wrong ✓; guesses do not affect one another ✓; p = 1/4 = 0.25 on every question ✓. So n = 6, p = 0.25, q = 0.75, x = 2.

Step 1 — the coefficient: ₆C₂ = 6! / (2! · 4!) = (6 × 5) / (2 × 1) = 30/2 = 15.
Step 2 — the successes: (0.25)² = 0.0625.
Step 3 — the failures: (0.75)⁴ = 0.31640625.
Step 4 — multiply: 15 × 0.0625 × 0.31640625 = 0.9375 × 0.31640625 = 0.2966.

So there is about a 29.7% chance of exactly 2 correct. Running the same formula for every value of x produces the complete distribution:

The column totals 1.0000, confirming this is a valid probability distribution — a good habit whenever you build a binomial table by hand. Compound questions now work exactly as in Section 2. "What is the probability of at least 1 correct?" Use the complement: P(X ≥ 1) = 1 − P(0) = 1 − 0.1780 = 0.8220. "At least 4 correct?" Add the tail: 0.0330 + 0.0044 + 0.0002 = 0.0376, under 4%.

Mean and standard deviation of a binomial

You could find μ and σ by running the Section 3 machinery on all seven rows — but for a binomial there are shortcuts, and they are exact, not approximations:

μ = np    σ² = npq    σ = √(npq)

For the guessing student: μ = np = (6)(0.25) = 1.5 correct, which is sensible — guess on 6 questions with a 1-in-4 chance and you average one and a half. And σ = √[(6)(0.25)(0.75)] = √1.125 = 1.06 correct.

Apply the range rule of thumb: μ ± 2σ = 1.5 ± 2.12, an interval from −0.62 to 3.62. Scores of 0, 1, 2, or 3 are ordinary for a guesser; a score of 4 or more is unusual. That verdict matches the exact tail probability we just computed, P(X ≥ 4) = 0.0376. The rule of thumb and the exact calculation agree, which is the point of having both: the rule is quick, the calculation is precise.

Section 5Other discrete distributions

The Poisson distribution

The binomial needs a fixed number of trials. But many real counts have no trials to fix: calls arriving at a help desk in an hour, typos on a page, potholes per kilometer of road. There is no n, because there is no upper limit on how many could occur. For these, the Poisson distribution models the number of events in a fixed interval of time, area, volume, or distance, given a known average rate.

P(X = x) = ( λ^x · e^−λ ) / x!    for x = 0, 1, 2, 3, …

Here λ (lambda) is the mean number of events per interval, and e ≈ 2.71828 is the natural base. The distribution needs three conditions: the events occur independently, the average rate λ is constant over the interval, and the interval is fixed. It has one striking property — a single parameter controls everything:

μ = λ    σ² = λ    σ = √λ

The mean and the variance are the same number. That is a genuinely useful diagnostic: if you have count data whose sample variance is far larger than its mean, the Poisson model is probably wrong for it.

Worked example. A help desk receives an average of 3 calls per hour. What is the probability of exactly 5 calls in a given hour?

Here λ = 3 and x = 5.
Step 1: λ^x = 3⁵ = 243.
Step 2: e⁻³ = 0.049787.
Step 3: x! = 5! = 5 × 4 × 3 × 2 × 1 = 120.
Step 4: P(X = 5) = (243 × 0.049787) / 120 = 12.0983 / 120 = 0.1008.

So about a 10.1% chance of exactly 5 calls. The chance of a completely quiet hour is P(X = 0) = (3⁰ · e⁻³)/0! = 0.049787, roughly 5% — recall that 3⁰ = 1 and 0! = 1. And the chance of at least one call is the complement: 1 − 0.0498 = 0.9502. The standard deviation is σ = √3 ≈ 1.73 calls.

One more use worth knowing: when a binomial has a large n and a small p — a common rule of thumb is n ≥ 100 and np ≤ 10 — the Poisson with λ = np closely approximates it. This is why Poisson models rare events so well: accidents at an intersection, rare defects on a production line, unusual claims filed with an insurer.

The geometric distribution

The geometric distribution flips the binomial question around. A binomial fixes the number of trials and lets the number of successes vary. A geometric fixes the number of successes at exactly one — the first one — and lets the number of trials vary. It answers: how many attempts until the first success?

P(X = x) = q^x−1 · p    for x = 1, 2, 3, …

The logic is transparent: to have the first success on trial x, you need x − 1 failures first (probability q^x−1) followed by one success (probability p). There is no coefficient because there is only one arrangement that works. The requirements are the binomial's minus the fixed n: independent trials, two outcomes, and a constant p. Its mean and standard deviation are:

μ = 1/p    σ² = q/p²    σ = √q / p

Worked example. A representative converts 20% of the calls she makes, independently. What is the probability her first conversion comes on the 4th call?

Here p = 0.20, q = 0.80, x = 4.
P(X = 4) = (0.80)³ × 0.20 = 0.512 × 0.20 = 0.1024, about 10.2%.

How many calls does it take on average to get that first conversion? μ = 1/p = 1/0.20 = 5 calls — the reciprocal, exactly as intuition suggests. Read that carefully: μ counts the successful call itself, so the average number of failed calls preceding the first conversion is 4, not 5. The spread is σ = √0.80 / 0.20 = 0.894/0.20 = 4.47 calls, which is large relative to the mean; waiting times of this kind are notoriously variable. And the probability she needs more than 3 calls is simply the probability the first 3 all fail: P(X > 3) = (0.80)³ = 0.512. So P(X ≤ 3) = 1 − 0.512 = 0.488.

One caution when reading other sources: some define the geometric variable as the number of failures before the first success, so it starts at 0 instead of 1 and has mean q/p. The mathematics is the same; only the starting point shifts. Always check which version a formula sheet or calculator is using before plugging in.

Choosing among the three

Nearly every discrete problem you meet at this level fits one of these three, and the fastest way to identify which is to ask what is fixed and what is random.

ReferenceWords worth knowing

A quick reference for the terms in this chapter — skim it now, or circle back whenever a word trips you up.

Bernoulli trial

A single trial with exactly two possible outcomes, labeled success and failure, with success probability p. A binomial experiment is n independent Bernoulli trials.

Binomial coefficient

The count _nC_x = n!/[x!(n−x)!] of the different arrangements that give x successes in n trials.

Binomial distribution

The distribution of the number of successes in a fixed number of independent trials with constant success probability; P(x) = _nC_xp^xq^n−x.

Continuous random variable

A random variable that can take any value in an interval; probabilities are assigned to intervals, and the probability of any single exact value is 0.

Cumulative probability

The probability that a random variable is at most a given value, P(X ≤ x), found by adding the probabilities of all values up to and including x.

Discrete random variable

A random variable whose possible values can be listed, such as 0, 1, 2, 3; usually the result of counting. The list may be infinite.

Expected value

Another name for the mean of a random variable, E(X) = μ = Σ[xP(x)]; the long-run average over many repetitions.

Factorial

The product n! = n(n−1)(n−2)…(1), with 0! defined as 1.

Geometric distribution

The distribution of the number of independent trials needed to obtain the first success; P(x) = q^x−1p, with mean 1/p.

Independent trials

Trials in which the outcome of one has no effect on the probabilities of any other.

Law of large numbers

The principle that as a chance process is repeated more and more times, the average of the results converges to the expected value.

Mean of a random variable (μ)

The probability-weighted average of the values, Σ[xP(x)]; the balance point of the distribution.

Memoryless property

The feature of the geometric distribution that past failures do not change the probability of success on the next trial.

Parameter

A number that describes a population or a probability model, such as n and p for a binomial or λ for a Poisson.

Poisson distribution

The distribution of the number of events occurring in a fixed interval at a constant average rate λ; P(x) = λ^xe^−λ/x!, with mean and variance both equal to λ.

Probability distribution

A listing, formula, or graph giving every possible value of a discrete random variable together with its probability.

Probability histogram

A bar graph of a discrete distribution in which the values sit on the horizontal axis and the bar heights are the probabilities.

Random variable

A variable whose numeric value is determined by the outcome of a chance process; written X, with x denoting a particular value.

Range rule of thumb

A rough convention that values farther than 2 standard deviations from the mean — outside μ ± 2σ — may be considered unusual.

Standard deviation of a random variable (σ)

The square root of the variance; a typical distance of the values from the mean, expressed in the original units.

Success probability (p)

The probability of the outcome being counted on a single trial; its complement q = 1 − p is the failure probability.

Variance of a random variable (σ²)

The probability-weighted average squared distance from the mean, Σ[(x−μ)²P(x)], equal to Σ[x²P(x)] − μ².

The Normal Distribution

Measure enough of almost anything that varies for lots of small, independent reasons — heights, birth weights, test scores, the error in a laboratory instrument — and the histogram tends to pile up in the middle and thin out symmetrically on both sides. That shape has a name: the normal distribution, or the bell curve. It matters far beyond its own good looks, because once you know a variable is roughly normal, two numbers — the mean and the standard deviation — let you answer questions about any value in the distribution. This chapter teaches you to read the curve, to reason with the 68–95–99.7 rule, to convert any score into a z-score, to turn that z-score into a probability, and to run the whole machine backward when you are handed a percentile instead of a score. There is arithmetic here, but it is all addition, subtraction, multiplication, and division — nothing harder.

Section 1The normal curve

What the shape is telling you

A normal distribution is a smooth, symmetric, single-peaked curve — the bell curve. It is not a picture of data you collected; it is a model, an idealized mathematical shape that real data can resemble more or less closely. When we say "adult female heights are approximately normal," we mean the histogram of real heights looks a lot like that curve, close enough that we can use the curve's mathematics to make good predictions.

Every normal curve is completely pinned down by exactly two numbers, called parameters. The mean μ (mu) sits at the center and locates the peak. The standard deviation σ (sigma) controls the spread: a small σ gives a tall, narrow curve where values huddle near the middle; a large σ gives a short, wide curve where values sprawl. Change μ and the whole curve slides sideways without changing shape. Change σ and the curve stretches or squeezes without moving its center. Statisticians write this compactly as X ~ N(μ, σ²), read "X follows a normal distribution with mean μ and variance σ²."

The equation behind the picture is f(x) = [ 1 ÷ (σ √(2π) ) ] · e^{−(x − μ)² / (2σ²)}. You will never have to evaluate it by hand, and nothing in this chapter depends on it. It is worth seeing once only so you can confirm the claim above: the only symbols in it besides x are μ and σ. Two numbers, one curve.

The properties, one at a time

Six features define the shape, and every technique in the rest of this chapter leans on one of them.

The fourth property deserves a slow read, because it is where most students first get uneasy. For a continuous variable — one that can take any value in a range, like height in inches carried out to as many decimals as you like — the probability of landing on exactly one specific value is zero. Nobody is exactly 64.000000… inches tall. So we never ask "what is the probability height equals 64?" We ask "what is the probability height falls between 63 and 65?", and the answer is the area under the curve over that stretch of the axis. Height of the curve is density, not probability. Area is probability.

A practical consequence: for continuous distributions, P(X < 64) and P(X ≤ 64) are the same number, because the single point 64 contributes zero area. That is not true for the discrete distributions of Chapter 6, and it trips people up on exams.

How close to normal does real data have to be?

Close enough, and no closer. Real heights are bounded below by zero and above by about nine feet; the normal curve technically extends to infinity in both directions. Real test scores are integers; the curve is smooth. None of that matters in practice, because the curve assigns so little area to the impossible regions that the error is negligible. What does matter is gross violations: strong skew, two peaks, or a hard boundary sitting close to the mean. Income is the classic bad fit — it is right-skewed, with a long tail of high earners, so its mean sits well above its median and the empirical rule you are about to learn will give you badly wrong answers.

Section 2The empirical rule in action

The rule, and the finer breakdown

Because every normal curve has the same shape, the proportion of area inside a given number of standard deviations is always the same, no matter what the units are. That constancy is the empirical rule, also called the 68–95–99.7 rule:

• About 68% of values fall within μ ± 1σ (exactly 68.27%).
• About 95% of values fall within μ ± 2σ (exactly 95.45%).
• About 99.7% of values fall within μ ± 3σ (exactly 99.73%).

Split those bands in half using symmetry and you get a map of the curve precise enough for most quick reasoning. Each half-band below is one side of the center.

Memorize the sequence 0.15, 2.35, 13.5, 34 — then 34, 13.5, 2.35, 0.15 going back out. Those eight numbers sum to 100% and answer a surprising share of exam questions with no table at all.

A worked example

Suppose adult female heights in a large population are approximately normal with μ = 64.0 inches and σ = 2.5 inches. First, build the ruler by marking off standard deviations from the center:

• μ ± 1σ: 64.0 − 2.5 = 61.5 and 64.0 + 2.5 = 66.5 → about 68% of women are between 61.5 and 66.5 inches.
• μ ± 2σ: 64.0 − 5.0 = 59.0 and 64.0 + 5.0 = 69.0 → about 95% are between 59.0 and 69.0 inches.
• μ ± 3σ: 64.0 − 7.5 = 56.5 and 64.0 + 7.5 = 71.5 → about 99.7% are between 56.5 and 71.5 inches.

Now answer four questions using only that ruler and the table above.

(a) What percent are taller than 69.0 inches? 69.0 is exactly μ + 2σ. The middle 95% lies inside ±2σ, so 100% − 95% = 5% lies outside. Symmetry splits that 5% evenly between the two tails: 5% ÷ 2 = 2.5%.

(b) What percent are shorter than 61.5 inches? 61.5 is μ − 1σ. Outside ±1σ lies 100% − 68% = 32%, split evenly: 32% ÷ 2 = 16%.

(c) What percent are between 66.5 and 69.0 inches? That is the band from μ + 1σ to μ + 2σ. Take the difference of the two symmetric bands and halve it: (95% − 68%) ÷ 2 = 27% ÷ 2 = 13.5%.

(d) What percent are between 59.0 and 66.5 inches? Split at the mean. From 59.0 (μ − 2σ) up to 64.0 is half of the 95% band = 47.5%. From 64.0 up to 66.5 (μ + 1σ) is half of the 68% band = 34%. Add them: 47.5% + 34% = 81.5%.

Notice the pattern in all four: draw the curve, mark μ and the three standard-deviation ticks on each side, shade what the question asks for, and add or subtract the block percentages. The sketch is not optional decoration — it is the method.

Where the rule stops working

The empirical rule is a fact about normal distributions only. Apply it to right-skewed data such as household income or hospital length of stay and it will overstate the left tail and understate the right, sometimes badly. For a distribution of unknown shape there is a weaker but universal backstop, Chebyshev's inequality: at least 1 − 1/k² of any distribution lies within k standard deviations of the mean, for any k > 1. That guarantees at least 75% within 2σ and at least 88.9% within 3σ — true always, but much looser than 95% and 99.7%. The precision of the empirical rule is the reward for the normal assumption being justified.

Section 3The standard normal

One curve to rule them all

There are infinitely many normal curves — one for every pair (μ, σ). Nobody wants infinitely many tables. The fix is to rescale every normal distribution onto a single common curve, the standard normal distribution: the normal distribution with μ = 0 and σ = 1, written Z ~ N(0, 1).

The rescaling is done with the z-score:

z = ( x − μ ) ÷ σ

Read it as two steps in plain English. Subtract the mean: this recenters the value, telling you how far above or below average it sits, in the original units. Divide by the standard deviation: this converts that distance from original units into "standard deviations," which have no units at all. A z-score of 1.5 means "one and a half standard deviations above the mean," whether the original data were inches, dollars, milliseconds, or exam points.

The sign carries meaning. Positive z means above the mean, negative z means below, and z = 0 means exactly at the mean. The magnitude carries the rest: |z| under 1 is thoroughly ordinary, around 2 is somewhat unusual, and 3 or more is genuinely rare — only about 0.3% of a normal distribution sits beyond ±3.

Worked example: one score, two ways

An exam has μ = 72 points and σ = 8 points. A student scores x = 86.

z = (86 − 72) ÷ 8 = 14 ÷ 8 = 1.75

The score is 14 points above average, and 14 points is 1.75 standard deviations. Now a second student scores x = 60:

z = (60 − 72) ÷ 8 = (−12) ÷ 8 = −1.50

That student sits one and a half standard deviations below the mean. Note that z = −1.50 and z = +1.50 are the same distance from center in opposite directions — a fact we will exploit constantly in Section 4.

Why standardizing is worth the trouble: comparing apples to oranges

The real power of a z-score is that it strips away the units and the scale, so scores from completely different measurements become comparable. Suppose a student takes two exams:

The raw scores say Exam B (91) was the best performance and Exam A (86) second. The z-scores say otherwise: relative to everyone else who took each test, Exam A was the strongest showing at +1.75, ahead of Exam B at +1.00. The 91 looks impressive until you notice the class averaged 85 with a tight spread, so a 91 was only one standard deviation up. This is exactly why z-scores appear on growth charts, standardized tests, and laboratory reference ranges — they answer "how unusual is this?" rather than "how big is this?"

The reverse conversion is just algebra. Multiply both sides of the z-score formula by σ and add μ:

x = μ + zσ

Check it on Exam A: x = 72 + (1.75)(8) = 72 + 14 = 86. It returns the original score, as it must. Section 5 is built entirely on this rearranged form.

Section 4Finding probabilities

What the table actually gives you

A standard normal table (or z-table) is a lookup of areas. The standard version used almost everywhere reports cumulative left-tail area: the entry for a given z is the proportion of the distribution lying below that z, written Φ(z) or P(Z < z). So the entry for z = 0.00 is 0.5000 — half the curve lies below center — and the entries climb toward 1 as z increases.

Reading it takes two coordinates. The row gives z to one decimal place; the column gives the hundredths digit. To find Φ(1.75), go down to the row 1.7 and across to the column .05:

Φ(1.75) = 0.9599. That single number means: 95.99% of a standard normal distribution lies below z = 1.75. Everything else in this section is arithmetic performed on numbers like that one.

The three question types

Every normal probability problem is one of three shapes, and each has a one-line rule. In all of them, first convert your raw value to a z-score, then look it up.

Two habits keep you out of trouble. Sketch first: draw the bell, mark the mean, mark your value, shade the region the question asks about. If your final answer is more than 0.5 but you shaded a small sliver, you have made an error. Sanity-check against the empirical rule: an area beyond z = 2 should be near 2.5%, an area below z = 0 should be exactly 0.5000.

Worked example: the same exam, three questions

Exam scores are approximately normal with μ = 72 and σ = 8.

(a) What proportion of students score below 86?
Step 1 — standardize: z = (86 − 72) ÷ 8 = 14 ÷ 8 = 1.75.
Step 2 — look up: Φ(1.75) = 0.9599.
Step 3 — answer: P(X < 86) = 0.9599, about 96.0% of students score below 86.

(b) What proportion score above 86?
Same z = 1.75. This is a right tail, so subtract from 1:
P(X > 86) = 1 − 0.9599 = 0.0401, about 4.0%. Sensible: (a) and (b) must add to 1, and 0.9599 + 0.0401 = 1.0000.

(c) What proportion score between 64 and 86?
Step 1 — standardize both ends.
Lower: z = (64 − 72) ÷ 8 = (−8) ÷ 8 = −1.00.
Upper: z = (86 − 72) ÷ 8 = 1.75.
Step 2 — look up both. Φ(1.75) = 0.9599. Φ(−1.00) = 0.1587.
Step 3 — subtract: 0.9599 − 0.1587 = 0.8012, about 80.1%.
Sanity check: the interval runs from 1σ below the mean to 1.75σ above, so it should comfortably exceed the 68% that sits inside ±1σ but stay under 95%. It does.

If your table lists only positive z values, get Φ(−1.00) from symmetry: Φ(−z) = 1 − Φ(z), so Φ(−1.00) = 1 − 0.8413 = 0.1587. Same number.

Worth committing to memory, because they recur throughout the rest of statistics:

Section 5Working backward

The question runs the other way

So far you have been handed a value and asked for a proportion. Many real questions arrive in the opposite order: What score puts a student in the top 10%? What birth weight is the 5th percentile? Where do we set the cutoff so that 95% of parts pass inspection? These are inverse problems, and they run the same machinery backward.

Begin with the vocabulary. The kth percentile is the value with k percent of the distribution below it. The 90th percentile is the value that 90% of the data falls under, and 10% exceeds. Percentiles are always stated as left-tail areas, which is convenient, because that is exactly what the z-table reports.

The three-step recipe:

1. Convert the wording into a left-tail area. "90th percentile" is already 0.9000. "Top 10%" is not 0.10 — the cutoff for the top 10% has 90% below it, so the area is 0.9000 as well. "Bottom 5%" is 0.0500.
2. Search the body of the table for that area and read off the z. This is the reverse lookup: you scan the interior probabilities for the one closest to your target, then read the z from its row and column.
3. Un-standardize with x = μ + zσ. That converts the position back into the original units.

Worked example: the 90th percentile

Same exam: μ = 72, σ = 8. What score is the 90th percentile?

Step 1 — target left-tail area = 0.9000.

Step 2 — hunt in the body of the table. The nearest entries are Φ(1.28) = 0.8997 and Φ(1.29) = 0.9015. The first is off by 0.0003, the second by 0.0015, so take z = 1.28.

Step 3 — un-standardize:
x = μ + zσ = 72 + (1.28)(8) = 72 + 10.24 = 82.24

A score of about 82.2 marks the 90th percentile: roughly 90% of students score below it, 10% above. Check the answer for plausibility — 82.24 is above the mean, as any percentile above 50 must be, and it is a bit more than one standard deviation up, which fits an area of 0.90 sitting between Φ(1.00) = 0.8413 and Φ(1.50) = 0.9332.

Two more, including a negative z

(a) What score separates the top 5%? The cutoff has 95% below it, so the target area is 0.9500 and z = 1.645 (the table straddles it: Φ(1.64) = 0.9495, Φ(1.65) = 0.9505, so 1.645 is the standard interpolated value).
x = 72 + (1.645)(8) = 72 + 13.16 = 85.16. Scores at or above about 85.2 are in the top 5%.

(b) What score is the 25th percentile (the first quartile)? Target area 0.2500, which is below the mean, so the z must be negative. By symmetry, Φ(−0.67) = 1 − Φ(0.67) = 1 − 0.7486 = 0.2514, and Φ(−0.68) = 1 − 0.7517 = 0.2483. The value 0.2500 sits between them; the conventional value is z = −0.67 (more precisely −0.6745).
x = 72 + (−0.67)(8) = 72 − 5.36 = 66.64. About a quarter of students score below 66.6.

The single most common error in inverse problems is dropping the minus sign. If the percentile is below 50, the z must be negative and the answer must land below the mean. If your arithmetic hands you a value above μ for the 25th percentile, you lost a sign somewhere.

These z values recur so often that they are worth keeping close:

Finding a symmetric middle interval

A close cousin: which two scores enclose the middle 95%? Symmetry does the work. If 95% is in the middle, 5% is left over, split into 2.5% in each tail. So the lower bound has left-tail area 0.0250 (z = −1.96) and the upper bound has left-tail area 0.9750 (z = +1.96). Then:

Lower: x = 72 + (−1.96)(8) = 72 − 15.68 = 56.32
Upper: x = 72 + (1.96)(8) = 72 + 15.68 = 87.68

The middle 95% of scores runs from about 56.3 to 87.7. This calculation is the direct ancestor of the confidence interval, and it is where the number 1.96 first earns its keep.

ReferenceWords worth knowing

A quick reference for the terms in this chapter — skim it now, or circle back whenever a word trips you up.

Area under the curve

The region between a density curve and the horizontal axis over some interval; for a probability model, this area is the probability of landing in that interval.

Bell curve

Informal name for the normal curve, from its symmetric, single-humped shape.

Chebyshev's inequality

The guarantee that at least 1 − 1/k² of any distribution lies within k standard deviations of the mean (k > 1); weaker than the empirical rule but always true.

Continuous random variable

A variable that can take any value in an interval; the probability of any single exact value is zero, so probabilities are computed over ranges.

Cumulative probability

The total probability at or below a given value — the left-tail area, written Φ(z) for the standard normal.

Density curve

A smooth curve whose total enclosed area equals 1, used to model the distribution of a continuous variable.

Empirical rule

The 68–95–99.7 rule: in a normal distribution, roughly 68%, 95%, and 99.7% of values lie within 1, 2, and 3 standard deviations of the mean.

Inflection point

The point where a curve changes its direction of bending; on a normal curve these sit exactly at μ − σ and μ + σ.

Lower-tail area

The proportion of the distribution below a stated value; what a standard normal table reports.

Mean (μ)

The center of a normal distribution; for this model it equals the median and the mode.

Normal distribution

A symmetric, unimodal, bell-shaped continuous probability model determined completely by μ and σ; written X ~ N(μ, σ²).

Parameter

A numerical characteristic of a population or model, such as μ or σ, as opposed to a statistic computed from a sample.

Percentile

The value below which a stated percentage of the distribution falls; the 90th percentile has 90% of values beneath it.

Probability density function

The equation that produces a density curve; its height is density, not probability.

Raw score

A value expressed in its original measurement units, before standardizing.

Standard deviation (σ)

The spread of a normal distribution; the distance from the mean to either inflection point.

Standard normal distribution

The normal distribution with μ = 0 and σ = 1, written Z ~ N(0, 1); the reference curve every other normal distribution is converted to.

Standardizing

Converting values to z-scores by subtracting the mean and dividing by the standard deviation; a linear rescaling that changes center and spread but not shape.

Symmetric distribution

One whose left and right halves are mirror images about the center.

Unimodal

Having exactly one peak.

z-score

The unitless quantity z = (x − μ) ÷ σ, giving how many standard deviations a value sits above (positive) or below (negative) the mean.

z-table

A table of standard normal cumulative probabilities, used forward to turn a z into an area and backward to turn an area into a z.

Sampling Distributions & the Central Limit Theorem

Every time you take a sample, you get a slightly different answer — and for a long time that fact looked like a fatal flaw in the whole enterprise of statistics. It is not. The variability of a statistic from sample to sample is itself lawful, predictable, and describable with a curve, and once you can describe it you can say exactly how far off your one real sample is likely to be. This chapter builds that idea from the ground up: what a sampling distribution is, where it is centered, how wide it is, and why — through one of the most surprising results in all of mathematics — it turns out to be bell-shaped almost no matter what the original population looks like. Nothing here requires you to love algebra. It requires you to keep three different distributions straight in your head, and the rest follows.

Section 1What a sampling distribution is

Three distributions, and why people confuse them

Almost every difficulty students have with this chapter comes from collapsing three different things into one word. Let us pull them apart before we do any arithmetic.

The population distribution is how the individual values are spread out in the whole group you care about — all adult resting heart rates, all household incomes, all widget diameters coming off a line. It has a true mean, written μ, and a true standard deviation, written σ. These are parameters: fixed numbers you almost never get to see.

The distribution of the sample data is what you actually hold in your hand: the n values you collected. Its histogram is an imperfect photograph of the population's histogram. If the population is right-skewed, your sample histogram will look right-skewed too, and it will look more like the population as n grows. Its mean is x̄ and its standard deviation is s. These are statistics: numbers computed from data.

The sampling distribution is the strange and powerful one. It is not a distribution of people, or incomes, or widgets. It is the distribution of a statistic — of x̄, or of s, or of a proportion — across every sample of size n you could possibly have drawn. One dot in a sampling distribution is one whole sample boiled down to one number.

Notice the last row. You will never observe a sampling distribution in real life, because in real life you take one sample, not a thousand. The sampling distribution is a thought experiment — but a thought experiment whose center and spread we can compute exactly, which is what makes inference possible at all.

A statistic is a random variable

Here is the conceptual hinge of the whole chapter. Because the sample was chosen at random, the value of x̄ was decided partly by chance. Draw a different random sample and you get a different x̄. A quantity whose value is determined by a random process is a random variable, and every random variable has a distribution. So x̄ has a distribution. That distribution is the sampling distribution, and the spread of it is what we call sampling variability — the ordinary, expected, non-scandalous fact that two honest samples from the same population give two different answers.

Building one by hand

Sampling distributions stop feeling mystical the moment you build one exhaustively. Take a toy population of four values: 2, 4, 6, 8. Its parameters are exact and easy:

μ = (2 + 4 + 6 + 8)/4 = 5, and σ² = [(2−5)² + (4−5)² + (6−5)² + (8−5)²]/4 = (9 + 1 + 1 + 9)/4 = 20/4 = 5, so σ = √5 ≈ 2.236.

Now list every sample of size n = 2 drawn with replacement. There are 4 × 4 = 16 of them, each equally likely, and each one has a mean. Tabulating those 16 means gives the complete sampling distribution of x̄:

Two things jump out. First, the original population was flat — four values, each equally likely — but the sampling distribution of x̄ is already peaked in the middle. There are four ways to land on a mean of 5 and only one way to land on a mean of 2, because an extreme mean requires every draw to be extreme in the same direction. That is the seed of the Central Limit Theorem, visible at n = 2.

Second, let us compute the center and spread of this new distribution. The mean of x̄ is

(2·1 + 3·2 + 4·3 + 5·4 + 6·3 + 7·2 + 8·1)/16 = (2 + 6 + 12 + 20 + 18 + 14 + 8)/16 = 80/16 = 5 — exactly μ.

And its variance is Σ(x̄ − 5)² · (probability):

[9(1) + 4(2) + 1(3) + 0(4) + 1(3) + 4(2) + 9(1)]/16 = (9 + 8 + 3 + 0 + 3 + 8 + 9)/16 = 40/16 = 2.5, so the standard deviation is √2.5 ≈ 1.581.

Compare that to the population: σ² = 5 and σ ≈ 2.236. The variance was cut exactly in half, and 2.236/√2 = 1.581. We did not approximate anything; the two rules of Section 2 fell out of pure counting.

Section 2The distribution of the sample mean

Where it is centered: the mean of the mean

For a simple random sample of size n from a population with mean μ:

μ_x̄ = μ

In words: the average of all possible sample means is the population mean. This holds for every sample size, from n = 2 to n = 2,000,000, and for every population shape, skewed or not. It is the property that makes x̄ an unbiased estimator of μ.

Be careful about what "unbiased" promises. It does not mean your particular x̄ equals μ — it almost certainly doesn't. It means the procedure has no systematic tendency to run high or low. The errors are centered on zero. Bias is a property of a method, not of a number.

How wide it is: the square-root law

For a simple random sample in which observations are independent:

σ_x̄ = σ/√n

Equivalently, the variance of the mean is σ²/n. This quantity is so important it gets its own name — the standard error — and its own section below. Two conditions earn it:

• Independence. Guaranteed by sampling with replacement. When sampling without replacement (the usual case), it is close enough as long as the sample is less than about 10% of the population. Otherwise multiply by the finite population correction √[(N−n)/(N−1)], which shrinks the spread — sampling a big fraction of a population tells you more, not less.
• Randomness. A convenience sample has no sampling distribution worth the name. No formula repairs a biased selection process.

The intuition is averaging: when you add up n independent values, the high draws and the low draws partially cancel. They do not cancel completely (that would give zero spread), and they do not fail to cancel at all (that would give σ). They cancel at exactly the rate √n.

And its shape

Shape has a clean rule and a famous one. The clean rule: if the population is normal, then x̄ is exactly normal for any n, even n = 2. The famous one — what happens when the population is not normal — is the Central Limit Theorem, and it is the next section.

Look down the middle two columns: the center and the spread are the same in all three rows. Population shape affects only the shape of the sampling distribution, never its center and never its width. That is worth memorizing, because exam questions love to test it.

A worked example

Adult resting heart rate in a large population is approximately normal with μ = 74 beats per minute and σ = 12 bpm. You draw a simple random sample of n = 36 adults. What is the chance the sample mean exceeds 77 bpm?

Step 1 — center. μ_x̄ = 74 bpm.

Step 2 — spread. σ_x̄ = σ/√n = 12/√36 = 12/6 = 2 bpm.

Step 3 — shape. The population is normal, so x̄ is exactly normal. No sample-size rule needed.

Step 4 — standardize. z = (77 − 74)/2 = 3/2 = 1.50.

Step 5 — read the table. The area to the right of z = 1.50 is 1 − 0.9332 = 0.0668, about a 6.7% chance.

Now feel the leverage. An individual adult above 77 bpm is unremarkable: z = (77 − 74)/12 = 0.25, an area of about 0.40, so roughly 40% of adults clear it. But an average of 36 adults above 77 happens only 6.7% of the time. Same threshold, same population — a group average is a far more stable, far harder-to-move quantity than one person's value. Every clinical trial and every quality-control chart runs on this fact.

Section 3The Central Limit Theorem

The statement

Let X₁, X₂, …, X_n be independent draws from any population with mean μ and finite standard deviation σ. Then as n grows, the distribution of the sample mean approaches a normal distribution:

x̄ ≈ Normal(μ, σ/√n)  — equivalently  z = (x̄ − μ) / (σ/√n) ≈ Normal(0, 1)

Read that carefully, because the word "any" is doing enormous work. The population may be right-skewed like income, left-skewed like exam scores, bimodal, spiky, discrete, or shaped like a comb. It does not matter. Average enough of it and the averages go bell-shaped. This is why the normal curve turns up everywhere in applied statistics: not because nature loves bells, but because we keep taking averages, and averages are drawn toward normality by a mathematical law.

The same result applies to sums, since a sum is just n times a mean: the total of n independent draws is approximately Normal(nμ, σ√n). Insurers, casinos, and warehouse planners live in that version.

What the CLT does not say

Four misreadings are worth killing on sight.

It does not say the population becomes normal. The population is whatever it is and never changes. Only the distribution of the statistic changes shape.

It does not say your data become normal. Collect 5,000 household incomes and the histogram of those 5,000 values will be gloriously right-skewed — more clearly skewed than a sample of 50, in fact, because more data reveals the population better. The CLT is about the histogram of many means, not the histogram of many observations. This is the deepest confusion in the chapter, and the figure above is worth staring at until the distinction is automatic.

It is not the Law of Large Numbers. The LLN says x̄ converges to μ as n → ∞ — it tells you where you're going. The CLT tells you the shape and scale of the error along the way. LLN gives the destination; CLT gives the wobble.

It does not always apply. The theorem requires a finite variance. A Cauchy distribution has none, and the mean of n Cauchy draws has exactly the same distribution as a single draw — averaging buys you nothing at all. Heavy-tailed data in finance and insurance sit uncomfortably close to this territory, which is one reason those fields treat the normal approximation with suspicion.

How large is "large enough"?

There is no single answer, because the honest answer depends on how skewed the population is. The familiar n ≥ 30 is a rule of thumb, not a theorem — a border checkpoint, not a law of physics.

A worked example

Support-call durations are strongly right-skewed — most calls are short, a few drag on for an hour. The population mean is μ = 4.0 minutes with σ = 6.0 minutes. (Note that σ exceeds μ, a signature of heavy right skew.) A supervisor samples n = 100 calls at random. What is the chance the sample averages more than 5.0 minutes?

Step 1 — shape check. The population is badly skewed, so x̄ is not normal for small n. But n = 100 is large, the calls are independent, and 100 is a tiny fraction of all calls. The CLT applies.

Step 2 — center. μ_x̄ = 4.0 minutes.

Step 3 — spread. σ_x̄ = 6.0/√100 = 6.0/10 = 0.60 minutes.

Step 4 — standardize. z = (5.0 − 4.0)/0.60 = 1.00/0.60 ≈ 1.67.

Step 5 — read the table. Area to the right of z = 1.67 is 1 − 0.9525 = 0.0475, about 4.8%.

The sum version is free: 100 calls averaging over 5.0 minutes is the same event as 100 calls totaling over 500 minutes, so there is also a 4.8% chance the shift's total call time exceeds 500 minutes. Staffing decisions are made on exactly this arithmetic.

Section 4Standard error

The definition, and the distinction that matters most

The standard error of a statistic is the standard deviation of that statistic's sampling distribution. For the sample mean:

SE(x̄) = σ/√n  — and when σ is unknown, as it nearly always is, we estimate it:  SE(x̄) ≈ s/√n

Now the distinction. A standard deviation answers "how spread out are the individuals?" A standard error answers "how spread out would my estimate be if I repeated the study?" One is a fact about the world; the other is a statement about the precision of a measurement procedure. They are not interchangeable, and quoting one when you mean the other misleads readers badly — SE is always the smaller number, so reporting "mean ± SE" when you meant to describe your subjects makes the data look far tidier than it is.

That third row deserves a moment. Doubling your sample does not make people more alike. The population's σ is a fixed feature of the world. What more data buys is a sharper picture of μ — a smaller SE — and nothing else.

The square-root law and its brutal arithmetic

Because n sits under a square root, precision improves slowly. Take σ = 20 and watch:

To cut the standard error in half, you must quadruple the sample. To cut it to a tenth, multiply the sample by a hundred. This is why enormous studies are expensive and why the jump from a 400-person poll to a 1,600-person poll buys less than people expect. It is also why, past a point, money is better spent reducing bias — better sampling frames, higher response rates — than on chasing decimal places that √n will not surrender.

A worked example, and the surprising irrelevance of population size

A sample of n = 49 finished-part diameters has s = 14.2 microns.

SE = s/√n = 14.2/√49 = 14.2/7 = 2.03 microns.

Engineering wants the standard error down to 1.0 micron. How many parts? Set 14.2/√n = 1.0, so √n = 14.2 and n = 14.2² = 201.64 → round up to n = 202. (Check: 14.2/√202 = 14.2/14.21 ≈ 0.999. ✓) Halving the SE cost roughly four times the sample — 49 to 202 — exactly as the square-root law promised.

Now the fact that startles nearly everyone: the population size N does not appear in the formula. As long as the sample is a small fraction of the population, a random sample of 1,000 estimates a parameter just as precisely in a country of 300 million as in a town of 30,000. Precision is governed by how many people you asked, not by what fraction of them you asked. "Only 1,000 people out of 300 million — how can that mean anything?" is a natural objection and a mistaken one; the soup does not care how big the pot is, provided you stirred it.

Section 5Sampling distribution of a proportion

A proportion is a mean in disguise

Categorical data seems like it should need a whole new theory. It doesn't — it needs a relabeling. Code each observation as 1 for "yes" and 0 for "no." Then the sample proportion

p̂ = x/n  (where x = number of successes)

is literally the mean of those 0s and 1s. And for a variable that takes only the values 0 and 1 with success probability p, the population mean is μ = p and the population standard deviation is σ = √[p(1−p)]. Substitute those into Section 2's two rules and the proportion results appear with no new work at all:

μ_p̂ = p   and   SE(p̂) = √[p(1−p)/n]

So p̂ is an unbiased estimator of p, and its spread still falls off as 1/√n. Same logic, same square-root law, different-looking formula.

The success–failure condition

The one genuinely new wrinkle is shape. The count x follows a binomial distribution, which is discrete and — when p is near 0 or 1 — badly skewed, because it is pressed up against a wall. If p = 0.02, most samples yield very few successes and the distribution cannot extend left past zero. The normal approximation needs enough room on both sides:

np ≥ 10  and  n(1−p) ≥ 10

In plain terms: you expect at least about 10 successes and at least about 10 failures. (Some texts use 5, some 15; the idea is identical — both tails need elbow room.) When the condition fails, do not fake it with a z-score — use exact binomial methods instead.

Two lessons live in that table. First, p(1−p) peaks at p = 0.5, so a proportion near a half is the hardest to pin down — which is precisely why close elections are hard to call and lopsided ones are easy. Second, because 0.5 is the worst case, planners who don't yet know p assume p = 0.5 and get a sample size that is guaranteed sufficient no matter what the truth turns out to be.

A worked example

Suppose 60% of a large population supports a measure, so p = 0.60. A pollster surveys n = 200 people at random. What is the chance the poll reports 55% support or less?

Step 1 — conditions. np = 200(0.60) = 120 ≥ 10 and n(1−p) = 200(0.40) = 80 ≥ 10. Both clear easily, and 200 people are far fewer than 10% of a large population, so the draws are effectively independent. Normal approximation approved.

Step 2 — center. μ_p̂ = 0.60.

Step 3 — spread. SE = √[(0.60)(0.40)/200] = √(0.24/200) = √0.0012 ≈ 0.0346, or about 3.46 percentage points.

Step 4 — standardize. z = (0.55 − 0.60)/0.0346 = −0.05/0.0346 ≈ −1.44.

Step 5 — read the table. The area to the left of z = −1.44 is about 0.075 — roughly a 7.5% chance.

Interpret it honestly: even with a perfectly executed random sample and no bias whatsoever, a poll of 200 will land at or below 55% about one time in thirteen when the truth is 60%. That gap is not fraud, incompetence, or a "shift in the electorate." It is sampling variability doing exactly what the formula predicts.

ReferenceWords worth knowing

These terms are the working vocabulary of inference — skim them now, and come back whenever a symbol or phrase stops making sense.

Bias (of an estimator)

A systematic tendency for a statistic to fall above or below the parameter it estimates. Bias is a property of a procedure, not of any single result.

Central Limit Theorem (CLT)

For independent draws from any population with finite standard deviation, the sampling distribution of x̄ approaches Normal(μ, σ/√n) as n grows.

Estimator

A rule or formula for producing an estimate of a parameter from sample data; x̄ is an estimator of μ.

Finite population correction

The factor √[(N−n)/(N−1)] applied to the standard error when the sample is a large fraction of a finite population; it makes the SE smaller.

Independence

The condition that one observation carries no information about another; satisfied by sampling with replacement, or approximately when n is under about 10% of the population.

Law of Large Numbers

The result that x̄ converges to μ as the sample size grows without bound. It gives the destination; the CLT gives the shape of the error along the way.

Margin of error

Roughly 2 × SE — the half-width of a typical 95% confidence interval. It accounts for random sampling variability only, never for bias.

Parameter

A fixed numerical feature of a population, such as μ, σ, or p. Usually unknown.

Population distribution

The distribution of individual values across the entire group of interest.

Sample proportion (p̂)

The fraction of successes in a sample, x/n; mathematically the mean of a set of 0s and 1s.

Sampling distribution

The distribution of a statistic across all possible samples of a fixed size n from the same population.

Sampling variability

The natural sample-to-sample fluctuation of a statistic. It is expected behavior, not a mistake.

Simple random sample (SRS)

A sample chosen so that every possible sample of size n is equally likely; the assumption behind every formula in this chapter.

Skewness

Asymmetry in a distribution. Heavier skew in the population requires a larger n before the CLT approximation becomes trustworthy.

Standard deviation

A measure of how spread out individual values are; σ for a population, s for a sample. It does not shrink as n grows.

Standard error (SE)

The standard deviation of a statistic's sampling distribution — a measure of the precision of an estimate. For the mean, σ/√n, estimated by s/√n.

Statistic

A number computed from sample data, such as x̄, s, or p̂. Because the sample is random, a statistic is a random variable.

Success–failure condition

The requirement np ≥ 10 and n(1−p) ≥ 10 before treating the sampling distribution of p̂ as approximately normal.

Unbiased estimator

An estimator whose sampling distribution is centered on the parameter, so its average value over all possible samples equals the truth. x̄ and p̂ are unbiased.

z-score (standardized score)

The number of standard deviations a value sits from its mean: z = (value − center)/(spread). For a sample mean, the spread used is the standard error.

Confidence Intervals

A sample can tell you what it saw, but it cannot tell you the truth about the whole population — not exactly. A confidence interval is the honest way to report that gap: instead of a single number that is almost certainly a little wrong, you report a range of plausible values, along with how confident the method is. The recipe is short enough to memorize in one line: take your best estimate and stretch it by a margin of error. This chapter builds that line from scratch, works the arithmetic for means and for proportions, shows exactly what makes an interval wide or narrow, and — the part most people get wrong — teaches you to say what “95% confident” actually means.

Section 1The idea of a confidence interval

Why a single number is never enough

Suppose you want to know the average number of hours adults in a small city sleep each night. You cannot ask all 24,000 of them, so you take a random sample of 25 and compute a sample mean of 6.8 hours. That 6.8 is a point estimate: a single number, computed from a sample, offered as your best guess for the parameter — the true population mean μ, which you will never observe directly.

Here is the uncomfortable truth about point estimates: yours is almost certainly wrong. Not badly wrong, and not wrong on purpose — just wrong, because a different random sample of 25 adults would have given 7.1, and another would have given 6.5. The estimate moves around from sample to sample. That movement is called sampling error, and it is not a mistake you made; it is an unavoidable consequence of looking at a part instead of the whole.

So reporting “6.8 hours” with a straight face overstates what you know. Reporting “somewhere between 6.1 and 7.5 hours, and here is how sure I am of that method” is honest. That second statement is a confidence interval: an interval estimate for a parameter, paired with a stated confidence level.

The one-line recipe

Every confidence interval at this level — for a mean, for a proportion, for a difference between two groups — is built from the same skeleton:

point estimate ± margin of error

and the margin of error is itself always the same two things multiplied together:

margin of error = (critical value) × (standard error of the estimate)

The standard error measures how much your estimate bounces around from sample to sample — it is the standard deviation of the estimate's sampling distribution. The critical value (written z^* or t^*) is a multiplier chosen so that the interval is wide enough to succeed as often as your confidence level demands. Want more confidence? Use a bigger multiplier. That is the entire machine. Everything that follows in this chapter is filling in those two blanks for a specific situation.

Why does this work at all? Because of the Central Limit Theorem from the previous chapters: for a large enough sample, the sampling distribution of the sample mean is approximately normal, centered on the true parameter. If the estimate lands within about 1.96 standard errors of the truth roughly 95% of the time, then reaching out 1.96 standard errors from the estimate will capture the truth roughly 95% of the time. The interval is the sampling distribution read backwards.

Reading an interval out loud

An interval has a center (your point estimate), a half-width (the margin of error), and two endpoints. If a poll reports “54% approve, margin of error 3 percentage points,” the interval is 51% to 57%, the center is 54%, and the half-width is 3. Notice that the width of the whole interval is twice the margin of error — a distinction that trips people up constantly. A margin of error of 3 points produces an interval 6 points wide.

One more habit worth building now: a confidence interval is a statement about a parameter, not about an individual. An interval of 6.1 to 7.5 hours is about the population's average sleep. It says nothing about whether any particular person sleeps in that range — most of them will not. The tool for individuals is a prediction interval, and it is much wider.

Section 2A confidence interval for a mean

When σ is known: the z interval

Start with the easier case, the one that almost never happens in real life but makes the logic visible. Suppose the population standard deviation σ is genuinely known — a filling machine whose long-run variability the manufacturer has measured over millions of bottles, say. Then the standard error of the sample mean is exactly σ/√n, and the interval is

x̄ ± z^* · (σ/√n)

where z^* is the value that cuts off the middle portion of the standard normal curve matching your confidence level. For 95%, that value is 1.96 — the number worth committing to memory.

Worked example. A bottling line fills 500 mL bottles. The machine's long-run standard deviation is known to be σ = 4 mL. A quality inspector pulls a random sample of n = 36 bottles and finds x̄ = 502 mL. Build a 95% confidence interval for the true mean fill volume μ.

• Standard error: σ/√n = 4/√36 = 4/6 = 0.6667 mL
• Critical value at 95%: z^* = 1.96
• Margin of error: 1.96 × 0.6667 = 1.31 mL
• Interval: 502 − 1.31 = 500.69 and 502 + 1.31 = 503.31

The 95% confidence interval is 500.69 mL to 503.31 mL. Because 500 mL falls just outside this interval, a true mean of exactly 500 mL is not among the plausible values this sample supports — the data do give evidence, at the 5% level, that the machine is off target. Notice how narrowly it misses: had the sample mean been 501 mL, the interval would have run 499.69 to 502.31 and 500 would have been comfortably inside.

When σ is unknown: the t interval

In practice you almost never know σ. You estimate it with the sample standard deviation s, which introduces a second source of uncertainty: not only is x̄ a noisy estimate of μ, but s is a noisy estimate of σ. If you simply swapped s for σ and kept using 1.96, your intervals would be a little too narrow and would capture the truth less than 95% of the time.

William Gosset solved this in 1908, publishing under the pen name “Student.” His fix was a new family of curves — Student's t distribution — that look like the normal curve but with heavier tails, which produces slightly larger critical values and slightly wider intervals. Exactly how much heavier depends on the degrees of freedom, df = n − 1. Small samples give few degrees of freedom, fat tails, and big multipliers; large samples give many degrees of freedom, and t converges on z.

x̄ ± t^*_n−1 · (s/√n)

Worked example. A random sample of n = 25 adults completes a reaction-time task. The sample mean score is x̄ = 78.4 with sample standard deviation s = 9.0. Build a 95% confidence interval for the population mean μ.

• Degrees of freedom: df = 25 − 1 = 24
• Standard error: s/√n = 9.0/√25 = 9.0/5 = 1.80
• Critical value: t^* at 95% with df = 24 is 2.064
• Margin of error: 2.064 × 1.80 = 3.72
• Interval: 78.4 − 3.72 = 74.68 and 78.4 + 3.72 = 82.12

The 95% confidence interval is 74.68 to 82.12. Notice what the correction cost: had you carelessly used z^* = 1.96, the margin would have been 3.53 and the interval 74.87 to 81.93 — noticeably too narrow, and too confident.

Conditions, and how t collapses into z

Any interval for a mean — the z version and the t version alike — rests on two assumptions, and you should state and check them before you trust the arithmetic in either worked example above. First, the data must come from a random sample (or a randomized experiment), and the observations must be independent — when sampling without replacement, a practical check is that the sample is less than 10% of the population. Second, the sampling distribution of x̄ must be roughly normal. That holds if the population itself is roughly normal, or, thanks to the Central Limit Theorem, if n is large enough (n ≥ 30 is the usual rule of thumb, and more is needed if the data are badly skewed or contain outliers).

The table below shows the 95% critical value shrinking toward 1.96 as the sample grows. This is why software uses t for everything: with n = 100 the difference between 1.984 and 1.960 is trivial, and with n = 10 it is not.

Section 3A confidence interval for a proportion