Numbers can strengthen stories by backing up claims or illustrating concepts, so as a journalist you may sometimes need to include and interpret statistics in your stories. Understanding where those numbers come from and what they mean can help you get them right. Here are six key concepts you don’t need a stats degree to understand:
1. A study’s sample size can tell you how well the study may represent the real world.
Sample size is the number of people or other data points (like animals or cells) in a study. Scientists often call it “n”.
Bigger is better, because the chances of a sample accurately reflecting a larger population generally increase with more data.
But how big is big enough depends on the study. Ask an outside expert whether a study’s sample size is big enough to justify its conclusions.
Scientists calculate a p-value, a number between 0 and 1, which indicates the chance of getting the results a study did if there is no actual effect in the real world.
For example: If a study shows that fewer people caught a disease after getting a vaccine, what are the chances of getting that result if the vaccine doesn’t actually work?
In most cases, scientists call results that have a p-value less than 0.05 statistically significant. This means there was a less than 5 percent (1 in 20) chance of getting that specific result if the vaccine didn’t work.
However, statistical significance can’t tell you whether results are actually meaningful (e.g., how much a drug actually helps people). To gauge this, ask experts about a result’s effect size (often written as “Cohen’s d” or “r-squared”) and avoid overselling findings with small effect sizes just because they are statistically significant.
A correlation represents the strength of a relationship between two things, such as UV exposure and risk of skin cancer.
When two things increase or decrease together, they have a positive correlation.
When one increases and the other decreases, they have a negative correlation.
Correlation strength (often called “r”) ranges from –1 to 1, with –1 being a perfect negative correlation, 1 being a perfect positive correlation, and 0 being no relationship at all.
Just because two things are correlated doesn’t mean that one causes or even influences the other. The relationship could be totally coincidental, or they could both be influenced by a third, separate thing.
4. In statistics, error isn’t about mistakes; it refers to how accurate a measurement is.
Many surveys and polls will report a margin of error—the researchers’ range of uncertainty around their estimate. Including this information in stories gives readers a sense of the reliability or certainty of a result.
For example: A pre-election poll might show one candidate ahead of another 52 percent to 47 percent, ± 3 percent.
If you add and subtract the margin of error from the estimate, you get the upper and lower bounds of that estimate’s confidence interval (CI), a key statistic that studies often report.
When the confidence intervals of two results overlap, there’s no statistical difference between them—something you’ll want to call out.
For example: With the above margin of error, the first candidate’s CI is 49 percent to 55 percent and the other’s is 44 percent to 50 percent. This means that the two are essentially tied.
5. “Risk” and “odds” might sound the same, but statistically they’re different.
Risk indicates how likely an event is. It’s what we think of when we say “percent chance,” as in “there’s a 25 percent chance of drawing a spade from a pack of cards.” It captures how likely an event is to occur, divided by all possible outcomes.
The term odds represents the same concept in a mathematically different way—it captures how likely an event is, divided by the likelihood of it not occurring.
A lot of scientific papers report odds, not risk. But since risk often feels more intuitive to readers, it’s useful to translate odds into risk.
You can convert odds to risk with a simple formula: odds/(1+odds) = risk
For example: Say 100 transplant patients take a new anti-rejection drug and 25 develop severe side effects while 75 don’t. The risk of severe side effects is .25 or 25 percent. The odds are 25:75 or 1:3 (.33). To convert odds to risk, you’d calculate: .33/(1+.33) = .25.
A percentage is a number represented as a fraction of 100 (e.g., 1/4 = 25/100 = 25 percent).
Percentage points refer to the absolute difference between two percents. It’s just subtraction!
For example: If 5 percent of people used to drive electric cars, and now 10 percent do, that’s a 5 percentage-point increase (10 – 5 = 5).
Percent change is the relative difference between two percents. To calculate it: Percent change = (percentage point difference / starting value) x 100.
So, for the electric car example, that’s a 100 percent increase (5/5 x 100 = 100 percent). Both “5 percentage point increase” and “100 percent increase” are true, so watch for instances when percentage language is used to exaggerate or downplay a result.
