P-value

P-value (Probability Value)

The probability of observing a result as extreme as the one measured, assuming the null hypothesis is true — a small P-value means the result would be surprising if nothing real were happening.

When you'd see it: A/B test results, scientific research, product experiment writeups. 'P < 0.05' is the conventional threshold for statistically significant — meaning less than a 5% chance of seeing this result by random chance if the null hypothesis were true.

Why it matters: The P-value is the standard output of most statistical tests and understanding it is the entry ticket to reading any experimental result. But it's also the most misinterpreted number in data analysis: a small P-value tells you the result is unlikely to be noise, not that the effect is large, important, or worth acting on.

Common mistakes: Treating P < 0.05 as automatic confirmation that a finding matters. Statistical significance and practical significance are different things. A large experiment can produce P < 0.001 for an effect too small to be commercially meaningful. Always ask for effect size alongside the P-value.

Related hypothesis testing, null hypothesis, statistical significance, confidence interval, p-hacking