Are confidence intervals better termed “uncertainty intervals”?

Yes—Andrew Gelman

Science reformers are targeting P values and statistical significance, and rightly so.123 It’s wrong to take P≤0.05 as indicating that an effect is real, and it’s wrong to take P>0.05 as a reason to act as though an effect is zero.

One proposed reform is to replace statistical significance with confidence intervals: instead of simply reporting whether the 95% interval contains zero or reporting a P value, report the entire interval. But this approach has problems too,4 in that there can be good reasons in some cases to think that the true effect is likely to be outside the interval entirely. Confidence intervals excluding the true value can result from failures in model assumptions (as we’ve found when assessing US election polls5) or from analysts seeking out statistically significant comparisons to report, thus inducing selection bias.6

Confidence intervals can be a useful summary in model based inference. But the term should be “uncertainty interval,” not “confidence interval,” for four key reasons.