Precision of findings: p-value and the confidence interval

Confidence interval

The precision of the findings.This is the range within which the predicted means of population may lie.

A confidence interval is the range of values within which the "actual" gods-own-truth result is found. Essentially, a CI of 95% means that if a trial was repeated an infinite number of times, 95% of the results would fall within this range of values.

  • 95% of sample means should lie between 1.96 standard error of the mean above & below their sample mean.
  • if the sample should represent the 95% CI for the population mean if:
    • the sample was large enough
    • the sample is evenly distributed
    • the sample was selected randomly
  • The CI gives an indication of the precision of the sample mean as an estimate of the "true" population mean
  • A wide CI can be caused by small samples or by a large variance within a sample.

The CI is a pain in the arse to calculate for the mathematic-averse Homo vulgaris. A good impression of the difficulty involved can form if one reads one of these two BMJ articles.


The probability of the observed result arising by chance

The p-value is the chance of getting the reported study result (or one even more extreme) when the null hypothesis is actually true.

For example, if your findings are that 1g of oral cyanide kills 95% of swallowers, a p-value of 0.05 would suggest that if cyanide was completely harmless and you repeated your experiment an infinite number of times, your results would be reproduced in 5% of those experiments.

Thus, a very small p-value - say, 0.0001- gives rise to the impression that the results are valid, because it suggests that if the investigator's hypothesis was in fact completely wrong, the chances of arriving accidentally at their "mistaken" results would be 0.1%.


Viera, Anthony J. "Odds ratios and risk ratios: what's the difference and why does it matter?." Southern medical journal 101.7 (2008): 730-734.

Szumilas, Magdalena. "Explaining odds ratios." Journal of the Canadian Academy of Child and Adolescent Psychiatry 19.3 (2010): 227.

Cook, Richard J., and David L. Sackett. "The number needed to treat: a clinically useful measure of treatment effect." Bmj 310.6977 (1995): 452-454.

Goodman, Steven N. "Toward evidence-based medical statistics. 1: The P value fallacy." Annals of internal medicine 130.12 (1999): 995-1004.

Morris, Julie A., and Martin J. Gardner. "Statistics in Medicine: Calculating confidence intervals for relative risks (odds ratios) and standardised ratios and rates." British medical journal (Clinical research ed.) 296.6632 (1988): 1313.

Campbell, Michael J., and Martin J. Gardner. "Statistics in Medicine: Calculating confidence intervals for some non-parametric analyses." British medical journal (Clinical research ed.) 296.6634 (1988): 1454.