In the history of CICM exams, this has only come up once: in Question 23 from the second paper of 2008, where we were called upon to define the types of error.

Random error

  • Results from a lack of precision in study conduct
  • Reduced by meticulous technique and large sample size

Type 1 error

  • This is a "false positive".
  • The null hypothesis is incorrectly rejected (i.e. there really is no treatment effect, but the study finds one)
  • The alpha value determines the risk of this happening. An alpha value of 0.05 - same as the p-value - so there is a 5% chance of making a Type 1 error.

Type 2 error

  • This is a "false negative"
  • The null hypothesis is incorrectly accepted (i.e. there really is a treatment effect, but you fail to find it)
  • The power (1-beta) determines the risk of this happening. Thus at a beta of 0.2, there is a 20% chance of making a Type 2 error.

Minimisation of error

  • Construct a well-designed RCT, which is the gold standard of research
  • Choose an appropriate sample size to power the study
  • Choose an appropriate effect size (which is an arbitrary decision, but one made rationally)
  • Minimise bias by blinding and allocation concealment
  • Conduct a sequential trial with a built-in interim analysis at a predetermined point in the study
  • Use correct tests to analyse the data
  • Report mean with standard deviation
  • Dont assume statistical significance equates to clinical significance
  • Give explicit details of the study design and statistical analysis
  • Publish even the negative results

References

There is an online Handbook of Biological Statistics which has an excellent overview of power analysis.

Kelley, Ken, and Kristopher J. Preacher. "On effect size." Psychological methods 17.2 (2012): 137.

Moher, David, Corinne S. Dulberg, and George A. Wells. "Statistical power, sample size, and their reporting in randomized controlled trials." Jama 272.2 (1994): 122-124.

Cohen, Jacob. "A power primer." Psychological bulletin 112.1 (1992): 155.

Dupont, William D., and Walton D. Plummer Jr. "Power and sample size calculations: a review and computer program." Controlled clinical trials 11.2 (1990): 116-128.

Eng, John. "Sample Size Estimation: How Many Individuals Should Be Studied? 1." Radiology 227.2 (2003): 309-313.