Types of error in medical research

In the history of CICM exams, this has only come up once: in Question 23 from the second paper of 2008, where we were called upon to define the types of error.

- Results from a lack of precision in study conduct
- Reduced by meticulous technique and large sample size

- This is a "false positive".
- The null hypothesis is incorrectly rejected (i.e. there really is no treatment effect, but the study finds one)
- The alpha value determines the risk of this happening. An alpha value of 0.05 - same as the p-value - so there is a 5% chance of making a Type 1 error.

- This is a "false negative"
- The null hypothesis is incorrectly accepted (i.e. there really is a treatment effect, but you fail to find it)
- The power (1-beta) determines the risk of this happening. Thus at a beta of 0.2, there is a 20% chance of making a Type 2 error.

- Construct a well-designed RCT, which is the gold standard of research
- Choose an appropriate sample size to power the study
- Choose an appropriate effect size (which is an arbitrary decision, but one made rationally)
- Minimise bias by blinding and allocation concealment
- Conduct a sequential trial with a built-in interim analysis at a predetermined point in the study
- Use correct tests to analyse the data
- Report mean with standard deviation
- Dont assume statistical significance equates to clinical significance
- Give explicit details of the study design and statistical analysis
- Publish even the negative results

There is an online Handbook of Biological Statistics which has an excellent overview of power analysis.

Kelley, Ken, and Kristopher J. Preacher. "On effect size." *Psychological methods* 17.2 (2012): 137.

Moher, David, Corinne S. Dulberg, and George A. Wells. "Statistical power, sample size, and their reporting in randomized controlled trials." *Jama* 272.2 (1994): 122-124.

Cohen, Jacob. "A power primer." *Psychological bulletin* 112.1 (1992): 155.

Dupont, William D., and Walton D. Plummer Jr. "Power and sample size calculations: a review and computer program." *Controlled clinical trials* 11.2 (1990): 116-128.

Eng, John. "Sample Size Estimation: How Many Individuals Should Be Studied? 1." *Radiology* 227.2 (2003): 309-313.