Bias, types of error and confounding factors

This chapter answers parts from Section A(d) of thePrimary Syllabus, "Describe bias, types of error, confounding factors and sample size calculations, and the factors that influence them". This topic was examined only once in Question 19 from the second paper of 2011. However, in the fellowship exam it has come up several times:

• Question 23 from the second paper of 2008 (types of error)
• Question 26 from the first paper of 2014  (types of bias)
• Question 5 from the second paper of 2013  (types of bias)

For this reason, most of the discussion takes place in the Part II Required Reading Section (see Types of error in medical research, Types of bias in medical research, Confounding in clinical trials and Interpretive bias).

Here, only a brief summary is offered:

Types of error

Random error

• Results from a lack of precision in study conduct
• Reduced by meticulous technique and large sample size

Type 1 error

• This is a "false positive".
• The null hypothesis is incorrectly rejected (i.e. there really is no treatment effect, but the study finds one)
• The alpha value determines the risk of this happening. An alpha value of 0.05 - same as the p-value - so there is a 5% chance of making a Type 1 error.

Type 2 error

• This is a "false negative"
• The null hypothesis is incorrectly accepted (i.e. there really is a treatment effect, but you fail to find it)
• The beta determines the risk of this happening. Where beta is 0.2 (a common setting), at a power of 0.8 (1-beta), there is a 20% chance of making a Type 2 error.

Types of bias

Selection bias:

• The selection of specific patients which results in a sample group which is not random, and which is not representative of a population. This can be avoided by randomising selection.

Detection bias:

• The observations in the treatment group are pursued more diligently than in the control group. This can be avoided by blinding.

Observer bias:

• The observer makes subjective decisions about the outcome. This can be avoided by blinding the observer, and making the outcome measures objective (eg. measuring mortality, rather than than measuring the warm fluffy sensation of internal wellbeing).

Recall bias:

• The patients know whether they were allocated to the treatment group or the control group, and this discolours their reporting of their symptoms. This can be avoided by blinding the patients.

Response bias:

• The patients enrol themselves in the trial, which results in a non-representative sample. This can be avoided by randomly sampling the population.

Publication bias:

• Publication bias is the influence of study results on the likelihood of their publication. Nobody likes to publish negative data, even though it is as valuable as positive data. This in turn influences the meta-analysis of all data (which cannot be accurate if the only published data is positive).

Regression to mean

• When random chance influences cause extreme variations in an initial measurement, the next measurement (unaffected by this random influence) will be closer to the mean, thus giving the apparance of a treatment effect. This is avoided by using control groups.

Hawthorne effect

• The process of follow-up and careful scrutiny influences the patient outcome. Patients who receive more attention may do better than patients who are ignored. The way to avoid this is to mask the intention of the study from the patients and observers.

Treatment selection bias

• The effects of a treatment are determined by confounders (such as differences in the patients or other co-interventions) rather than than the treatment itself.

The effect of industry sponsorship

• We would like to think that this does not affect people, but it clearly does.