Measures of effect size, risk and odds

Risk and benefit are conjoined in statistics as expressions of the chances of a bad thing happening vs not happening. Risk ratio (RR), odds ratio (OR), hazard ratio (HR) and numbers needed to treat (NNT) are some of the measures applied to this concept. Many other concepts such as "effect size" are also affiliated. Question 23 from the second paper of 2008 asked the candidates to define effect size, and Question 2 from the first paper of 2025 and Question 4 from the second paper of 2025 both asked them to calculate OR, RRR and NNT; the later version also asked what additional information would help decide whether to add the intervention to an ICU formulary. 

In short:

Where

a = events in treatment group
b = non-events in treatment group,

c = events in control group,

d = non-events in control group,

The variables can be calculated as

• AR = a/(a+b)
• ARR = a/(a+b) - c/(c+d)
• NNT = 1/ARR
• RR = a/(a+b) / c/(c+d)
• RRR= (1-RR)
• NNN= 1/RRR
• OR = (a/b) / (c/d)

This whole thing falls into the impossibly vast "Aspects of statistics including: ...[all of statistics]"  from Section 2.5.1 in the second edition of the CICM Syllabus for the Second Part Examination. The successful candidates who reflect on this expectation and determine that they have not met it should feel neither shame nor guilt. There are several ways of looking at this, and the purist would contest that statistics is a part of our professional landscape and we deprioritise this aspect of our training to our own peril; if a paper reports a statistical test and you use the result in patient care then you are responsible for making sure you understood the thing you are using (analogous to knowing the pharmacology of a drug you prescribed). The pragmatist would counter by demonstrating their own smartphone, where being unable to comprehend the inner workings of its electronics poses absolutely no barrier to being an effective user. We will demonstrate this concept by posing snippets of R, which, as millions of people have surely by now discovered, can be used to vibe-code their own statistical analysis without years of training or leet computer skills. For the reader tired of reading one million websites that patronise them with this exact method of explanation, the excellent and highly accessible paper by Scott (2008) is recommended as the single necessary reference.

Risk

Percieved intuitively as the chance of something bad occurring, "risk" is a concept which mutates considerably depending on context in the health sciences. Ortwin Renn (1992) and  Jacobs (2000) delve deeper into the analysis of this than any CICM exam candidate could possibly need.  Mallette, in Psyche, suggests that we owe the term to the Latin "resicum", deriving either from the word "resecare", meaning "to cut off" like "resect", or from the Arabic rizq, meaning "provision". The original use of the term appears to have been in the context of banking and insurance, where in the the Middle Ages it was applied to the sum (the cut) paid out to an financier to incentivise investment in a venture with a chance of failure (in its first documented use, for a shipping run to Valencia from Genoa, contracted on 26 April 1156). 

In epidemiology, risk can be colloquially defined as:

"...relationships which are objective, depersonalized, quantitative, and scientifically measured and suspected causal agents are understood as "factors".

- Gifford, 1986

"Risk ...expresses the degree of belief we have concerning the probability that an event will occur"

- Last, 1983, the Dictionary of Epidemiological Concepts

Risk does not exist mathematically in a raw form, and in health statistics "risk" usually refers to either absolute or relative risk. 

Absolute risk (AR)

AR is the actual event rate in the group (treatment or placebo). Essentially, it is the incidence rate.

The equation is:

where

If you assemble a table of outcomes, where

Then AR is

Or, in R,

Absolute risk, from a patient's point of view, is the most relevant measure of risk when it comes to their expectations for what is likely to happen to them. For a clinician, it is occasionally necessary to compare the absolute risk of one group to the absolute risk of another group

Absolute risk reduction (ARR)

ARR  is the difference between the absolute risk in the treatment group, subtracted from the abvso0lute risk in the conrtol group. 

If you assemble a table of outcomes, where

then  ARR can be calculated as

Or, in R, 

By this, we can determine how many members of the population (c+d) we need to treat to achieve the same effect as was seen in the population (a+b).

Number needed to treat (NNT)

The definition of this concept is:

"the number of patients you need to treat to prevent one additional bad outcome"

- CEBM

NNT is the reciprocal of ARR, which makes logical sense. If the risk of an outcome is reduced by 10% in absolute terms, the chances of the outcome for each individual are reduced by 10%. Because of how chance works, this means that, in order to develop a series with a 100% probability of at least one individual not having that outcome, the number of individuals needs to be at least 10.

In other words:

NNT is an excellent concept for putting across the risk and benefit assessment to a group of colleagues, if one is able to combine it with a number needed to harm (NNH) for a therapeutic agent. For example, for intermediate risk pulmonary embolism receiving thrombolysis,

Therefore, NNT to prevent mortality = 59, and NNH to cause intracranial haemorrhage = 79, presenting a balance still in favour of thrombolysis. Still: when you are anticoagulated, you only have a roughly one in 500 chance of bleeding randomly intro your brain, whereas after thrombolysis that chance increases to one in eighty. This illustrates that NNT and NNH cannot just be compared numerically: the clinical meaning of the risk also needs to be taken into context. 

Anbother limitation is that NNT is unidimensional, because the ARR which it is derived from is calculated on the basis of a binary outcome,  response or nonresponse. Whereas of course one may respond, or one may really respond, and this richness of data is lost when everything gets flattened into statistics. Hutton (2010) is an excellent resource which continues the conversatioin about the limitations of the NNT and NNH.

Relative risk (RR)

RR is defined as:

"the rate of events in the treatment group, divided by the rate of events in the control group."

The college descrives it as "the difference in event rates between 2 groups expressed as proportion of the event rate in the untreated group". Essentially, it is a likelihood ratio: how much more likely is the outcome to occur in the treatment group as opposed to the control group.

It can be calculated as:

or, 

It is a concept notable for its misinterpretation. A RR of 1.0 does not mean that the chance of the outcome is 100% - it means the chance is 0%. Anything over 1.0 is an increase in risk over the baseline,  anything under 1.0 is a decrease in risk. As Schmidt & Kohlmann (2008) point out, it is often impossible to offer a RR with certainty in real life studies as the risk of the control population for a given outcome may not be known. 

Relative risk reduction (RRR) 

RRR is the comparison between groups, the same as ARR but for relative risk:

"Relative risk reduction is the difference in event rates between two groups expressed as a proportion of the event rate in the untreated group"

Except of course in the control group there is no change in relative risk (as there was no treatment), which means it cannot involve the subtraction of one risk from another like ARR. 

Onemust therefore assume that in the control group the risk was 1.0, and then calculate the RRR  by subtracting relative risk (RR) from 1:

Like the ARR, this can be reciprocalled, giving the number needed to harm (NNH):

Odds and odds ratio (OR)

The odds ratio is defined as:

"the odds that an outcome will occur given a particular exposure, compared to the odds of the outcome occurring in the absence of that exposure."

Odds are not risk, and OR is not the same as the relative risk reduction, though computationally they may lead to the same result. To be precise:

“Risk”  refers to the probability of occurrence of an event or outcome.

“Odds” refers to the probability of occurrence of an event, divided by the probability of the event not occurring.

- Ranganathan et al (2015)

How is that different? Observe:

Or,

So, where a is much smaller than b, AR and OR are relatively similar, because a/b ends up being very similar to a/a+b (i.e. 1/1000 is very close to 1/1001).  However as the frequency of the event increases, the two metrics begin to diverge, as illustrated in this excellent graph from Schmidt & Kohlmann (2008). Odds grow much faster than risk as the incidence increases.  

The main issue in medical research is that frequently we have no way of assessing the size or the incidence rate of the control population, leading to a situation where RR cannot be calculated, leaving us only with ORs.