Question 24

College answer

a)

An active control trial which tests whether an experimental treatment is not worse than the control treatment by more than a specified margin. Originally conceived as “a safe alternative” treatment.

b)

The null hypothesis states that the primary end point for the new treatment is worse than that of the active control by a prespecified margin, and rejection of the null hypothesis at a prespecified level of statistical significance permits a conclusion of noninferiority

c)

Typically, a placebo controlled trial would be considered unethical as an established treatment already exists.

The investigators may consider the experimental treatment unlikely to be superior to established treatment or the current treatment is highly effective.

The experimental treatment may offer advantages such as safety (reduced adverse effects), better compliance, lower cost or more convenience. 

d)

Proving that two treatments are equivalent could mean that they are both ineffective or even harmful.  Could lead to the acceptance of progressively worse treatments if noninferiority is blindly accepted with repeated noninferiority trials ('biocreep'). 

Conditions and practice may have changed since the original placebo trial of the current standard treatment.

Equipoise is more complex.

Analysis is more complex

A poorly conducted study tends to “noninferiority” as missing data and protocol violations favour noninferiority.

The margin by which non-inferiority is determined is arbitrarily decided by the researchers and may not be clinically appropriate

Sample sizes larger than placebo controlled trials

Examiners Comments:

 

Very poorly answered. Evidence based medicine is an important part of the curriculum and the examiners were concerned at the low level of knowledge displayed. Some candidates appeared to list unrelated phrases from the EBM literature without any appearance of understanding.

 The level of detail given in the template was not required to obtain a passing mark in this question.

Discussion

a) What is a noninferiority trial?    

• Superiority trials aim to demonstrate that there is a difference between treatments, i.e. that one treatment is better than another
• Equivalence trials aim to demonstrate that the effects differ by no more than a specific amount (the "equivalence margin"). 
• Non-inferiority trials aim to demonstrate that an experimental treatment is not worse than an active control by more than the equivalence margin

b) What is the null hypothesis in a noninferiority trial?    

In superiority trials, the hypothesis is that the experimental treatment is different (better) to the standard treatment, and two-sided statistical tests are used to test the null hypothesis (because the experimental treatment could be better or worse). The null hypothesis is therefore that there really is no difference. In equivalence trials the null hypothesis is that the treatments are significantly different, by a specified margin (the "equivalence margin"). In non-inferiority trials the null hypothesis is that the experimental treatment is worse than the standard treatment - and the pre-specified equivalence margin determines how much worse. 

The diagram below is borrowed and modified from Ian A Scott (2009), and demonstrates the results and confidence interval ranges expected of the three different types of trials, when they have demonstrated that the null hypothesis is false.

Superiority trials have to have their results well over to the "favours experimental treatment" side, usually by a pre-specified margin. Equivalence trials need to have their results and confidence intervals within that margin to confirm that the two treatments are in fact equivalent. Non-inferiority trials also need to have their results within that margin, but there is no need to prove that the treatment is superior (i.e the confidence intervals and results simply need to remain within the not much worse margin, the "+1%" line in the diagram).

c) Why would a noninferiority trial be undertaken instead of a superiority trial?  

A non-inferiority trial is appropriate when:

• A placebo treatment is unethical
• The standard treatment is exceptionally effective
• The experimental treatment is thought to be equivalent or at least not worse but not superior to the current treatment (i.e. everybody is convinced that a superiority trial would show no difference)
• The experimental treatment is expected to be similar to the standard treatment in terms of the primary outcome, but has other unrelated advantages (eg. is cheaper, less invasive or more convenient) in which case it would be helpful to demonstrate that its' efficacy is not worse.

d) What are the limitations of noninferiority trials?    

• The standard of care you test against may be more harmful than placebo.
• Because you are not testing against placebo, a situation may arise where both treatments are similarly harmful, and you have merely demonstrated that your experimental treatment is not any more harmful than the current harmful standard of care.
• Because you are not testing against placebo the effect size difference is smaller, and in order to achieve satisfactory power the sample size needs to be larger (and your trial becomes more expensive).
• If the effect of the standard treatment is very close to the effect of a placebo, then the effect of the supposedly non-inferior experimental treatment may end up being very close to the placebo.
• If you test one treatment and prove that it is not much worse, and then test another treatment proving that it is not much  worse than the last, you may eventually come to a point where after multiple noninferiority trials you have demonstrated that your terrible useless treatment is not  much worse than the other terrible useless treatment, something described as "biocreep", or the acceptance of progressively worse treatments.
• Equipoise is ethically necessary to run these trials, but there may be no equipoise with regards to non-inferiority (i.e. some may genuinely believe that the standard treatment is substantially superior to the experimental treatment). Considering that the null hypothesis is that the experimental treatment is much worse, some ethicists may argue that true equipoise is impossible. You basically end up consenting your enrolled patients to agree that they may be randomised to a treatment which is believed to be inferior, or which at best might turn out to be no better.
• A poorly conducted superiority trial (i.e. with many protocol violations and drop-outs) will have a result which trends towards non-inferiority because through intention-to-treat analysis the effect size of the experimental treatment will be diluted.
• The investigators are in control of the equivalence margin, which means they could have decided on an inappropriately wide margin. If the margin is established after the results become available, the experimental treatment could appear not much worse by manipulating how much worse you would accept as a threshold. Even pre-specified margins might be completely arbitrary and inappropriate. There is some pressure to select an inappropriately wide limit - the wider the limit, the smaller the sample size you will require, and the cheaper your trial. This may lead to truly ridiculous conclusions. For instance, Silvio Garattini (2007) describes the COMPASS trial where "the thrombolytic saruplase was judged equivalent to streptokinase for post-myocardial infarction, even though the saruplase group had 50% more deaths than the control group".
• For a drug company, to prove non-inferiority of a new drug is less risky than to try to demonstrate their superiority. Failure to demonstrate superiority may stop the product from making its way into the market, and doesn't look as good on the promotional literature.