Question 22

 

b)

Non-inferiority trial (10%)

The null hypothesis in a noninferiority study states that the primary end point for the experimental treatment is worse than that for the positive control

treatment by a specified margin. Rejection of the null hypothesis supports a claim of noninferiority the control treatment

Advantages: (20%)

Allows investigation of a new therapy to be compared to an existing accepted therapy Does not require a placebo group, where this may be unethical

Allows cheaper or less toxic therapies to the introduced in place of older therapies

Disadvantages: (20%)

Does not prove efficacy of tested therapy Relies upon known / accepted benefit of control

Needs to be performed under similar conditions in which the active control has demonstrated benefit No clear consensus on what margin of noninferiority should be accepted

Repeated noninferiority trial may lead to acceptance of inferior therapies ‘biocreep’

Examiners Comments:

Significant knowledge gap. Disappointing, since several important trials have followed these designs.
 

Discussion

The disappointment felt at the 4.5% pass rate for this question underscores the need to promote formal training in statistics and literature analysis. Other colleges have already moved to such a strategy, where their trainees may dispense with the increasingly pointless formal project (a mandatory requirement to generate meaningless papers) by satisfying their research requirements though a university unit of study in interpretation of evidence-based medicine.

In summary:

Features of a cluster-randomised trial:

• Groups of patients rather than individuals are randomised
• A group may be as large as a hospital or an ICU
• This is done because sometimes, it would be totally impractical to randomise an intervention to each individual patient; for example where the intervention is a large scale organisational change
• The number of patients in each cluster does not matter as much as the total number of clusters, and power design involves deciding how many clusters one requires (patients within a cluster are more likely to have similar outcomes).
• The outcome for each patient can no longer be assumed to be independent of that for any other patient, 

Advantages of a cluster-randomised trial:

• Able to test interventions applied to whole services or communities
• Increased logistical convenience (less difficulty than individual randomisation)
• Greater acceptability by participants (when something viewed as a worthwhile intervention is delivered to a large group rather than to individuals)
• Both the direct and indirect effects of an intervention can be captured in a population, i.e. the study is more pragmatic (a good example is a study of infectious disease: not only do the randomised participants benefit from a decontaminatingtreatment, but also the population who are exposed to them)
• This increases the external validity

Disadvantages of a cluster-randomised trial:

• The statistical power of a cluster randomised trial is greatly reduced in comparison with a similar sized individually randomised trial (Campbell & Grimshaw, 1998)
• The number of patients required  may be twice or thrice that of a comparable individually randomised trial
• To calculate the power of such a trial requires a specialised approach. The intracluster correlation coefficient needs to be taken into account, as standard power calculations will lead to an underpowered trial if it is analysed taking clustering into account.
• Analysis needs to take into account the cluser design: "If the clustering effect is ignored p values will be artificially extreme, and confidence intervals will be over-narrow, increasing the chances of spuriously significant findings and misleading conclusions". Apparently, this adjustment does not routinely happen.

Features of a non-inferiority trial

• Non-inferiority trials aim to demonstrate that an experimental treatment is not worse than an active control by more than the equivalence margin. 
• In superiority trials, the hypothesis is that the experimental treatment is different (better) to the standard treatment, and two-sided statistical tests are used to test the null hypothesis (because the experimental treatment could be better or worse). The null hypothesis is therefore that there really is no difference. In equivalence trials the null hypothesis is that the treatments are significantly different, by a specified margin (the "equivalence margin"). In non-inferiority trials the null hypothesis is that the experimental treatment is worse than the standard treatment - and the equivalence margin determines how much worse. 

Advantages of a non-inferiority trial:

A non-inferiority trial is appropriate when:

• A placebo treatment is unethical
• The standard treatment is exceptionally effective
• The experimental treatment is thought to be equivalent or at least not worse but not superior to the current treatment (i.e. everybody is convinced that a superiority trial would show no difference)
• The experimental treatment is expected to be similar to the standard treatment in terms of the primary outcome, but has other unrelated advantages (eg. is cheaper, less invasive or more convenient) in which case it would be helpful to demonstrate that its' efficacy is not worse.

Disadvantages of non-inferiority trials

• The standard of care you test against may be more harmful than placebo.
• Because you are not testing against placebo, a situation may arise where both treatments are similarly harmful, and you have merely demonstrated that your experimental treatment is not any more harmful than the current harmful standard of care.
• Because you are not testing against placebo the effect size difference is smaller, and in order to achieve satisfactory power the sample size needs to be larger (and your trial becomes more expensive).
• If the effect of the standard treatment is very close to the effect of a placebo, then the effect of the supposedly non-inferior experimental treatment may end up being very close to the placebo.
• If you test one treatment and prove that it is not much worse, and then test another treatment proving that it is not much  worse than the last, you may eventually come to a point where after multiple noninferiority trials you have demonstrated that your terrible useless treatment is not  much worse than the other terrible useless treatment, something described as "biocreep", or the acceptance of progressively worse treatments.
• Equipoise is ethically necessary to run these trials, but there may be no equipoise with regards to non-inferiority (i.e. some may genuinely believe that the standard treatment is substantially superior to the experimental treatment). Considering that the null hypothesis is that the experimental treatment is much worse, some ethicists may argue that true equipoise is impossible. You basically end up consenting your enrolled patients to agree that they may be randomised to a treatment which is believed to be inferior, or which at best might turn out to be no better.
• A poorly conducted superiority trial (i.e. with many protocol violations and drop-outs) will have a result which trends towards non-inferiority because through intention-to-treat analysis the effect size of the experimental treatment will be diluted.
• The investigators are in control of the equivalence margin, which means they could have decided on an inappropriately wide margin. If the margin is established after the results become available, the experimental treatment could appear not much worse by manipulating how much worse you would accept as a threshold. Even pre-specified margins might be completely arbitrary and inappropriate. There is some pressure to select an inappropriately wide limit - the wider the limit, the smaller the sample size you will require, and the cheaper your trial. This may lead to truly ridiculous conclusions. For instance, Silvio Garattini (2007) describes the COMPASS trial where "the thrombolytic saruplase was judged equivalent to streptokinase for post-myocardial infarction, even though the saruplase group had 50% more deaths than the control group".
• For a drug company, to prove non-inferiority of a new drug is less risky than to try to demonstrate their superiority. Failure to demonstrate superiority may stop the product from making its way into the market, and doesn't look as good on the promotional literature.