Viva 2

This viva will examine knowledge of evidence based medicine and clinical trial design
Describe a Classification of levels of evidence. Subsequent discussion involved research design, types of bias, randomisation, end-points and drug development studies.

What is meant by the term, "levels of evidence"?
  • Levels of evidence is a hierarchical system of classifying studies according to their methodology.
  • It is a necessary oversimplification which acts a shortcut to aid decisionmaking. 
What are the NHMRC levels of evidence?
  • Level I (evidence obtained from a systematic review of all (at least 2) relevant randomized controlled trials),
  • Level II (evidence obtained from at least one properly designed randomized controlled trial,
  • Level III (evidence obtained from other well-designed experimental or analytical studies (not RCCT’s),
  • Level IV (evidence obtained from descriptive studies, reports of expert committees or from opinions of respected authorities based on clinical experience).  
What other grading systems are there to classify quality of evidence?

Several are available:

The original 1979 Canadian Task Force levels

  • Level I-Evidence from at least one randomized controlled trial,
  • Level II1-Evidence from at least one well designed cohort study or case control study, i.e. a controlled trial which is not randomized
  • Level II2-Comparisons between times and places with or without the intervention
  • Level III-Opinions of respected authorities, based on clinical experience, descriptive studies or reports of expert committees.

The United States Preventive Services Task Force (1989) levels

  • Level I: Evidence obtained from at least one properly designed randomized controlled trial.
  • Level II-1: Evidence obtained from well-designed controlled trials without randomization.
  • Level II-2: Evidence obtained from well-designed cohort or case-control analytic studies, preferably from more than one center or research group.
  • Level II-3: Evidence obtained from multiple time series designs with or without the intervention. Dramatic results in uncontrolled trials might also be regarded as this type of evidence.
  • Level III: Opinions of respected authorities, based on clinical experience, descriptive studies, or reports of expert committees.

Oxford Centre for Evidence-based Medicine  Levels of Evidence (2009)

  • Levels:
    • 1a: Systematic reviews (with homogeneity) of randomized controlled trials
    • 1b: Individual randomized controlled trials (with narrow confidence interval)
    • 1c: All or none randomized controlled trials
    • 2a: Systematic reviews (with homogeneity) of cohort studies
    • 2b: Individual cohort study or low quality randomized controlled trials (e.g. <80% follow-up)
    • 2c: "Outcomes" Research; ecological studies
    • 3a: Systematic review (with homogeneity) of case-control studies
    • 3b: Individual case-control study
    • 4: Case series (and poor quality cohort and case-control studies)
    • 5: Expert opinion without explicit critical appraisal, or based on physiology, bench research or "first principles"
  • Grades:
    • A – consistent level 1 studies
    • B – consistent level 2 or 3 studies or extrapolations from level 1 studies
    • C – level 4 studies or extrapolations from level 2 or 3 studies
    • D – level 5 evidence or troubling inconsistent or inconclusive studies of any level

US Agency for Healthcare Research and Quality (AHRQ, 2014) 

Grade Definition
High High confidence that the evidence reflects the true effect. Further research is very unlikely to change our confidence in the estimate of effect.
Moderate Moderate confidence that the evidence reflects the true effect. Further research may change our confidence in the estimate of effect and may change the estimate.
Low Low confidence that the evidence reflects the true effect. Further research is likely to change the confidence in the estimate of effect and is likely to change the estimate.
Insufficient Evidence either is unavailable or does not permit a conclusion.
What is the rationale for classifying studies into"levels"?
  • The quality of the research must factor into the decision of whether or not the findings will be incorporated into clinical practice
  • Evaluation of research quality takes into account research methodology, and this is represented by the grading of evidence into levels.
  • Time is limited, and classification systems help clinicans whittle down their otherwise unmanageable reading lists so as to include only the highest quality of evidence.
  • Using a system of graded evidence to include or exclude studies in meta-analysis increases the confidence of the clinician in the quality of the meta-analysis 
  • The use of a simple grading system is quick and intuitive
  • There seems to be good inter-observer agreement in applying these grading systems (at least within the same system and the same journal - eg Bhandari et al, 2004)
What might be the disadvantages of such a classification system?
  • In many cases, sound clinical practice is based on evidence which cannot be collected prospectively for ethical or logistic reasons - eg. CPR in cardiac arrest parachutes in skydiving, or pretty much all major advances in orthopaedic surgery. In fact Harvie et al (2005) found that during a ten-year period to 2002 only 19 papers on shoulder surgery were RCTs, whereas 538 were single-centre case series which would have graded very low according to the grading system.
  • Labeling an article with a level of evidence creates a "pre-reading bias" which may produce prejudice against the paper
  • Studies of otherwise high quality and scientific merit may not find their way to publication of editors insist on publishing only high-level studies