Define levels of evidence with respect to Evidence Based Medicine (EBM). (30% of marks)
Discuss the strengths and weaknesses of meta-analysis. (70% of marks)
For a good answer candidates were expected to list the following levels of evidence, eg Level I (evidence obtained from a systematic review of all (at least 2) relevant randomized controlled trials), Level II (evidence obtained from at least one properly designed randomized controlled trial, Level III (evidence obtained from other well-designed experimental or analytical studies (not RCCT’s), Level IV (evidence obtained from descriptive studies, reports of expert committees or from opinions of respected authorities based on clinical experience).
Candidates were expected to define a meta-analysis (process of combining the results of different (randomised) trials to derive a pooled estimate of effect) and a systematic review (process of obtaining and evaluating all relevant trials, their statistical analyses and interpretation of results). In relation to strengths, a good answer required mentioning increased power of pooled data, analysis and conclusions based on inclusion of high quality trials (weighting of trial quality), overcomes the uncertainty associated with single-centre trials, robust methodology; combines similar patient groups, interventions and end-points to inform the analysis and established methods to find all relevant trials. In relation to weaknesses, a good answer required mentioning publication bias, heterogeneity of included trials, pooled result may be biased toward the largest included trials, historical (outdated) data, and that positive results generally require confirmation by a large RCT.
Syllabus: EBM 2a
References: Myles & Gin Statistical methods for Anaesthesia and Intensive Care, pg114-118
This question is virtually identical to Question 8 from the second paper of 2013. The discussion sections for both questions are therefore very similar.
a) Define levels of evidence with respect to Evidence Based Medicine (EBM).
There are in fact several systems. The NHMRC classification system is discussed in the document "NHMRC additional levels of evidence and grades for recommendations for developers of guidelines". Instead of wading through the entire 23-page morass, the time-poor candidate is invited to explore Table 3 on page 15. In brief:
b) Discuss the strengths and weaknesses of meta-analysis.
From the college answer, we observe that the candidates were "expected to define a meta-analysis ... and a systematic review", but it is unclear how this expectation could be determined from actually reading the question. The question clearly asks to "discuss the strengths and weaknesses" and nothing more. No mention whatsoever is made of systematic review. However, if the college insist that some hidden meaning is encoded into their SAQ, so we must play along and answer it.
There are several possible definitions in addition to the canonical college answer:
Now, to answer the actual question as it was asked:
Advantages of meta-analysis
Disadvantages of meta-analysis
Sackett, David L., et al. "Evidence based medicine: what it is and what it isn't." (1996): 71-72.
Brown, Gary C., Melissa M. Brown, and Sanjay Sharma. "Value-based medicine: evidence-based medicine and beyond." Ocular immunology and inflammation 11.3 (2003): 157-170.
Bartolucci, Alfred A., and William B. Hillegass. "Overview, strengths, and limitations of systematic reviews and meta-analyses." Evidence-Based Practice: Toward Optimizing Clinical Outcomes. Springer Berlin Heidelberg, 2010. 17-33.