Give a classification for levels of evidence with respect to Evidence Based Medicine (EBM). (20% of marks)
Discuss the strengths and weaknesses of meta-analysis. (80% of marks)
There is more than one classification system for level of evidence with respect to EBM, and any of those was acceptable. Most candidates struggled with the second part of the question. The strengths/weaknesses listed by candidates were often not related to the scientific or statistical principles/properties of a meta-analysis. A good answer would include a discussion about publication bias, duplicate publication, heterogeneity (different interventions), inclusions of outdated studies, inclusions of non-randomised trials, that pooled result may be biased towards the largest included trials as potential weaknesses and increased sample size and that more variables, sub-groups and outcomes can be examined can be considered as strengths.
This question is virtually identical to Question 19 from the second paper of 2010, except that in 2010 for whatever reason the "levels of evidence" question was weighed higher, as 30% of the marks. Also, during the first appearance of it, the examiners gave more of an answer- actually listing the NHMRC levels of evidence, for example. Even a text reference was offered (Myles & Gin Statistical methods for Anaesthesia and Intensive Care, pg114-118).
a) Define levels of evidence with respect to Evidence Based Medicine (EBM).
There are in fact several systems. The NHMRC classification system is discussed in the document "NHMRC additional levels of evidence and grades for recommendations for developers of guidelines". Instead of wading through the entire 23-page morass, the time-poor candidate is invited to explore Table 3 on page 15. In brief:
b) Discuss the strengths and weaknesses of meta-analysis.
From the college answer, we observe that the candidates were "expected to define a meta-analysis ... and a systematic review", but it is unclear how this expectation could be determined from actually reading the question. The question clearly asks to "discuss the strengths and weaknesses" and nothing more. No mention whatsoever is made of systematic review. However, if the college insist that some hidden meaning is encoded into their SAQ, so we must play along and answer it.
There are several possible definitions in addition to the canonical college answer:
Now, to answer the actual question as it was asked:
Advantages of meta-analysis
Disadvantages of meta-analysis
Sackett, David L., et al. "Evidence based medicine: what it is and what it isn't." (1996): 71-72.
Brown, Gary C., Melissa M. Brown, and Sanjay Sharma. "Value-based medicine: evidence-based medicine and beyond." Ocular immunology and inflammation 11.3 (2003): 157-170.
Bartolucci, Alfred A., and William B. Hillegass. "Overview, strengths, and limitations of systematic reviews and meta-analyses." Evidence-Based Practice: Toward Optimizing Clinical Outcomes. Springer Berlin Heidelberg, 2010. 17-33.