a) With respect to meta-analysis of randomised controlled trials, what is a funnel plot?
b) In the funnel plot above:
i. What do the outer dashed lines indicate?
ii. To what does the solid vertical line correspond?
c) List three factors that result in asymmetry in funnel plots.
a) A funnel plot is a scatter plot of the effect estimates from individual studies against some measure of each study’s size or precision. The standard error of the effect estimate is often chosen as the measure of study size and plotted on the vertical axis with a reversed scale that places the larger, most powerful studies towards the top. The effect estimates from smaller studies should scatter more widely at the bottom, with the spread narrowing among larger studies.
Outer dashed lines-triangular region where 95% of studies are expected to lie
Solid vertical line- no intervention effect
- Size of effect differs according to study size
- Clinical differences
- Methodological differences
ii) Reporting bias
- Publication bias- delayed publication, language, citation, multiple publication bias
- Selective outcome reporting
- Selective analysis/inadequate analysis reporting
- Poor design
It was expected that candidates regularly attending journal club would have the knowledge to answer this question but overall it was not well answered and explanation of terms was poor
The abovedepicted plot is not the gospel plot from the CICM paper, but one which I have confabulated myself. Hopefully, it bears some resemblance to the original.
a) is answered by the college in a manner which precisely reflects the wording of the Cochrane Handbook. That is indeed " a simple scatter plot of the intervention effect estimates from individual studies against some measure of each study’s size or precision ".
The lines? what do they mean? Said best by the laconic college:
- Outer dashed lines-triangular region where 95% of studies are expected to lie. This triangle is centred on a fixed effect summary estimate, and extens 1.96 standard errors in each direction. If no bias is present, this triangle will include about 95% of studies, provided the true treatment effect is the same in each study (i.e. none were using some sort of dodgy home-made levosimendan, for instance).
- Solid vertical line- no intervention effect. This corresponds to an OR of 1.00.
Causes of assymmetry are well summarised by Sterne et al (2011), whose Box 1 I have shamelessly stolen:
Poor methodological quality
DerSimonian, Rebecca, and Nan Laird. "Meta-analysis in clinical trials."Controlled clinical trials 7.3 (1986): 177-188.
Rockette, H. E., and C. K. Redmond. "Limitations and advantages of meta-analysis in clinical trials." Cancer Clinical Trials. Springer Berlin Heidelberg, 1988. 99-104.
Walker, Esteban, Adrian V. Hernandez, and Michael W. Kattan. "Meta-analysis: Its strengths and limitations." Cleveland Clinic Journal of Medicine75.6 (2008): 431-439.
Sterne, Jonathan AC, et al. "Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials." Bmj 343 (2011): d4002.