Question 21

College Answer

Candidates struggled to identify, and differentiate the various symbols (e.g. significance of 
diamond compared to the square symbol, as well as their relative sizes). Essentially 
candidates were expected to include in their answer that the X axis is the Odds ratio, the 
vertical line from 1 on the x axis is the line of no effect, the results of the individual trials are 
shown as boxes with the size of the box relating to the size of the trial, the position of the 
box relates to the result of the trial, the horizontal lines are the 95% confidence intervals, 
the diamond at the bottom of the diagram represents the combined result of the trial, the 
size of the diamond represents the combined numbers from all the trials and that the 
results can be considered statistically significant if the confidence intervals of the combined 
result do not cross the line of no effect.

Discussion

In summary:

• The x-axis: odds ratio
• The y-axis: a list of studies. How you list them is a lawless free-for-all; Anzures-Cabrera reports that "no general recommendation is appropriate for the order in which the studies should be presented in a forest plot. Many authors default to an alphabetical ordering for ease of cross-referencing with tables and reference lists"
• The vertical line: line of "no effect", OR=1.0
• The horizontal lines: the confidence interval of the individual study
• The position of the square: a point estimate of the odds ratio (OR)
• The size of the square: the weight of the study according to the weighing rules of the meta-analysis, likely representing the sample size and statistical power. 
• The diamond at the bottom: the combined result of the trial
• Results can be considered statistically significant if the confidence intervals of the combined result do not cross the line of no effect

Methodological Expectations of Cochrane Intervention Reviews

Schriger, David L., et al. "Forest plots in reports of systematic reviews: a cross-sectional study reviewing current practice." International journal of epidemiology39.2 (2010): 421-429.

Lewis, Steff, and Mike Clarke. "Forest plots: trying to see the wood and the trees." Bmj 322.7300 (2001): 1479-1480.

Anzures‐Cabrera, Judith, and Julian Higgins. "Graphical displays for meta‐analysis: An overview with suggestions for practice." Research Synthesis Methods 1.1 (2010): 66-80.

Cochrane: "Considerations and recommendations for 
figures in Cochrane reviews: graphs of statistical data" 4 December 2003 (updated 27 February 2008)

Reade, Michael C., et al. "Bench-to-bedside review: Avoiding pitfalls in critical care meta-analysis–funnel plots, risk estimates, types of heterogeneity, baseline risk and the ecologic fallacy." Critical Care 12.4 (2008): 220.

DerSimonian, Rebecca, and Nan Laird. "Meta-analysis in clinical trials."Controlled clinical trials 7.3 (1986): 177-188.

Biggerstaff, B. J., and R. L. Tweedie. "Incorporating variability in estimates of heterogeneity in the random effects model in meta-analysis." Statistics in medicine 16.7 (1997): 753-768.

The Cochrane Handbook: 9.5.4 "Incorporating heterogeneity into random-effects model"