Question 8

A systematic review of the literature was undertaken comparing proton pump inhibitors with H2-receptor blockers for the prevention of gastro-intestinal bleeding in ICU patients.


a)  Name the type of graph illustrated in the above figure. (10% marks)

b)  What does it show? (25% marks)

c)  What are the benefits of this type of analysis?  (25% marks)

d)  What are the disadvantages of this analysis? (40% marks)

[Click here to toggle visibility of the answers]

College Answer


Forest plot


Combining the trials together, PPI use results in an odds ratio of 0.35 or reduction in the risk of bleeding compared to H2RA. Alternatively, PPI use results in 65% reduction (1- 0.35) in bleeding.


Combines small studies with limited power, increasing the number and thus the ability to pick up a positive effect. Small studies with low power (due to small effect, small numbers) run the risk of a Type II error.


Individual studies might have different patient populations (with different risk of bleeding) or different definitions of outcome.

Individual studies might have been conducted with different degrees of rigour (blinding, etc.)

There is publication bias to positive studies so that negative studies are not reported. Need full disclosure how the studies were selected, their scientific grading, subgroup analyses and assessment of heterogeneity.


I have no idea whether the college actually used this exact image, but certainly the paper was correctly identified by LITFL. My hat is off to Chris Nickson, who managed to track down the exact PPI vs H2A study which had this exact forest plot and OR / RRR. It was indeed the Alhazzani study from 2013.


a) and b) are reviewed in greater detail in the chapter on forest and box-and-whisker plots. In short:

  • This is a forest plot.
  • The horizontal lines – confidence intervals of the OR
  • The position of the square – point estimate of the OR
  • The size of the square – the weight of the study
  • The vertical line: OR of 1 (no association)
  • If the CI of the summed results crosses the vertical line, the treatment is no more effective than control.
  • This study shows that PPIs are better than H2As in reducing the risk of bleeding.

c) and d)

Advantages of meta-analysis

  • A more objective appraisal of evidence
  • Reduces the probability of false negative results
  • May explain heterogeneity between the results of different studies
  • Avoids Simpson’s paradox, in which a consistent effect in constituent trials is reversed when results are simply pooled.

Disadvantages of meta-analysis

  • Population samples may be too heterogeneous to combine into a summed result
  • The trials may span a large period of time, over which "standard care" for the control group has changed, which means the results can't be summed
  • The validity of the meta-analysis is undermined by the inclusion of studies that were well powered but poorly designed, as the results lack validity independently of the n value
  • Selection of studies may be biased:
    • Language bias, where only English-language studies are selected
    • Publication bias, where only published (i.e. positive) studies are selected
  • Negative studies are rarely published, and unless the authors of unpublished trials were contacted, these may not be included
  • The meta-analysis uses summary data rather than individual data 
  • Heterogeneity in methodology makes it difficult to combine summed outcome results, and metaanalysis only deals with summed results, as a  re-analysis of individual data is usually not performed


Alhazzani, Waleed, et al. "Proton pump inhibitors versus histamine 2 receptor antagonists for stress ulcer prophylaxis in critically ill patients: a systematic review and meta-analysis*." Critical care medicine 41.3 (2013): 693-705.

Methodological Expectations of Cochrane Intervention Reviews

Schriger, David L., et al. "Forest plots in reports of systematic reviews: a cross-sectional study reviewing current practice." International journal of epidemiology39.2 (2010): 421-429.

Lewis, Steff, and Mike Clarke. "Forest plots: trying to see the wood and the trees." Bmj 322.7300 (2001): 1479-1480.

Anzures‐Cabrera, Judith, and Julian Higgins. "Graphical displays for meta‐analysis: An overview with suggestions for practice." Research Synthesis Methods 1.1 (2010): 66-80.

Cochrane: "Considerations and recommendations for 
figures in Cochrane reviews: graphs of statistical data"
 4 December 2003 (updated 27 February 2008)

Reade, Michael C., et al. "Bench-to-bedside review: Avoiding pitfalls in critical care meta-analysis–funnel plots, risk estimates, types of heterogeneity, baseline risk and the ecologic fallacy." Critical Care 12.4 (2008): 220.

DerSimonian, Rebecca, and Nan Laird. "Meta-analysis in clinical trials."Controlled clinical trials 7.3 (1986): 177-188.

Biggerstaff, B. J., and R. L. Tweedie. "Incorporating variability in estimates of heterogeneity in the random effects model in meta-analysis." Statistics in medicine 16.7 (1997): 753-768.

The Cochrane Handbook: 9.5.4 "Incorporating heterogeneity into random-effects model"