a) What is a Standardised Mortality Ratio (SMR) and how is it calculated? (20% marks)
b) The SMR in your ICU has increased from 0.95 to 1.05 in the past 12 months. Outline the possible causes. (80% marks)
a) Overview of SMR (20% marks)
SMR is one of the quality indicators that reflect the performance of an ICU.
Definition of SMR = ratio of observed deaths in the study group to expected deaths in the general population based on APACHE or other severity of illness
SMR values of 1 indicate expected performance, whereas values below 1 and above 1 indicate respectively better and worse performances than expected
b) Causes for increase (80% marks)
Lower than expected predicted mortality
Errors in predicted/expected mortality due to gaps in data, changes in case-mix etc
Change in data collection systems or personnel – e.g., change in the way the expected mortality is estimated
Lead-time bias (pre-ICU care) – patients transferred from other facilities may have become more stable after receiving appropriate management at the original hospital.
Increases in observed mortality
Based on hospital mortality, not ICU mortality – therefore, influenced by pre-ICU and post ICU care in the hospital
Change in case-mix, so changes in case mix may account for increase in SMR and increased other hospital admissions
One-off events such as mass disasters, epidemics etc
Variations in practice, changes in clinical protocols either in the hospital or in the ICU Changes in personnel – e.g., new intensivist, new surgeon etc
Changes in staffing levels and training
New services introduced such as ECMO etc.
Examiner’s Comments:
The candidates rarely considered the denominator. Often wrote "admitted sicker patients" without considering these likely to also have higher predicted mortality. Rarely any structure.
In brief:
Causes for an elevation of the SMR were separated into two categories by the college; either the predicted mortality has dropped, or the actual mortality has increased. Another way of looking at this is whether the SMR elevation is "true", or whether it is spurious, i.e. where the change in SMR is not representative of a change in the quality of care being provided by the ICU.
Young, Paul, et al. "End points for phase II trials in intensive care: Recommendations from the Australian and New Zealand clinical trials group consensus panel meeting." Critical Care and Resuscitation 15.3 (2013): 211. - this one is not available for free, but the 2012 version still is:
Young, Paul, et al. "End points for phase II trials in intensive care: recommendations from the Australian and New Zealand Clinical Trials Group consensus panel meeting." Critical Care and Resuscitation 14.3 (2012): 211.
Suter, P., et al. "Predicting outcome in ICU patients." Intensive Care Medicine20.5 (1994): 390-397.
Martinez, Elizabeth A., et al. "Identifying Meaningful Outcome Measures for the Intensive Care Unit." American Journal of Medical Quality (2013): 1062860613491823.
Tipping, Claire J., et al. "A systematic review of measurements of physical function in critically ill adults." Critical Care and Resuscitation 14.4 (2012): 302.
Gunning, Kevin, and Kathy Rowan. "Outcome data and scoring systems." Bmj319.7204 (1999): 241-244.
Woodman, Richard, et al. Measuring and reporting mortality in hospital patients. Australian Institute of Health and Welfare, 2009.
Vincent, J-L. "Is Mortality the Only Outcome Measure in ICU Patients?."Anaesthesia, Pain, Intensive Care and Emergency Medicine—APICE. Springer Milan, 1999. 113-117.
Rosenberg, Andrew L., et al. "Accepting critically ill transfer patients: adverse effect on a referral center's outcome and benchmark measures." Annals of internal medicine 138.11 (2003): 882-890.
Burack, Joshua H., et al. "Public reporting of surgical mortality: a survey of New York State cardiothoracic surgeons." The Annals of thoracic surgery 68.4 (1999): 1195-1200.
Hayes, J. A., et al. "Outcome measures for adult critical care: a systematic review." Health technology assessment (Winchester, England) 4.24 (1999): 1-111.
RUBENFELD, GORDON D., et al. "Outcomes research in critical care: results of the American Thoracic Society critical care assembly workshop on outcomes research." American journal of respiratory and critical care medicine 160.1 (1999): 358-367.
Turnbull, Alison E., et al. "Outcome Measurement in ICU Survivorship Research From 1970 to 2013: A Scoping Review of 425 Publications." Critical care medicine (2016).
Solomon, Patricia J., Jessica Kasza, and John L. Moran. "Identifying unusual performance in Australian and New Zealand intensive care units from 2000 to 2010." BMC medical research methodology 14.1 (2014): 1.
Liddell, F. D. "Simple exact analysis of the standardised mortality ratio." Journal of Epidemiology and Community Health 38.1 (1984): 85-88.
Ben-Tovim, David, et al. "Measuring and reporting mortality in hospital patients." Canberra: Australian Institute of Health and Welfare (2009).
McMichael, Anthony J. "Standardized Mortality Ratios and the'Healthy Worker Effect': Scratching Beneath the Surface." Journal of Occupational and Environmental Medicine 18.3 (1976): 165-168.
Wolfe, Robert A. "The standardized mortality ratio revisited: improvements, innovations, and limitations." American Journal of Kidney Diseases 24.2 (1994): 290-297.
Kramer, Andrew A., Thomas L. Higgins, and Jack E. Zimmerman. "Comparing observed and predicted mortality among ICUs using different prognostic systems: why do performance assessments differ?." Critical care medicine 43.2 (2015): 261-269.
Spiegelhalter, David J. "Funnel plots for comparing institutional performance." Statistics in medicine 24.8 (2005): 1185-1202.
Teres, Daniel. "The value and limits of severity adjusted mortality for ICU patients." Journal of critical care 19.4 (2004): 257-263.