a)    What is a Standardised Mortality Ratio (SMR) and how is it calculated?    (20% marks)

b)    The SMR in your ICU has increased from 0.95 to 1.05 in the past 12 months. Outline the possible causes.    (80% marks)
 

[Click here to toggle visibility of the answers]

College answer

a)    Overview of SMR (20% marks)

SMR is one of the quality indicators that reflect the performance of an ICU.
Definition of SMR = ratio of observed deaths in the study group to expected deaths in the general population based on APACHE or other severity of illness
SMR values of 1 indicate expected performance, whereas values below 1 and above 1 indicate respectively better and worse performances than expected
 

b)    Causes for increase (80% marks)

Lower than expected predicted mortality
Errors in predicted/expected mortality due to gaps in data, changes in case-mix etc

Change in data collection systems or personnel – e.g., change in the way the expected mortality is estimated

Lead-time bias (pre-ICU care) – patients transferred from other facilities may have become more stable after receiving appropriate management at the original hospital.

Increases in observed mortality

Based on hospital mortality, not ICU mortality – therefore, influenced by pre-ICU and post ICU care in the hospital

Change in case-mix, so changes in case mix may account for increase in SMR and increased other hospital admissions

One-off events such as mass disasters, epidemics etc

Variations in practice, changes in clinical protocols either in the hospital or in the ICU Changes in personnel – e.g., new intensivist, new surgeon etc

Changes in staffing levels and training

New services introduced such as ECMO etc.

Examiner’s Comments:

The candidates rarely considered the denominator. Often wrote "admitted sicker patients" without considering these likely to also have higher predicted mortality. Rarely any structure.

Discussion

In brief:

  • SMR is the ratio of the observed mortality vs. predicted mortality for a specified time period.
  • The formula is SMR = observed number of deaths / expected number of deaths,  where the expected number of deaths is predicted by an illness severity scoring system
  • One can use this to compare hospitals and ICUs
  • One needs to first calculate the predicted hospital mortality using an illness severity scoring system.
  • An SMR of 1 means the mortality is as expected.
  • An SMR of < 1 is better than expected, and >1 is worse than expected.

Causes for an elevation of the SMR were separated into two categories by the college; either the predicted mortality has dropped, or the actual mortality has increased. Another way of looking at this is whether the SMR elevation is "true", or whether it is spurious, i.e. where the change in SMR is not representative of a change in the quality of care being provided by the ICU. 

  • Spurious elevation of SMR
    • Poor data entry (i.e. true illness severity is not captured by lazy registrars failing to dutifully record every last drop of urine in the APACHE form)
    • "Lead time bias" - treatment received prior to ICU admission may result in artifically normalised acute physiology scores
    • "Healthy worker effect" - a change towards selective ICU admission practices may be favouring patients who score low on illness severity scales, eg. young elective surgical patients
  • True elevation due to internal ICU issues
    • A new staffing model is in place (inexperienced staff)
    • Understaffing has impaired patient care
    • Junior people are not following unfamiliar protocols, or the new protocols are of a poor quality
    • New equipment or technique is less useful than advertised
  • True elevation due to external problems
    • Increase in the pre-hospital morbidity of admitted patients (eg. increased acuity, where you suddenly become a trauma centre or an organ transplant service)
    • Play of chance, eg. mass casualty event 
    • Deterioration of the quality of pre-ICU care
    • Parameters which govern ICU admission have changed (eg. administrative pressure is being placed on the ICU to rapidly admit ED patients who have had little management or workup)
    • Discharge arrangements have changed (eg. a local palliative care ward had shut down, and you keep dying patients in the ICU because it would be insensitive to transfer them to the next nearby palliative care unit)

References

References

Young, Paul, et al. "End points for phase II trials in intensive care: Recommendations from the Australian and New Zealand clinical trials group consensus panel meeting." Critical Care and Resuscitation 15.3 (2013): 211. - this one is not available for free, but the 2012 version still is:

Young, Paul, et al. "End points for phase II trials in intensive care: recommendations from the Australian and New Zealand Clinical Trials Group consensus panel meeting." Critical Care and Resuscitation 14.3 (2012): 211.

Suter, P., et al. "Predicting outcome in ICU patients." Intensive Care Medicine20.5 (1994): 390-397.

Martinez, Elizabeth A., et al. "Identifying Meaningful Outcome Measures for the Intensive Care Unit." American Journal of Medical Quality (2013): 1062860613491823.

Tipping, Claire J., et al. "A systematic review of measurements of physical function in critically ill adults." Critical Care and Resuscitation 14.4 (2012): 302.

Gunning, Kevin, and Kathy Rowan. "Outcome data and scoring systems." Bmj319.7204 (1999): 241-244.

Woodman, Richard, et al. Measuring and reporting mortality in hospital patientsAustralian Institute of Health and Welfare, 2009.

Vincent, J-L. "Is Mortality the Only Outcome Measure in ICU Patients?."Anaesthesia, Pain, Intensive Care and Emergency Medicine—APICE. Springer Milan, 1999. 113-117.

Rosenberg, Andrew L., et al. "Accepting critically ill transfer patients: adverse effect on a referral center's outcome and benchmark measures." Annals of internal medicine 138.11 (2003): 882-890.

Burack, Joshua H., et al. "Public reporting of surgical mortality: a survey of New York State cardiothoracic surgeons." The Annals of thoracic surgery 68.4 (1999): 1195-1200.

Hayes, J. A., et al. "Outcome measures for adult critical care: a systematic review." Health technology assessment (Winchester, England) 4.24 (1999): 1-111.

RUBENFELD, GORDON D., et al. "Outcomes research in critical care: results of the American Thoracic Society critical care assembly workshop on outcomes research." American journal of respiratory and critical care medicine 160.1 (1999): 358-367.

Turnbull, Alison E., et al. "Outcome Measurement in ICU Survivorship Research From 1970 to 2013: A Scoping Review of 425 Publications." Critical care medicine (2016).

Solomon, Patricia J., Jessica Kasza, and John L. Moran. "Identifying unusual performance in Australian and New Zealand intensive care units from 2000 to 2010." BMC medical research methodology 14.1 (2014): 1.

Liddell, F. D. "Simple exact analysis of the standardised mortality ratio." Journal of Epidemiology and Community Health 38.1 (1984): 85-88.

Ben-Tovim, David, et al. "Measuring and reporting mortality in hospital patients." Canberra: Australian Institute of Health and Welfare (2009).

McMichael, Anthony J. "Standardized Mortality Ratios and the'Healthy Worker Effect': Scratching Beneath the Surface." Journal of Occupational and Environmental Medicine 18.3 (1976): 165-168.

Wolfe, Robert A. "The standardized mortality ratio revisited: improvements, innovations, and limitations." American Journal of Kidney Diseases 24.2 (1994): 290-297.

Kramer, Andrew A., Thomas L. Higgins, and Jack E. Zimmerman. "Comparing observed and predicted mortality among ICUs using different prognostic systems: why do performance assessments differ?." Critical care medicine 43.2 (2015): 261-269.

Spiegelhalter, David J. "Funnel plots for comparing institutional performance." Statistics in medicine 24.8 (2005): 1185-1202.

Teres, Daniel. "The value and limits of severity adjusted mortality for ICU patients." Journal of critical care 19.4 (2004): 257-263.