What is a Standardised Mortality Ratio? What are the limitations of using this ratio to compare the performance of Intensive Care Units?
College Answer
Standardised Mortality Ratio is defined as the observed mortality rate/expected mortality rate. Need to estimate expected mortality rate using a scoring system (eg. APACHE II or III, SAPS II or MPM). Better than comparison of non-adjusted mortality data.
The potential limitations of the system are multiple including: inconsistencies and inaccuracies associated with collection of data and scoring (eg. GCS, recording of parameters); problems of missing data limiting inclusion of all patients; problems of patient mix not adequately accounted for by the original population used for calculation of formulae (eg. transferred patients or delays before admission); small numbers of patients (increasing the error of the SMR estimate); accuracy of the prediction model; relying on mortality as a surrogate marker for quality of care; cost of use of proprietary system; etc.
Discussion
This question closely resembles Question 30 from the second paper of 2006.
Definition of the SMR
- This is the ratio of the observed hospital mortality vs. predicted hospital mortality for a specified time period.
- One can use this to compare hospitals and ICUs
- One needs to first calculate the predicted hospital mortality using an illness severity scoring system.
- An SMR of 1 means the mortality is as expected.
- An SMR of < 1 is better than expected, and >1 is worse than expected.
Limitations of the SMR
- Acceptable deviations from the SMR are not defined
- Suffers from inaccuracies associated with data collection
- SMR may be influenced by ICU admission and discharge practices (eg. discharging patients who are palliated, or admitting patients who are inevitably going to die).
- Accuracy of the SMR as a quality assessment tool may be influenced by patients who have been predominantly cared for at another ICU, and who have been received as a transfer.
- Mortality is not a surrogate for quality of care
- The populations used to calculate the predicted hospital mortality are potentially non-representative (i.e. the population may also contains a number of dying critically ill patients, or it may contain an unusually large proportion of people in robust health).
Limitations of comparing ICUs with the SMR:
- The SMR assumes all pre-ICU care is identical
- Ignores differences in case mix
- Sample sizes need to be large enough to obey the laws of logistic regression
- Data is assumed to be flawless and complete
References
Liddell, F. D. "Simple exact analysis of the standardised mortality ratio."Journal of Epidemiology and Community Health 38.1 (1984): 85-88.
Wolfe, Robert A. "The standardized mortality ratio revisited: improvements, innovations, and limitations." American Journal of Kidney Diseases 24.2 (1994): 290-297.
Gaffey, William R. "A critique of the standardized mortality ratio." Journal of Occupational and Environmental Medicine 18.3 (1976): 157-160.
Jones, Michael E., and Anthony J. Swerdlow. "Bias in the standardized mortality ratio when using general population rates to estimate expected number of deaths." American journal of epidemiology 148.10 (1998): 1012-1017.
van Gestel, Yvette RBM, et al. "The hospital standardized mortality ratio fallacy: a narrative review." Medical care 50.8 (2012): 662-667.
Combes, Alain, et al. "Adverse effect on a referral intensive care unit's performance of accepting patients transferred from another intensive care unit*."Critical care medicine 33.4 (2005): 705-710.