Question 25

With reference to intensive care outcomes, discuss the advantages and limitations of each of the following endpoints as a measure of quality of care:

ICU mortality
Hospital mortality
90 day mortality
1 year functional outcome

[Click here to toggle visibility of the answers]

College Answer

ICU mortality

Advantages:

simple, single metric
concrete endpoint which is already available in hospital databases
death is an important endpoint
aggregation of a large number of diagnoses with a small number in each increases power to detect variation
variation over time may reflect institutional and organisational events or characteristics- budget cuts, bed pressure etc. and be able to detect true quality deficiencies
· May be useful when combined as part of an overall quality program

Disadvantages:

Definition of ICU is very hospital specific which can influence mortality (e.g. non-ICU stepdown areas in some hospitals)
As a consequence can be ‘gamed’ e.g. transfers out to die in the ward or other units
Poor correlation between mortality and quality of care in some diagnoses - alternatives available e.g. diagnosis specific risk models e.g. EuroSCORE for CABG, APACHE SMR, trauma scores. Can mask problems in low volume diagnostic groups
Difficult to draw hospital comparisons and or allow league table construction
False conclusions can be drawn unless robust statistical methods used

Hospital Mortality

Compared to ICU mortality, avoids many problems of censoring at ICU discharge. Hospital mortality can often be 50% higher than ICU mortality, and is a reasonable surrogate (90%) for 90 days mortality.

Advantages – gets over differences in definition of ICU and ICU discharge thresholds. Still a simple and robust endpoint which is easy to obtain from exisiting hospital databases.

Disadvantages – can confound intensive care outcomes with deficiencies in ward or other post ICU care. Does not address in any way functional outcomes [so discharge from hospital to a nursing home in a vegetative state is counted as a positive outcome]

90 day mortality
Simple robust endpoint which addresses the issue of ongoing mortality after hospital discharge (though this difference is about 10% relative in recent large trials).

Advantages – simple robust endpoint; data may be available by linkage with external registries (e.g. Births, Deaths and Marriages)

Disadvantages – still an arbitrary time point [while 28 days is clearly inadequate, 90 days may still be insufficient to accurately measure the attributable mortality from an episode of critical illness]. Problems with loss to follow up after ICU discharge. Ethical implications of contacting patients after discharge (especially for research studies).

1 year functional outcome

Advantages – a ‘POEM’ – (patient oriented endpoint that matters). Takes into account disability and true long-term consequences of critical illness.

Disadvantages – no ideal soring tool available – existing tools all have problems; some measure particular functional domains well; problems with face validity. All functional outcome measures are time consuming to apply. Problems with loss to follow up. Time consuming, labour intensive, costly face to face vs. phone vs. mail follow up. Depending on disease may reflect more the natural history of the disease rather than the ICU care per se.

Discussion

The above answer is used as a backbone of the LITFL article on this topic. A good discussion of "at what point do we measure mortality" can be found at the AIHW statement on Measuring and reporting mortality in hospital patients (page 6, ch. 2.4.2).

As a good guideline for other ICU outcome measures which should be used to assess the outcomes of Phase II trials, ANZICS has published this statement in 2012 (the 2013 update is for some reason not available for free).

A good discussion of ICU outcome measures and some of their limitations can also be found in the1999 workshop publication of the American Thoracic Society.

A question like this probably lends itself better to a tabulated answer.

A Comparison of Outcome Measures in Intensive Care Research
Outcome measure	Advantages	Disadvantages
ICU mortality	Mortality is simple and cheap to measure It is an important outcome measure It is already being recorded in hospital databases It can be used to track the performance of an ICU, as it may detect true deficiencies in quality of care	The definition of "ICU" is different across different hospitals ICU mortality neglects the influence of pre-hospital and emergency medical care on mortality Perimortem patients can be discharged from the ICU before they die, thus "shifting" the statistics out into the hospital wards. Selection of low-risk patients in order to improve the statistics for mortality is known among cardiac surgeons. Conversely, critically ill peri-mortem patients can be transferred to the ICU, increasing ICU mortality, thus shifting the mortality statistic into the ICU. This is called "transfer bias". Mortality does not necessarily equate with quality of care - some patients receive good-quality appropriate palliation in ICU
Hospital mortality	Avoids the statistic-skewing practice of discharging palliated patients out of ICU Avoids the problem posed by different definitions of what an "ICU" is. Reflects the performance of the whole hospital, rather than just the ICU Reasonable surrogate for 90day mortality	Many effects of hospital care on mortality do not become evident until after discharge from hospital Like ICUs, hospitals may discharge poor prognosis patients home, thus reducing in-hospital mortality artificially Hospital mortality as a measure of ICU care quality brings in confounders- ward care might negatively influence outcomes after ICU discharge Mortality is not a surrogate for functional outcome - hospitals may discharge patients who are alive, but who are in a state of severe functional impairment (eg. persistent vegetative state).
90-day mortality	Avoids the statistic-skewing practice of discharging palliated patients out of ICU and out of hospital Easy to measure through the record of births and deaths	The 90 day timeline is completely arbitrary 90 days may not be an adequate duration during which the full effects of ICU and hospital care manifest themselves Some patients may be lost to follow-up Confounders such as quality of home care and community follow-up are introduced, which affect mortality
1-year functional outcome	Patient-centered outcome measure (i.e. it matters to the patients) A more accurate estimate of the long-term health cost of critical illness	Scoring systems of functional outcome are not without their flaws Functional outcome scores may score some functional domains better than others, and broadly speaking they all have poor validity. Much of the time focus is on respiratory and cardiovascular function surrogate measures (such as exercise tolerance and FEV₁) Some patients may be lost to follow-up This sort of data collection is neither cheap not easy This is an invasive data collection technique - patients need to be contacted 1 year after their diascharge, which may be an unethical invasion of their privacy for the purposes of research The natural history of the disease acts as a confounder, as it may influence functional outcome. The influence of ICU care and hospital care may become obscured by the progression of the disease.

References

Caron, Guy, et al. "Submental endotracheal intubation: an alternative to tracheotomy in patients with midfacial and panfacial fractures." Journal of Trauma and Acute Care Surgery 48.2 (2000): 235-240.

Young, Paul, et al. "End points for phase II trials in intensive care: Recommendations from the Australian and New Zealand clinical trials group consensus panel meeting." Critical Care and Resuscitation 15.3 (2013): 211. - this one is not available for free, but the 2012 version still is:

Young, Paul, et al. "End points for phase II trials in intensive care: recommendations from the Australian and New Zealand Clinical Trials Group consensus panel meeting." Critical Care and Resuscitation 14.3 (2012): 211.

Suter, P., et al. "Predicting outcome in ICU patients." Intensive Care Medicine20.5 (1994): 390-397.

Martinez, Elizabeth A., et al. "Identifying Meaningful Outcome Measures for the Intensive Care Unit." American Journal of Medical Quality (2013): 1062860613491823.

Tipping, Claire J., et al. "A systematic review of measurements of physical function in critically ill adults." Critical Care and Resuscitation 14.4 (2012): 302.

Gunning, Kevin, and Kathy Rowan. "Outcome data and scoring systems." Bmj319.7204 (1999): 241-244.

Woodman, Richard, et al. Measuring and reporting mortality in hospital patients. Australian Institute of Health and Welfare, 2009.

Vincent, J-L. "Is Mortality the Only Outcome Measure in ICU Patients?."Anaesthesia, Pain, Intensive Care and Emergency Medicine—APICE. Springer Milan, 1999. 113-117.

Rosenberg, Andrew L., et al. "Accepting critically ill transfer patients: adverse effect on a referral center's outcome and benchmark measures." Annals of internal medicine 138.11 (2003): 882-890.

Burack, Joshua H., et al. "Public reporting of surgical mortality: a survey of New York State cardiothoracic surgeons." The Annals of thoracic surgery 68.4 (1999): 1195-1200.

Hayes, J. A., et al. "Outcome measures for adult critical care: a systematic review." Health technology assessment (Winchester, England) 4.24 (1999): 1-111.

RUBENFELD, GORDON D., et al. "Outcomes research in critical care: results of the American Thoracic Society critical care assembly workshop on outcomes research." American journal of respiratory and critical care medicine 160.1 (1999): 358-367.

[Submit a comment or correction]