Question 7

Angoff score

4.50

Highest mark

8.25

Choose ONE of the following ICU illness severity scoring systems:

1. Australian and New Zealand Risk of Death (ANZROD) or

2. Acute Physiology and Chronic Health Evaluation (APACHE) III/IV

For your chosen scoring system:

a) List the key components. (2 marks)

b) Outline the current applications / usage in intensive care. (5 marks)

c) List the limitations of your chosen illness severity scoring system. (3 marks)

[Click here to toggle visibility of the answers]

College answer

Syllabus topic/section:

2.1.2 Decision Making: Severity scoring and outcome prediction: L1

Discussion:

This question focused on the core topic of ICU scoring systems. We recommend that candidates have good knowledge of the principles and details of common ICU severity scoring systems. The question was generally answered well if candidates took a broad approach.
Part a) required a list of the key components (acute physiological scores, age and chronic health conditions) rather than an extensive list of the individual acute physiological components. Many candidates missed simple marks by not mentioning the 24-hour timeframe for the data acquisition of the physiological scores or the impact of relevant chronic conditions.

Part b) was answered well if candidates provided a detailed outline of several applications rather than a superficial list. Candidates also did well if they used broad headings (safety and quality, clinical or research for example) and then added detail with subheadings to ensure they included a breadth of applications. Please note the glossary terms outline vs list.

Part c) required a simple list, and candidates scored well if they could provide several limitations. The marking rubric is included to aid the candidate’s future study.

	Below standard	At standard	Above standard
a) Key Components	Only included 0-2 components or didn’t recognise that physiology scores are within 24 hours of ICU admission or included incorrect components	Included all 3 components and noted physiology should be within 24 hours. Also included some detail around chronic health conditions or the physiological variables	At standard plus Included all 3 components plus some detail around chronic health conditions and physiological variables. Also included some of the info specific to the chosen scoring system to score full marks
(2 marks)	0-0.5 marks	1 mark	1.5 - 2 marks
b) Current Applications/ Clinical Usage	Recognises a couple of applications with some detail to each OR Superficial knowledge of 3 different areas	Demonstrates sound knowledge of the applications with an understanding of most of the components listed. Does not need to include all 4 areas but understands that they are used in benchmarking, outcome prediction and research	Demonstrates advanced knowledge of applications with good detail. Coverage of ALL applications is not required for advanced marks, but should be a broad range across the categories with good understanding demonstrated
(5 marks)	0-2 marks	2.5-3.5 marks	4-5 marks
c) Limitations	Only a couple of limitations are identified, or incorrect factors are included	A reasonable number of limitations are identified with minimal detail included	Majority of the limitations are identified with some detail included
(3 marks)	0-1 mark	1.5-2 marks	2.5-3 marks

Discussion

We have been here before. Question 11 from the first paper of 2009 asked the candidates to compare APACHE with SOFA, and Question 4 from the second paper of 2005 asked about the principles of scoring systems in general. And then, when examiners asked about scoring systems in Question 18 from the cursed first paper of 2023, whereas the intention was clearly to get answers about APACHE and SOFA, some candidates wrote about the GCS and the Child-Pugh score, which actually answered the question, and had to be marked. The specific choices offered to the candidates in this sitting appear to be a deliberate effort to guide them towards a specific answer.

What follows is a repurposed answer from Question 18. APACHE-IIIj is used here mostly because it is very similar to APACHE IV (only a couple of extra things were added, and the disease-specific coefficients have been updated to change the calculated risk). ANZROD is also basically APACHE IIIj with some adjustments to the weights, to reflect that the American critical care mortality is higher than Australian.

Components of APACHE III-j

Major medical and surgical disease categories
Acute physiologic abnormalities in the first 24 hours (17 variables)
Age
Preexisting functional limitations
Major comorbidities
Treatment location immediately prior to ICU admission

Advantages of APACHE III-j

Of APACHE-III-j specifically:
- Easy to collect data (computerised information systems can calculate the score automatically without much human input)
- Well-validated and internationally familiar
- Makes comparison between health services easier
- Is an improvement on the APACHE-II score, as some of the physiological variables have been re-weighed
Of such scoring systems in general:
- Can be used to standardise quality assurance studies and research
- Can help perform comparisons between health services
- Usually user-friendly and dependent on variables which are already being collected for patient care purposes

Applications of APACHE III-j in the ICU

Used in ICUs around the world to predict mortality for critically ill patients
Permits comparison and benchmarking of critical care services worldwide, including the analysis of endpoints in trials
Allows the standardisation of illness severity scoring, permitting comparison between patient populations in research
- Easy to collect data (computerised information systems can calculate the score automatically without much human input)
- Well-validated and internationally familiar
- Makes comparison between health services easier
- Is an improvement on the APACHE-II score, as some of the physiological variables have been re-weighed
Allows administrators to assess illness acuity in health services and to use this information to determine the allocation of resources and staff
Predicts mortality, which may have implications for decisionmaking

Limitations of APACHE III-j

Of APACHE III-j specifically:
- Does not incorporate frailty (but APACHE-IV does!)
- Older, and drifting in calibration in terms of predictive value (last validated in 2003!)
- Not a sequential score (thus, cannot be used to track response to therapy)
Of these systems in general:
- Generally these are poor predictors of individual patient outcome
- They are susceptible to coding errors, particularly where one variable is subjective (eg. the "diagnosis" category in the APACHE score)
- There is a variation in recording of data - not everyone is equally accurate at filling out the forms, and computerised systems can record spurious readings uncritically
- There are differences in patient groups which influence "illness severity" which are not measured by the scoring system
- Some data goes missing
- Delay to ICU admission affects the initial score
- Outcomes may not be related to ICU alone - the whole hospital is involved

A lot of these advantages and disadvantages were extracted from the excellent 2008 paper by Shann et al.

References

Paul, Eldho, et al. "The ANZROD model: better benchmarking of ICU outcomes and detection of outliers." Critical care and resuscitation 18.1 (2016): 25-36.

Paul, E., et al. "Assessing contemporary intensive care unit outcome: development and validation of the Australian and New Zealand risk of death admission model." Anaesthesia and intensive care 45.3 (2017): 326-343.

[Submit a comment or correction]