Critique of obsolete definitions for sepsis and SIRS

 

[toc]

This chapter discusses the now-defunct  "Sepsis 2" criteria, as well as the series of criteria which went beofre them. It is largely of historical interest, and is therefore non-essential for the struggling exam candidate. New "Sepsis 3" criteria are discussed in a dedicated chapter, and being the hot new thing they will likely form a sadistic SAQ in one of the upcoming fellowship exam papers. An excellent overview of these old criteria be found at LITFL, where Chris Nickson dissects both the old and the new definitions. Published articles which were used to construct this summary included:

• Beesley et al, 2015 ("Why we need a new definition of sepsis ")
• Kaukonen et al (2015), a massive retrospective study of SIRS validity
• P.Marik, 2002 - "Not quite time to dump SIRS?"
• Gaieski et al (2014) - a highly critical expert review of SIRS
• Shankar-Hari et al (2015) - "Judging quality of current septic shock definitions and criteria" - this could easily be the sole reference for this topic, and the time poor candidate would wisely limit their reading to this single article.

Rationale for establishing definitions for sepsis

Theoretical arguments for the utility of strict definitions

• Theoretical medicine requires strict definitions to separate patients into a "disease" group, to compare them to a "no disease" group
• Standardisation of definitions is required to ensure a common language, to facilitate communication between clinicans and researchers
• Standard definitions allow comparative evaluation of different therapies; without such definitions comparison between studies and their meta-analysis would be difficult.

Pragmatic arguments for defining sepsis

• Sepsis is a disease entity with a significant mortality, and this mortality decreases with early detection and treatment.
• Ergo, some of this mortality is due to a failure of early detection, and early detection would be facilitated by a set of diagnostic criteria which can be used to identify sepsis

Arguments against strict definitions

• In Australia, nobody uses these definitions to make a diagnosis of sepsis, at least not obviously.
• In Australia, mortality from sepsis and septic shock seems to be comparable to (or better than) much of the world, which implies that we dont need no stinking definitions.
• Changing the definitions frequently and erratically makes it difficult to compare modern research to older research which used old definitions.
• Why did it take so long to create definitions? Prior to the advent of "modern" intensive care, the management of sepsis had consisted only of antibiotics and supportive care, which trivialised the need for precise definitions. During the era contemporary to Bone et al (1992), there was a surge of interest in novel therapies for sepsis, including steroids, monoclonal antibodies, high-flux dialysis, and various others. Now that these therapies have been discredited, one could make the argument that we again have no need for precise definitions (i.e. we are back to antibiotics and FASTHUG).

History of the attempts ot define sepsis.

The original (Sepsis-1) definitions came about following the ACCP-SCCM conference of 1991 (Bone  et al, 1992). The criteria to define SIRS and sepsis were also discussed in a more detailed manner by Bone et al in the same issue of Chest.

How did they come up with these definitions?

The authors offered a series of definitions, which were propagated from the early 1990s into the present day (see the old Surviving Sepsis definitions from 2015). We are grateful to Dr Lawrence Lynn, who linked his own 2014 paper in a comment to Chris Nickson's LITFL article. The article points out some significant flaws in the foundations of these early definitions. In summary, the definitions arose as the result of a perceived need for consensus, rather than on the basis of any sort of emerging scientific reseach. No new discovery or breakthrough was made; instead a bunch of people got together at a conference and decided to come up with a unified position, mainly because previously there wasn't one.

Lynn goes on to discuss the flaws of threshold decision making (Pauker et al, 1980), the seemingly logical technique of assigning a significant value to a laboratory test which identidies the patient as having the disease and requiring treatment. Recognised field leaders made guesses as to what those thresholds might be according to heterogenous trial data, and combined them into an agreed-upon package, hoping that the effect of having multiple cross-disciplinary experts would somehow enhance the validity of the guesswork. Obviously, there were several problems with the end product.

Old (Sepsis 2)  definitions of sepsis and SIRS

The chapter "Severe sepsis" by A Raffaele De Gaudio (Ch. 69 from Oh's Manual)  presents a nice blue box (Box 69.1 on page 717) listing all the various internationally agreed-upon criteria for sepsis contemporary to that edition of the manual. The content of this box is based heavily on the reported conclusions of a 2001 consensus conference. Though the 2001 consensus criteria are more extensive than Oh's Box 69.1, the Box still supercedes international published criteria because:

• It is in Oh's Manual.
• The Manual is written by CICM examiners.
• CICM examiners determine which responses are marked "correct" in the fellowship exam.

Thus, Box 69.1 is definitive. Here it is, with minimal modification:

2001 Consensus Diagnostic Criteria for Sepsis

SIRS criteria Heart rate >90 Temperature <36.0 or >38.3°C Blood glucose >7.7 mmol/L Resp rate >20/min WCC <4 or >12 Acutely altered mental state

Organ dysfunction SBP <90... or MAP < 65 or lactate > 2.0 mmol/L (after initial flid challenge) INR >1.5 or a PTT >60 s Bilirubin >34 µmol/L Urine output <0.5 mL/kg/h for 2 h Creatinine >177 µmol/L Platelets <100 ×109/L SpO2 <90% on room air

Criteria and mortality Sepsis: SIRS + infection mortality: 10-15% Severe sepsis: sepsis + organ failure mortality: 17-20% Septic shock: severe sepsis + hypotension mortality: 43-54%

For completeness, there is the (essentially identical) definition quoted by the  Surviving Sepsis campaign:

What can be said in support of the old definitions

• Better than nothing: Prior to the first definition, there were no definitions whatsoever, and the quality of sepsis research suffered as a consequence. Why were there no definitions before? Bone et al (1992) explain:
  • Sepsis and MOSF were relatively new entities in the 1980s and 90s: prior to that, septic shock patients just died. These days we are able to keep patients alive for longer.
  • Previously, studies of sepsis focused on bacteraemia, which is not found in a majority of patients (we now realise)
  • Many of the older studies were performed on surgical or trauma patients, which cannot be generalised to the bulk of the septic population
• Decent sensitivity: SIRS criteria (presence of 2) were at first found to have good sensitivity, which supported their ongoing use. In 2006, an analysis of SOAP study data (Sprung et al, 2006) found that t he frequency of having three or four SIRS criteria vs. two was higher in infected than non-infected patients, and that organ system failure and mortality increased as the number of SIRS criteria increased. Modern studies which have cast doubt on the validity of the old SIRS criteria have their own limitations (Beesley et al, 2015). More on that later.
• Simplicity: SIRS criteria are cardinal vital signs, collected at every triage window and monitored continuously in critically ill patients - this forms a part of their appeal (i.e. they are not esoteric biomarkers or difficult-to-find physical signs).
• Feasibility: SIRS criteria offer a compromise between validity and reliability. By reliably over-diagnosing sepsis, they have the potential to prevent mortality (which is important in high-mortality conditions such as sepsis). A high false positive rate should probably be tolerated because missed cases have a high likelihood of ending up dead.

Disadvantages of the old definitions

This section was best informed by the excellent review article from Shankar-Hari et al (2015), using the 2015 retrospective audit by Kaukonen et al as the main source of anti-SIRS-criteria evidence. In summary, the problems of the old SIRS-based definition are as follows:

Criticism of the process of criteria design

• The parameters are arbitrary. In fact, the cut-offs (eg. a heart rate of 90, a white cell count of 10) were arrived at in a fairly haphazard manner, probably in some smoke-filled room full of senior intensivsists. Certainly, none of the cut-offs have any sort of physiological significance. Nothing happens at a heart rate of 90 that was not happening at a heart rate of 85.
• No attempt to reflect disease severity: the odds ratio for mortality increases incrementally with additional SIRS criteria (from 1, to 2, to 3, to 4) without a step up when the definition of SIRS is met (i.e. once you meet 2 criteria, you do not suddenly enter a new prognostic category). But then again, SIRS was not designed to measure illness severity. It was designed to be exquisitely sensitive in not missing patients with sepsis.
• No attempt to account for progression of disease: SIRS criteria do not account for the dynamic time-course of sepsis (e.g. rise and fall in white cell count over time, fluctuations in vital signs, etc). One might reasonably expect a patent with raidly improving sepsis (eg. urosepsis) to be in a different prognostic category to the patient with an infected pancreatic pseudocyst, but the SIRS criteria would lump them all together.

Lack of sensitivity

• Sensitivity is the ability of a test to recognize true positives. The SIRS criteria consistently fail to recognise truly septic patients.
• 1 in 8 ICU patients with infection and organ dysfunction did not have 2 or more SIRS criteria in the 2015 retrospective audit by Kaukonen et al.
• As a result, septic patients who do not meet criteria may have their treatment delayed (increasing their mortality). For example, such patiens would not have been enrolled into the Rivers' EGDT trial.

Lack of specificity

• Specificity is the ability of a test to recognize true negatives. The SIRS criteria persistently fails to identify truly non-septic patients.
• 4 in 5 ICU patients without infection had ‘SIRS’ criteria in the 2015 retrospective audit by Kaukonen et al. Of course they were all tachycardic and tachypnoeic, that's critical illness for you. Doesn't help the diagnosis of sepsis though. J.L Vincent (1997) famously pointed out that he could satisfy the SIRS criteria merely by going out for a jog. This is not news, people.
• As a result, non-septic patients may receive inappropriate antibiotics (also delaying definitive management of their aseptic SIRS).

Lack of face validity

• "Face validity" is the "transparency or relevance of a test as it appears to test participants ", i.e.,  the test looks like it is going to measure what it’s supposed to measure. This parameter describes the relationship between the SIRS criteria and what "looks septic" in the steely eyes of an intensive care veteran.
• This form of validity is clearly lacking, as the SIRS criteria were frequently abandoned by clinicians in the Kaukonen study (2015). Patients identified by clinicians as having sepsis were defined as sepsis-negative by the criteria. The abovementioned veterans easily recognised that it is possible to be shocked without hypotension, and to be hypotensive without shock. The criteria did not.
• The Kaukonen study will probably do little to change practice, because (as it appears) most people don't use the SIRS criteria in the diagnosis of sepsis anyway.

Failure to accurately describe the disease process

• A major criticism is that the criteria failed to identify the most important issue - that a key feature of sepsis is a host response to infection that is maladaptive, damaging the host’s own organs and tissues. This represents a descriptive failure of the definition, i.e. it does not incorporate any components which might accurately describe the underlying disease process. This is probably why the SIRS criteria don't look like they measure what they are supposed to measure.

Lack of content validity

• Content validity assesses whether the clinical criteria encompass all components of the illness.
• Sepsis has three major components:
  • Cardiovascular dysfunction
  • Hypoperfusion
  • Cellular dysfunction
• The SIRS criteria fail to account for any of these except for cardiovascular dysfunction (by way of hypotension). Where is lactate? Where is oliguria, acidosis, increased or decreased oxygen extraction ratio, cardiomyopathy, delirium?

Lack of  concurrent validity

• Concurrent validity would ordinarily refer to the extent to which the application of these SIRS and sepsis criteria correspond to some sort of previously established gold standard for sepsis. It is difficult to determine the concurrent valididy of the SIRS criteria because there is no such gold standard for sepsis, but generally people have used such unequivocally sepsis-y findings as bacteraemia and organ dysfunction. In relation to the SIRS criteria, concurrent validity refers more to the clinical effectiveness of the criteria in discriminating patients into groups, eg. septic vs non-septic.
• SIRS criteria do not discriminate patients into such discrete groups. Because of the poor sensitivity and specificity, there is significant overlap between SIRS and non-SIRS patients in terms of any measurable parameters, be they mortality, organ dysfunction, presence of bacteraemia, etc. As the constantly referred-to Kaukonen study (2015) has shown, 1 in 8 patients admitted to critical care units in Australia and New Zealand with infection and new organ failure did not have the requisite minimum of 2 SIRS criteria to fulfill the definition of sepsis, but went on to have sepsis-like illnesses.

Lack of predictive validity

• Predictive validity is the ability to predict outcomes. SIRS and sepsis criteria lack such predictive validity. For esample, a 2014 cohort study (Leligdowicz et al) found that among their 8000 septic shock patients, the mortality was better predicted by anatomical site of infection than the SIRS criteria. Of course different sources of infection are associated with different mortality rates - the patient with a urinary tract infection will fare beter than the patient with necrotising fasciitis.

Lack of construct validity

• Construct validity indicates the degree to which a test measures what it purports to measure. i.e. how well the definitions are converted into measurable criteria to identify septic shock in clinical practice. Construct validity consists of two main domains, discriminant validity and convergent validity.
• Discriminant validity  refers to how well the criteria correspond to conditions which are unrelated to sepsis, i.e. whether or not the criteria correlate with tests for totally unrelated physiological conditions. A test with good discriminant validity would not correlate with other tests which look for something totally unrelated. This is plainly failed by the SIRS criteria, which correlate just as easily with haemorrhagic shock, cardiogenic shock, and even vigorous exercise.
• Convergent validity refers to how well the criteria correlate with other tests for the presence of sepsis. A system of sepsis criteria with good convergent validity would prospectively agree with the clinician's opinion, presence of bacteraemia, high procalcitonin, raised CRP, and objective (eg. autopsy) findings of a source of sepsis. This is also failed by the SIRS criteria.

New definitions for sepsis (Sepsis-3)

Advantages and disadvantages of these new definitions are discussed in the next chapter: Critique of the Modern Definitions of Sepsis.