Critique of the modern definitions for sepsis (Sepsis-III)

 

" I can't define obscenity, but I know it when I see it."

- United States Supreme Court Justice Potter Stewart, 1964

The topic of defining sepsis and SIRS has finally appeared in the CICM Part II as Question 22 from the second paper of 2021. The new definition is based on organ dysfunction rather than old laboratory definitions of SIRS such as fever and leukocytosis. Much reliance is upon the SOFA organ dysfunction scoring system, and its qSOFA abbreviation. The new criteria for sepsis are a vast improvement over the old SIRS paradigm, which was similar in validity to the classical Greek meaning of the word ("a range of processes characterised by putrefaction and a foul smell").

An excellent overview can also be found at LITFL, where Chris Nickson dissects the old and new definitions. The time-poor candidate is directed there, for rapid revision. This chapter does to aim to supercede that resource, but rather acts as a series of footnotes to it. Because of the novelty of this issue, no RCT-level evidence exists at this stage, and the published resources used to create this summary are composed largely of the following opinionated editorials:

• Singer et al (2016) - the actual Sepsis 3 definitions
• Seymour et al (2016) - how they created their clinical criteria for sepsis
• Shankar-Hari (2016) - how they created their clinical criteria for septic shock
• J.Marshall (2016) - "What is the meaning of a definition" (Crit Care Med)
• S.Q.Simpson (2016) - "A change we should not make" (CHEST)

FOAM multiverse resources beyond LITFL are well prepresented by the piece in PulmCrit ("Top ten problems with the new sepsis definition"). At the end of the article is a list of resources which includes other notable FOAM blog entries (Rebel EM,  FOAMcast, St Elmyn’s) as well as statements from groups opposing Sepsis-3.

Wherever possible, and operating within the intellectual limitations of this author, these resources have been mixed and distilled into the summary offered here.

New (Sepsis 3) definitions of sepsis

Given how much the old definitions sucked, ESICM and SCCM assembled a Justice League style assault team of 19 specialists and launched them into the task of repairing the definitions of sepsis. "Unrestricted funding support" was granted to this crack squad, so that they could meet face-to-face four times between January 2014 and January 2015.  At the end of these meetings, these people produced a document which redefines sepsis and septic shock for the 21st century.

"Sepsis-3" is what they called it.

Major barriers identified by the team were:

• How the hell do you achieve concurrent validity in the absence of a gold standard? Typically a set of clinical diagnostic criteria is measured according to its correlation with another, better test. In this case, there is no such test- no single diagnostic parameter can be used to validate the sepsis definitions. Faced with this, the ESICM/SCCM illuminati abandoned the concept altogether, and went for a definition which  "fulfilled multiple domains of usefulness" instead. Presumably, the gold standard of diagnosis in sepsis is a senior intensivist who looks the patient over and says "hmm, looks septic". In that sense, "useful" criteria are those which help lesser practitioners achieve the same diagnostic acumen.
• Sepsis is protean: the manifestations of sepsis from different organisms and at different anatomical sources among different patients with different backgrounds is so varied that it is virtually impossible to create a definition which would recognize every possible variant. A broad and necessarily vague definition would be required, and false positives would have to be tolerated.

How these definitions were generated

A detailed description of what exactly happened can be seen in Seymour et al (2016). In brief, the process involved trawling though 1.3 million patient encounters from twelve large hospitals in southwest Pennsylvania. Out of these 1.3 million, 148 907 patients had "suspected infection" (i.e. somebody took blood cultures and gave antibiotics). Then, the predictive validity of both SIRS and SOFA were tested against records of hospital and ICU mortality, as well as length of ICU stay.

Specifically, the team found that in a population of "suspected infection" patients, a SOFA score increase by > 2 points was associated with a 10% overall mortality risk. The criteria-makers went on to compare this with STEMI (an 8.4% overall mortality risk) to point out how dangerous and scary this sepsis thing really is.

To simplify things for the out-of-ICU environment, an abbreviated scoring system was established. The group looked at which combination of variables had the greatest predictive value for mortality, and by means of a complex Bayesian method arrived at respiratory rate, altered mentation and a low-ish systolic blood pressure. In this manner, the qSOFA scoring system was created.

As for septic shock definitions, they were arrived at by a fairly democratic process (Shankar-Hari, 2016). The committee used the Delphi method to arrive at a consensus list of definitions, which (by majority of 11 out of 19 members) ended up being lactate, a MAP under 65 and the need for vasopressors. Then they went deep into the Surviving Sepsis international registry database and extracted 18840 patients who met these criteria as well as the old  SIRS definitions.  They found that the combination of hypotension and  lactate were more predictive of death (mortality ~ 40%) than each alone (mortality around 20%-ish).

As far as the "definition" of sepsis goes, the new wording ("organ dysfunction",  "dysregulated host response" etc) has much better face validity, as the new clinical criteria measure organ dysfunction (i.e. the test looks like it measures what it is supposed to measure). The definition now better represents the current (incomplete) understanding of this disease process.

Advantages of the new definitions

Advantages of SOFA

• Well-validated scoring system
• Well known within the critical care community
• Can be scored retrospectively and is susceptible to automated scoring
• Superior to SIRS (among the population of south-west Pennsylvania):  AUROC= 0.74, as compared to SIRS (AUROC=0.64). The population investigated was narrow and thoroughly First-world, but one must agree that these were basically normal human organisms, genetically and phenotypically similar to all other human organisms insofar as their response to infection goes.

Advantages of qSOFA

• Simple to apply
• Variables are cardinal vital signs: already being collected routinely
• Validated on the basis of the new Seymour et al (2016) data set.
• Better than SOFA in the out-of-ICU environment (AUROC=0.81 vs AUROC=0.74)

Integration into published "care bundles"

• The Surviving Sepsis Campaign has published a response to the Sepsis-3 definition. In short, the response took the form of complete uncritical support.

Disadvantages of the new definitions

Limitations of SOFA

• Use of SOFA and qSOFA is questionable: these are not tests for sepsis, but rather mortality predictors which are totally unrelated to sepsis.
• The cut-offs for organ dysfunction scores were arrived at arbitrarily. For the most, they do not represent any genuine physiologically relevant thresholds.
• Several components (eg. bilirubin) require laboratory testing and may not be immediately available. Waiting for a patient to satisfy the SOFA criteria for sepsis could deay lifesaving treatment. In contrast, SIRS criteria are immediately available (as they consist of vital signs).
• SOFA is not well known outside of the ICU community, which limits the utility of the new sepsis criteria (how is an ED physician supposed to apply these criteria without knowing what the SOFA score is?). The answer to this problem was supposed to be the qSOFA  abbreviated scoring system.
• Outside the ICU, Seymour et al (2016) found SOFA was only as good as the old crappy SIRS (AUROC=0.79 vs. AUROC= 0.76).\
• Neither the SOFA nor the qSOFA make any attempt to distinguish between new and pre-existing organ dysfunction (and everybody would probably agree that new creatinine rise to over 300  is more important than a long-standing history of the same).
• Hilariously, the process of validating SOFA for the new definition of sepsis can be decribed as a non-sequitur. Seymour et al used a retrospective analysis to show that the  presence of organ dysfunction (SOFA) detects life-threatening organ dysfunction (sepsis). "The logic of this approach, in terms of saving lives, is not evident"  complains Steven Simpson.

Limitations of qSOFA

• Similar in predictive validity to SIRS in the in-ICU environment (AUROC=0.66 vs. AUROC =0.64)
• The qSOFA system sacrificed predictive validity to simplicity (for example, the Bayesian algorithm used to nominate the diagnostic criteria was designed to punish solutions with too many criteria). A more complex system would have had a better predictive validity.
• qSOFA lacks concurrent validity. In comparison with other well-validated mortality-predicting scoring systems, it grossly overestimates mortality. PulmCrit give one example where it is compared with the CURB65 score for pneumonia. A patient with a CURB65 score of 0 (i.e. send home with oral antibiotics) would qualify for sepsis and ICU admission according to the qSOFA score. 

Limitations of the criteria design process

• Delphi method of consensus-building is designed to obliterate outlier opinions by encouraging anonymously polled experts to repeatedly revise their opinions so as to fall in line with the opinions of the group. This encourages a trend towards mediocrity.
• There was significant areas of disagreement within this team of experts. Specifically, not everybody agreed about lactate. Some wanted lactate to be a part of the sepsis criteria ("cryptic shock", they called it). Others wanted lactate removed from the septic shock criteria (not specific enough for sepsis, they said).

Limitations of the overall criteria

• These criteria were validated in the US health system, localised to a tiny nest of WASPs. It is unclear how these criteria can be extrapolated to the local environment. Resource poor areas will have no access to lactate measurement and no way to score SOFA without biochemistry. Most sepsis patients live and die outside of major metropolitan hospitals. Should the rural and regional providers have to rely on the old defunct criteria?
• The criteria-makers only looked at patients in whom infection was either suspected or documented (i.e cultures sent, antibiotics given). There is obviously a whole other population of septic patient sin whom sepsis is not even remotely considered until it is too late.
• The whole "suspected infection" criterion was suspiciously subjective - one might suspect infection in a whole range of situations, depending on the prevailing levels of paranoia. That level seems to be quite high, as the guidelines recommend infection be suspected in any unexplained dysfunction of any organ.
• Just like with the old SIRS criteria, there is no accomodation for the dynamic course of sepsis. The patient may go on to develop a more severe organ dysfunction later in the course of their disease, or they may arrive with severe organ failure and improve with resuscitation by the time you get around to applying the criteria.
• There is only limited therapeutic stratification (into sepsis and septic shock). What the ICU community want is a way to stratify patients by risk, so that therapies can be targeted appropriately.
• It is unclear how patients benefit from a re-definition of sepsis which is more predictive of mortality. For instance, mortality for AMI was 14% in the pre-thrombolysis era, and is now down to around 5% - so should we redefine AMI because it is losing its predictive validity?
• The studies used to validate the new definitions were performed retrospectively, on patients who were diagnosed and treated according to the old definitions. Some might argue that prospective studies should be performed instead, and the definitions changed only if they improve clinically significant outcomes.
• In spite of the well-acknowledged crappyness of the SIRS criteria, sepsis mortality has been on a decline worldwide. How was that a problem?

Nobody asked us, and we don't agree

• Nobody made any attempt to include emergency physicians in the design of the criteria.
• Nobody involved ACEP or ACEM in the process, even though the qSOFA is designed for use by members of the emergency community.
• ANZICS is listed as one of the endorsing societies, but they were not involved in the creation of the guidelines (which is bizarre, as they generate some of the highest quality literature on sepsis).
• The American College of Chest Physicians did not endorse the change, and in fact Chest published a highly critical editorial by S.Q.Simpson which spoke strongly against the adoption of the new definition.
• The UK Sepsis Trust were not consulted about the new ESICM definition, even though they are still techinically part of Europe. Still, the UKST were glad to see the end of the old SIRS definitions, and extended a lukewarm welcome to the new definitions.  However their statement went on to offer an extensive criticism of the SOFA and qSOFA systems and then recomended that British providers continue to use the local (supposedly superior ) NEWS screening tool. SOFA and qSOFA are for reasearch only, they implied.
• The Latin America Sepsis Institute also were not involved in the decisionmaking, and published a statement of non-endorsement (which cannot be found on their website any more). Without being able to actually read their objections, it seems they mainly had a problem with the first world focus of the new definition, and asked how the SOFA laboratory scores and lactate measurements could be expected from healthcare services in low and middle-income countries. Might as well ask for serial procalcitonin measurements, you rich white bastards.

Now what do we do with all these studies

• Everything to date has been using the SIRS criteria for about two decades. How will we compare the existing body of knowledge to studies which use the new criteria? Meta-analysis is going to be difficult.