Ground truth labels challenge the validity of sepsis consensus definitions in critical illness

Background Sepsis is the leading cause of death in the intensive care unit (ICU). Expediting its diagnosis, largely determined by clinical assessment, improves survival. Predictive and explanatory modelling of sepsis in the critically ill commonly bases both outcome definition and predictions on clinical criteria for consensus definitions of sepsis, leading to circularity. As a remedy, we collected ground truth labels for sepsis. Methods In the Ground Truth for Sepsis Questionnaire (GTSQ), senior attending physicians in the ICU documented daily their opinion on each patient’s condition regarding sepsis as a five-category working diagnosis and nine related items. Working diagnosis groups were described and compared and their SOFA-scores analyzed with a generalized linear mixed model. Agreement and discriminatory performance measures for clinical criteria of sepsis and GTSQ labels as reference class were derived. Results We analyzed 7291 questionnaires and 761 complete encounters from the first survey year. Editing rates for all items were > 90%, and responses were consistent with current understanding of critical illness pathophysiology, including sepsis pathogenesis. Interrater agreement for presence and absence of sepsis was almost perfect but only slight for suspected infection. ICU mortality was 19.5% in encounters with SIRS as the “worst” working diagnosis compared to 5.9% with sepsis and 5.9% with severe sepsis without differences in admission and maximum SOFA. Compared to sepsis, proportions of GTSQs with SIRS plus acute organ dysfunction were equal and macrocirculatory abnormalities higher (p < 0.0001). SIRS proportionally ranked above sepsis in daily assessment of illness severity (p < 0.0001). Separate analyses of neurosurgical referrals revealed similar differences. Discriminatory performance of Sepsis-1/2 and Sepsis-3 compared to GTSQ labels was similar with sensitivities around 70% and specificities 92%. Essentially no difference between the prevalence of SIRS and SOFA ≥ 2 yielded sensitivities and specificities for detecting sepsis onset close to 55% and 83%, respectively. Conclusions GTSQ labels are a valid measure of sepsis in the ICU. They reveal suspicion of infection as an unclear clinical concept and refute an illness severity hierarchy in the SIRS-sepsis-severe sepsis spectrum. Ground truth challenges the accuracy of Sepsis-1/2 and Sepsis-3 in detecting sepsis onset. It is an indispensable intermediate step towards advancing diagnosis and therapy in the ICU and, potentially, other health care settings.

Discriminatory performance of mid-upper arm circumference for identifying thin and severely thin adolescents: a secondary data analysis using Comprehensive National Nutrition Survey

Background: Timely identification of adolescents with undernutrition is of utmost importance, and recently, mid-upper circumference (MUAC) had been considered as an alternative to body mass index (BMI) and BMI for age z-score (BAZ) for its screening. However, little is known about the MUAC cut-offs, specific to age and sex. The study was planned to assess the discriminatory performance of MUAC in identifying thin and severely thin adolescents and estimating age specific MUAC cut-offs, separately for males and females, against BAZ as the gold standard. Methods: The Comprehensive National Nutrition Survey (CNNS), India data was used for this analysis. The Receiver Operating Characteristic curve (ROC), area under curve (AUC), and Youden Index were used to estimate MUAC cut-off values for thin (BAZ < -2) and severely thin (BAZ < -3) adolescents. The current analysis was done on 31471 adolescents. Results: The MUAC cut-offs for identifying thin adolescents were: for 10-14 years – 19.2/19.4 cm, for 15-19 years – 22.9/21.7 cm for males and females respectively; and for severe thinness were: for 10-14 years – 18.4/18.3 cm, for 15-19 years – 21.9/20.2 cm for males/females. For thinness, the cut-off varied between 17.4-24.5 cm (for 10-19 years) among males, and for females, it varied between 17.5 -20.9 cm (for 10-19 years). For severe thinness, MUAC cut-off ranged between 16.4-23.7 cm (for 10-19 years) among males, and for females, between 17.3-20.7 cm (for 10-19 years). Conclusion: Thus, age- and sex-specific cut-offs could be considered for screening thin and severely thin adolescents.

Quantitative Estimate of Protein-Protein Interaction Targeting Drug-likeness

The quantification of drug-likeness is very useful for screening drug candidates. The quantitative estimate of drug-likeness (QED) is the most commonly used quantitative drug efficacy assessment method proposed by Bickerton <i>et al</i>. However, QED is not considered suitable for screening compounds that target protein-protein interactions (PPI), which have garnered significant interest in recent years. Therefore, we developed a method called the quantitative estimate of protein-protein interaction targeting drug-likeness (QEPPI), specifically for early-stage screening of PPI-targeting compounds. QEPPI is an extension of the QED method for PPI-targeting drugs and developed using the QED concept, involving modeling physicochemical properties based on the information available on the drug. QEPPI models the physicochemical properties of compounds that have been reported in the literature to act on PPIs. Compounds in iPPI-DB, which comprises PPI inhibitors and stabilizers, and FDA-approved drugs were evaluated using QEPPI. The results showed that QEPPI is more suitable for the early screening of PPI-targeting compounds than QED. QEPPI was also considered an extended concept of "Rules of Four" (RO4), a PPI inhibitor index proposed by Morelli <i>et al</i>. To compare the discriminatory performance of QEPPI and RO4, we evaluated their discriminatory performance using the datasets of PPI-target compounds and FDA-approved drugs using F-score and other indices. Results of the F-score of RO4 and QEPPI were 0.446 and 0.499, respectively. QEPPI demonstrated better performance and enabled quantification of drug-likeness for early-stage PPI drug discovery. Hence, it could be used as an initial filter for efficient screening of PPI-targeting compounds, which has been difficult in the past.<br>

Quantitative Estimate of Protein-Protein Interaction Targeting Drug-likeness

The quantification of drug-likeness is very useful for screening drug candidates. The quantitative estimate of drug-likeness (QED) is the most commonly used quantitative drug efficacy assessment method proposed by Bickerton <i>et al</i>. However, QED is not considered suitable for screening compounds that target protein-protein interactions (PPI), which have garnered significant interest in recent years. Therefore, we developed a method called the quantitative estimate of protein-protein interaction targeting drug-likeness (QEPPI), specifically for early-stage screening of PPI-targeting compounds. QEPPI is an extension of the QED method for PPI-targeting drugs and developed using the QED concept, involving modeling physicochemical properties based on the information available on the drug. QEPPI models the physicochemical properties of compounds that have been reported in the literature to act on PPIs. Compounds in iPPI-DB, which comprises PPI inhibitors and stabilizers, and FDA-approved drugs were evaluated using QEPPI. The results showed that QEPPI is more suitable for the early screening of PPI-targeting compounds than QED. QEPPI was also considered an extended concept of "Rules of Four" (RO4), a PPI inhibitor index proposed by Morelli <i>et al</i>. To compare the discriminatory performance of QEPPI and RO4, we evaluated their discriminatory performance using the datasets of PPI-target compounds and FDA-approved drugs using F-score and other indices. Results of the F-score of RO4 and QEPPI were 0.446 and 0.499, respectively. QEPPI demonstrated better performance and enabled quantification of drug-likeness for early-stage PPI drug discovery. Hence, it could be used as an initial filter for efficient screening of PPI-targeting compounds, which has been difficult in the past.<br>

A BASELINE GENE EXPRESSION-BASED PROGNOSTIC FOR ANTI-TNFα THERAPY RESPONSE IN PATIENTS WITH INFLAMMATORY BOWEL DISEASE

Background Although anti-TNFα therapies have revolutionized the management and care of IBD, their administration and usage remain suboptimal because 1) over 50% of patients do not have a lasting therapeutic response, 2) they increase risk of infections, liver problems, arthritis, and lymphoma, and 3) they are expensive. With approximately 1.6 million people suffering from IBD in the US and global prevalence of IBD on the rise, a predictive test for anti-TNFα response would greatly improve the efficacy and cost-to-benefit ratio of these biologics. Methods We hypothesized that a multicohort analysis of the publicly available IBD gene expression datasets would yield a robust set of mRNAs for distinguishing anti-TNFα responders vs non-responders in the IBD patient population prior to treatment. We identified 5 datasets (n = 160) where whole-genome transcriptomic data was derived from colonic mucosal biopsies of IBD patients who were then subjected to anti-TNFα therapy and subsequently adjudicated for response. We used the MetaIntegrator framework which leverages a leave-one-study-out cross-validation technique in conjunction with effect size and FDR adjusted p-value to identify significant differentially expressed (DE) genes associated with a patient’s predisposition to a response outcome. DE genes were subjected to a greedy forward search to derive a parsimonious gene signature for a response score (geometric mean of the expression level for all positive mRNAs minus the geometric mean of the expression level of all negative mRNAs, multiplied by the ratio of counts of positive to negative genes). Area under the receiver operating characteristic curve (AUC) was subsequently calculated in a leave-one-study-out manner to assess discriminatory performance. Results We first identified 170 genes that were present in at least 40% of cohorts and significantly differentially expressed between responders and non-responders with effect size &gt; 0.8 and q value &lt; 0.1. A score based on these genes predicts responder vs non-responder across the 5 discovery cohorts with AUC of 0.82. Optimizing the variables with a greedy forward search algorithm allowed us to downselect to 7 genes from the set, and a score based on this parsimonious set of 7 genes improved the discriminatory performance to an AUC of 0.87. Choosing a high sensitivity (90%) for a rule-in scenario, the score had moderate specificity (60%); alternatively choosing a high specificity (90%) for a rule-out scenario, the score still had a good sensitivity (80%). Conclusions These initial findings suggest that there is a strong signal for predicting anti-TNFα response in colonic biopsies. In particular, we showed using the leave-one-study-out approach that a predictive signature using mRNA can be generalizable (works in independent cohorts). These initial results warrant further investigation.

Performance of prediction models for short term outcome in COVID-19 patients in the emergency department: a retrospective study

IntroductionCoronavirus disease 2019 (COVID-19) has a high burden on the healthcare system and demands information on the outcome early after admission to the emergency department (ED). Previously developed prediction models may assist in triaging patients when allocating healthcare resources. We aimed to assess the value of several prediction models when applied to COVID-19 patients in the ED.MethodsAll consecutive COVID-19 patients who visited the ED of a combined secondary/tertiary care center were included. Prediction models were selected based on their feasibility. The primary outcome was 30-day mortality, secondary outcomes were 14-day mortality, and a composite outcome of 30-day mortality and admission to the medium care unit (MCU) or the intensive care unit (ICU). The discriminatory performance of the prediction models was assessed using an area under the receiver operating characteristic curve (AUC).ResultsA total of 403 ED patients were diagnosed with COVID-19. Within 30 days, 95 patients died (23.6%), 14-day mortality was 19.1%. Forty-eight patients (11.9%) were admitted to the MCU, 66 patients (16.4%) to the ICU and 152 patients (37.7%) met the composite endpoint. Eleven models were included: RISE UP score, 4C mortality score, CURB-65, MEWS, REMS, abbMEDS, SOFA, APACHE II, CALL score, ACP index and Host risk factor score. The RISE UP score and 4C mortality score showed a very good discriminatory performance for 30-day mortality (AUC 0.83 and 0.84 respectively, 95% CI 0.79-0.88 for both), for 14-day mortality (AUC 0.83, 95% CI: 0.79-0.88, for both) and for the composite outcome (AUC 0.79 and 0.77 respectively, 95% CI 0.75-0.84). The discriminatory performance of the RISE UP score and 4C mortality score was significantly higher compared to that of the other models.ConclusionThe RISE UP score and 4C mortality score have good discriminatory performance in predicting adverse outcome in ED patients with COVID-19. These prediction models can be used to recognize patients at high risk for short-term poor outcome and may assist in guiding clinical decision-making and allocating healthcare resources.

Placental growth factor for the prognosis of women with preeclampsia (fullPIERS model extension): context matters

Background The fullPIERS risk prediction model was developed to identify which women admitted with confirmed diagnosis of preeclampsia are at highest risk of developing serious maternal complications. The model discriminates well between women who develop (vs. those who do not) adverse maternal outcomes. It has been externally validated in several populations. We assessed whether placental growth factor (PlGF), a biomarker associated with preeclampsia risk, adds incremental value to the fullPIERS model. Methods Using a cohort of women admitted into tertiary hospitals in well-resourced settings (the USA and Canada), between May 2010 to February 2012, we evaluated the incremental value of PlGF added to fullPIERS for prediction of adverse maternal outcomes within 48 h after admission with confirmed preeclampsia. The discriminatory performance of PlGF and the fullPIERS model were assessed in this cohort using the area under the receiver’s operating characteristic curve (AUROC) while the extended model (fullPIERS +PlGF) was assessed based on net reclassification index (NRI) and integrated discrimination improvement (IDI) performances. Results In a cohort of 541 women delivered shortly (< 1 week) after presentation, 8.1% experienced an adverse maternal outcome within 48 h of admission. Prediction of adverse maternal outcomes was not improved by addition of PlGF to fullPIERS (NRI: -8.7, IDI − 0.06). Discriminatory performance (AUROC) was 0.67 [95%CI: 0.59–0.75] for fullPIERS only and 0.67 [95%CI: 0.58–0.76]) for fullPIERS extended with PlGF, a performance worse than previously documented in fullPIERS external validation studies (AUROC > 0.75). Conclusions While fullPIERS model performance may have been affected by differences in healthcare context between this study cohort and the model development and validation cohorts, future studies are required to confirm whether PlGF adds incremental benefit to the fullPIERS model for prediction of adverse maternal outcomes in preeclampsia in settings where expectant management is practiced.

Determination of Red Cell Indices Cut-off for Identifying Iron Deficiency Anaemia among Pre-school Children with Sickle Cell Anaemia

Nigerian pre-school children have a high risk of developing iron deficiency and there is no consistent evidence that patients with sickle cell anaemia are protected from iron deficiency anaemia. The objective is to explore red cell indices cut-off values useful as surrogate for detecting iron deficiency in children with sickle cell anaemia. Ninety-seven children with sickle cell anaemia were recruited from Children Outpatient. Reference intervals were developed using the 2.5th – 97.5th, 3.0rd – 97.0th, 5 – 95th, and 10th – 90th percentile intervals for MCV and MCH. The discriminatory performance of the proposed red cell indices criterion was assessed by use of sensitivity, specificity, accuracy, likelihood ratio and predictive values. The 2.5th, 3rd, 5th, 10th, 90th, 95th, 97th, and 97.5th percentile values were: MCV (62.7, 63.6, 66.5, 69.6, 86.3, 87.7, 89.5, and 90.1fl), and MCH (19.0, 19.5, 20.8, 21.4, 28.2, 29.1, 29.5 and 29.7pg). The various calculated cut-off points for the MCV and MCH had lower sensitivity but a higher specificity for detecting iron deficiency than the standard reference values for the general population. The calculated cut-off point for the study subjects below the 10th percentiles had the best discriminatory performance. The cut-off for iron deficiency was 69.6fl for MCV and 21.4pg for MCH either use singly or in combination. In conclusion, standard reference cut-offs of MCV and MCH based on results from western individuals without sickle cell anaemia of the same age are not in agreement with the estimated values for children with sickle cell anaemia in Nigeria.