Original research

Early clinical predictors of infected pancreatic necrosis: a multicentre cohort study

Abstract

Background Infected pancreatic necrosis (IPN) exacerbates complications in patients with acute pancreatitis (AP), increasing mortality rates if not treated promptly. We aimed to evaluate the predictive value of clinical characteristics within 24 hours of admission for IPN prediction.

Methods We conducted a retrospective, multicentre cohort study including 3005 patients with AP from eight hospitals in China. Clinical variables collected within 24 hours after admission were analysed using least absolute shrinkage and selection operator regression (10 cross-validations) for variable selection, followed by multivariate logistic regression to develop an IPN prediction model. Internal cross-validation of the development set and validation of the validation set were performed to ensure robustness. Decision curve analysis was used to evaluate its clinical utility.

Results IPN occurred in 176 patients (176/3005, 5.9%). The final model included temperature, respiratory rate, plasma calcium ion concentration, serum urea nitrogen and serum glucose. The area under the receiver operating characteristics curve (AUC) was 0.85 (95% CI 0.81 to 0.89), outperforming widely used severity scoring systems. The model demonstrated robust performance on the internal validation cohort (mean AUC: 0.84) and external validation cohort (AUC: 0.82, 95% CI 0. 77 to 0.87).

Conclusion We developed a simple and robust model for predicting IPN in patients with AP, demonstrating strong predictive performance and clinical utility.

What is already known on this topic

  • Infected pancreatic necrosis (IPN) remarkably increases the risk of persistent multiple organ failure and mortality if left untreated.

  • However, accurate and practical tools for the early prediction of IPN are still lacking.

What this study adds

  • Our multicentre cohort study used readily available clinical indicators to predict IPN early identifying temperature, respiratory rate, plasma calcium ion concentration, serum urea nitrogen and serum glucose levels as independent risk factors for IPN.

  • The prediction model underwent validation with decision curve analysis demonstrating significant clinical value.

How this study might affect research, practice or policy

  • This study provides a simple tool for early prediction of IPN, assisting clinicians in personalising treatment strategies to reduce mortality in patients with acute pancreatitis.

Introduction

Acute pancreatitis (AP), the most prevalent pancreatic disease, is associated with substantial morbidity and mortality.1 Annually, approximately 34 out of every 100 000 individuals experience AP, which continues to rise,2 likely due to the increasing prevalence of metabolic syndrome and obesity.1 The overall mortality rate of AP is around 1%.1 However, approximately 18% of patients with AP may develop infected pancreatic necrosis (IPN), increasing the risk of persistent multiple organ failure, and further escalating the mortality.3–6 IPN also significantly prolongs hospital stays, imposing a considerable socioeconomic burden.

IPN often requires effective antimicrobial therapy and invasive interventions.7 While antimicrobial therapy, particularly antibiotics, plays a critical role in IPN treatment, the frequency of antibiotic use in AP ranges from 31% to 82%.8 Timely identification of IPN is essential for improving the prognosis of patients with AP. Along with confirmation by microbiology or imaging, the initiation of antibiotic therapy is also recommended when IPN is highly suspected, such as in the presence of sepsis or clinical deterioration.9 10 However, the overuse of antibiotics, especially broad-spectrum antibiotics, can lead to the emergence of antibiotic resistance without providing desired benefits. Additionally, the inappropriate use of antimicrobial drugs may even harm the patients.11 Previous research suggests that procalcitonin-based algorithms can reduce antibiotic overuse without increasing the risk of infections or adverse events.12 Therefore, to reduce the disease burden caused by IPN and minimise the overuse of antibiotics, accurately predicting IPN in patients with AP at an early stage is highly desirable for clinicians.

However, accurate predictors for the early detection of IPN are still lacking. Based on a rigorous prospective clinical trial design, the TRACE Trial13 modified the pancreatitis activity scoring system (PASS), resulting in the modified PASS-4 (mPASS-4). While the mPASS-4 has shown decent performance in predicting IPN, it remains unsatisfactory (area under the receiver operating characteristics (ROC) curve (AUC) 0.75). Therefore, using our multicentre cohort data, we aimed to determine the significance of early clinical characteristics within 24 hours of admission for the early prediction of IPN and to provide evidence when IPN is highly suspected. Decision curve analysis was performed to evaluate the clinical value of our model.

Methods

Cohort design and study sample selection

This multicentre cohort of patients with AP included eight hospitals from various regions of China (Peking Union Medical College Hospital, Beijing; Fangshan District Liangxiang Hospital, The Sixth Hospital, Beijing; The Fourth Affiliated Hospital of Harbin Medical University, Heilongjiang; The First People’s Hospital of Longquanyi District, People’s Liberation Army The General Hospital of Western Theatre Command, Sichuan; Affiliated Hospital of Hebei University, Hebei; First Affiliated Hospital of Nanchang University, Jiangxi) from January 2017 to July 2023. The diagnosis and severity of AP were based on the revised Atlanta classification.9 Patients admitted to hospitals within 48 hours of AP onset were included. The following patients were excluded: those under 18 years of age, those with chronic or recurrent pancreatitis and those discharged early due to personal preference.

Outcome definition

The primary outcome was the occurrence of IPN. IPN was defined as a positive culture result from samples obtained via fine-needle aspiration, drainage or surgical procedures; or the presence of gas in the pancreas and/or peripancreatic tissue on CT.9

Clinical information collection

Demographic and clinical data were collected within the first 24 hours of admission for each patient from the electronic medical record system as candidate variables. These included age, sex, heart rate, respiratory rate (RR), systolic blood pressure, temperature, Glasgow Coma Scale Score, whether the patient had peritoneal irritation signs, leucocyte count, blood platelet count, haematocrit in peripheral blood, and concentrations of plasma potassium, total calcium (Ca), sodium ions, blood urea nitrogen (BUN), serum creatinine, and serum glucose (Glu) level.

The severity of AP was evaluated using widely accepted severity scores, systemic and local complications, period of hospital and intensive care unit stay, and mortality. In addition to IPN, local complications included acute peripancreatic fluid and acute necrotic collections.14 Systemic complications included systemic inflammatory response syndrome (SIRS), organ failure (cardiovascular or renal) and acute respiratory distress syndrome. We also collected information on whether patients underwent endoscopic or interventional drainage treatment, or surgery for AP during their hospitalisation. Severity scores included the Beside Index for Severity in Acute Pancreatitis (BISAP) Scores,15 Acute Physiology and Chronic Health Evaluation II,16 Sepsis-related Organ Failure Assessment,17 and Modified Marshall9 measured within the first 24 hours of admission.

Statistical analysis

Continuous variables were presented as medians (25th percentile to 75th percentile), while categorical data were expressed as numbers and percentages. In the univariate analysis, continuous variables were compared using the t-test or Mann-Whitney U test, whereas categorical variables were compared using the χ2 test or Fisher’s exact test. Patients with missing data were excluded from the analysis. Multiple logistic regression analysis was used for model derivation. Statistical significance was set at values of p<0.05.

Model derivation

The model was designed to predict IPN in all the patients with AP. First, the data set was randomly divided into development and validation sets (7 : 3). The development set was used for model derivation and internal validation.

For variable selection, least absolute shrinkage and selection operator (LASSO) regression, and multivariate logistic regression were performed. To limit the number of potential predictor variables in logistic regression analysis, variables with a value of p<0.1 in the univariate analysis were included in the LASSO regression to select variables and apply a penalty to the model coefficients to mitigate overfitting.18 19 Further, 10-fold cross-validation was used to tune the λ parameter. The variables selected in the LASSO regression were included in the multiple logistic regression analysis. Finally, predictors with values of p<0.05 were used to develop the final prediction model.

The linear relationship between the continuous independent variables and the logit transformation of the dependent variable was assessed using the Box-Tidwell test. Additionally, multicollinearity among the independent variables was diagnosed using tolerance and variance inflation factors. The discriminatory ability using the AUC, and an ROC curve was plotted.20 Model calibration was assessed using the Hosmer-Lemeshow goodness-of-fit test and visualised using a calibration plot.21

Model validation

First, we performed internal validation using a 10-fold cross-validation in the development set to provide an unbiased estimate of model performance. The prediction accuracy of the IPN models was then determined on the validation set by computing the AUC and 95% CI.

The analyses were conducted using R software V.3.6.1 (R Foundation for Statistical Computing, Vienna, Austria) using the package ‘glmnet’ for LASSO regression, ‘caret’ for 10-fold internal cross-validation, ‘rms’ for model derivation. Furthermore, clinical value of the model was assessed through decision curve analysis using the package ‘rmda’, which helped visualise benefits and risks under different prediction thresholds.22 23

Results

Demographic and clinical characteristics

A total of 3491 patients with AP from a multicentre cohort met the inclusion criteria. After excluding patients with missing data, the final analysis included 3005 patients with AP. The mean age of the patients was 49 years, with 643 patients aged over 65 years. There were 1924 male patients, accounting for 64.0% of all patients with AP. The selected cohort comprised 1246 biliary patients with AP, 1062 cases with hypertriglyceridemic AP and 284 cases of alcohol-induced AP.

There were 730 patients with severe AP (SAP), accounting for 24.3% of all patients with AP. In total, 176 patients with AP were diagnosed with IPN, comprising 5.9% of all patients with AP. Among patients with SAP, IPN was diagnosed in 19.3% of patients (141 out of 730), whereas it occurred in 3.5% of patients with moderately severe AP (35 out of 992). The mortality rates were 7.1% (52 out of 730) for patients with SAP and 0.2% (2 out of 992) for patients with moderately severe AP.

Regarding severity and complications, patients with IPN demonstrated significantly worse outcomes compared with those without IPN in the univariate analysis (table 1). Significant differences were obtained in baseline vital signs and laboratory examinations within the first 24 hours of admission between the two groups,

Table 1

Comparison of clinical characteristics between patients with IPN and patients without IPN

Furthermore, for patients with AP exhibiting acute necrotic collection on their first CT, we compared the clinical characteristics of IPN with those of sterile necrotising pancreatitis (online supplemental table 1).

Model derivation

The development set consisted of 2103 patients with AP, 113 of whom had IPN. We selected λmin (0.005) as the most appropriate tuning parameter for LASSO regression (online supplemental figure 1). Age, sex, concentrations of plasma sodium ion, blood platelet count and haematocrit in peripheral blood were excluded from the LASSO regression, while other variables were included in the multivariable logistic regression model, adjusted for age and sex. Serum creatinine levels were excluded because of their correlation with BUN.

Finally, as presented in table 2, temperature (OR 1.71; 95% CI 1.28 to 2.27; p<0.001), RR (OR 1.04; 95% CI 1.00 to 1.08; p=0.042), Ca (OR 0.32; 95% CI 0.17 to 0.62; p<0.001), BUN (OR 1.08; 95% CI 1.03 to 1.12; p<0.001) and Glu (OR 1.15; 95% CI 1.01 to 1.19; p<0.001) were incorporated into the final multivariate model.

Table 2

Multivariate logistic regression analysis for the variables selected by univariate analysis and LASSO regression

The equation for the final IPN prediction model is as follows: probability (IPN)=exp(Y)/(1+exp(Y)). Y=0.05 * RR+0.57* Temperature+0.15* Glu+0.09* BUN-1.26* Ca-25.10. The model is visualised as a nomogram in figure 1.

Figure 1
Figure 1

Nomogram of our infected pancreatic necrosis (IPN) prediction model. BUN, blood urea nitrogen; Ca, total calcium; Glu, serum glucose; RR, respiratory rate.

The AUC for the model was 0.85 (95% CI 0.81 to 0.89), with a sensitivity of 0.75 and a specificity of 0.85 (positive predictive value: 76.7%, negative predictive value: 95.7%, pretest probability: 5.4%, post-test probability: 50.1%). We then compared the IPN prediction model with the severity scores of AP, which are widely used in clinical practice, as depicted in figure 2. In terms of prediction accuracy, our models outperformed other severity scores. Regarding the calibration ability, the value of p for the Hosmer–Lemeshow goodness of fit test was 0.58. The calibration plot depicted a good agreement between the predicted and actual probabilities of IPN (online supplemental figure 2).

Figure 2
Figure 2

ROC comparison between our infected pancreatic necrosis (IPN) model and other severity scores. Brown line (model) = ROC of our IPN prediction model, which had the best prediction scores. Cyan line (APACHE II) = ROC drawn based on APACHE II Scores. Orange line (Marshall) = ROC of Marshall Scores. Khaki line (SOFA) = ROC of SOFA Scores. Blue line (BISAP) = ROC of the BISAP Scores. The AUC of APACHE-II Score, Marshal Score, SOFA Score and BISAP Score are 0.64 (95% CI 0.59 to 0.69), 0.62 (95% C1 0.56 to 0.67), 0.57 (95% C1 0.51 to 0.62), 0.76 (95% CI 0.71 to 0.81), respectively. The performance of the IPN model differed significantly from that of the BISAP Scores (p=0.007). APACHE II, Acute Physiology and Chronic Health Evaluation II; AUC, area under the ROC curve; BISAP, Beside Index for Severity in Acute Pancreatitis; ROC, receiver operating characteristics; SOFA, Sepsis-related Organ Failure Assessment.

Model validation

In the 10-fold internal cross-validation, the IPN model achieved a mean AUC of 0.84, with a maximum AUC of 0.96 and minimum of 0.67. The validation set comprised 878 patients with AP, 55 of whom had IPN. A comparison of the demographic and clinical characteristics between the development and validation sets is presented in online supplemental table 2. In validation set, the final model also demonstrated good discrimination ability, with an AUC of 0.82 (95% CI 0.77 to –0.87), outperforming the best severity score, BISAP (AUC 0.76, 95% CI 0.70 to 0.83), indicating the robustness of our model.

In the decision curve analysis, the IPN model exhibited a positive net benefit for predicted probability thresholds above approximately 2% compared with treating all the patients with AP as if they all had IPN or none of them would have IPN (figure 3). Between 75% and 85%, the net benefit of the model was occasionally negative, owing to random noise.22 In the comparison of our model with other widely used severity scores in the derivation and validation sets, the IPN model provided a greater net benefit compared with those over a wide range of probability thresholds, suggesting a better clinical decision-making value (figure 3; online supplemental figure 3).

Figure 3
Figure 3

Decision curve analysis using the infected pancreatic necrosis (IPN) prediction model comparing with other scores in derivation set. Black line (none): net benefit when we assume that no patients with acute pancreatitis (AP) will have the outcome (IPN) and do not intervene; grey line (all): net benefit when we assume that all patients with AP will have the IPN and intervene; red line (IPN model) = net benefit when we manage patients with AP according to the predicted risk of the IPN estimated by the IPN prediction model; orange line (BISAP Score): net benefit when we manage patients with AP according to the predicted risk of the IPN estimated by BISAP Score; blue line (APACHE II Score): net benefit when we manage patients with AP according to the predicted risk of the IPN estimated by APACHE II Score; cyan line (Marshall Score): net benefit when we manage patients with according to the predicted risk of the IPN estimated by Marshall Score; brown line (SOFA Score): net benefit when we manage patients with AP according to the predicted risk of the IPN estimated by SOFA Score. APACHE II, Acute Physiology and Chronic Health Evaluation II; BISAP, Beside Index for Severity in Acute Pancreatitis; SOFA, Sepsis-related Organ Failure Assessment.

Discussion

We observed that early clinical characteristics have significant discriminatory value for IPN in a multicentre cohort. The prediction model, based on early clinical characteristics (temperature, RR, Ca, BUN and Glu) demonstrated robust performance in the derivation (AUC 0.85, 95% CI 0.81 to 0.89), internal cross-validation (mean AUC 0.84), and validation sets (AUC 0.82, 95% CI 0. 77 to 0.87). For high-risk patients identified by our model, subsequent monitoring and vigilance for the occurrence of IPN are recommended. In addition, our model serves as evidence supporting the initiation of antibiotic therapy, if there is the presence of sepsis or clinical deterioration, and IPN is highly suspected.10

In our model, the risk of IPN can be defined. Using a model score of 0.068 at the optimal AUC as the cut-off value, the IPN probability threshold was 0.51. The sensitivity and specificity of the model were 0.75 and 0.85, respectively. Increasing the probability threshold to 0.60 reduces sensitivity but increases specificity (to 0.23 and 0.99, respectively), thus lowering the risk of overtreatment while raising the risk of missed diagnoses. Conversely, if higher sensitivity is desired, the model threshold can be lowered. For instance, when the cut-off model score was 0.03, the sensitivity reached 0.84, but the specificity dropped to 0.67, increasing the risk of overtreatment. Therefore, a threshold that maximises the classification accuracy can be determined based on acceptable levels of false positives and false negatives. Thus, the model allows flexible selection of thresholds to define risk groups.

In the decision curve analysis, the introduction of the model significantly increased net benefits, reduced the probability of unnecessary interventions, saved healthcare resources and reduced medical costs. Given that IPN without appropriate treatment significantly increases the mortality rate of AP, the prevention of SAP and IPN is important. Currently, Vege et al24 and Ke et al25 have actively explored this field using pentoxifylline and early immune-enhancing thymosin α1, respectively. As the results were not satisfactory, further research is required. Therefore, if preventive measures for IPN are confirmed in the future, identifying high-risk patients using a model might enable early preventive treatment.

In our model, we included the first measurements taken within 24 hours of admission for RR, temperature, BUN, Ca and Glu levels. Both RR and temperature are considered evaluation indicators for SIRS reflecting the level of inflammation in the body.26 A study by Talukdar et al27 also indicated that an increase in BUN within 48 hours of admission in patients with AP can serve as a predictive indicator for IPN, consistent with our research findings. It has been believed that decreased blood calcium levels are associated with the severity of AP, which explains the rationale for incorporating this indicator in our study.28 Czapári et al suggested that glucose level at the time of admission is an independent risk factor associated with postdischarge mortality.29 Additionally, other studies have indicated that patients with on-admission hyperglycaemia are more likely to require antibiotic treatment during hospitalisation, and are associated with a higher incidence of acute necrotic collection and major infection.30 31 These findings explain why glucose level was included as an independent risk factor for IPN in our final model.

Currently, few studies are available on the early identification of IPN.13 32–35 However, these studies have relatively small sample sizes and some lack validation, which limits the reliability and generalisability of their findings. Despite providing instructive results, the predictive efficacy of these models may not always be satisfactory. Chen et al33 and Wiese et al34 focused specifically on necrotising pancreatitis, with their models exhibiting good performance, with AUCs of 0.79 and 0.82, respectively. However, it is important to note that the impairment of pancreatic perfusion and signs of peripancreatic necrosis evolved over several days.9 Therefore, the use of these models requires that there be no missed diagnoses of necrotising pancreatitis. In contrast, our model can be applied to all adult patients with AP, offering greater convenience, and has shown superior performance in both the training and validation data sets. Similarly, Song et al35 focused on a specific group of patients with AP (moderately severe and severe AP). Zhu et al36 proposed an IPN prediction model based on a modified CT Severity Index, neutrophil-to-lymphocyte ratio and procalcitonin on the seventh day postadmission. In terms of predictive performance, their model demonstrated a high AUC of 0.92 on the training data set. Despite robust performance, their model has not been validated, presenting a risk of overfitting. Furthermore, our study included indicators that might be obtained on the day of admission, enabling more rapid and convenient IPN prediction. Mao et al13 demonstrated the effectiveness of mPASS-4 for predicting IPN in a prospective clinical trial including 508 cases of acute necrotising pancreatitis, achieving an AUC of 0.75. This research underscored the potential of PASS for predicting IPN.37 However, due to the lack of certain data types in our cohort, we were unable to compare PASS, mPASS-4 and our model. Future research should focus on comparative studies of these existing models using larger sample sizes. Apart from predicting IPN, Trikudanathan et al38 developed a model to predict the need for intervention in acute necrotising pancreatitis after discharge, with an AUC of 0.88. This model provides a valuable tool for managing patients with IPN from different perspectives.

However, our study has limitations. It was retrospective, which may have introduced selection bias, as the inpatients may have severe disease, resulting in a higher incidence of IPN. Additionally, the incidence of hypertriglyceridemic pancreatitis in China is higher than that in Europe and USA, as observed in our cohort.39 40 Therefore, when applying the model to regions such as Europe and USA, it is important to consider population-specific factors and conduct additional external validation beyond China. Furthermore, in our multicentre study, we did not include other inflammatory markers, such as interleukin-6, or additional peripheral blood information, including proteomics, radiomics and other potential factors. This may have reduced the predictive accuracy of our model. This area will be further explored in the future to enhance the accuracy of predictive models.

Conclusion

We developed a simple model for early prediction of IPN based on a multicentre cohort, exhibiting good performance and robustness. The model can assist clinicians in devising personalised treatment strategies, potentially helping to reduce AP-related healthcare costs and mortality rates.