The revised Atlanta Criteria classify acute pancreatitis on a scale of mild to severe with severe pancreatitis being associated with organ failure persisting greater than 48 hours.1 In the most severe cases, pancreatitis can advance to acute necrotizing pancreatitis (ANP), which is associated with a high mortality rate ranging from 13% for sterile necrosis to 28% for infected necrosis.2 Over the past 20 years, the surgical management options for ANP have expanded from an open surgical necrosectomy to now including the use of percutaneous drainage, endoscopic debridement, and minimally invasive surgery. These changes have been supported by the pancreatitis, necrosectomy versus step up approach (PANTER) trial, which demonstrated improvements in the rate of major complications and mortality for patients managed with a therapeutic “step-up” approach that emphasizes first attempting minimally invasive techniques and moving to more invasive techniques if the less invasive techniques were not sufficient.3 Long-term follow-up has also demonstrated improved endocrine and exocrine function in those patients managed with the step-up approach.4

Since these early studies, both surgical and gastroenterological societies have developed guidelines that emphasize a multidisciplinary management approach that uses the step-up approach and supports delaying pancreatic debridement until out of the acute inflammatory phase, when able.5,6 As mortality continues to improve among patients with ANP, there is a new surge of research comparing a broad range of management approaches and outcomes, including, but not limited to, specific techniques, nutritional management, and antimicrobial management.7,8 Unfortunately, because of the heterogeneity of this disease process and inconsistencies in reported outcomes, the literature can be difficult to compare and synthesize, making it less likely to impact clinical practice.

The purpose of this project is to create a core outcome set (COS) that establishes a minimum set of outcomes for future high-quality studies on ANP. Importantly, a COS does not restrict future research from exploring additional outcomes but rather serves as a suggested minimum framework to maximize the number of studies that can be compared or pooled in systematic reviews and meta-analyses to guide complex management decisions for ANP. The Eastern Association for the Surgery of Trauma (EAST) created a COS task force to identify and develop COS in high-value areas related to acute care surgery. Acute necrotizing pancreatitis was recognized as a high-priority topic that would benefit from a COS. We hypothesized that, by identifying experts on ANP and using a modified Delphi method, we could identify a list of outcomes to serve as a COS for future research.

PATIENTS AND METHODS

The ANP COS study was developed following the Core Outcome Measures in Effectiveness Trials tool and conducted in accordance with recommendations from the Core Outcome Set—Standards for Development and Reporting.9–11 The study was registered with the Core Outcome Measures in Effectiveness Trials database and carried out under the oversight of the EAST COS task force. This study was unfunded and deemed exempt by our institutional review board. Informed consent was assumed if a participant responded to the modified Delphi survey.

Panel members were identified based on their contribution to peer-reviewed literature on ANP. Eastern Association for the Surgery of Trauma maintains a list of heavily cited and potentially practice-changing articles for trauma, critical care, and emergency general surgery topics. As with previously published EAST COS projects, the “EAST Landmark Papers” were evaluated for content experts.12 Since the EAST Landmark Papers did not include literature from recent years and in an effort to broaden the expert panel, a limited PubMed literature search was also completed. This secondary search was focused on ANP literature published within the past 2 years. The first and last authors of the identified publications, excluding review articles, were invited to participate through contact information published with the manuscripts. Lastly, a public call for volunteers through the National Pancreatitis Foundation seeking individuals with personal experience either as a survivor of ANP or as a personal caretaker or family member for someone with ANP was completed. The final pool of invited content experts included both academic and nonacademic clinicians with expertise in surgery, gastroenterology, and infectious disease as well as patients with a personal history of ANP. For the purposes of this study, those experts identified through published literature were referred to as “medical experts,” and those identified through the National Pancreatitis Foundation were referred to as “patient experts.” Patient experts were given the opportunity with each round to ask questions for points of clarification or any explanation of terminology from the principal investigator. The identity of each panelist was kept blinded to the rest of the panel until completion of the Delphi process.

In the first round, ANP content experts were queried for suggested research outcomes that they felt should be included in future clinical research on ANP. The stated goal was to identify outcomes they felt were likely to either shape clinical practice or impact physician/patient decision making. The outcomes were intended to be generalizable, so they could be included in future research, without dictating the study design or comparison groups. These responses were collected, and duplicates were removed. In the second round, experts were asked to score each outcome, independent of the other outcomes, based on how important they felt each outcome was to be included in future research. This was completed on a Likert scale, based on the Grading and Recommendations Assessment, Development and Evaluations scale of 1 to 9.13 In this scale, 1 to 3 signifies a less important variable, 4 to 6 indicates an important but not critical variable, and 7 to 9 indicates a critically important variable. As in previous modified Delphi studies conducted by the task force, consensus defined a priori as >70% of scores ranging from 7 to 9 and <15% of scores ranging from 1 to 3. Similar to recent COS publications, outcomes were excluded if they received <40% of scores ranging from 7 to 9 and >15% of scores ranging from 1 to 3.14–17

In the second round, after reviewing the list of suggested outcomes, panelists were given an additional opportunity to add outcomes they felt should be considered in future rounds. In each round, the outcomes were presented in random order to minimize bias. Only those participants who completed the previous round were invited to participate in the subsequent round. Each participant was provided 6 to 8 weeks to complete each round, and three individual reminder emails were sent during this period to maximize participation. Beginning in round 3, each panelist was shown their responses from the previous round, as well as a histogram of the entire panel's de-identified aggregate responses for each outcome (Supplemental Digital Content, Supplementary Data 1, https://links.lww.com/TA/D582). Panelists were then given the opportunity to revise their ratings. Interclass correlation (ICC) was calculated for each round, beginning with round 3. This process was continued until the ICC suggested that it was statistically unlikely to achieve further consensus. This was defined as either a failure to increase the ICC between rounds or an ICC below 0.50. A flow diagram outlining this process can be seen in Figure 1.

Figure 1:

Core outcome set for ANP flow chart.

Statistical assessment was performed using the ICC function to assess interrater test-retest reliability, in a two-way mixed-effects model scoring for absolute agreement, with a 95% confidence interval (CI). Interclass correlation estimate and their 95% CI were calculated using R statistical package “ICC” version 2.3. Accepted definitions of agreement were as follows: <0.50, poor; 0.50 to 0.74, moderate; 0.75 to 0.90, good; and 0.90 to 1.00, excellent. All statistical tests were performed in the R version 4.3.0 environment.18–20 The Core Outcome Set—Standards for Development and Reporting and Standards for Reporting Diagnostic accuracy studies guidelines were used to ensure proper reporting of methods, results, and discussion (Supplemental Digital Content, Supplementary Data 2, https://links.lww.com/TA/D583).

RESULTS

Twenty-four medical professionals were invited to participate in this project beginning January 17, 2023. Of the 17 who responded, 16 (67%) agreed to participate with the one abstention being secondary to availability. The panel included individuals from North America, South America, and Europe. All of the medical experts actively care for patients with ANP, and the panel included seven acute care surgeons, five interventional gastroenterologists, and four hepatobiliary surgeons spanning from early to late career with 15 being male. While most of the medical experts were identified through the EAST Landmark Papers, the limited PubMed search did result in four of the panelists including two nonacademic acute care surgeons, one academic minimally invasive hepatobiliary surgeon, and one academic gastroenterologists. A call for volunteers through the National Pancreatitis Foundation resulted in the identification of three additional experts with personal experience as survivors of ANP. The patient experts included two females and one male ranging from 25 to 58 years of age, all of whom had personal experiences of being hospitalized multiple times for ANP related issues. In round 1, all 19 participants submitted proposed core outcomes with a 100% response rate. After removing duplicate items, 43 unique responses were compiled for further consideration in subsequent rounds (Table 1).

TABLE 1 - Submissions From the Expert Panel for Consideration as Core Outcomes Mortality Mortality — in hospital Need for surgical necrosectomy (open or minimally invasive) Need for open surgical necrosectomy Need for minimally invasive surgical necrosectomy — video-assisted retroperitoneal dissection Need for minimally invasive surgical necrosectomy — laparoscopic transabdominal Need for minimally invasive necrosectomy — sinus tract endoscopy Need for endoscopic necrosectomy Need for endoscopic catheter drainage Return to oral only diet Need for nutritional supplements Acute organ failure — single-organ system Acute organ failure — multiorgan system New onset organ failure (single or multi-system) Organ failure — transitory/resolved Chronic/persistent organ failure Number of disease specific interventions (percutaneous, endoscopic, or surgical) Development of infected necrosis Development of multidrug-resistant organisms Need for percutaneous drainage Percutaneous aspiration culture results compared with endoscopic or surgically obtained cultures Changes to antimicrobial plan based on endoscopic or surgically obtained cultures Enterocutaneous fistula requiring drainage Development of pancreatic leak/fistula Duration of external pancreatic fistula Eventual cholecystectomy technique (laparoscopic vs. open vs. none) Complications of subsequent cholecystectomy Recurrence of pancreatitis Development of bile duct stricture Endocrine insufficiency (development of diabetes mellitus) Exocrine insufficiency (need for enzyme replacement) Bleeding requiring invasive intervention Development of portosplenic thrombus Long term development of pancreatic cancer Gastrointestinal perforation requiring intervention Need for emergency laparotomy without necrosectomy (i.e., bleeding, abdominal compartment syndrome, etc.) Cost of care Return to predisease health Return to activities of daily living Quality of life Chronic pain Hospital length of stay ICU length of stay *Need for necrosectomy (any/all techniques) *Need for TPN in hospital *Need for TPN at discharge *Need for enteral nutrition *Number of CT scans performed

*Outcomes are displayed as proposed by content experts in round 1 with outcomes received after round 2.

CT, computed tomography; TPN, total parenteral nutrition.

In round 2 (March 3, 2023), the expert panel was presented with the compiled list of proposed outcomes, in random order. One patient expert withdrew from the study for reasons unrelated to the project resulting in a 95% participation rate. Of the 43 proposed outcomes, 7 achieved consensuses for inclusion, and 8 outcomes were eliminated per predefined criteria. The seven included outcomes were as follows: mortality, mortality (in hospital), acute organ failure (single-organ system), acute organ failure (multiple-organ systems), chronic/persistent organ failure, number of interventions for treatment of collection, and bleeding requiring invasive intervention. In addition to scoring the previously proposed outcomes on the Likert scale, four new outcomes/variables were proposed for future consideration. These included the following: need for total parenteral nutrition in hospital, need for total parenteral nutrition at discharge, need for enteral nutrition, and number of computed tomography scans performed. In addition, many panelists noted that multiple proposed outcomes were procedural in nature and requested an all-encompassing “need for necrosectomy (any/all techniques)” be added for consideration to potentially combine the procedural outcomes (Table 1).

In round 3 (April 20, 2023), the expert panel was asked to re-evaluate the 28 outcomes that had failed to meet inclusion/exclusion criteria and the additional outcomes proposed in round 2. One medical expert failed to respond during this round and was removed from the study (17 of 19 [90%], overall retention). Of the 17 remaining participants, 16 changed at least one score. Eight additional outcomes reached consensus for inclusion, and eight outcomes were eliminated. The eight included outcomes were as follows: need for necrosectomy (any/all techniques), need for surgical necrosectomy (open or minimally invasive), need for endoscopic necrosectomy, development of infected necrosis based on positive cultures from fine-needle aspiration or from first drainage procedure/operation or presence of gas in the fluid collection on a contrast-enhanced computed tomography, number of disease specific interventions, cost of care, quality of life, return to activities of daily living. As “need for necrosectomy any/all techniques” achieved consensus, all procedural outcomes were grouped under this outcome with an additional emphasis placed on the type of necrosectomy and number of procedures utilized. An ICC analysis of the remaining outcomes was completed and demonstrated an ICC of 0.59 (95% CI, 0.32–0.79), indicating moderate reliability. Therefore, we proceeded with a fourth round.

This process was repeated in round 4 (June 26, 2023). One medical expert failed to respond during this round and was removed from the study (16 of 19 [84%], overall retention). Of the 16 remaining participants, 15 changed at least one score. Three outcomes reached consensus for inclusion, and five outcomes were eliminated. The three included outcomes were as follows: hospital length of stay, development of endocrine insufficiency defined as the development of diabetes mellitus, and development of pancreatic leak/fistula. The remaining outcomes demonstrated an ICC of 0.517 (95% CI, 0.12–0.79). The modified Delphi process was deemed complete because the ICC did not improve with the fourth round. The final list of consensus items included 16 core outcomes that the panel felt warranted routine inclusion in future ANP studies (Table 2).

TABLE 2 - Set of Outcomes Achieving Consensus as “Core Outcomes” for ANP Mortality Outcomes  Overall  In-hospital Organ Failure Outcomes  Acute organ failure — single-organ system  Acute organ failure — multiple-organ systems  Chronic/persistent organ failure Complication Outcomes  Development of infected pancreatic necrosis  Development of pancreatic leak/fistula  Development of endocrine insufficiency  Development of bleeding requiring invasive intervention Management/Interventions Outcomes  Hospital length of stay  Number of disease specific interventions (including percutaneous, endoscopic, or surgical)  Need for necrosectomy (any/all types)  Type of necrosectomy required/performed (e.g., endoscopic, minimally invasive, or open surgical) Social Outcomes  Cost of care  Quality of life  Return to predisease activities of daily living
DISCUSSION

A multidisciplinary and international panel of content experts and ANP survivors identified a set of core outcomes that included hospital outcomes, organ failure, rates of disease progression/complications, need/type of interventions, and social outcomes. This COS was identified with the goal of setting a minimum framework for future studies to allow for comparisons of data and to facilitate any future data pooling or meta-analyses. Notably, the COS is not intended to restrict future researchers from exploring additional questions; rather, we encourage future work to include these outcomes while also expanding beyond the COS.

The management of ANP often benefits from a multidisciplinary approach, and we intentionally sought out a multidisciplinary team of experts for this study. As may have been expected, the first set of outcomes that reached consensus was those that impacted the care provided from a surgical, gastroenterological, and patient perspective. Specifically, the outcomes focusing on mortality and organ failure achieved early agreement. All-cause mortality is an important endpoint in clinical trials related to ANP, as it is an easily measured, binary, patient centered outcome. It was not limited to ANP-specific causes of death because secondary systemic insults are often the cause of death. Because ANP may require a prolonged management period and since we recognize that future ANP management practices or the particular question being addressed in future research may make certain mortality timelines more appropriate, no strict mortality timeline was stated, but since most patients who require a procedure will receive it within 6 months of diagnosis, it is reasonable to consider all-cause mortality with respect to in-hospital/first presentation mortality and within 6 months of diagnosis.21

Similarly, organ failure outcomes reached consensus early in this modified Delphi process. Organ failure is a defining characteristic of severe pancreatitis and accounts for 55% to 87% of deaths in patients with ANP.22–24 Because the COS was designed as a framework that should be amenable to changes in clinical practices and definitions, we did not want to restrict future research with specific organ systems or diagnostic criteria, but we strongly recommend that researchers clearly state the definitions used in their studies. Still, based on current literature, we support the definitions from the Atlanta Criteria, which use the Modified Marshall Score to evaluate the respiratory, cardiovascular, and renal systems and define persistent organ failure as organ failure >48 hours from presentation.1

Notably, bleeding requiring invasive intervention also reached early consensus, but multiple panelists noted that they scored it highly because they thought it rare but felt it would potentially impact their future decisions. Interestingly, while most studies do report hemorrhage rates around 1%, some studies report it as high as 23%, which highlights how its inclusion in the COS may help identify underemphasized risks and potentially influence practice patterns.25

The second set of outcomes that achieved consensus was largely procedurally focused with many experts selecting procedures that were used in their personal medical practice. Because there is variability in management practices for ANP, rather than prioritizing certain techniques over others, these outcomes were regrouped under an all-encompassing “need for necrosectomy” outcome. We do recommend future researchers specifically address the approach used for any necrosectomy. Given both the medical and patient experts acknowledged the strain placed on patients through multiple procedures, we encourage the number and the type of interventions to be reported to help better understand this burden and to effectively compare outcomes.

The last set of outcomes to reach consensus was focused on long-term complications of the disease process, specifically the development of a pancreatic leak/fistula and endocrine insufficiency. Many medical experts commented in earlier rounds that these outcomes were an expected part of the disease process as opposed to patient experts who acknowledged personal frustration with these conditions. Still, they are both relatively common complications with leak/fistula occurring in 13% and endocrine insufficiency occurring in 37% of patients with ANP, and because both severity of disease and management approaches may impact their rate of occurrence, tracking the rate of occurrence is an important step in optimizing disease management.26 We recognize that the diagnostic criteria for pancreatic leak/fistula may change, and for that reason, we did not specifically state the criteria in this COS. Still, we believe that, based on current literature, the definition used by the International Study Group of Pancreatic Surgery is reasonable. This definition indicates any measurable volume of fluid with an amylase level >3 times the upper limit of institutional normal serum amylase activity, associated with a clinically relevant condition.27 Given the multiple available definitions, as with other listed outcomes, we strongly encourage future researchers to clearly state the diagnostic criteria used in their studies.

Social outcomes were also included in this COS. These include cost of care, quality of life, and returning to predisease activity. Interestingly, only three medical experts initially proposed social outcomes, but all patient experts proposed at least one social outcome. The outcomes proposed by the medical experts and patient experts did overlap except for “chronic pain” that was included by a patient expert. Furthermore, almost all proposed social outcomes did achieve consensus, suggesting that medical professionals recognize the importance of patient-centered outcomes but may overlook them for clinically focused outcomes. We hope that, by including social outcomes in this COS, more data will become available that will help providers assist patients in making informed decisions for the management of this complex disease process.

Perhaps just as important as the accepted outcomes are the rejected outcomes that saw a divergence between the medical and patient experts. The originally proposed outcomes of “eventual cholecystectomy technique” and “long-term development of pancreatic cancer” were both eliminated in round 2. Most medical experts scored both outcomes between 1 and 3, indicating that they were “less important,” and many commented that the cholecystectomy was beyond the scope of ANP research and that the development of pancreatic cancer was too rare to be reasonably captured in most research. Alternatively, all patient experts scored these outcomes between 7 and 9, indicating that they were “critically important.” While this divergence was limited to only these two outcomes, it does serve as an example of how patients may have concerns that extend beyond the acute management, and there may be an opportunity for providers to better educate on both short- and long-term components of their care.

This study has several limitations inherent to remote, asynchronous, iterative consensus studies. While we did invite 24 medical experts to participate, we only had a 67% acceptance rate. Many experts did not respond, despite multiple attempts to contact them through email, social media, and personal contacts. Unfortunately, this was most pronounced with international experts. In addition, we recognize that, by identifying experts through published literature, we may limit nonacademic viewpoints. While it would have been desirable to continue with additional rounds until all outcomes met either the inclusion or exclusion criteria, we felt that four rounds provided ample opportunity to achieve consensus and that further rounds were unlikely to change the final COS, especially given the decreasing ICC. Lastly, while we do suggest definitions for many outcomes, we recognize that this COS does not specifically define criteria for each outcome. This was done intentionally to allow future researchers to adapt to changing definitions, and so this COS may remain appropriate as definitions change. We do recommend that future researchers clearly define the criteria used for each item in the COS.

One of the greatest strengths of this COS is the high retention rate of the multidisciplinary and international panel of experts that included surgeons, gastroenterologists, and ANP survivors with varying practice patterns. This mix of medical and patient experts provided the opportunity to include both clinically relevant and patient-centered outcomes. The panelists received multiple weeks to complete each round, which allowed for greater participation. Also, the modified-Delphi study design used had clear, predefined, inclusion and exclusion criteria that have been used in multiple prior studies.

Finally, we do recognize that researchers may wish to expand beyond this COS, and we encourage them to not be restricted to this COS. We hope that future researchers will use this as a minimum framework when designing their studies and intermittently reassess this COS as clinical practice changes.

CONCLUSION

An international multidisciplinary panel of content experts, including ANP survivors, achieved consensus upon a core set of outcomes that are recommended to be included in future research to better facilitate data pooling and meta-analyses.

AUTHORSHIP

M.S.F. performed the literature review. M.S.F., R.B.G., and D.D.Y. assisted with the study design. M.S.F. performed the data collection. S.B. assisted with data analysis. M.S.F., R.B.G., D.D.Y., and S.B. performed data interpretation. M.S.F. wrote the manuscript. M.S.F., A.A., S.B., P.F., R.B.G., F.A.G., J.G., K.H., D.J., C.L., N.M., B.R.H.R., A.R., V.K.S., A.K.S., S.S.V., G.T., B.C.V., R.P.V., and D.D.Y assisted with identifying outcomes and critical revisions of the manuscript.

DISCLOSURE

Conflicts of Interest: Author Disclosure forms have been supplied and are provided as Supplemental Digital Content (https://links.lww.com/TA/D584).

REFERENCES 1. Banks PA, Bollen TL, Dervenis C, Gooszen HG, Johnson CD, Sarr MG, et al. Classification of acute pancreatitis—2012: revision of the Atlanta classification and definitions by international consensus. Gut. 2013;62(1):102–111. 2. Werge M, Novovic S, Schmidt PN, Gluud LL. Infection increases mortality in necrotizing pancreatitis: a systematic review and meta-analysis. Pancreatology. 2016;16(5):698–707. 3. van Santvoort HC, Besselink MG, Bakker OJ, Hofker HS, Boermeester MA, Dejong CH, et al. A step-up approach or open necrosectomy for necrotizing pancreatitis. N Engl J Med. 2010;362(16):1491–1502. 4. Hollemans RA, Bakker OJ, Boermeester MA, Bollen TL, Bosscha K, Bruno MJ, et al. Superiority of step-up approach vs open necrosectomy in long-term follow-up of patients with necrotizing pancreatitis. Gastroenterology. 2019;156(4):1016–1026. 5. Baron TH, DiMaio CJ, Wang AY, Morgan KA. American gastroenterological association clinical practice update: management of pancreatic necrosis. Gastroenterology. 2020;158(1):67–75.e1. 6. Mowery NT, Bruns BR, MacNew HG, Agarwal S, Enniss TM, Khan M, et al. Surgical management of pancreatic necrosis: a practice management guideline from the Eastern Association for the Surgery of Trauma. J Trauma Acute Care Surg. 2017;83(2):316–327. 7. Sion MK, Davis KA. Step-up approach for the management of pancreatic necrosis: a review of the literature. Trauma Surg Acute Care Open. 2019;4(1):e000308. 8. Van Veldhuisen CL, Sissingh NJ, Boxhoorn L, van Dijk SM, van Grinsven J, Verdonk RC, et al. Long-term outcome of immediate versus postponed intervention in patients with infected necrotizing pancreatitis (POINTER): multicenter randomized trial. Ann Surg. 2023. 9. Kirkham JJ, Gorst S, Altman DG, Blazeby JM, Clarke M, Devane D, et al. Core Outcome Set—STAndards for Reporting: the COS-STAR statement. PLoS Med. 2016;13(10):e1002148. 10. Kirkham JJ, Davis K, Altman DG, Blazeby JM, Clarke M, Tunis S, et al. Core Outcome Set—STAndards for Development: the COS-STAD recommendations. PLoS Med. 2017;14(11):e1002447. 11. Farrell M. Core outcomes in emergency general surgery: development of a core outcome set for acute necrotizing pancreatitis. Available at: https://www.comet-initiative.org/Studies/Details/2513. Accessed January 10, 2023. 12. Eastern Association for the Surgery of Trauma. Landmark Papers in Trauma and Acute Care Surgery. Available at: https://www.east.org/education-resources/landmark-papers-in-trauma-and-acute-care-surgery. Accessed January 5, 2023. 13. Guyatt GH, Oxman AD, Kunz R, Atkins D, Brozek J, Vist G, et al. GRADE guidelines: 2. Framing the question and deciding on important outcomes. J Clin Epidemiol. 2011;64(4):395–400. 14. Knaapen M, Hall NJ, Moulin D, van der Lee JH, Butcher NJ, Minneci PC, et al. International core outcome set for acute simple appendicitis in children: results of a systematic review, Delphi study, and focus groups with young people. Ann Surg. 2022;276(6):1047–1055. 15. Byerly S, Nahmias J, Stein DM, Haut ER, Smith JW, Gelbard R, et al. A core outcome set for damage control laparotomy via modified Delphi method. Trauma Surg Acute Care Open. 2022;7(1):e000821. 16. Gelbard RB, Nahmias J, Byerly S, Ziesmann M, Stein D, Haut ER, et al. Establishing a core outcomes set for massive transfusion: an Eastern Association for the Surgery of Trauma modified Delphi method consensus study. J Trauma Acute Care Surg. 2023;94(6):784–790. 17. Ziesmann M, Byerly S, Yeh DD, Boltz M, Gelbard R, Haut ER, et al. Establishing a core outcome set for blunt cerebrovascular injury: an EAST modified Delphi method consensus study. Trauma Surg Acute Care Open. 2023;8(1):e001017. 18. Team RC. R: A Language and Environment for Statistical Computing. Vienna, Austria: GNU Software; 2020. 19. Wolak ME, Fairbairn DJ, Paulsen YR. Guidelines for estimating repeatability. Methods Ecol Evol. 2012;3:129–137. 20. Portney LG, Watkins MP. Foundations of Clinical Research: Applications to Practice. 2nd ed. Hoboken, NJ: Prentice Hall Health; 2000. 21. Trikudanathan G, Dirweesh A, Faizi N, Schat R, Suryavwanshi G, Lunos S, et al. Predicting need for intervention in acute necrotizing pancreatitis following discharge — a single center experience in 525 patients. Pancreatology. 2022;22(8):1063–1070. 22. Schepers NJ, Bakker OJ, Besselink MG, Ahmed Ali U, Bollen TL, Gooszen HG, et al. Impact of characteristics of organ failure and infected necrosis on mortality in necrotising pancreatitis. Gut. 2019;68(6):1044–1051. 23. Padhan RK, Jain S, Agarwal S, Harikrishnan S, Vadiraja P, Behera S, et al. Primary and secondary organ failures cause mortality differentially in acute pancreatitis and should be distinguished. Pancreas. 2018;47(3):302–307. 24. Le Mée J, Paye F, Sauvanet A, O'Toole D, Hammel P, Marty J, et al. Incidence and reversibility of organ failure in the course of sterile or infected necrotizing pancreatitis. Arch Surg. 2001;136(12):1386–1390. 25. Evans RP, Mourad MM, Pall G, Fisher SG, Bramhall SR. Pancreatitis: preventing catastrophic haemorrhage. World J Gastroenterol. 2017;23(30):5460–5468. 26. Leonard-Murali S, Lezotte J, Kalu R, Blyden DJ, Patton JH, Johnson JL, et al. Necrotizing pancreatitis: a review for the acute care surgeon. Am J Surg. 2021;221(5):927–934. 27. Bassi C, Marchegiani G, Dervenis C, Sarr M, Abu Hilal M, Adham M, et al. The 2016 update of the International Study Group (ISGPS) definition and grading of postoperative pancreatic fistula: 11 years after. Surgery. 2017;161(3):584–591.