Scope

The COS will include the most relevant outcome measures for NEC (defined as Bell’s stage ≥ 2) for all stakeholders involved in NEC, including patients and their families, from moment of diagnosis into adulthood. This COS will be designed to standardise the reporting of outcomes in NEC treatment trials, as opposed to diagnostic or prevention studies.

Key objectives 1.

Achieve consensus between stakeholders on an international COS, focused on reporting NEC outcomes in scientific literature

2.

Compare outcomes prioritised by three stakeholder groups (lay, neonatal and non-neonatal panels)

3.

Identify possible differences between low-middle-income and high-income countries

Design

An international steering committee was established by contacting major research groups involved in NEC. The steering committee consists of two parent representatives, nine paediatric surgeons, four neonatologists, and two PhD candidates, from several continents. The steering committee agreed on the protocol and will provide input throughout the project. Within the steering committee, a smaller study management group (JBFH, EMWK, OCvV, and DHK) will meet regularly in between steering group meetings.

The Medical Ethics Review Board of the University Medical Center Groningen, The Netherlands, has reviewed this study and decided that this is not clinical research with human subjects as meant in the Dutch Medical Research Involving Human Subjects Act (WMO). Further ethics approval was therefore waived. With this decision, each participating country or research group will be asked to ascertain local ethical board approval or confirm that this project does not need ethical board approval. Electronic informed consent will be obtained from all participants. They will be informed about the nature of the study before voluntary registration and starting the first round. Participants will be encouraged to complete all rounds of the study but can withdraw at any moment.

The study was registered with the COMET initiative on 17 August 2021, study number 1920. This protocol is constructed in accordance with the COS-STAP statement; the checklist can be found in Additional file 1 [14]. Steps followed to develop this COS can be seen in Fig. 1.

Fig. 1

Flow chart including all steps that will be followed for developing this COS

Systematic review

We performed a systematic review, identifying all outcomes used in intervention studies and systematic reviews aimed at improvement of NEC outcomes published in the past 6 years. The review has been registered in PROSPERO (CRD42022302712) and has been submitted for publication separately. In this review, we identified sixty-four unique outcomes which we mapped to 52 outcome terms (e.g. wound infection, fistula, and stoma prolapse were mapped to ‘postoperative surgical complications’). Outcomes were subsequently excluded because of being general NICU outcomes (e.g. neonatal pneumonia or antibiotic regimen) or specific use in a treatment trial (e.g. skin burning due to use of the intervention). Ultimately, 31 were adopted into a preliminary outcomes list for the COS. The identified outcomes were categorised by two independent researchers (DHK and OCV) according to Taxonomy for Outcomes in Medical Research [15]. After review of the initial list, steering group members added fourteen terms resulting in a total of 45 candidate outcomes. A detailed list of all excluded and added outcomes can be found in Additional file 2. These outcomes will be used as the starting point for establishment of the COS with different stakeholder groups.

The steering committee and the independent Dutch neonatal patient organisation (Care4Neo) reviewed the final list of outcomes entered into the first Delphi round, to evaluate wording and add lay terms before starting the first Delphi rounds. This was done to ensure all outcomes are appropriately presented to both former patients, parents, and experts.

Participants

Clinicians involved in the initial treatment of NEC may have a different set of priorities compared to former NEC patients and their parents or paediatricians who treat infants at NEC follow-up consultations. To ensure that this COS represents the views of former patients and their parents, clinicians active in the neonatal period and experts involved outside the neonatal period, these stakeholders will be separated into three panels [16].

1.

Lay panel—parents of infants diagnosed with NEC as well as adults who experienced NEC in infancy

2.

Neonatal panel—clinicians and researchers involved in NEC management in the neonatal (NICU) period. This group will include neonatologists, (paediatric) surgeons, gastroenterologists, paediatricians, researchers, nurses, physician assistants, and fellows.

3.

Non-neonatal panel—clinicians and researchers responsible for follow-up after NEC treatment, outside of the neonatal period. This group will also include rehabilitation therapists, physiotherapists, and (neuro)psychologists. If experts are involved in both the neonatal period and afterwards in the non-neonatal period, we (arbitrarily) opted to include them into the neonatal panel only.

For the lay panel, ex-NEC-patients and parents will be recruited through NEC patient organisations such as the NEC UK Charity and the Brazilian Instituto Pequenos Grandes Guerreiros, as well as through general neonatal societies such as the Dutch Care4Neo, the European EFCNI, and the Tunisian Neonatal association. Involvement of parents and patients’ representatives will be described using the GRIPP2 (Guidance for Reporting on Involvement of Patients and Public) short form checklist [17].

We will identify experts in the second and third panel using Scival Trend Analysis. Scival is a bibliometric tool that enables identification of the most prolific authors by continent. The basis for the NEC analysis is a set of publications from MEDLINE as result of a search for the MESH-term “Enterocolitis, Necrotizing” over the years 2016–2021 The set was imported into Scival for a Trend analysis, focusing on top authors and institutions in all regions of the world, looking at numbers of publications, but also at the (averaged) Field-Weighted Citation Impact (FWCI).

We will approach around 125 experts from low-middle and high-income countries. Countries are categorised based on the latest Organisation for Economic Co-operation and Development classification [18]. We will approach the top 20 authors per continent, with a maximum of one author per affiliation. In addition, the top 50 authors worldwide will be added if they were not present in their respective continental list or removed because another member of their affiliation is already present.

The invitation letters to potential participants will be developed separately for the professional stakeholder panels (neonatal and non-neonatal) and the lay panel, using appropriate language agreed on by the steering group (thereby including patient representatives). Patient organisations will also be encouraged to recruit former patients and parents or caretakers through social media. The invitation letter will include a standardised social media statement.

Delphi process: phase 1

We aim to include at least 40 experts divided over the two professional panels and an equal number of participants in the lay panel, for both low-middle and high-income countries. We recognise that finding former patients, parents and experts may be more difficult in low-middle-income countries. If necessary, we will invite participants from low-middle-income countries through the personal networks of steering group members. The invitation letter to all potential participants will contain a link to an online registration form in the customised Delphi Manager Tool. After confirming participation, participants will provide information on their country of work, gender, current role or function, years of work experience with NEC and amount of NEC cases encountered on average per year.

The DelphiManager software, developed and maintained by the COMET Initiative, is used to undertake the e-Delphi surveys [13]. In the first phase, participants will be presented with the list of outcomes identified through the previously conducted systematic review. Outcomes will be presented using the order of the Taxonomy for Outcomes in Medical Research, with functional outcomes presented first. Participants will score outcomes on a 9-point Likert scale, with 1–3 labelled ‘not important’, 4–6 labelled ‘important but not critical’, and 7–9 labelled ‘critical’. All participants are encouraged to keep in mind that we aim to report between 8 and 15 outcomes in the final COS. Clarifications for scientific terms including lay terms will be available in the questionnaire. As a final question in the phase 1 questionnaire, participants can add any outcomes they consider important in determining the success of NEC treatment that have not been identified in the previous outcome list. There will be no maximum to the number of additions that a participant can enter.

To limit attrition, reminder emails will be sent two weeks after the initial contact. If participants have not completed the questionnaire after 4 weeks, they will be contacted again by email to enquire if they are having difficulties in completing the questionnaire or have decided to end their participation in the study. Participants who have not completed the questionnaire within four weeks of the phase starting will be deemed not to have completed that phase and will be removed from any following phases.

The response rate from each panel group will be recorded and the median score as well as the interquartile range (IQR) will be calculated for all outcomes. Additional outcomes provided by participants will be reviewed independently by two members of the study management group to ensure they represent new outcomes and will be included in phase 2 if they were proposed by at least two participants and deemed suitable by the steering group.

Delphi process: phase 2

Participants who completed phase 1 will be invited to participate in phase 2. All outcomes will be carried forward to phase 2. Participants will be presented individually with their own scores and a graphical description of the distribution of scores of the three different panels. This allows participants to consider the views from their and other stakeholder groups’ before rescoring the outcomes. Subsequently, they will be asked to rescore each outcome. Participants will also be asked to score any new outcomes identified after phase 1.

Data analysis described for phase 1 will be repeated. Bias from loss of participants between phases will be assessed by looking for differences in median scores between participants who have completed both phases and participants who only completed phase 1. Any outcomes that meet the criteria of ‘consensus out’ in a minimum of two panels will be removed from the outcomes list prior to phase 3 (see the ‘consensus out’ definition below). All other outcomes from phase 2 will be carried forward to phase 3.

Delphi process: phase 3

Participants who completed phases one and two will be invited to participate in phase 3. The data collection process described for phase 2 will be repeated. The graphical description of the scores from all three panels is shown again, now also for the outcomes added after round 1. This allows all outcomes to undergo at least one round of stakeholder group feedback [19]. The data analysis process described for phase 2 is repeated.

Consensus definition in Delphi rounds

‘Consensus in’ is defined as ≥ 70% of participants rating the outcome 7–9 and ≤ 15% rating it 1–3. ‘Consensus out’ is defined as ≥ 70% participants rating an outcome 1–3 and ≤ 15% rating it 7–9. Outcomes not meeting these definitions will be classified as ‘no consensus’.

Final COS consensus meeting

The aim of the consensus meeting is to confirm outcomes where consensus ‘in’ has been achieved and to finalise the COS. Participants who have completed all three rounds Delphi are invited to attend an online consensus meeting. We aim to have a minimum of 20 stakeholders confirm their attendance with equally weighted panels and disciplines. Representatives from all stakeholder groups are required for the meeting to take place.

During the meeting, stakeholders are given an overview of the results of phase 3, including presentation of the top ten outcomes reaching consensus ‘in’ in the lay stakeholder group as well as the top ten outcomes consensus ‘in’ in the combined neonatal and non-neonatal stakeholder group. They will be shown how these outcomes were scored by each stakeholder panel and its consensus status. Following moderated discussion, these outcomes will be anonymously rescored using the same scoring system as the Delphi process. Outcomes that reach consensus ‘out’ or no consensus at all in phase 3 are only discussed after a unanimous decision by all attending participants. Following rescoring at the consensus meeting, outcomes reaching ‘consensus in’ will be included in the finalised COS. All others will be excluded.

Finalising the COS

We will incorporate the OMERACT 2.0 framework in this COS and aim to include at least one outcome for every domain (death, pathophysiological or clinical manifestations, life impact, adverse events and contextual factors and resource use) [20]. Overall, we aim to develop a manageable COS, including between 8 and 15 outcomes. A separate meeting will be held to identify measurement definitions for each outcome included in the COS.

Following the online consensus meeting, a consensus document containing the final COS will be written and distributed amongst all participants. This document will be presented as appropriate during international meetings and submitted as a manuscript for publication in a peer-reviewed journal.

Sub-group analysis

To evaluate whether the views on outcomes’ importance vary substantially between the three stakeholder groups, we will perform a sub-group analysis comparing phase 3 scores from the neonatal panel with phase 3 scores from the non-neonatal panel and the lay panel. In addition, scores from different specialties (paediatric surgeons, neonatologists, gastroenterologists etc.) will be compared. Finally, scores from participants working and/or living in low-middle-income countries will be compared with participants from high-income countries.

Data management

Participants will enter data directly into the customised and secured DelphiManager database when they complete each questionnaire at each phase of the Delphi process. All participants will receive a unique participant number and anonymised data will be stored securely. Only members of the study management group can access this data.