The present modified Delphi consensus process included representatives from all different PICUs across Canada, and identified 72 measurable data elements from six domains, for inclusion in a national PICU database that is currently being operationalized. Active recruitment and engagement of participants across the pediatric critical care community of practice resulted in high response rates from a diverse and heterogenous group of experts, and emphasizes the importance of this work. In contrast to national PICU data sets in other countries created by a small group of individuals, we have shown that development of consensus within a diverse group of experts was possible and resulted in a broad and inclusive group of data elements. Furthermore, we have established a framework to apply this methodology to the creation of other data registries.

The modified Delphi method phase used an initial small focus group of physicians and nurses from various centres to generate a list of variables based on literature, expertise, and other PICU databases. The larger Delphi panel of experts then suggested seven further variables to be added, but none of these were ultimately included in the mandatory data set based on consensus. The small focus group therefore likely appropriately reviewed evidence to select the variables, with their expertise in clinical research, data quality, and data acquisition likely strengthening this selection. The modification of this step in the Delphi process is therefore an efficient and appropriate method of variable preselection prior to consensus generation.

The exclusion of certain data elements in the consensus highlights whether relying solely on consensus methodology is the best approach for selecting a minimal data set. While consensus methodology effectively aggregates the anonymous opinion of the majority, other valuable but less commonly collected or reported data elements may be omitted. Furthermore, there is not a “correct” answer in a Delphi process, but rather a reflection of what the selected participants deem is important. In this case, the majority of selected elements reflected diagnostic categories, advanced medical care delivered, and PICU complications. These choices may reflect the potential of certain interventions to cause significant unintended harm or long-term impacts in patients.

A striking feature of the Delphi results was the lack of consensus regarding inclusion of elements describing social determinants of health. While race, sex, gender, and the first three digits of the postal code (geographic location) were included, elements such as minority status, indigenous status, primary language, family income, education, and ethnicity did not achieve consensus. There may be several reasons for reticence to include these data elements in the data set, including challenges with standardizing data collection, concerns for patient privacy, or lack of understanding of the item’s importance in the child’s health status. Furthermore, physiologic clinical variables may be as easier to standardize and collect than self-reported data of social constructs such as race and ethnicity. Regardless, these data elements remain important determinants of child health inequities in Canada,26 and serve as a foundation for future work in pediatrics in Canada.

Despite the significant increase in research on understanding outcomes after PICU, no scales associated with global or cerebral functional status were included. In addition, despite data to suggest the importance of transport on PICU patient outcomes,27 none of the transport data elements were included. Given that these pre-ICU admission data are not a part of the ICU electronic medical record, participants may not have included these due to perceived feasibility challenges related to collection and accuracy. Lastly, although patient weight is an included data element, there are no included measures of ideal body weight, body mass index, or height. Given the importance that obesity can have on outcomes of critical illness and long-term health, this exclusion will remain an important variable for future consideration. Data elements that did not achieve the vote of all members (e.g., “master ID” or “primary diagnosis”) highlight the heterogeneity, selective expertise, and knowledge bases of the panel members; each with their own use for the database. In this case, a hybrid approach for common data element inclusion, with oversight of a larger steering committee for example, may be necessary to avoid the omission of key data elements in addition to all those voted on by panel members.

Our study had some limitations. First, a low test-retest value (0.5) suggests a low reliability with equal true and error-related variance. It is unclear whether retesting would produce the same results given the possibility of measurement error.28 Second, despite the diversity of participants and their areas of expertise, the participant numbers were likely over represented by physicians, and limited in allied health professionals such as social workers and nutritionists. An increased presence of stakeholders, patients, and members specialized in equity, diversity, and inclusion may have balanced the perspectives of the group with regard to social determinants of health items. To overcome this limitation, we consulted the CIHI and the Public Health Agency of Canada to discuss adding important items for exploring social determinants of health, such as ethnicity. Improved understanding of the social determinants of health is essential to illuminate health care inequalities that exist for children in Canada, and to inform policy changes to improve health care access and quality for marginalized children. Ensuring that routinely collected data elements reflect the needs of a diverse range of stakeholders will amplify the sustainability of data collaboration. The aim to have a national, population-level data coverage, engaging public health and policy-makers has been emphasized as a means to strengthen our processes. Last, family perspectives were limited in the study as only one patient-partner was involved in this Delphi process.

Ongoing work includes clear ontogeny, definitions, and timing of data collection for the selected data elements, as some of these are not well established in children. In conjunction with pediatric members of the CCCTG, who conducted multiple randomized trials in children, we are harmonizing definitions for included data elements based on multiple recent trials29,30 to standardize data element inclusion. A parallel group has created a governance structure for how national data would be validated, accessed, and used for research and quality improvement. In terms of infrastructure, national partners in data management and adult critical care are creating an infrastructure for an open-source platform at each hospital. To overcome barriers preventing record-level data from leaving the provinces, data storage and analysis would be conducted behind a firewall at each individual site, and only aggregate data would exit the sites.

The core data elements selected in this Delphi study will be instrumental in assessing the determinants, clinical outcomes, and quality of care, of critically ill children in Canada. By pooling granular, standardized, anonymous data that goes beyond what is submitted by hospitals using diagnostic codes, we will have an unprecedented ability to ask targeted questions and learn from and improve our practices through quality improvement initiatives and research. Ensuring that variable coding can be harmonized with other data sources and is standardized to align with pre-existing data standards and international collaborations16 will ease linkage, sharing, and comparability across sites and registries. Overall, this will lead to greater insight in the effectiveness, safety, and quality of critical care.