Study selection

Following screening of 6119 unique articles, 519 studies reporting data on SSI-CRAN outcomes were included in the final dataset (Fig. 1). The reliability of study selection was substantial at both the title and abstract (k = 0.78), and full text stages (k = 0.87) [27].

Fig. 1

PRISMA flow diagram for studies included and excluded from the systematic review

Characteristics of included studies

The characteristics of the 519 primary studies, including their Digital Object Identifiers (DOIs), are presented in Supplementary Table 3.

A hundred and three (19.8%) of the included studies were in neuro-oncology, 92 (17.7%) in neurotrauma, 56 (10.8%) in paediatric neurosurgery (10.8%), 52 (10.0%) in functional surgery), 38 (7.3%) in neurovascular, 37 (7.1%) in skull base, 11 (2.1%) in CSF (cerebrospinal fluid) dynamics (2.1%), and 149 (28.7%) in general neurosurgical cases. The remaining were of overlapping subspecialities.

On assessing the risk of bias using the JBI checklist, 167 (32.2%) studies attained a full score of 11. Seventy (13.5%) studies attained a score of 10, 204 (39.3%) attained a score of 9, 48 (9.3%) attained a score of 8, 24 (4.6%) scored 7, and six (1.1%) scored 6.

Whether SSI-CRAN was defined in the study

A hundred and sixty-nine (32.6%) of the 519 included studies provided a definition of SSI-CRAN in their methods, with 163 (31.4%) studies specifying indicators that represented SSI-CRAN. Across the 519 studies reporting SSI-CRAN, a definition of SSI-CRAN was not provided in 350 (67.4%) studies reporting an SSI-CRAN outcome. There was no association between studies with higher overall quality and clearer definitions of SSI-CRAN (Supplementary Table 4).

Indicators used to define SSI-CRAN

It is well established that different signs are relevant at various depths of SSI. The major categories of wounds – superficial, deep and organ space infections – should be considered separately to minimise the possibility of overlooking the presence of infection. Superficial infections typically present with localised signs that correspond to the following SSI indicators: erythema and purulence, while deep infections involving underlying tissue layers present with systemic signs that correspond to SSI indicators such as fever and increased white blood cell count. Organ space infections have been characterised by indicators such as meningitis or ventriculitis.

Eighty-seven (51.5%) of the included studies defining SSI-CRAN referred to the varied signs of SSI-CRAN at different depths following cranial surgery.

The indicators used to define SSI-CRAN in the 163 studies (31.4%) that reported the use of one or more specific indicators are presented in Supplementary Table 5. Twenty-six different indicators were used to define SSI-CRAN across all these studies (Fig. 2). The median number of indicators across the included studies was nine, with a range of one to 16 indicators.

Fig. 2

Number of studies employing the relevant indicators to diagnose SSI-CRAN (n = 519 studies)

From the studies defining SSI-CRAN, the most frequently reported indicators were the presence of pus or purulent discharge (n = 112; 66.3% of studies defining SSI), presence of bacteria in the wound, identified from swab of pus or exudate (n = 109; 64.5%), wound dehiscence or wound reopened by surgeon (n = 108, 63.9%), change in colour or erythema of wound (n = 99; 58.6%), oedema (n = 86; 50.9%), pyrexia (n = 98; 58.0%), pain (n = 91; 53.8%) and positive CSF culture (n = 113; 66.9%).

Seven (4.3%) of the 163 studies defining indicators for SSI-CRAN used indicators that represented a group of signs and symptoms: cellulitis was used as an indicator to define SSI-CRAN in five (3.0%) studies and clinical symptoms or findings in two (1.2%) studies.

Several studies employed non-specific indicators including ‘clinician diagnosis of SSI-CRAN’ (n = 87, 53.0%), ‘cellulitis’ and/or ‘meningitis’ (n = 96, 58.9%) and ‘clinical signs’ which were not specified (n = 2, 1.2%).

Administration of antibiotics was used to define presence of SSI-CRAN in 12 of the 163 (7.4%) studies.

Seventeen (10.4%) studies reported the use of only one indicator. Of these, 10 (58.5%) used a positive culture (eight for CSF culture, two for wound swab culture) as an indicator of SSI. From the remaining seven studies, six (85.7%) used the requirement for reoperation and one (14.3%) used blood results including CRP as the sole indicator for SSI.

Whether the same indicators were used to define SSI-CRAN across studies

Of the 169 studies providing a definition for SSI-CRAN, 82 (48.5%) studies used the same definition of SSI-CRAN, guided by the Centers for Disease Control and Prevention (CDC) guidelines (definitions provided in Supplementary Table 6). These guidelines outlined 15 indicators and covered three categories of SSI: superficial SSI, deep SSI and organ-space SSI [28]. It requires a follow-up of 30 days post-operatively, extended to 90 days if an implant is present. However, it was noted that many studies that applied these guidelines frequently altered this follow-up period. It is also important to note that the CDC guidelines include a ‘diagnosis made by surgeon or attending physician’ as part of their criteria which may contribute to the variations in the incidence of SSI reported across these studies.

Other guidelines used by authors included the National Surgical Quality Improvement Program (NSQIP) guidelines for SSI following deep brain stimulator insertion (n = 1, 0.6%), the UK Health Protection Agency guidance (n = 1, 0.6%), and the Protocol for Surveillance of SSI published by Public Health England (n = 2, 1.2%) which assessed for 11 indicators covering superficial, deep, and organ-space SSIs (Supplementary Table 6).

In 10 of the 17 (58.8%) studies that used a single indicator to define SSI-CRAN, it was defined as the presence of an organism isolated from culture (eight based on CSF culture and two on wound swab or tissue culture). Among the 17 studies that used two indicators to define SSI-CRAN, two (11.8%) used the same indicators to define it: wound dehiscence or wound requiring reoperation alongside the initiation of an antibiotic course to treat suspected infection.

Numerical values for indicators used to determine the presence of SSI-CRAN

Out of the 109 studies using a positive wound swab or tissue culture as an indicator for SSI-CRAN, only one (0.9%) study stated that SSI-CRAN was defined as > 105 colony forming units per gram of tissue. The remaining studies did not clearly report what numerical values were used to detect the presence of bacteria in the wound.

Whether a method for combining data from several indicators to determine presence of SSI-CRAN was specified (e.g. a count of the number of indicators present or a weighted scoring system)

A total of 144 (27.7%) studies used more than one indicator to define SSI-CRAN. Of these, none reported a method for rating or combining data from the multiple indicators used to determine whether SSI-CRAN was present.

Primary study conclusions about intervention effectiveness and impact of heterogeneity of SSI-CRAN definition on them

Of the 169 primary studies that defined indicators for SSI-CRAN, 160 (95.0%) reported the effectiveness of the intervention assessed in their study, of which 21 (13.1%) studies suggested that their results should be interpreted with caution due to insufficient quality of the data, limiting the ability to draw effective conclusions surrounding SSI-CRAN outcomes. Four (2.4%) primary studies, which defined SSI-CRAN, did not specify conclusions on the effectiveness of the intervention for SSI-CRAN.

Of the 169 primary studies that reported definitions for SSI-CRAN, 18 (11.0%) addressed the heterogeneity of SSI-CRAN definitions in their conclusions summarising the effectiveness of the assessed intervention. In 145 studies that reported criteria for SSI-CRAN, the authors did not take into consideration the varying definitions of SSI across literature in their findings.

Three of the 350 (0.8%) studies which provided no definition of SSI addressed this heterogeneity of SSI-CRAN definitions in their findings. Three main types of conclusions were identified across the 21 studies that referred to heterogeneity of SSI-CRAN definition in their conclusions. Across the 21 studies that acknowledged the varying definitions of SSI-CRAN, three main types of conclusions were drawn. In two studies, the authors specified they could not accurately analyse the findings of the intervention’s efficacy for SSI-CRAN due to the discrepancies in SSI definition present across literature. In 11 studies, the authors advised to interpret their conclusions with caution due this heterogeneity. The conclusion of two studies indicated no significant effect of the intervention was found, with the authors suggesting this may have resulted from the various differences in SSI-CRAN definitions.

Of the 350 studies that did not specify any indicators for SSI-CRAN, 175 (50.0%) stated the effectiveness of the intervention for SSI-CRAN, of which three (1.7%) mentioned that their results surrounding SSI-CRAN should be interpreted cautiously due to the insufficient quality of the data.

No conclusion regarding the effectiveness of the intervention was drawn from the other 181 (51.7%) studies.

From the 498 studies that did not mention data in their results regarding variations of SSI-CRAN definition across literature, only two (0.4%) authors considered the variation in the indicators used in their own study. In two studies, the authors specified that they could not assess their findings as a result of the discrepancy in SSI-CRAN definition, whilst in 11 studies, the authors advised to cautiously review their conclusions due to this heterogeneity in definition.