Baseline characteristics of the alerts included in this analysis

Thirty seven out of the 187 laboratories were included. Out of 1,966,003 AKI alerts representing 532,884 patients received by the UKRR from the beginning of the NHS England algorithm deployment from December 2014 to the end of September 2020, 1,579,663 alerts—from 475,634 individuals—were available for analysis; exclusions are shown in Fig. S1. Out of the 37 laboratories considered to have high data completeness, eight were excluded for other reasons. Two laboratories inconsistently sent monthly data leading to frequent missing SCr values over time. There was evidence of missing pre-alert SCr data for five laboratories. One laboratory was excluded because only SCr data were sent but not any alerts. Patient, alert and SCr characteristics are shown in Table 1. Laboratories included in this analysis were well represented across England (Fig. S2).

Table 1 Characteristics of alerts that were included in this analysisCharacteristics of the centrally-derived alerts

The central code computed a greater number of AKI non-zero alerts (1,646,850) than those received by the UKRR, indicating possible under-ascertainment or incomplete submission of alerts. Compared to local laboratory-generated alerts displayed in Table 1, the SCr at central alert was higher than local laboratory-generated alerts for AKI 1 (median 140 vs 132 μmol/L) and AKI 2 (median 197 vs 181 μmol/L) but was 53 μmol/L lower for AKI 3 (median 361 vs 414 μmol/L).

Reference Value 1, which could potentially be drawn during the prodromal illness before AKI is recognised, was lower than Reference Value 2 for AKI 0 (78 vs 86 μmol/L), but higher than Reference Value 2 for all other stages—AKI 1: 109 vs 90, AKI 2: 122 vs 89, and AKI 3: 161 vs 88 μmol/L. The index creatinine (C1): Reference Value ratio, which is used to calculate relative changes in SCr, averaged 1.62 (IQR 1.53; 1.75) for AKI 1, 2.30 (IQR 2.12; 2.56) for AKI 2 and 3.81 (IQR 3.21; 5.13) for AKI 3. Although the baseline (Reference Value 1) was higher, the median index creatinine: Reference Value ratio for AKI 1 from between 0 and 7 days was < 1.5 suggesting that the relative change in SCr that flagged AKI 1 was from the 8–365-day period or an absolute increase in SCr of > 26 μmol/L in 48 h.

Local laboratory versus central generated alerts

Table 2 shows a cross-tabulation of local laboratory-generated alerts compared with those derived centrally. The local laboratory and central alerts mostly concurred. The majority were AKI 0.

Table 2 Matrix of raw numbers of laboratory versus centrally derived alerts

Laboratory 0 alerts were assumed on the basis that an alert was not generated for a given SCr value. This was only assumed for alerts generated after the first-ever alert for that laboratory was submitted as a proxy for the date of activation of the algorithm. Note that the number of alerts here reflects each individual SCr result available for analysis, hence the very many 0 AKI alerts.

Inter-rater analysis of local laboratory versus central alerts

In general, agreement was almost perfect—overall Gwet’s AC1 was 0.98 and 26 laboratories computed a Gwet’s AC1 of > 0.80 (Table 3). Some alerts generated within laboratories showed only slight agreement with the central code; Gwet’s AC1 ranged from 0.17 to 0.23 in three. This was driven by two patterns of misclassification in these three laboratories: (1) central AKI 0 was misclassified as laboratory AKI 3 in 38–55% of central AKI 0 alerts and (2) central AKI 0 was misclassified as laboratory AKI 1 in 88–92% of central AKI 0 alerts.

Table 3 Agreement coefficients for anonymised individual laboratories

Table S2 shows that assessment of agreement was also almost perfect (> 0.80) across all three laboratory information management system providers; Gwet’s AC1 ranged from 0.97 to 0.98. The laboratory information management system provider was unknown for 16/29 laboratories. As such, the provider was only available for 43% of local laboratory alerts. Of the three laboratories demonstrating slight agreement, two of these utilised the laboratory information management system 1 provider, previously shown to be performing well at the laboratory information management system-level. This suggests a local laboratory rather than a laboratory information management system inconsistency. Cumulatively, these represented a small proportion of the total alerts (8%).

Exploratory analyses

Firstly, given that there were laboratories that submitted incomplete alert data, these 11 laboratories were excluded. Table S3 showed that by excluding laboratories with patchy missing data, agreement was no higher (Gwet’s AC1 was 0.97).

The UKRR does not receive AKI 0 alerts and so local laboratory-generated alerts were not exactly comparable with alerts generated by the central code. For this reason, AKI 0 alerts were assumed on the basis that an alert was not received. Agreement, between nonzero AKI 1/2/3 alerts only, was once again almost perfect, though substantially lower, when local laboratory alerts were not recoded to AKI 0 – Gwet’s AC1 was 0.83 versus 0.97. Alert agreement is compared in Table S4 (number of alerts) and Table S5 (agreement coefficients).

An exploratory analysis, presented in Table S6, found that Gwet’s AC1, per calendar year, from 2015 to September 2020 was similar even during the 2020 Coronavirus-2019 pandemic (0.96 compared to 0.97 the previous year): laboratory implementation of the NHSE algorithm was sequential and the syntax was not altered over time. Serum creatinine data submission was incomplete for seven laboratories in 2020 compared to one in 2019.

Over quartiles of baseline SCr, agreement was high at lower SCr but decreased substantially at higher baseline SCr values. Gwet’s AC1 was 0.98 for quartile 1 but dropped to 0.88 for quartile 4 as shown in Table S7. Moreover, for the highest baseline SCr, quartile 4, the frequency of AKI 1 was greatest for central compared to local laboratory-generated alerts (18% [central] versus 15% [local]) but lower for stage 3 alerts (4% [central] versus 9% [local]). This suggests that severe AKI might be under-reported in people with pre-existing CKD because alerts are suppressed by laboratories.

Finally, agreement was found to be almost perfect (> 0.80) and consistent in younger and older people. In Table S8, the median age for each quintile ranged from 45 to 88 years and Gwet’s AC1 was between 0.81 – 0.85.