In this retrospective study, we found a strong association between mortality, transfer to the ICU from the ED, and/or transfer to the operation room, and high triage level with the modified SATS. As expected, the triage level was more strongly associated with 24-h mortality than with 30 day mortality. The sensitivity and specificity to detect severe illness were 85.7% and 76.2%, respectively. Overtriage was relatively common, especially in children and surgical patients, while undertriage was uncommon (0.9%). This is, to our knowledge, the first study to assess the validity of the SATS in adults in a high-resource setting.

We found that the NPV, i.e., the ability of a low triage level to exclude a serious outcome was excellent and above 98% in all subgroups. However, this comes at the cost of a large proportion of patients assigned to high triage levels, who did not have the outcomes measured in this study. This does not necessarily mean that all these patients were overtriaged. Some may have needed urgent interventions that were not recorded in this study such as fibrinolytic therapy for ischemic stroke, antimicrobial therapy for serious infections not requiring ICU admission, and acute interventions not performed in the operating room such as vascular interventions performed in the Radiology Department. Furthermore, undertriage is a major concern as it can lead to delayed or inadequate treatment, which can result in poor outcomes and even death. When patients are not triaged properly and are not given the appropriate level of care in a timely manner, it can lead to a deterioration of their condition and a greater likelihood of complications. On the other hand, overtriage can lead to unnecessary testing and treatment, increased healthcare costs and longer wait times, but it does not put the patient's health at the same risk as undertriage. Compared to the findings in other studies, our study shows that the SATS-N performs as well as or better than other triage systems used in high-resource settings [20,21,22]. A systematic review found that most studies of emergency department triage systems reported a low sensitivity (< 80%) in identifying patients who had critical illness outcomes or died during hospitalization [1]. In a recent systematic review of triage systems in emergency care, the authors found that common triage systems had a sensitivity for ICU admission ranging from 58 to 88% in adults and 71–93% in children. The specificity of triage systems to accurately classify patients discharged home as low urgency ranged from 64 to 98% in adults and 69–96% in children [13]. It is, however, difficult to compare the validity of triage systems between different studies, as the definitions of over and undertriage are very heterogeneous [23]. Other studies of SATS, that were performed in low-resource settings have found a sensitivity from 91 to 96%. However, the studies used hospital admission as the gold standard for high acuity, and the reported mortality in the studies is very low [15, 18, 24].

All validation studies on triage systems are also subject to limitations due to the absence of a gold standard of urgency [23, 25]. We also have very limited knowledge about the effect of triage levels on clinical outcomes. Relevant clinical outcomes could span from patient satisfaction, resource utilization, patient harm due to delay in or improved health due to prompt treatment and assessment, to mortality. No single outcome can measure all of these factors. To validate triage systems it is necessary to construct the best proxy for the reference standard applicable to the whole spectrum of patients in the ED [2]. We chose 24 h mortality, admittance to the ICU or direct transfer to asurgical operating theatre (including for coronary angiography) as a proxy for true high acuity. Not all patients with true high acuity may meet these criteria. Hence, our strict definition of true high acuity might have excluded patients with severe conditions. Regarding true low acuity, which in this study was defined as discharge from the ED, we did not collect data on readmission within a short period of time after discharge. Hence, some patients might have been misclassified as low acuity patients, even though their clinical condition indicated more severe illness.

This study has some limitations concerning generalizability. First, the hospitalization rate in our hospital system is higher than that in many countries. In other studies on triage performance, the hospitalization rate ranges from 4 to 58% [13]. In Norway, the organization of the emergency health care system differs from many other countries in that it is based on a two-tiered system where the municipalities are responsible for the first part, and the hospital trusts are responsible for the more specialized part of the system. In a study of Norwegian emergency admissions, Blinkenberg and collaborators found that a out-of-hospital physician referred 57% of patients presenting to emergency departments were referred by an out-of hospital physician and 43% of patients presented directly, mainly by Emergency medical services [26]. Therefore, most patients who arrive in the ED are admitted to the hospital. Indeed, in our study, 81.7% of the patients were hospitalized. In this study, the findings for true low-urgency patients may therefore not be generalizable to other health systems with higher discharge rates directly from the ED.

Second, our ED only takes care of children with surgical conditions, and our findings may not be applied to children with medical conditions. However, we have previously published a study on the modified SATS in children with medical conditions, and a similar sensitivity was found in that study [9]. This was also a single centre study, and the performance of SATS-N may be different in other EDs.

A strength of our study is the use of data from the National Population Register to ensure that no deaths were missed. Mortality is a definite endpoint, and the ideal triage system would classify those with a high mortality risk into the highest acuity levels. Most studies that have looked at mortality in relation to triage level have used either ED mortality or in-hospital mortality as the endpoint [13]. The disadvantage of using these endpoints is that deaths that occur shortly after discharge are missed. Indeed, in our study, 11 patients discharged from the ED died within 24 h after ED presentation, and 76 and 220 of the patients who were not admitted died after 7 and 30 days respectively. Another strength of our study is that our definitions and assessment of triage performance are based on objective patient outcome markers and not on subjective expert opinions, as is the case in many studies on triage performance [23, 27,28,29,30].

In this singe centre study, we validated the SATS-N for a few large groups of patients in the ED. Future multicentre studies, preferably including other emergency settings, should expand on the results from this study to further establish the validity of the SATS-N. Moreover, effort should be made to identify possible smaller subgroups that may be at risk of undertriage by the triage system.