In this study, we aimed to determine the link between the level of experience in raters and the reliability of the assessment of NCCT markers. Our previous results demonstrated good-to-excellent levels of inter- and intrarater reliability and contribute to the results presented by Dowlatshahi et al. [6,7]. However, the different levels of experience amongst raters with a radiological background may result in significant interobserver variability and differences in the diagnostic accuracy for predicting acute HE. Therefore, the results of our analysis highlight several novel important findings. Firstly, the reliability of NCCT markers varied among raters with different levels of experience. Nevertheless, eight out of nine NCCT markers showed substantial-to-almost-perfect agreement, whereas a moderate agreement was only found for the swirl sign. The illustrative examples shown in Figure 1 demonstrate that especially very nuanced density changes, such as the streak-like morphology of the swirl sign (Figure 1A) or scattered, primarily satellite-suggestive hematoma with yet subtle connections to the main hematoma (Figure 1D), were difficult to identify. Moreover, the strict encapsulation of the hypodense area within the hematoma (hypodensities) was false positively rated, especially in cases of small hematomas with very nuanced NCCT feature attributes, as shown in Figure 1I. Semiquantitative measurements were error-prone for calculating the correct diameter of the hemorrhage for assessing the satellite sign (Figure 1C,D) and density differences for the black hole sign or blend sign (Figure 1E–G). In line with this, raters from different clinical backgrounds also tended to obtain higher proportions of positive ratings of NCCT markers according to a recently published study [27]. Variabilities in the ratings of the IRR shape, heterogenous density, and island sign may be further influenced by differences in the slice position of the region of interest (ROI), which should be placed on the axial slice with the largest cross-sectional area of the hematoma (Figure 1B,J) [5]. This is of clinical importance, as measurement error may potentially obscure the true predictive effects. Evaluated AUC differences for acute HE prediction were minor and found in only two out of nine NCCT markers. Our study had some limitations. Firstly, our study offered only limited conclusions to whether variances of agreements were attributed to the level of experience, as this would require a larger sample size within each category of experience included. Furthermore, given that the imaging analysis in an emergency setting is often much more rushed than in an elective case, accounting for the reading time may have resulted in additional variability of the readings. Secondly, our findings were derived from a retrospective analysis and require prospective confirmation. Finally, the imaging protocol was not standardized across participating sites. Nevertheless, there is no evidence that the NCCT acquisition technique influences NCCT markers’ detection [5].