Expecting Intra-Class Correlation (ICC) coefficients higher than 0.7, an accuracy of 14% was achievable using twenty patients and twenty-two observers. Even when assuming an ICC of 0.65 or 0.60 an accuracy of 16% or 17% was achievable, respectively [21]. Thus, twenty patients below fifteen years who sustained a proximal humeral fracture between 2019 and 2022 were retrieved from our database. Their radiographs were completely anonymized, with only the age (in full years) written on them. The mean age was 9.8 years (± 3.3). Half of the patients were ten years old or younger, and the other half were older than ten years. The population included fifteen females and five males.

Twenty-two senior members from the “Section of Pediatric Traumatology of the German Association for Trauma Surgery” (“Sektion Kinder-Traumatologie”, SKT der Deutschen Gesellschaft für Unfallchirurgie (DGU)) were asked to participate in the study. All committee members are fellowship-trained trauma surgeons with long-time expertise in pediatric traumatology and work in Level-one trauma centers in Germany.

On each radiograph, observers were asked to determine the fracture angulation. Purposely, there was no obligation to apply a specific measurement technique. Besides, observers were asked to choose an operative (ESIN, K-wire, other) or non-operative (Gilchrist, Cuff/Collar, other) procedure. They could state if more imaging was necessary to make a treatment decision.

The recently introduced online-tool Tyche™ v1.0 (Mainz, Germany) [22,23,24] was utilized to facilitate a multi-center study including experts from different hospitals. Fully anonymized images were temporarily uploaded in JPEG format to Tyche, where only dedicated observers had temporary access via encrypted connections. Images were analyzed blinded, in random order, and with means to store results online on the same window. Observers could use standard imaging tools like zoom and contrast and the standard and Cobb angle tools (Fig. 1). Results were immediately merged and visible to the project manager.

Image analysis was performed blinded and in random order using the online-tool Tyche

Twenty-two experts in pediatric traumatology assessed twenty radiographs twice using the online-tool Tyche. Images were shown in random order with standard tools for analysis. On the same window was an input field to store the fracture angulation and a single-choice question with seven answers to choose operative and non-operative treatment. Results were immediately visible to the project manager

Mean values with standard deviations (SD) were calculated for the fracture angulation. The accuracy of single measurements was calculated as described by Popović et al. [25]: For every image, the SD between the observers was calculated. Then, the average of these SDs was calculated and termed the “mean of individual standard deviations”. Lower values indicate higher accuracy.

To assess the measurement reliability of the fracture angulation, Intra-Class Correlation (ICC) coefficients were calculated. For interobserver reliability, ICC(3,k) was used; for intraobserver reliability, ICC (3,1) was used [26]. To estimate agreement on the non-metrical results, Fleiss’ Kappa coefficients were calculated. ICC and Fleiss’ Kappa coefficients were interpreted as shown in Table 1. For statistical analysis, SPSS 27 (IBM, Armonk, USA) and Prism 9.4 (GraphPad Software, California, USA) were used.

An ordinal scale was created to evaluate treatment decisions by giving every non-operative treatment decision the number one and every operative treatment decision the number two. Thus a “mean treatment decision” ranging between one and two was calculated between all experts for every fracture. For mean values below 1.2, a consensus for non-operative treatment was concluded. Likewise, for mean values above 1.8, a consensus for operative treatment was supposed.