The dataset consists of 1514 ECG-gated, non-contrast CT scans performed on patients enrolled in the multicenter randomized controlled DISCHARGE trial [16,17,18] (NCT02400229). Examinations were performed at 26 clinical sites across Europe using CT scanners with at least 64-slice detectors from four different vendors. An Agatston score was determined using non-contrast cardiac CT scans with a slice thickness of 3.0 mm. We included all non-contrast CT scans with slice thicknesses ranging from 2.4 to 3.0 mm and excluded CT scans with smaller slice thicknesses because they might affect calcium scoring reproducibility due to increased noise. It is important to note that calcium scans with metal artifacts or anatomical abnormalities were not excluded from the dataset. A flowchart of the data selection process is presented in Fig. 1. Baseline characteristics of the selected patients are compiled in Table 1.

Flowchart of dataset selection from the DISCHARGE dataset

The dataset containing 1514 calcium scoring CTs was randomly divided into three parts: 60% (908/1514) for training, 10% (151/1514) for validation, and 30% (455/1514) for testing.

Coronary segmentation diagrams [19], such as the SCCT coronary segmentation diagram [6], were defined based on the visibility of coronary artery segments in contrast-enhanced CT angiography (CCTA). Defining a segmentation diagram for non-contrast calcium scoring CT is challenging due to the lack of contrast between coronary arteries and surrounding tissue. Differentiation of side branches such as segment 12 (first obtuse marginal) and segment 14 (second obtuse marginal) or segments 16a (posterior descending artery from left circumflex artery, PDA-LCx) and 16b (posterior-lateral branch from left circumflex artery, PLB-LCx) is very challenging on non-contrast CT. Consequently, we modified the SCCT coronary segmentation diagram [6] for CAC scoring on the segment level as follows.

Coronary vessels were divided into proximal, mid, distal, and side branches. The adapted SCCT coronary segmentation diagram consists of 13 segments, including the left main (LM) and proximal, mid, distal, and side branches of the right coronary artery (RCA), left anterior descending (LAD), and left circumflex (LCX). The ramus intermedius, diagonal 1, and diagonal 2 are defined as LAD side branches (s-LAD). The first obtuse marginal (OM1) and second obtuse marginal (OM2) are defined as LCX side branches (s-LCX). Segments PLB-LCx and PDA-LCx are defined as LCX distal segments (d-LCX). The posterior-lateral branch of the right coronary artery (PLB-RCA) and the posterior descending artery from the right coronary artery (PDA-RCA) is defined as the RCA distal segment (d-RCA). The adapted SCCT coronary segmentation diagram is shown in Fig. 2.

Adapted SCCT coronary segmentation diagram for segment-level calcium scoring. All side branches of the RCA are summarized as s-RCA, while segments 3, 4a, and 4b are labeled as d-RCA. Segments 9, 10, and 17 are combined into s-LAD. Segments 12 and 14 are combined into s-LCX, and segments 15, 16a, and 16b are combined to segment d-LCX

The reference standard for the training, validation and test set of the model for the segmentation of CAC on the segment level (main task) was provided by a trained reader with three years of experience.

To evaluate the performance of the model in comparison with a human observer, we performed an interobserver variability analysis between the first observer (reference standard) and a second observer. The second observer was a trained reader with one year of experience who performed further segment-level CAC segmentation of the test set on the segment level.

Our goal was to ultimately develop a multi-task model for automated segmentation of coronary calcifications and coronary artery regions. To learn coronary artery regions not only from CTs with CAC but also from CT scans without CAC, we performed weak annotations of coronary segment regions (provided by a trained imaging scientist) using 3D Slicer [20] and made the algorithm learn this information as an auxiliary task. An example of an axial CT image that was annotated and the segmentations predicted by the model are presented in Fig. 3. Details of the annotation process are provided in Supplementary Material B.

75-year-old female patient with coronary calcifications. A Image slice with coronary calcifications in the LM, p-LAD, m-LAD, d-LAD, and s-LAD. B Annotations for segment-level calcium scoring provided by the first observer. C Annotations for segment-level calcium scoring provided by the second observer. D Model prediction of segment-level calcium scores. E Model prediction of segment regions. Calcifications that were inconsistently assigned to the p-LAD and m-LAD by the two observers and small, misclassified calcifications that were incorrectly assigned to the p-LAD by the model are highlighted by arrows

A multi-task DL model was developed that would perform segmentation of coronary calcifications on the segment level (main task) and segmentation of segment regions (auxiliary task) at the same time inspired by Föllmer et al [13]. The input tensor of the network was created by concatenating five consecutive slices, which were normalized via min-max normalization between − 2000 and 1300 HU (2.5D approach) [21]. Additionally, a candidate lesion mask was appended, which was created by thresholding the CT images with a constant threshold of 130 HU [2]. In the initial training round, weak segment region annotations were provided for only 100 randomly selected image slices. We used the uncertainty-weighted loss method proposed by Cipolla et al [22] to combine the loss of the main task, \(_}\), and the loss of the auxiliary task, \(_}\), for multi-task model training (Fig. 4). To address the issue of annotation imbalance, we trained the model with a batch of 12 slices with and without calcifications, as well as with and without region annotations. Each batch comprises 12.5% of slices with both calcifications and region annotations, 12.5% of slices with calcifications and without region annotations, 37.5% of slices without calcifications but with region annotations, and 37.5% of slices without calcifications and region annotations. For samples without region annotations, we set the loss for region segmentations to zero.

Diagram of the multi-task deep-learning procedure for segmentation of coronary calcium on the segment level and weak segmentation of coronary artery segment regions. The model was trained using the normalized image slices (2.5D) and the candidate lesion mask as input. The model selected the most informative samples from the unlabeled segment region set in multiple sampling rounds. The annotator corrected pseudo-labels of predicted segment regions, which served as training samples for the next training round. After completion of the active learning procedure, the zero-CAC module was trained to distinguish slices with and without CAC in order to improve the identification of zero-CAC patients

To build the annotated dataset for weak region annotations, we used active learning [23,24,25,26,27] and increased the size of the labeled dataset in ten active learning rounds by labeling only most informative samples [28]. The most informative CT slices for weak annotation were selected using a sampling method that combines uncertainty-based sampling [25] and Fisher information correlation [26] between the main task and the auxiliary task. During each sampling round, a subset of the most uncertain samples identified for the main task \(}}}}_}}}} }}}}}_}}}}\) and the most uncertain samples identified for the auxiliary task \(}}}}_}}}}}}}}}_}}}}\) were selected from the unlabeled dataset using Monte Carlo dropout for uncertainty estimation [25]. Based on the Fisher maximum information correlation, samples \(\widetilde\) are iteratively selected \(\widetilde=}}}}_}}}}_}}}}}_\left(}}}}_}}}},\,}}}}_}}}}\cap x\right)\) for the batch \(}}}}_}}}}\). The Fisher information correlation is defined as

$$_\left(}}}}_}}}},\,}}}}_}}}}\cap \widetilde\right)=\frac__\right)}__}}__\right)}__}__\right)}__}}}$$

with \(__\right)}__}\) and \(__\right)}__}\) being the Fisher information of the samples \(}}}}_}}}}\) and \(}}}}_}}}}\), respectively. \(__)}__}\) is the average scalar product between the Fisher scores of the samples \(}}}}_}}}}\) and \(}}}}_}}}}\). The sample selection strategy is described in detail in Supplementary Material A. After each sampling round, the model was fine-tuned using the labeled dataset from the preceding round. After completion of the active learning procedure, we retrained the model from scratch to ensure generalizability and avoid overfitting by model warm start [29]. Compared with the earlier version proposed by Föllmer et al [13], we extended the network by the addition of a subsequential zero-CAC patient [14, 30] classification network, which explicitly learned to distinguish between slices with CAC and without any CAC. The model was trained using the image slices and the predictions of the multi-task model to learn the identification of zero-CAC patients from noisy images. Additionally, ablation experiments were conducted on the zero-CAC module, with the results presented in Supplementary Table S5. The model diagram and a detailed overview of model architecture are presented in Fig. 4 and Supplementary Fig. S1, respectively. The model hyperparameters we used are compiled in Supplementary Table S1. Training of this model was performed using the masks of calcifications on the segment level from the entire training set and weak annotations from the small, annotated dataset of calcifications and segment regions.