A total of 45 paired CMRs (1.5 T vs. 3 T) scans acquired from 15 healthy participants were utilized in this analysis. Figure 2 shows a representative pair of mid-cavity short-axis T1 maps.

A sample paired T1 map acquired at different field strengths. Representative mid-short axis LV parametric maps acquired from a 33-year-old male participant and the contours drawn in one segmentation run. A is a 1.5 T T1 map, B is a 3 T T1 map. Apparent in the image is the difference in signal intensity levels. The 3 T map appears to have a higher signal intensity level, indicating that features such as mean T1 are not the same for both images

The agreement between manual segmentation runs is illustrated in Fig. 3. In 3A, the segmentation masks extracted from a selected scan at both 1.5 T and 3 T are displayed, showcasing the extent of overlap between all three segmentation runs (3), the additional area common to only two runs (2), and the region unique to each run (1) for each field strength. This visualization provides a clear depiction of the agreements among segmentation runs.

A figure showing the extent of agreement between the three segmentation runs. Panel A shows the overlap between the case example maps drawn by the three segmentation runs, color coding the areas that have achieved consensus between the three runs (3), two runs (2), and no agreement (1). Panel B shows the proportion of ROI that is agreed on by three, two, and only one segmentation run for the scan in Panel A, an average for the entire dataset

The IoU values for the masks in Fig. 3A and the average IoU values across the entire dataset for each field strength are shown in Fig. 3B. The IoU values for the scan in Fig. 3A are intended to assist our readers in interpreting the average values across the dataset.

For the 1.5 T scans, the average IoU values across the entire dataset were as follows: 0.725, or 72.5%, of the pixels captured by all segmentation runs, were common to run1, run2, and run3, indicating a substantial agreement (Fig. 3B, all scans, 1.5 T). Additionally, 13.7% of the pixels captured in all segmentation runs were agreed upon by two segmentation runs, while 13.8% did not reach consensus.

A similar level of consensus was observed among the 3 T masks. An average IoU value of 0.738 or 73.8% was achieved, signifying a robust agreement across all segmentation runs. Moreover, 13.1% of the segmented pixels were agreed on by two segmentation runs, while 13.1% showed disagreement among all segmentation runs. There were no observed differences in IoU3 values across 1.5 T and 3 T scans (0.725 vs 0.738).

A total of 1023 RTFs were initially extracted and evaluated for reproducibility across the 1.5 T and 3 T scanners. However, “lbp-2D_firstorder_Maximum,” was identified as numerically unstable, consistently yielding the same value across all scans and field strengths. Consequently, this feature was excluded from further analysis, resulting in 1022 RTFs for subsequent investigation.

No RTFs demonstrated excellent reproducibility, defined by ICC values exceeding 0.95. The analysis revealed that 76 RTFs (7.44% of the 1022 RTFs) displayed good reproducibility. Additionally, 402 RTFs (39.33%) exhibited moderate reproducibility, while 544 features (53.23%) demonstrated poor reproducibility (Table 1). Notably, the feature with the highest reproducibility was identified as “gldm_GrayLevelNonUniformity” with an ICC of 0.879 (95% CI = 0.79–0.93).

Filter Class Level: Fig. 4A shows the reproducibility at the filter class level, ranked from the highest to the lowest by the percentage of features with good reproducibility. No filter class produced an RTF with excellent reproducibility across 1.5 T and 3 T scanners. The percentage of RTFs with good reproducibility varied across filter classes, ranging from approximately 1% (exponential, n = 1) to 20% (lbp-2D, n = 63). Features with moderate reproducibility ranged from 8% (exponential, n = 7) to 73% (gradient, n = 68), while those with poor reproducibility ranged from 12% (gradient, n = 14) to 91% (exponential, n = 85). Table 2 highlights the most reproducible features in each filter class. The glrlm_GrayLevelNonUniformity exhibited dominance in reproducibility, being the most reproducible feature in 6 filter classes (original, squareroot,wavelet-HL and LH, logarithm, and square). The gldm_DependenceNonUniformity and gldm_GrayLevelNonUniformity were the most reproducible features in 2 filter classes each: (lbp-2D, exponential) and (gradient,wavelet-LL). The glrlm_RunLengthNonUniformity was the most reproducible feature in the wavelet-HH filter class.

Reproducibility of RTFs. A display of the proportion of RTFs from A each filter class, B each feature class that fall in each category of reproducibility

Figure 4B further breaks down the results at the feature class level. The percentage of RTFs with good reproducibility ranged from 3.4% (glcm) to 16% (glrlm). Similarly, the percentage of features with moderate reproducibility ranged from 23% (ngtdm) to 46% (firstorder), while features with poor reproducibility ranged from 46% (firstorder) to 63% (glszm).

Figure 5 shows the results for the effects of segmentation variability on the repeatability of RTFs, illustrating a discernible trend. As the dropout probability (dp) increases, there is a corresponding decrease in the proportion of ROI pixels achieving consensus across the three segmentation runs (IoU3). This, in turn, results in a decline in RTFs with excellent repeatability for both 1.5 T and 3 T scans. The graph shows that with dp = 0, all three masks are perfectly Identical (IoU3 = 1.0), resulting in 100% or (proportion = 1.00) of all RTFs being repeatable. This proportion gradually decreases in both 1.5 T and 3 T images. At dp = 0.3, where IoU3 approaches 0.5, only about 5% or (proportion = 0.05) of RTFs produce excellent repeatability.

A plot of the results showing the sensitivity of RTFs to variations in image segmentation. The bars show the proportion of the segmented ROI that attained consensus between three, two and only one segmentation runs. The lines show the corresponding number of RTFs that fell in each category of repeatability

Table 2 (Repeatability) compares RTFs extracted from three manual segmentation runs (run1, run2, and run3) for repeatability. From the 1.5 T scans, the proportion of features, 32.68% (n = 334), exhibited excellent repeatability. Furthermore, 40.51% (n = 414) displayed good repeatability, 17.22% (n = 176) and 9.58% (n = 31) demonstrated moderate and poor repeatability, respectively. The most repeatable feature was gradient_gldm_GrayLevelNonUniformity, with an ICC of 0.993 (95% CI 0.990–1.000).

For the 3 T RTFs, we observed a similar yet slightly improved repeatability profile. A total of 31.56% (n = 322) of features displayed excellent repeatability, showcasing high consistency in the 3 T scans. Additionally, 54.11% (n = 553) exhibited good repeatability, while 12.81% (n = 131) and 1.56% (n = 16) showed moderate and poor repeatability, respectively. The most repeatable RTF from the 3 T scans was gradient_gldm_LargeDependenceLowGrayLevelEmphasis, with an ICC of 0.995 (95% CI 0.990–1.000). The repeatability patterns were further analyzed for each field strength at the filter class and feature class level.

Breaking down the repeatability of RTFs from 1.5 T scans across all segmentation runs, we observed varying degrees of repeatability with excellent repeatability at the feature class level, ranging from approximately 8% (exponential, n = 7) to 41% (lbp-2D/wavelet-HL, n = 38). The proportion of features with good repeatability ranged from 19% (squareroot, n = 18) to 59% (wavelet-LL, n = 55), while the percentage of RTFs with moderate or worse repeatability varied from 2% (lbp-2D, n = 2) to around 48% (original, n = 45). At the feature class level, the range of features with excellent repeatability extended from 18% (ngtdm) to 42% (glrlm), while features with good repeatability ranged from 31% (grlm) to 48% (ngtdm) (Fig. 6). The most repeatable feature in each filter category is detailed in Table 3. The “firstorder_Median” emerged as the most repeatable feature in most filter classes, including squareroot, original, and wavelet-LL.

Repeatability of RTFs. A chart showing the proportion of 1.5 T RTFs from each A filter class and B feature class that fall in each category of repeatability

The percentage of 3 T RTFs with excellent repeatability ranged from about 16% (logarithm, n = 15) to 42% (wavelet-HL /wavelet-LH, n = 39). Features with good repeatability ranged from 40% (wavelet-HH, n = 38) to 68% (logarithm, n = 63), while RTFs with moderate or worse repeatability fell between 3% and about 28% (square, n = 26). At the feature class level, the features with excellent repeatability extended from 21% (ngtdm) to 45% (glrlm). Features with good repeatability were observed at percentages ranging from 35% (grlm) to 65% (glcm) (Fig. 7). Notably, “gldm_GrayLevelNonUniformity” emerged as the most repeatable feature in most filter classes, including squareroot, square, gradient, logarithm, and all wavelet filters.

Repeatability of RTFs. A chart showing the proportion of 3 T RTFs from each A filter class and B feature class that fall in each category of repeatability