1 INTRODUCTION

Coronary heart disease (CHD) is the leading cause of death worldwide (Global Health Estimates 2016, 2018). People at high risk for CHD can benefit from lifestyle or pharmacologic interventions, even if the risk is of genetic origin (Abraham et al., 2016; Damask et al., 2020; Khera et al., 2016; Mega et al., 2015). Therefore, reliable prediction models are needed. Genetic information may add predictive value to established clinical risk models like the Framingham risk score (FRS) (D'Agostino et al., 2008). In the last few years, high-throughput genotyping has become more and more cost-effective (Schwarze et al., 2018; Wetterstrand, 2020). Genetic prediction modeling has several advantages compared to prediction based on blood biomarkers, such as proteins and metabolites (Herder, Karakas, et al., 2011). Genotyping needs to be carried out only once in a lifetime and its results can be used to determine the risk of various diseases. Moreover, risk assessment can be performed early in life, when lifestyle changes are likely to have the greatest impact.

The field is evolving rapidly and there is an ongoing debate on how genetic information, provided as single-nucleotide polymorphisms (SNPs), can be most effectively integrated into predictive models for complex diseases like CHD. The genetic information can be integrated in the form of (weighted) genetic risk scores (GRS) consisting of preselected variants (Antiochos et al., 2016; Beaney et al., 2017; Elliott et al., 2020; Inouye et al., 2018; Iribarren et al., 2016; R. W. Morris et al., 2016; Paynter et al., 2010; Said et al., 2018; Tada et al., 2016; de Vries et al., 2015). Machine Learning approaches without score formation are alternatively applied (Boulesteix et al., 2020; Dogan et al., 2018; Gola et al., 2020; Okser et al., 2014). Apart from the method of inclusion, studies vary regarding the utilized genotyping tool. Predictive SNPs may stem from genome-wide SNP array data or specialized genotyping arrays such as the Metabochip, which focuses on SNPs associated with cardiometabolic outcomes (Voight et al., 2012).

Until recently, genetic risk prediction studies on complex diseases focused on variants identified as significantly associated in genome-wide association studies (GWAS) or candidate gene association studies (Müller et al., 2016). Studies on cardiovascular disease-related events included only a few literature-based confirmed SNPs in their analyses to investigate the predictive value of genetic information in addition to established risk prediction models (Antiochos et al., 2016; Beaney et al., 2017; Iribarren et al., 2016; R. W. Morris et al., 2016; Paynter et al., 2010; Said et al., 2018; Tada et al., 2016; de Vries et al., 2015). More recent studies showed that taking into consideration a larger number of SNPs tends to give more accurate prediction results beyond established risk factors than including only genome-wide significant SNPs (Abraham et al., 2016; Elliott et al., 2020; Inouye et al., 2018; Mars et al., 2020). Khera et al. (2018) derived a polygenic risk score for coronary artery disease, based on genome-wide SNP array data. It has been repeatedly evaluated in several different studies and can therefore be regarded as a benchmark in coronary artery disease risk prediction (Aragam et al., 2020; Dikilitas et al., 2020; Hindy et al., 2020; Mosley et al., 2020; Wünnemann et al., 2019). Another comprehensive polygenic risk score previously derived by Gola et al. (2020) based on Metabochip data, was reported to yield even better prediction results. However, the Gola et al. (2020) results lack sufficient external investigation and comparison.

The aim of the present study is to compare the ability of different genetic risk quantification approaches to improve the accuracy of CHD prediction beyond the established FRS variables, including the recently published Gola score.

2 METHODS 2.1 Study populations

We used data from a prospective case-cohort study incorporated within the population-based Monitoring of Trends and Determinants in Cardiovascular Diseases/Cooperative Health Research in the Region of Augsburg (MONICA/KORA) study as a training data set. This study was initiated in the early 1980s by the World Health Organization. Details of design and procedures of this study have been described elsewhere (Herder, Baumert, et al., 2011; Koenig et al., 2006; Thorand et al., 2005; Tunstall-Pedoe & WHO MONICA Project Principal Investigators, 1988). Briefly, for the original cohort study, three independent cross-sectional population-based surveys were conducted in the years 1984/1985 (Survey S1), 1989/1990 (S2), and 1994/1995 (S3) in Augsburg and two adjacent counties in Southern Germany. In total 13,427 individuals (6725 men and 6702 women) aged 25–64 (S1) or 25–74 years (S2, S3) participated at baseline.

As test data set, we used data from the population-based KORA S4 cohort study. The cross-sectional baseline survey was performed in 1999–2001 in the same study region as the MONICA/KORA S1–S3 cohort study. In total 4261 individuals (2090 men and 2171 women) aged 25–74 years participated at baseline (Ward-Caviness et al., 2017).

Data on sociodemographic and lifestyle variables were collected through standardized interviews in both studies. Data on medication use was assessed based on the packaging of all medication, which the study participants had taken during the last week before the investigation and brought along due to request in advance. In addition, standardized medical examinations were carried out and blood samples were collected (Holle et al., 2005). All participants were prospectively followed within the KORA framework.

Because of the low incidence of CHD in those younger than 35 years, we restricted the source populations of the present studies to persons between 35 and 74 years at baseline. Further exclusion criteria consisted of missing blood samples, withdrawn consent, double examinations and prevalent CHD, yielding two cohorts of interest comprising 9299 persons in the MONICA/KORA S1–S3 cohort study and 3289 persons in the KORA S4 cohort study. Because participants in S3 were followed for a maximum of 14–15 years, we limited the follow-up time of both studies to 14 years. After the restriction of follow-up time, a case-cohort study was formed in the surveys S1–S3. The case-cohort design of the present study has been described in detail before (Huth et al., 2020; Thorand et al., 2020). Briefly, a random sample subcohort of 2163 participants (1154 men and 1009 women) was drawn from the cohort of interest, stratified by sex and survey. The subcohort was extended by all further cases occurring within 14 years in the cohort of interest. In the following, we excluded persons with missing values, persons without genotyping informed consent and persons who failed preimputation genotype quality control (QC) in both studies. The finally analyzed MONICA/KORA S1–S3 case-cohort study consisted of 451 incident CHD cases and 1488 noncases (“MONICA/KORA trainingset”; median follow-up time: 14.0, 25th percentile: 10.3, 75th percentile: 14.0 years). The KORA S4 cohort study included 160 incident CHD cases and 2749 noncases (“KORA testset”; median follow-up time: 14.0, 25th percentile: 14.0, 75th percentile: 14.0). All study individuals are German residents and carry a German passport. For details on the selection of participants see Figure 1.

Flowchart showing sample sizes and reasons for exclusions. CHD, coronary heart disease; QC, quality control

Written informed consent was given by all study participants, the study protocols were approved by the Ethics Committee of the Bavarian Chamber of Physicians. The studies were carried out in accordance with the Declaration of Helsinki.

2.2 Assessment of prevalent and incident CHD

The outcome CHD was a combined endpoint of nonfatal myocardial infarction as well as coronary death and sudden death (International Classification of Disease 9th Revision: 410–414 and 798).

Until December 2000, the diagnosis of a major, nonfatal myocardial infarction and coronary death was based on the MONICA algorithm (H. Tunstall-Pedoe et al., 1994) in which a diagnosis of a major CHD event was based on symptoms, cardiac enzymes (creatine kinase, aspartate aminotransferase, and lactate dehydrogenase), serial changes from 12-lead electrocardiograms (ECGs) evaluated by Minnesota coding (Prineas et al., 1982), necropsy results and history of CHD in fatal cases. Since January 1, 2001, the diagnosis of myocardial infarction was based on the European Society of Cardiology and American College of Cardiology criteria (Alpert et al., 2000).

Incident events were identified through follow-up questionnaires or through the MONICA/KORA myocardial infarction registry, which monitors the occurrence of all in- and out of-hospital fatal and nonfatal myocardial infarctions among the 25–74-year-old inhabitants of the study region (Löwel et al., 1991). Follow-up questionnaires were sent to all MONICA/KORA S1–S3 study participants in 1997/1998, 2002/2003, and 2008/2009 and to the KORA S4 study participants in 2008/2009 and 2016. The MONICA/KORA myocardial infarction registry covers the study region only. Therefore, subjects who had moved outside the study area reported incident events through follow-up questionnaires sent to their new addresses, which were traced by address research. Additionally, all participants from S1 were invited to participate in a follow-up examination in 1987/1988. For the KORA S4 study participants, two follow-up examinations were conducted in 2006–2008 and 2013/2014. Initially identified self-reported incident cases and the self-reported date of diagnosis not covered by the MONICA/KORA myocardial infarction registry, were validated by hospital records or by contacting the patient's treating physician. Deaths from myocardial infarction were validated by death certificates, autopsy reports, chart reviews, or information from the last treating physician.

2.3 Genotyping and assessment of risk score variables used for benchmarking

Details on genotyping, genetic QC, and genetic imputation can be found in Supporting Information Methods, Table S1, and Figure S1.

Genetic information was added to two reference clinical models: (1) age, sex, and survey; (2) sex, survey, and the FRS variables according to D'Agostino et al. (2008) with a minor modification. The original FRS contains the variables age, total cholesterol, high-density lipoprotein (HDL) cholesterol, systolic blood pressure, antihypertensive medication use, diabetes status, and current smoking. All metric variables, except for age, were naturally logarithmically (ln) transformed and (0,1)-standardized. Prevalent diabetes at baseline was defined as any type of self-reported diabetes that could be validated by the responsible physician or medical chart review or as current self-reported use of glucose-lowering medication. As a minor modification, we included former smokers in addition to current smokers as a separate category and used never smokers as the reference category. As we determined the combined hazard for men and women, we additionally included sex as a predictor variable, as well as a survey indicator, to account for the study design. During baseline examinations, a venous blood sample was collected while sitting. Samples were centrifuged within 120 min, refrigerated at 4–8°C and shipped on refrigerant packaging within 2–4 h to the laboratory of the Augsburg Central Hospital (now university hospital of Augsburg) for measurement of serum HDL cholesterol and total cholesterol (routine enzymatic method cholesterol oxidase phenol 4-aminoantipyrine peroxidase; Boehringer Mannheim [Roche from 1997 on]). HDL cholesterol, total cholesterol as well as several other collected biomarkers and variables (not used in the present analyses) had some missing values in the MONICA/KORA S1–S3 case-cohort study (HDL and total cholesterol: 0.04%). Therefore, to enable unbiased analyses and to use the available data as efficiently as possible, we replaced missing values using 20-fold multiple imputations by chained equations (MICE) (R version 3.2.3 and R package MICE version 2.25 [van Buuren & Groothuis-Oudshoorn, 2011; van Buuren et al., 1999]). As there was only a small number of missing values in the KORA S4 cohort study, we excluded persons with missing values from the KORA testset (Figure 1).

2.4 Compared genetic models—Main analyses

The prediction performance of all main analysis models was evaluated in the KORA testset based on regression coefficients (ln-transformed hazard ratios) estimated in the MONICA/KORA trainingset (Figure 2). We compared three different approaches to take genetic information into account. The first two were based on all Metabochip SNPs published to be genome-wide significantly (p < 5 × 10−8) associated with CHD. These were extracted from the literature received by a systematic Pubmed literature search on January 21, 2019, using the terms (metabochip OR cardiochip OR cardiometabochip). In the first approach, we used these SNPs to build a weighted GRS (GRSMetabo). In the second approach, we selected the most predictive SNPs from the identified genome-wide significant variants using a newly developed method called priority-Lasso, where reference model variables were prioritized over the SNP variables (PLMetabo) (Klau et al., 2017, 2018). Prediction included the unpenalized regression coefficients from the reference variables together with the shrunk regression coefficients from the selected SNPs. In the third approach, we evaluated a weighted GRS published by Gola et al. (2020) (GRSGola). Briefly, Gola et al. (2020) performed a random forest approach to preselect variants with corrected Gini importance greater 0. This was followed by a hyperparameter tuning to evaluate the optimal number of SNPs and the necessity of weights. Metabochip data imputed with the Haplotype Reference Consortium reference panel were used. Their final score comprised 50,633 weighted SNPs. The authors of the Gola et al. (2020) publication provided us with variant names, positions, chromosome numbers, weights, weighted alleles, and non-weighted alleles of these 50,633 SNPs, which we used to form GRSGola in the MONICA/KORA trainingset and the KORA testset.

Analyses workflow. GRS, genetic risk score; PL, priority-Lasso; SNP, single-nucleotide polymorphism

We added the genetic information of all three approaches to the two reference models. Regression coefficients of reference models alone and with the addition of PLMetabo, GRSMetabo, and GRSGola were calculated in the MONICA/KORA trainingset and the predictive value of each model was evaluated in the KORA testset. GRSMetabo and GRSGola were also evaluated without additional variables (crude models). Priority-Lasso regression is not possible without reference variables of first priority. Therefore no crude PLMetabo model was estimated.

Only SNPs available in both the MONICA/KORA trainingset and the KORA testset were used. We excluded SNPs with Hardy–Weinberg equilibrium (HWE) test p < 10−20 and minor allele frequency (MAF) < 1% as in Gola et al. (2020). We further excluded SNPs with imputation quality R² < 0.3, which was not as strict as in Gola et al. (2020), where a threshold of 0.8 was chosen, to keep as much genetic information as possible (Table S2).

2.5 Compared genetic models—Sensitivity analyses

Four sensitivity analyses were carried out via 10-fold cross-validation in the KORA testset only (Figure 2 and Table S2).

(1)

To verify that the genetic data quality of the MONICA/KORA trainingset was sufficient, GRSGola was evaluated in the KORA testset, according to the same protocol as in the main analysis.

(2)

Furthermore, to recreate the original risk score by Gola et al. (2020) as closely as possible, we used all available SNPs in the KORA testset, including also those which were unavailable in the MONICA/KORA trainingset, after exclusion of SNPs with HWE test p < 10−20, MAF < 1%, and imputation quality R² < 0.8 (termed GRSGolaOrig).

(3)

To take full advantage of the more comprehensive genetic data in the KORA testset and to compare the performance of GRSGola to an established benchmark GRS, we additionally built the weighted GRS, developed by Khera et al. (2018). This score was derived from a GWAS and two genome-wide genotyped validation studies, imputed with 1000 g reference panel. It includes 6,630,150 genome-wide distributed SNPs. We built this GRS variable after applying two different QC schemes. On the one hand, we used only imputation quality R² < 0.3 as an exclusion criterion, as in the original publication by Khera et al. (2018) (termed GRSKheraOrig).

(4)

On the other hand, we applied the same criteria as for GRSGolaOrig, namely HWE p < 10−20, MAF < 1%, and imputation quality R² < 0.8 (termed GRSKheraQCGola).

All GRS variables were added separately to the variables of the two reference models used for the main analyses. Crude models were also evaluated.

2.6 Statistical methods

Descriptive analyses of baseline characteristics were computed separately for cases and noncases. For the MONICA/KORA trainingset, weighting was incorporated using the survey- and sex-specific sampling weights to account for the case-cohort design. Multiple results of the 20 imputations were summed up as medians for percentiles and arithmetic means for proportions. Descriptive analyses for the MONICA/KORA trainingset were calculated using SAS (Version 9.4; SAS Institute Inc).

To assess the association of risk factor variables with incident CHD, Cox proportional hazard regression was applied. To take the case-cohort design into consideration, regression coefficients (ln-transformed hazard ratios) in the MONICA/KORA trainingset were calculated using Barlow's weighting method (Barlow et al., 1999). Standard error estimates for the regression coefficients in the MONICA/KORA trainingset were obtained from robust variance estimation (Barlow, 1994). Additional variation due to imputation was incorporated using Rubin's rules for Model 2 (Rubin, 1987). There were no missing values in the Model 1 variables, which had therefore not been imputed.

Priority-Lasso, which was used for the selection of predictive SNP variables, was extended by Barlow's weighting method to account for the case-cohort design of the MONICA/KORA trainingset. Details of priority-Lasso have previously been described in Klau et al. (2018). Briefly, priority-Lasso is an enhancement of the standard Lasso method with a fixed block structure of the variables. Each variable belongs to exactly one block and each of these blocks is assigned a priority level. Unpenalized Cox regression is applied to all variables of the first block. Subsequently, a Lasso regression is conducted among all variables of the second block, using the linear predictor of the first step as offset. Through this method, variables from blocks of lower priority are only included in the model, if they explain variability that was not explained by blocks of higher priority. We assigned the variables of our reference models the first priority (Block 1) and all SNP variables the second priority (Block 2). The R package priority-Lasso (Klau et al., 2017) was used after having been adapted to the case-cohort design, which includes also an adapted glmnet version. The glmnet version can be downloaded from https://github.com/mu4bu2/glmnet. The tuning parameter lambda was chosen via 10-fold cross-validation in the MONICA/KORA trainingset. We used the default cross-validation procedure as described in Simon et al. (2011). Briefly, we evaluated 100 different lambda values, with λmin and λmax defined as in Simon et al. (2011). Deviance, which uses the partial-likelihood for the Cox model, was used as the loss function. That lambda resulting in the minimal mean cross-validated prediction error was selected as the final tuning parameter and used in the complete MONICA/KORA trainingset for priority-Lasso estimations.

To take the uncertainty of genetic imputation into account, instead of best-guess genotypes (“hard calls”), the estimated alternate allele dosage for each SNP was used and treated as metric quantity. For priority-Lasso we included the (0,1)-standardized dosage of each SNP as a separate variable. For the construction of all GRS (GRSMetabo, GRSGola, GRSGolaOrig, GRSKheraOrig, GRSKheraQCGola), dosages were adapted to the published regression coefficients (two-dosage, if the regression coefficient referred to the other allele). The GRS variables were built by multiplying the adapted dosage with the published regression coefficient (ln-transformed odds ratio) and by summing across all SNPs, followed by (0,1)-standardization. Khera et al. (2018) published positive regression coefficients and therefore solely risk alleles were weighted. All other published regression coefficients used in the present analyses were a mixture of positive and negative values (Deloukas et al., 2013; Gola et al., 2020; Howson et al., 2017; van der Harst & Verweij, 2018).

The added predictive value of the genetic information was evaluated by three performance measures: (1) Harrel's concordance index (C-index) and the difference between the reference and the genetically extended C-indices (delta C-index) (Harrell et al., 1982); (2) the category-free net reclassification index (Pencina et al., 2011) calculated overall (cfNRI) and separately for cases (cfNRIcases) and controls (cfNRIcontr); (3) the absolute integrated discrimination improvement (IDI) and additionally the relative IDI (IDIrel), as the IDI depends on the incidence of the outcome (Pencina et al., 2010).

The performance measures in the KORA testset were calculated from the regression coefficients derived in the MONICA/KORA trainingset. As there were no missing values in the reference model 1 variables, the linear predictor was built as usual. The regression coefficients of Model 2 alone and with extension of GRSMetabo or GRSGola were averaged over the 20 imputations. Confidence intervals (CIs) were calculated with standard errors of an approximate normal-distribution for the C-index using the R package compareC (Kang et al., 2015) and via 1000-fold percentile bootstrapping for cfNRI and IDI using the R packages nricens and boot (Canty & Ripley, 2020; Davison & Hinkley, 1997; Inoue, 2018); for the PLMetabo—extended reference model 2, it was not possible to take the averaged regression coefficients due to different tuning parameters in the 20 imputations. Therefore, each performance measure was calculated separately in each imputation and the results were combined using Rubin's Rules (Rubin, 1987).

In the cross-validation of the sensitivity analyses, the arithmetic mean of all performance measures was calculated over the 10 folds. The CIs of the mean performance measures were calculated via 1000-fold percentile bootstrapping using the R package boot (Canty & Ripley, 2020; Davison & Hinkley, 1997).

Test results with two-sided p < 0.05 were considered statistically significant and all statistical analyses were performed with R version 4.0.0 unless specified otherwise.

3 RESULTS 3.1 Baseline characteristics

Table 1 shows baseline characteristics of the 451 incident CHD cases and 1488 noncases of the MONICA/KORA trainingset and the 160 incident CHD cases and 2749 noncases of the KORA testset. In both studies, the cases comprised more men than women. They were on average older and had a higher systolic blood pressure despite a more frequent use of antihypertensive medication. They were more likely to have diabetes at baseline and to be current or former smokers. Furthermore, the cases had higher levels of total cholesterol and lower levels of HDL cholesterol compared to noncases. Medians of GRSGola and GRSGolaOrig were substantially higher in cases than in noncases. There were only slight differences in GRSMetabo, GRSKheraOrig, and GRSKheraQCGola.

Table 1. Baseline characteristics of incident CHD cases and noncases in the MONICA/KORA trainingset and the KORA testset MONICA/KORA trainingseta KORA testset Characteristics CHD cases n = 451 Noncases n = 1488 CHD cases n = 160 Noncases n = 2749 Male (%) 73.4 46.9 75.6 46.5 Age (years) 62 (55, 68) 51 (43, 60) 64 (54, 69) 53 (44, 63) Survey 1 (%) 21.3 28.8 – – Survey 2 (%) 43.7 36.3 – – Survey 3 (%) 35.0 35.0 – – Survey 4 (%) – – 100.0 100.0 Framingham risk score variables Prevalent diabetes mellitus at baseline (%) 17.7 4.3 20.6 4.6 Systolic blood pressure (mmHg) 144.0 (130.0, 158.0) 130.0 (120.0, 143.0) 140.0 (126.4, 152.2) 128.5 (117.0, 141.5) Antihypertensive medication use (%) 37.9 14.2 41.3 19.7 Total cholesterol (mg/dl) 253.1 (223.3, 286.0) 233.0 (207.0, 265.9) 238.7 (212.4, 265.7) 229.7 (203.5, 260.2) HDL cholesterol (mg/dl) 48.0 (40.5, 58.4) 55.3 (45.7, 67.3) 50.0 (41.8, 61.7) 56.2 (45.7, 68.7) Smoking Former (%) 36.8 27.3 36.3 33.5 Current (%) 30.8 23.1 28.8 22.7 Never (%) 32.4 49.6 35.0 43.8 Genetic risk scores GRSMetabo 0.26 (−0.13, 0.68) 0.12 (−0.24, 0.54) 0.22 (−0.21, 0.59) 0.16 (−0.24, 0.54) GRSGola 4.45 (−0.10, 8.60) 3.15 (−0.67, 6.97) 2.85 (−2.95, 8.04) 0.30 (−5.22, 5.86) GRSGolaOrig – – 1.42 (−5.41, 6.58) −1.71 (−8.14, 4.60) GRSKheraOrig – – 17.60 (17.54, 17.67) 17.58 (17.52, 17.64) GRSKheraQCGola – – 13.85 (13.79, 13.91) 13.83 (13.77, 13.88) Note: Categorical variables are presented as proportions and continuous variables as medians (25th; 75th percentile). Abbreviations: CHD, coronary heart disease; GRS, genetic risk score; HDL, high-density lipoprotein. aFor the MONICA/KORA trainingset, weighting was incorporated using the survey- and sex-specific sampling weights to account for the case-cohort design. 3.2 Literature search for Metabochip SNPs associated with CHD

Our literature search yielded three studies reporting genome-wide significant association results between Metabochip SNPs and CHD. These were Deloukas et al. (2013), Howson et al. (2017), and van der Harst and Verweij (2018) who conducted large-scale, consecutive association studies based only or mainly on Metabochip data. All three studies were extensions of the largest GWAS meta-analysis at that time, performed by the Coronary ARtery DIsease Genomewide Replication and Meta-analysis (CARDIoGRAM) Consortium (Schunkert et al., 2011). We included all Metabochip SNPs that were reported to be genome-wide significantly associated with CHD either directly in one of the three studies, or in cited studies. The 138 identified SNPs were then used for selection in the priority-Lasso analysis as well as for building GRSMetabo, choosing the most recent published association estimate as weight, as the more recent studies had the larger sample sizes. The 138 extracted SNPs are listed with their weight (ln[odds ratio]) in Table S3.

3.3 Main analyses

The hazard ratios of the reference models, calculated in the MONICA/KORA trainingset and used for the prediction analyses in the KORA testset, are shown in Table S4. Except for survey and former smoking, all variables were significantly associated with incident CHD.

In the PLMetabo models, 10 SNPs were selected on top of Model 1; 7 of these also on top of Model 2 (Table S5).

The GRSMetabo was associated with incident CHD in the crude model (hazard ratio [95% CI] per SD: 1.2341 [1.1137, 1.3676]) and in Model 1 (1.2126 [1.0766, 1.3659]) but not in Model 2 (1.1300 [0.9989, 1.2784]); Table S6). The GRSGola was associated with CHD in all three models (crude: 1.2516 [1.1203, 1.3982]; Model 1: 1.2297 [1.0804, 1.3996]; Model 2: 1.1580 [1.0109, 1.3265]; Table S6).

The prediction performance measures in the KORA testset are presented in Tables 2 and S7. In the crude models, the GRSGola yielded a C-index of 0.5878 [95% CI: 0.5446, 0.6310] and thus performed better than the GRSMetabo (C-index: 0.5253 [0.4803, 0.5702]). Generally, the C-index is a measure to determine the power of a model to discriminate between cases and noncases based on predicted survival probabilities. A C-index of 0.5 indicates that the model is not able to distinguish between cases and noncases at all. A C-index significantly higher than 0.5 points at a better discrimination ability of the model than random guessing. On top of both reference models, there was no evidence that the SNPs that were selected by priority-Lasso, or the GRSMetabo improved the prediction. The extended models did not produce delta C-indices, cfNRIs nor IDIs significantly higher than 0. Whereas the delta C-index compares the discrimination ability of a reference and an extended model, cfNRI and IDI compare the reclassification ability of both. The cfNRI describes the proportion of people for whom the expanded model improved the prediction (predicted risk higher for cases, lower for noncases). The IDI quantifies the difference between a change on average sensitivity and a change on average 1–specificity. In IDIrel, this difference is considered in relation to the reference model to avoid dependence on incidence. In contrast to PLMetabo and GRSGola, the GRSGola in both models yielded improved C-indices (Model 2 delta C-index: 0.0087 [0.0044, 0.0130]) and IDI estimates (Model 2 IDIrel: 0.0509 [0.0131, 0.0894]). The overall cfNRI showed an improved reclassification in Model 1, although it was not statistically significant in Model 2.

Table 2. Performance measures [95% CI] of GRSMetabo, PLMetabo, and GRSGola in the KORA testset GRSMetaboa(138 SNPs) GRSMetabo Model 1 Model 1 + GRSMetabo Model 2 Model 2 + GRSMetabo C-index 0.5253 [0.4803, 0.5702] 0.7537 [0.7189, 0.7886]* 0.7571 [0.7234, 0.7908]* 0.7905 [0.7604, 0.8207]* 0.7920 [0.7622, 0.8219]* ∆C-index, cfNRIcases, cfNRIcontr, IDIabs, and IDIrel were not significant (Table S7). PLMetabob(Model 1: 10 SNPs, Model 2: 7 SNPs) PLMetabo