WHAT IS ALREADY KNOWN ON THIS TOPICWHAT THIS STUDY ADDS

Type 2 diabetes can be prevented but a 75 g oral glucose tolerance test (OGTT) is needed to identify people with impaired glucose tolerance (IGT), the main beneficiaries of diabetes prevention programs. Inclusion of routine renal biochemistry (serum potassium, creatinine, and urea) and FPG to a validated non-invasive risk score can significantly discriminate subjects with abnormal glucose tolerance (2-hour plasma glucose ≥7.8 mmol/L) who will benefit from early intervention.

HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICYIntroduction

Type 2 diabetes (T2D) could be delayed and prevented by lifestyle modification and/or pharmacotherapy in individuals with impaired glucose tolerance (IGT).1 2 In the China Daqing study, the rate of T2D conversion was reduced by 51% at 6 years via lifestyle modification which was translated to reductions in retinopathy and cardiovascular disease during 30-year follow-up.1 3 In the latest China Diabetes Prevention Program, the combined use of lifestyle modification and metformin reduced progression to diabetes by 18% compared with lifestyle modification only, mainly in subjects with IGT.4 The majority of Chinese individuals have isolated IGT.5 A significant proportion of subjects with diabetes mellitus are only identified by abnormal 2-hour plasma glucose (2hPG) during 75 g oral glucose tolerance test (OGTT), with normal fasting plasma glucose (FPG) and HbA1c (glycated hemoglobin A1c).6 7 Apart from progression to diabetes, elevated post-OGTT plasma glucose strongly predicts cardiovascular events.8 9 Given the resource implication in performing OGTT, developing a method to identify subjects with abnormal glucose tolerance (AGT) including IGT and undiagnosed diabetes will reduce the number of OGTT and improve the cost-effectiveness of diabetes prevention and early intervention programs.

There are only a few non-invasive risk scores used to identify Chinese subjects at high risk of having diabetes and pre-diabetes based on 75 g OGTT. With a modified cut-off point of 27 out of 51, the New Chinese Diabetes Risk Score (NCDRS, including age, sex, body mass index (BMI), systolic blood pressure, waist circumference, family history of diabetes) was used to predict undiagnosed diabetes and pre-diabetes based on the American Diabetes Association (ADA) criteria with a sensitivity of 0.694 and positive predictive value (PPV) of 36.2%.10 The Risk-Understanding-By-Yourself (RUBY) risk score consists of self-assessment items (including age, sex, BMI, family history of diabetes, gestational diabetes, hypertension, and dyslipidemia) which had been validated in Chinese adults of working age for predicting T2D.11

Many individuals with diabetes or pre-diabetes have concomitant conditions such as hypertension, dyslipidemia, or cardiovascular disease managed in an electronic health records (EHR) system. These individuals have regular blood tests which may provide opportunities for early diagnosis of dysglycemia. In the Atherosclerosis Risk in Communities (ARIC) study, the area under receiver operating characteristics (AUROC) of a diabetes risk score increased from 0.71 to 0.80 (p<0.001) when FPG and lipids were added to non-invasive measurements.12 In the Framingham Offspring study, the inclusion of FPG, high-density lipoprotein cholesterol (HDL-c), and triglycerides (TG) increased the AUROC of a diabetes risk score from 0.72 to 0.85.13 In the European Prospective Investigation into Cancer and Nutrition (EPIC)-Potsdam study, the addition of FPG, lipids and liver enzymes to non-invasive measurements improved the discriminative power of a diabetes risk score.14 In a prediction model for diabetes in Chinese, the inclusion of FPG, lipids and white blood cell (WBC) count also improved its performance.15 Other risk scores that included laboratory measurements were used to predict AGT (diabetes and IGT). In the National Health and Nutritional Examination Survey, a risk score including clinical risk factors, FPG and TG had a total score of 8. The lowest cut-off point of 2 yielded high sensitivity (0.86) and the highest cut-off point of 6 yielded high specificity (0.96) with an AUROC of 0.74 for predicting AGT.16

Few studies to date have evaluated the addition of routine laboratory measurements, especially renal parameters, in improving discrimination for AGT in the Chinese population. In this study, we investigated whether the inclusion of routine biochemical measurements into a validated non-invasive diabetes risk score could enhance the performance and discrimination of AGT in community-dwelling Chinese adults.

MethodsStudy population

A total of 1,938 community-dwelling subjects were curated from the Better Health for Better Hong Kong (BHBHK) Survey and the Hong Kong Family Diabetes Study (HKFDS). Both cohorts were established in 1998–2003 to define the phenotypes of diabetes in Hong Kong Chinese adults. The BHBHK cohort was established during a city-wide health promotion and screening campaign targeting the workforce.17 The HKFDS recruited first-degree (parents and children) and second-degree relatives of patients with T2D, the majority of whom were diagnosed before 40 years old.18 In this cross-sectional study, we included 1,452 subjects who underwent OGTT in the BHBHK-HKFDS cohort. We excluded participants with known diagnoses of diabetes or using glucose-lowering drugs or less than 18 years old (n=99). We excluded 60 subjects with undiagnosed diabetes based on FPG≥7.0 mmol/L. A number of 1,293 individuals were included in this analysis (figure 1).

Figure 1

Flow chart of the study. BHBHK, Better Health for Better Hong Kong; FPG, fasting plasma glucose; HKDFS, Hong Kong Family Diabetes Study; OGTT, oral glucose tolerance test; OGLD, oral glucose lowering drugs.

Data collection

All participants in the BHBHK-HKFDS cohort underwent structured assessments including anthropometric measurements, demographic status, medical history, lifestyle information, clinical measurements, and laboratory tests. Age, sex, body height, and weight (for calculation of BMI), waist circumference, blood pressure, smoking status, family history of diabetes, and medical conditions including hypertension, dyslipidemia, and gestational diabetes were documented. After an overnight 8-hour fast, all participants underwent standard 75 g OGTT with measurement of PG at 0, 60, and 120 min. Lipid profiles (total cholesterol (TC), TG, HDL-c and calculated low-density lipoprotein cholesterol (LDL-c)), renal function tests (serum sodium, potassium, urea, creatinine levels, and urinary albumin creatinine ratio (uACR)), total protein and albumin were also measured.

All laboratory assays with external accreditation were performed at the Departments of Chemical Pathology and Haematology Laboratory of the Prince of Wales Hospital. Plasma glucose was measured by a glucose oxidase method (Diagnostic Chemicals reagent Kit). Plasma TC and TG were measured enzymatically with commercial reagents (Dimension; DuPont Instruments, Delaware, USA) and HDL-c was measured by the same enzymatic assay after precipitation of HDL-c by the heparin/manganese method. LDL-c was calculated using Friedewald et al equation.19

External validation

We used data from 550 subjects without diabetes diagnosis or FPG≥7.0 mmol/L for external validation. This included data was from three diabetes prevention-related studies, a lifestyle modification study (NCT04588896, n=324), a polycystic ovary syndrome follow-up study,20 (n=158) and a study evaluating traditional Chinese medicine in people with pre-diabetes (NCT04441216, n=68). We used anthropometric characteristics, medical history, and biochemical parameters collected before any intervention. All subjects similarly had a 75 g OGTT with FPG and 2hPG measurements after an 8-hour overnight fast. -

Statistical analysis

Data were summarized as mean±SD or median (IQR) as appropriate. We compared anthropometric and demographic characteristics, as well as the laboratory parameters between the three groups (stratified by 2hPG<7.8, 7.8–11.0, and ≥11.1 mmol/L) using analysis of variance or Kruskal-Wallis test for continuous data, and χ2 test for categorical data.

Construction of the extended RUBY risk score with laboratory variables

The original RUBY included age, sex, BMI, family history of diabetes, hypertension, dyslipidemia, and gestational diabetes in the non-invasive risk score. Hypertension was defined as known hypertension with or without taking anti-hypertensive drugs, or blood pressure ≥140/90 mm Hg. Dyslipidemia was defined as the history of any lipid abnormalities (including TC≥5.2 mmol/L, TG≥2.3 mmol/L, and/or HDL-c<1.0 mmol/L) with or without lipid-modifying drugs. The original RUBY score was derived from community-dwelling adults in Hong Kong with AUROC 0.735 (0.705 to 0.765) for the prediction of diabetes. A cut-off point of 16 out of 30 had 40% PPV for detecting undiagnosed diabetes. It was externally validated in two cohorts of 3,743 subjects with known risk factors for diabetes with an AUROC of 0.681 and another cohort (n=1,513) with an AUROC of 0.772.11

To identify laboratory variables for inclusion in the extended RUBY models, we used a forward stepwise logistic regression to select relevant laboratory predictors for AGT. These included renal function tests (serum levels of sodium, potassium, creatinine, urea, and uACR), total protein, albumin, and complete blood test (red blood cell, platelet, WBC). Multicollinearity diagnostics were tested among the final chosen prediction factors by comparing the variance inflation factor (VIF). A VIF≥10 indicates multicollinearity. The importance of each risk factor was estimated by the partial χ2 statistic minus the predictor df (Wald-df). The best model was defined by the Bayesian information criterion (BIC), which had a stronger penalty for additional parameters than the Akaike information criteria.

10-fold cross-validation for internal validation

We performed 10-fold cross-validation of the extended RUBY model within the BHBHK-HKDFS cohort for internal validation. First, the whole data was randomly separated into 10-folds. Second, in the first loop, an arbitrary onefold data was selected as the first test set to examine the first trained model built by the other ninefold data to yield the first receiver operating characteristic (ROC). Third, we ran the second loop. The next fold data was set as the test set and the remaining ninefold data was used to build the trained model to generate the second ROC curve. During the 10 loops, each fold had the opportunity to be the tested data to yield their respective ROCs. This method was used to control overfitting and an AUROC of 0.7 to 0.8 was considered to be good to excellent. Kappa statistic is a measurement of the accuracy of a model while considering randomness. The closer to 1, the better the model with 0.4–0.6 regarded as having moderate performance.21

Comparison of extended models against other non-invasive risk scores for AGT

The discriminative value was assessed using AUROC. Integrated Discrimination Index (IDI) and net reclassification improvement (NRI)22 were used to compare the extended models and the original RUBY score. In this study, the AUROC and the IDI were used to compare the performance of different models. A comparison of AUROCs was done using the Delong test. NRI was used to assess the improvement of the best extended RUBY model compared with the original RUBY score. We defined three risk categories a priori: low risk: <30%, intermediate: 30–60%, and high risk: >60%. Calibration measures the agreement between predicted probabilities and observed risks and was assessed by the Hosmer-Lemeshow test. Sensitivity, specificity, PPV, and negative predictive value (NPV) at different cut-off points were presented. We assessed screening efficiency by defining the number needed to screen (NNS) as the number of OGTTs needed to be performed to detect one case of AGT=1/positive case detection rate of AGT.

Decision curve analysis (DCA) was used to assess the utility of models, including the default strategies of treating all or no patients for decision-making. It combines the number of true positives and false positives into a single one as a net benefit to determine whether the model is superior to any other models across the full range of reasonable threshold probabilities.23 24 A risk threshold of 0.25 was used based on the prevalence of IGT and T2D based on the report of the International Diabetes Federation 2021,25 which is consistent with the percentage of AGT in our cohort.

We compared the performance of the extended RUBY models versus the NCDRS which included age, gender, BMI, waist circumference, hypertension, and family history of diabetes as risk predictors. The NCDRS was derived from 7,675 community residents of Eastern China. The NCDRS used a cut-off point of 27 out of 51 to predict undiagnosed diabetes and pre-diabetes.10 Youden Index was used to determine the optimal cut-off for maximal sensitivity and specificity for the extended RUBY models to estimate the proportion AGT in the extended models (0–100%). We used the original RUBY to predict AGT with a modified cut-off point determined by the Youden Index. We compared the discrimination expressed as sensitivity, specificity, PPV, and NPV of the extended RUBY models using different thresholds for risk prediction. We also compared the extended RUBY models versus the NCDRS.

A p value<0.05 was considered significant. The analysis was conducted using IBM SPSS Statistics V.23.0 and R software V.4.1.1.

ResultsParticipant characteristics

Among the 1,293 adults without known diabetes or FPG≥7.0 mmol/L, 317 had AGT, including 85 with 2hPG≥11.1 mmol/L and 232 with 2hPG between 7.8 mmol/L and 11.0 mmol/L (figure 1). Anthropometric measurements were compared among the three groups with 2hPG<7.8, 7.8–11.0, ≥11.1 mmol/L. The 2hPG≥11.1 mmol/L were the oldest, had the highest BMI, and highest rate of family history of diabetes. They were most likely to be active smokers and treated with anti-hypertensive drugs (all p<0.001). They also had higher plasma total protein, TC, TG, LDL-c, and uACR but lower HDL-c and serum potassium (p=0.016) (table 1).

Table 1

Cross-sectional participant characteristics in Better Health for Better Hong Kong_Hong Kong Family Diabetes Study cohort

Selection of variables in the extended model

Apart from variables in the original RUBY score, FPG, serum creatinine, WBC count, serum urea, potassium, and total protein were significant in the multivariable logistic regression model without collinearity (online supplemental table S1).

The performance of the original RUBY score in predicting AGT (2hPG≥7.8 mmol/L) was 0.698 (95% CI, 0.662 to 0.733). We constructed extended RUBY model 1 and added FPG to the original RUBY with an increased AUROC to 0.756 (95% CI, 0.722 to 0.791, p<0.001) and improvement of IDI by 20.1% (p<0.001). In extended RUBY model 2, we included serum creatinine, potassium, and urea to extended RUBY model 1 with an AUROC of 0.778 (95% CI, 0.744 to 0.812, p=0.002) and improved IDI by 6.4% (p<0.001). However, when further including WBC or total protein to extended RUBY model 2, the AUROC and IDI were not significantly improved (online supplemental table S2).

Although the extended RUBY model with FPG, creatinine, potassium, urea, and WBC had the lowest BIC (online supplemental table S3), however, since most individuals do not always have both WBC and renal biochemistry measured in routine clinical settings, the optimal model including RUBY score, and biochemistry parameters of FPG, creatinine, potassium and urea was chosen for practical purposes (table 2).

Table 2

Beta-coefficients and OR of extended RUBY models in predicting AGT

Discrimination of extended RUBY models compared with the original RUBY score

Figure 2 shows ROC curves of four models. Compared with the original RUBY score, the extended RUBY models including FPG showed better discrimination. The extended RUBY model 2 including additional potassium, urea, and creatinine showed best. There was no difference in discrimination of AGT between the model with FPG only and the original RUBY risk score. The AUROC for a model containing RUBY+creatinine+urea+potassium without FPG was 0.725 (95% CI, 0.690 to 0.761) which still showed improved discrimination compared with RUBY alone (IDI 0.047 (95% CI, 0.024 to 0.069, p<0.001) online supplemental table S2).

Figure 2

Receiver operating characteristic curves of the performance of original RUBY, FPG only, extended RUBY model 1 and 2 in predicting abnormal glucose tolerance in the Better Health for Better Hong Kong-Hong Kong Family Diabetes Study cohort. AUROC, area under receiver operating characteristics; FPG, fasting plasma glucose; RUBY, Risk-Understanding-By-Yourself.

In the DCA, the extended RUBY model 2 achieved the highest net benefit across a range of risk thresholds for AGT. At the risk probability of 0.25, the net benefit was superior in extended RUBY model 2 than the original RUBY, and the extended model 1 was intermediate (online supplemental figure S5).

Calibration, re-classification and cross-validation compared with an original RUBY score

Online supplemental figure S1 compares observed and predicted probabilities of AGT by extended RUBY model 2 as assessed by the Hosmer-Lemeshow test with good calibration (χ2=11.315, p=0.1845, F=8). After 10,000 bootstraps, compared with the original RUBY model, the extended RUBY model improved net reclassification by 31.9% (p<0.001), with 30.8% among persons with AGT and 1.1% among persons without AGT (online supplemental figure S2). Online supplemental figure S3 shows 10-fold cross-validation of the extended RUBY model 2. The average AUROC was 0.814 and the kappa was 0.415 indicating no overfitting in the extended RUBY model 2 construction.

Comparison of extended model 2 against other non-invasive risk scores

We compared the performance of extended RUBY model 2 versus other non-invasive risk scores. We selected the cut-off value using the Youden Index to yield high PPV and sensitivity (online supplemental table S4). With a modified cut-off point of 12 out of 30, the original RUBY had a sensitivity of 0.485 and 2.4 NNS. The NCDRS had a sensitivity of 0.603 and 2.6 NNS. The extended RUBY model 2 achieved the highest sensitivity of 0.635 with NNS 1.9, as well as the best specificity, PPV, and NPV (table 3).

Table 3

Performance of two extended RUBY models in predicting AGT and compared with NCDRS at optimal cutoffs in the Better Health for Better Hong Kong_Hong Kong Family Diabetes Study cohort (n=1054)

External validation

For external validation, we included 550 adults at high risk for diabetes but without known diabetes or FPG≥7.0 mmol/L during 75 g OGTT. Their mean (±SD) age was 53±10 years and their BMI was 26.1±4.7 kg/m2 (online supplemental table S5). Of these, 242 participants had AGT with 2hPG≥7.8 mmol/L. In the external validation set, the AUROC for the extended RUBY model 1 was 0.716 (95% CI, 0.674 to 0.759), while the extended RUBY model 2 (including FPG, serum potassium, urea, creatinine) achieved an AUROC of 0.722 (95% CI, 0.680 to 0.764). Both extended RUBY models had higher AUROCs than the original RUBY of 0.637 (95% CI, 0.591 to 0.683). Applying NCDRS to the external cohort yielded an AUROC of 0.632 (95% CI, 0.586 to 0.679) (online supplemental table S6). The Hosmer and Lemeshow test showed good calibration (χ2=3.972, p=0.860, F=8) of the extended RUBY model 2 in the external cohort (online supplemental figure S4).

Discussion and conclusions

The efficiency of the implementation of diabetes prevention and early treatment program relies on the effective identification of individuals with AGT (2hPG≥7.8 mmol/L) detected by OGTT. In this study, we have shown that the inclusion of renal parameters to validated non-invasive risk scores can improve the reclassification of AGT by 31.9%. To our knowledge, this is the first study to validate a model incorporating clinical risk factors, FPG, and renal parameters to predict AGT in Chinese people.

By adding FPG to the original RUBY model, we can substantially enhance its performance in predicting AGT. Other studies have also shown that clinical parameters combined with FPG and lipid profiles achieved the best performance in diabetes or IGT prediction.12 14 16 We did not include lipid profiles in our extended RUBY models as the history of dyslipidemia was already contained in the original RUBY score.

Additional inclusion of serum potassium, creatinine, and urea on top of FPG further improved the performance of extended RUBY model 2 with low serum potassium being associated with an increased risk of AGT. In the 9-year prospective ARIC study, adults with serum potassium levels lower than 4.0 mmol/L had an adjusted HR of incident diabetes of 1.64 (95% CI, 1.29 to 2.08) compared with those with high-normal serum potassium level (5.0–5.5 mmol/L).26 In the Toranomon Hospital Health Management Center Study 1 (TOPICS 1), during 5-year follow-up, the lowest tertile of serum potassium (2.8–3.9 mmol/L) was independently associated with incident diabetes with an HR of 1.57 (95% CI, 1.15 to 2.15) compared with the highest tertile (4.2–5.4 mmol/L) in Japanese.27 Low serum potassium may be associated with poorer beta cell function and impaired insulin secretion.28 In addition, potassium homeostasis is regulated by dietary intake and renal excretion, the activity of the renin-angiotensin-aldosterone system, and transmembrane cellular shift which can be activated by hyperinsulinemia and sympathetic nervous system.29 30 Potassium supplements and herbal medicines, such as licorice, might affect serum potassium. Mineralocorticoid antagonist and potassium-sparing diuretics have been associated with a lower risk of diabetes31 while thiazide diuretics have been associated with higher risk. We did not capture the details of medications or dietary intake in our cohort. However in this cohort of working age, 4.5% of subjects reported using anti-hypertensive drugs which was less likely to contribute to the risk association of low serum potassium and AGT in our study.

Mechanistically, serum creatinine and urea are inversely associated with a higher risk of AGT via different pathways. The lower serum creatinine might reflect hyperfiltration as an early marker of nephropathy which has been reported in people with pre-diabetes.32 33 Serum creatinine is a product of muscle catabolism. A low muscle mass might be a proxy of low physical activity, poor peripheral insulin sensitivity, and glucose disposal. In a 7-year prospective study including 36,304 Koreans who did not have diabetes, the lowest sex-specific skeletal muscle index tertile measured by bioelectrical impedance was associated with increased risk of incident T2D (HR 1.31, 95% CI, 1.18 to 1.45).34 On the other hand, urea is the primary metabolite derived from dietary protein and tissue protein turnover. A higher serum urea might reflect higher dietary protein intake, which has been associated with a lower risk of development of diabetes in some randomized controlled feeding studies.35 36 Taken together, the association of renal parameters notably low serum potassium with AGT are biologically plausible and warrant further investigations. Future studies may examine the association between serum potassium, creatinine, and urea and body composition as well as measured insulin resistance.

These routine biochemical tests are commonly performed among people receiving medical care in clinical settings. By incorporating clinical and laboratory measurements to automatically generate risk scores, such as the extended RUBY model 2, we can increase the efficiency of diagnosing AGT based on OGTT. In a separate territory-wide EHR-based Hong Kong Diabetes Surveillance Database with 1,028,931 people with pre-diabetes, 96% of individuals had an FPG, 96% had routine biochemistry testing performed, and 93% had both FPG and serum creatinine before pre-diabetes diagnosis.37 Since these are routinely captured data, this would not incur additional laboratory or blood-taking costs. Application of extended RUBY 2, as compared with original RUBY would reduce the number needed to screen from 2.4 to 1.9 (PPV 51.6% vs 42.4%) to diagnose one case of AGT. Although a formal cost-effectiveness analysis is beyond the scope of this paper, assuming a hypothetical population of 1 million, OGTT costs of $200 Hong Kong dollars (US$25) per test, and a prevalence of AGT of 24%, we estimate the extended RUBY model 2 would reduce the cost per case identified by 19%.

Our study has several limitations. First, our sample size was small although external validation of the extended RUBY model 2 showed good discrimination. Second, HbA1c was used for diagnosis of diabetes only since 2003 while the BHBHK-HKFDS cohort was established in 1998–2003. Thus, baseline HbA1c was not available in our analysis. However, the use of HbA1c to diagnose AGT is challenging in resource-restrained settings, apart from issues related to standardization of assays, there are other confounding factors such as ethnicity, anemia, hemoglobinopathy, or other coexisting conditions.38 In our study, the BHBHK was a territory-wide community-based survey that covered 50% of the working population including manual and unskilled workers.17 Nevertheless, selection criteria based on OGTT may have led to a potential selection bias, favoring participants with higher adherence to medical protocols. Selecting such a cohort might lead to an underestimation of AGT prevalence, as healthier habits could underestimate the incidence of glucose intolerance. Consequently, the actual prevalence in a more general population might be higher. Our study was conducted in middle-aged Chinese community-dwelling individuals which might limit its generalizability to other ethnicities or age groups.

In summary, the inclusion of routine renal parameters (serum potassium, urea, creatinine) and FPG, greatly improved the performance of a non-invasive risk score in identification of subjects with AGT. Pending further studies, the use of our extended RUBY model may improve the yield of using OGTT to diagnose subjects with IGT and diabetes with high 2hPG for early intervention.

Data availability statement

Data are available upon reasonable request. Anonymised data are available upon reasonable request to the corresponding author.

Ethics statementsPatient consent for publication

Not applicable.

Ethics approval

The study has received ethical approval from the Joint New Territories East Cluster-Chinese University of Hong Kong ethics committee (CREC 2009.421). Participants gave informed consent to participate in the study before taking part. The studies have also received local ethical approvals (2019.310).

Acknowledgments

We thank all study participants, physicians and nurses at the Diabetes and Endocrine Centre at the Prince of Wales Hospital. Special acknowledgment is extended to Ms Cherry Chiu, the nursing officer who coordinated the recruitment of the BHBHK-HKFDS Cohorts.