Selection process

We retrieved 108 hypertension-related Cochrane reviews up to December 31, 2020, of which 35 were excluded for the following reasons: 1 was withdrawn from publication, 1 was inactive, 30 were intervention-protocols, and 3 reviews were without RCTs. Seventy-three Cochrane reviews included at least one RCT, and 2185 RCTs were included in these reviews, of which 622 shared the same/similar “author name & year” (e.g., Bruni 2003 vs. Bruni 2003/ACCORD 2010 vs. ACCORD BP 2010). We manually checked RCTs that shared the same reference in different reviews and excluded 314 RCTs for the following reasons: 140 RCTs with the same/similar “author name & year” from one Cochrane review, 38 RCTs with similar “author name & year” had different references, and 136 RCTs with same “author name & year” had different references. A total of 308 RCTs shared the same reference. Among the 308 RCTs, 101 RCTs that reviews’ authors did not use the 2011 version of the RoB tool. Forty-two RCTs were assessed the RoB in only one Cochrane review. A total of 165 RCTs from 26 Cochrane reviews were included and matched with 78 groups. Among the 165 RCTs, 111 RCTs in 51 groups were evaluated for “random sequence generation”. A total of 165 RCTs in 78 groups were evaluated for “allocation concealment”. All outcomes in our study were objective, so the RoB assessments of blinding were not affected by the outcomes, despite their subtle differences. A total of 165 RCTs in 78 groups were evaluated for “blinding of participants and personnel”. And 85 RCTs in 39 groups were evaluated for “blinding of outcome assessment”. A total of 161 RCTs in 76 groups were evaluated for “incomplete outcome data”. Twenty-six RCTs in 13 groups focused on different outcomes in different reviews. Therefore, only 113 RCTs in 63 groups were analysed for “incomplete outcome data”. A total of 152 RCTs in 72 groups were evaluated for “selective outcome reporting”. Ninety RCTs in 44 groups were evaluated for “other sources of bias”. Fig. 1 shows the selection process.

Fig. 1Characteristics of the included Cochrane reviews

The characteristics of the included Cochrane reviews are shown in Table 1. Of the 26 reviews included in this study, Canada produced the most reviews (61.5%), followed by the UK (7.6%), Costa Rica (7.6%) and China (2.7%). The median number of RCTs included in the Cochrane reviews was 16. The median number of participants in the Cochrane reviews was 11,789. The tool for assessing all RCTs in this study was the 2011 version of the RoB, not the RoB 2.0. The publication trends of the reviews are shown in Fig. 2.

Table 1 Characteristics of included the Cochrane reviewsFig. 2

The publication trends of the included Cochrane reviews

Characteristics of the included RCTs

A total of 78 RCTs were included in the study. Most RCTs were published in the Lancet (11.5%), followed by the Journal of Hypertension (10.3%), Hypertension (7.7%), BMJ- British Medical Journal (7.7%), American Journal of Cardiology (3.8%), Therapeutic Research (2.6%), JAMA (2.6%), and European Journal of Clinical Pharmacology (2.6%). Details of the journals published by the included RCTs are shown in Supplementary Material 1. The median impact factor for journals was 4.2. Of the included RCTs, 92.3% were included in two reviews, 6.4% were included in three reviews, and 1.3% were included in four reviews. Most RCTs were from the USA (33.3%), followed by the UK (15.4%). In terms of research topics, 69.2% examined drug therapy, and 30.8% examined non-pharmacologic treatment (Table 2).

Table 2 Characteristics of RCTs in Cochrane reviewsAssessment of agreements and disagreementsAssessment of RCT level

The assessment results were agreement in 44 (56.4%) RCTs and disagreement in 34 (43.6%) RCTs at the trial level. In the agreement group, high risk vs. high risk accounted for 90.9%. In the disagreement group, unclear risk vs. high risk and low risk vs. unclear risk accounted for 52.9% and 38.2%, respectively. The distribution of agreement and disagreement of the RoB assessment at the RCT level is shown in Fig. 3.

Fig. 3

The distribution of agreement and disagreement of the RoB assessment at the RCT level. A: The distribution of agreement of the RoB assessment at the RCT level. B: The distribution of disagreement of the RoB assessment at the RCT leve

Assessment of each domain in the RoB

“Random sequence generation” was assessed in 51 RCTs, and the assessment results were agreement in 12(23.5%) RCTs and disagreement in 39(76.5%) RCTs. In the disagreement group, there were 9 (17.6%) low risk vs. unclear risk and 30 (58.8%) unclear risk vs. high risk (Fig. 4).

Fig. 4

Distribution of agreement and disagreement for different RoB domains

“Allocation concealment” was assessed in 78 RCTs, of which 19 (24.4%) RCTs showed agreement of the RoB assessment. The disagreements included 27 (34.6%) low risk vs. unclear risk and 32 (41.0%) unclear risk vs. high risk (Fig. 4).

“Blinding of participants and personnel” was assessed in 78 RCTs, and the assessment results were agreement in 37 (47.4%) RCTs and disagreement in 41 (52.6%) RCTs. In the disagreement group, there were 18 (23.1%) low risk vs. unclear risk, 16 (20.5%) unclear risk vs. high risk, 4 (5.1%) low risk vs. high risk, and 3 (3.8%) low risk vs. unclear risk vs. high risk (Fig. 4).

“Blinding of outcome assessment” was assessed in 39 of RCTs, and the assessment results were agreement in 37 (94.9%) RCTs and disagreement in 2 (5.1%) RCTs. In the disagreement group, there were 1 (2.6%) unclear risk vs. high risk and 1 (2.6%) low risk vs. high risk (Fig. 4).

“Incomplete outcome data” were assessed in 63 RCTs that focused on the same outcomes in different reviews. The assessment results were agreement in 47 (74.6%) RCTs and disagreement in 16 (25.4%) RCTs. In the disagreement group, there were 8 (12.7%) low risk vs. unclear risk, 1 (1.6%) unclear risk vs. high risk, 5 (7.9%) low risk vs. high risk, and 2 (3.2%) low risk vs. unclear risk vs. high risk (Fig. 4).

“Selective outcome reporting” was assessed in 72 RCTs, and the assessment results were agreement in 57 (79.2%) RCTs and disagreement in 15 (20.8%) RCTs. In the disagreement group, there were 2 (2.8%) low risk vs. unclear risk, 2 (2.8%) unclear risk vs. high risk, 9 (12.5%) low risk vs. high risk, and 2 (2.8%) low risk vs. unclear risk vs. high risk (Fig. 4).

“Other sources of bias” were assessed in 45 RCTs, and the assessment results were agreement in 34 (75.6%) RCTs and disagreement in 11 (24.4%) RCTs. In the disagreement group, there were 5 (11.1%) low vs. unclear, 4 (8.9%) unclear vs. high, 1 (2.2%) low vs. high and 1 (2.2%) low vs. unclear vs. high (Fig. 4).

Possible reasons for disagreement in the RoB assessment

At the trial level, there was no significant difference in the proportion of the year of publication ≤ 1996 and impact factor between the agreement and disagreement groups. At the domain level, the “allocation concealment” and “blinding of participants and personnel” had higher proportion of publication year ≤ 1996 in the agreement group (P = 0.008 and P < 0.001, respectively). For the “blinding of participants and personnel”, the impact factor was higher in the agreement group (P < 0.001) (Table 3). We analyzed the support text, and found that the most common reason for disagreement was related to extracting different information in the article. The other reason was that the reviewers considered differently in same or similar text, 41.0% for “random sequence generation”, 30.5% for “allocation concealment”, 29.3% for “blinding of participants and personnel”, 50.0% for “blinding of outcome assessment”, 6.7% for “selective outcome reporting”, and 9.1% for “other sources of bias”. The main reasons for differences in support text for each domain are reported in Table 4.

Table 3 The year of publication and impact factor of the journal between the agreement and disagreement groups for the RoB assessmentTable 4 Main reasons for disagreements in assessment domains for RoB