In total, 212 and 255 complete responses were obtained in 2016 and 2021, respectively. Detailed characteristics of the respondents in both groups are presented in Table 1. Prior to PSM, most respondents in 2016 were researchers (72.2%), resided in Eastern China (78.3%), and had educational backgrounds in medicine (51.9%). The most common highest degree obtained was a bachelor’s degree (47.2%). The average age of the respondents was 38.9 years (± 8.7 years), and the average years of work experience was 10.61 years (± 6.5 years).

Among the 255 respondents in the 2021 group, 52.9% were researchers. The majority of participants had an educational background in medicine (57.6%), and the most common highest degree obtained was a bachelor's degree (48.2%). Most respondents resided in Eastern China (86.3%). The average age of the respondents was 37.8 years (± 9.1 years), and the average years of work experience was 7.9 years (± 7.2 years). There were no significant differences regarding gender, age, or region between the two groups (P > 0.05).

The PSM process yielded a matched sample of 183 respondents in both the 2016 and 2021 groups, using a 1:1 nearest neighbor matching algorithm. Following PSM adjustment, no significant differences were observed between the two matched groups regarding the propensity score (generated by the PSM method, as shown in Additional file 2: Figure S2). This indicates that the baseline characteristics were effectively balanced between the two groups.

As indicated in Table 2, the overall score for HTA development in China was higher in 2021 than in 2016 prior to applying PSM. Furthermore, the mean scores across various domains, including institutionalization, priority setting, assessment, appraisal, reporting, dissemination of findings and conclusions, and implementation in policy and practice., suggest notable progress in HTA development over time. However, the identification domain identified in 2021 had a relatively lower overall mean score than that in 2016.

Within the “Institionization” domain, most respondents (63.4% in 2016 and 71.0% in 2021) indicated that China lacked an HTA agency that met the specified survey criteria. These criteria encompass aspects such as reporting to a Minister of Health/human resources or other relevant authorities, generating and/or endorsing HTA reports, and informing decisions regarding the introduction, reimbursement, and disinvestment of health technologies. Additionally, the respondents assessed the presence of essential elements for establishing a formal HTA program in China (see Table 3).

The mean score for ‘interest in HTA expressed by government/policy makers which can be retrieved in official documents’ (item I.1, t = −4.455, P < 0.05), ‘commitment toward HTA from government/policy makers and it is expressed in official documents’ (item I.2, t = −2.134, P < 0.05), ‘support for HTA from various stakeholders’ (item I.5, t = −3.856, P < 0.05), and ‘the availability of human resource development’ (item I.8, t = −3.289, P < 0.05) were significantly higher in 2021 than in 2016.

The ‘Identification’ domain focuses on the implementation of emerging technologies in need of assessment or those identified in the early monitoring system(s). The mean score of the indicator related to the existence of monitoring system(s) for emerging technologies (item II.1) is significantly higher in 2016 (1.98 out of 3) than in 2021(1.51 out of 3), (t = 4.237, P < 0.05). Regarding the performance of other activities involving identification (item II.2), the mean scores is 1.99 out of 3 in 2016 and 1.98 out of 3 in 2021.

Regarding the characteristics of China’s priority setting process, most indicators had a mean score of nearly 2 out of 3, demonstrating that these indicators were largely present. The indicator related to stakeholder involvement (item III.3) consistently received the highest score in both 2016 and 2021. However, respondents noted that the clarity of information on priorities (item III.4, t = 2.144, P < 0.05) and the extent of available literature (item III.5, t = 5.484, P < 0.05) were more explicit in 2016 than in 2021.

The ‘Assessment’ domain comprises 16 indicators categorized into five dimensions, inclusion of goal and scope, description of alternative technologies, aspects of assessment contents, standardized methods incorporation, and generalizability of the HTA scheme. The mean score of indicators concerning healthcare problems, patient population, and practitioners or users (item IV.1) was higher in 2016 than in 2021 (P < 0.05).

Regarding describing the technical characteristics of health technologies under study and their alternatives (item IV.2), the mean score in 2021 was lower than that in 2016 (t = 4.469, P < 0.05). Respondents reported that HTA activities in 2016 focused more on safety, clinical effectiveness, cost, and cost-effectiveness (item IV.3) than those in 2021 (P < 0.05). However, the organizational analysis showed the opposite trend (t = −2.408, P < 0.05).

Across all three indicators in the fourth dimension (Do HTAs incorporate standardized methods, item IV.4), the scores were below 2, indicating that this aspect of the HTA process was considered less developed by respondents. The indicator assessing whether HTA plans in China addressed generalizability and transferability (item IV.5) received the highest mean score among all 16 indicators in both 2016 (2.30 out of 3) and 2021 (2.23 out of 3).

The ‘Appraisal’ domain investigated whether a transparent and deliberative appraisal system, according to the participants, was in place. The respondents in 2021 believed that the appraisal process was more explicit, transparent, and replicable than those in 2016 (item V.1, t = −3.279, P < 0.05).

The ‘Reporting’ domain related to the utilization of the best practice guidelines in conducting and reporting HTA (item VI.1) received a score of 2.28 out of 3 in 2016 and 2.34 out of 3 in 2021, indicating that this aspect of HTA reporting was considered well developed in China over the five-year period.

Respondents reported a mean score of 2.93 out of 4 in 2016 and 2.70 out of 4 in 2021 for the number of HTA reports produced per year (item VI.2a, t = 2.564, P < 0.05), indicating a declining trend from 2016 to 2021. In contrast, the mean score for the number of HTA reports related to the NRDL per year (item VI.2b) was 3.09 out of 4 in both 2016 and 2021.

The ‘Dissemination of findings and conclusions’ domain related to the timeless of HTA report dissemination to decision makers and some dissemination strategies. It is worth noting that all indicators related to dissemination strategies (item VII.2) in 2021 showed significant improvement compared to those for 2016 (P < 0.05).

The ‘Implementation in policy and practice’ domain is relevant to the policy and practice information provision and the HTA impact measurement. The mean scores for each indicator were above 2 out of 3, showing that these indicators were largely present. These findings suggest that HTA implementation in policy and practice is moderately well developed in China.

Overall, the level of HTA development in China was higher than that in middle-income countries (Argentina, Brazil, India, Indonesia, Malaysia, Mexico, and Russia) and lower than that in high-income countries (Australia, Canada, and the United Kingdom) in 2016 and 2021 (see Additional file 3: Table S1). In our previous research conducted in 2016, we found that China scored lower than all ten countries regarding institutionalization level based on the views of survey respondents. However, the current results indicate that China has made significant advancements in institutionalization, although in 2021 the score for China is still lower than that for middle-income countries.