Introduction

Antimicrobial resistance is one of the major challenges faced by public health, resulting in prolonged hospital stays, increased mortality rates, and augmented medical expenditures.1–4 The main driving factor behind the development of antimicrobial resistance is the inappropriate use of antimicrobial agents.1,5 Recent data show that over 50% of antimicrobial prescriptions worldwide are inappropriate.6–8 The inappropriate use of antimicrobial agents has been found to be more prevalent in underdeveloped regions and middle-income countries compared to high-income countries, as indicated by various studies.6,9–14 In China, a study on antimicrobial prescriptions for outpatient visits at primary care institutions found the inappropriate antimicrobial prescription rate (IAPR) of 79.8%.15 A national survey revealed that 68.9% of the inappropriate antimicrobial prescriptions in primary care institutions were attributed to diagnoses of upper respiratory tract infections, acute bronchitis, and noninfectious gastroenteritis.16

Relevant studies have previously proposed diverse intervention strategies to mitigate the inappropriate use of antimicrobial agents. Several studies have found that providing physicians with multifaceted intervention and communication training, as well as establishing an antimicrobial prescription peer review council, has a significant and long-term impact on reducing antimicrobial prescriptions.17–19 We conducted a cluster randomized cross-over controlled trial of antimicrobial prescription feedback intervention in 2021.20 Feedback intervention refers to the act of providing clinics, physicians, and healthcare workers with knowledge regarding their behavioral or performance outcomes.21–24 The trial results demonstrated the effectiveness of the feedback intervention measures in controlling both the IAPRs and the quantity of antimicrobial prescriptions in primary care institutions. In group A, during the period of receiving feedback intervention (phase 1 of the trial), the IAPR and the antimicrobial prescription rate (APR) decreased by 12.3% and 11.9%, respectively. In group B, during the period of receiving feedback intervention (phase 2 of the trial), there was a decrease in these rates by 15.1% and 11.7%, respectively.20 The existing studies have not assessed the long-term impacts of antimicrobial agents feedback intervention, and the feedback intervention has been employed as a means to regulate antimicrobial prescriptions in numerous studies.22,25–27 Therefore, it is imperative to provide empirical evidence regarding the long-term impacts of feedback intervention.

Drawing on findings from previous cluster randomized cross-over controlled trials,20 we present here the disparities in IAPRs and APRs between two groups of physicians who received the intervention following a 12-month follow-up period. The objective is to investigate any differences between two groups of physicians after a 12-month follow-up period, aiming to assess whether our intervention produced long-term impacts in IAPRs and APRs within primary care institutions in China. This study contributes theoretical evidence towards achieving long-term control of antimicrobial prescriptions within primary care institutions.

Methods Previous Intervention

Our previous study comprehensively reported the trial protocol, intervention details, and outcomes.20,28 A cluster randomized cross-over controlled trial of antimicrobial prescription feedback intervention was conducted in primary care institutions in Guizhou province from April to September 2021, based on the Health Information System (HIS). The feedback intervention comprised three measures: a real-time pop-up warning message, a 10-day antimicrobial prescription feedback, and distribution of educational manuals. Our research team was responsible for the overall design and implementation of all three intervention measures. To push the real-time pop-up messages and the links for 10-day antimicrobial prescription feedback, Guizhou Lianke Weixin Technology Co., LTD. (LWTC), a technical service company specializing in HIS development, has developed an antimicrobial prescription intervention early warning plug-in. With authorization and approval from the Information Center of Guizhou Provincial Health Commission (Additional file 1), LWTC has opened its data interface for us to collect data.

The first measure of the feedback intervention involved a real-time pop-up warning based on the HIS to identify inappropriate antimicrobial prescriptions. The physician’s computer screen will promptly display a reminder window if an inappropriate antimicrobial prescription is prescribed, providing a brief explanation for its inappropriateness. Consistent with our previous study,20 the appropriateness of antimicrobial prescriptions was evaluated according to the following four criteria: (1) the Guiding Principles for Clinical Use of Antimicrobial Agents issued by the National Health Commission of China in 2015, (2) the guidelines for antimicrobial use provided by the United States Centers for Disease Control and Prevention (CDC), (3) our previous research,14 and (4) the insights from 17 clinical and pharmaceutical experts well-versed in the situation of primary care institutions in China. The appropriate use of antimicrobial agents was categorized into two types: (1) preferred medication: the optimal choice of drug, and (2) antimicrobial agents can be used or substituted: available but not the optimal choice. Inappropriate antimicrobial prescriptions were classified into three categories: (1) unnecessary use: such as prescribing antimicrobial agents to patients diagnosed with viral infections, (2) incorrect spectrum of antimicrobial agents: such as prescribing nitroimidazoles or aminoglycosides for group A streptococcal infections, (3) combination of antimicrobial agents without indication: referring to the concurrent use of diverse categories of systemic antimicrobial agents during a single visit without any indication, such as the combination of amoxicillin and cefixime.

The second feedback measure consisted of providing physicians with updated antimicrobial prescription feedback every 10 days. The information primarily included: the APR and its ranking within institutions, the top 5 main diseases using antimicrobial agents, the top 5 antimicrobial agents prescribed along with their frequencies, as well as contraindications and precautions associated with antimicrobial agents. A hyperlink for the prescription feedback was displayed at the bottom of the physician’s computer screen, allowing them to access it whenever necessary. This regularly refreshed information is confidential and can be prioritized or disregarded freely by the physician.

The third component of the intervention was the distribution of the educational manual titled “Guidance and Suggestion Manual for Clinical Use of Common Antimicrobial Agents in Primary Care Institutions”. The manual originated from a previous Delphi study29 and provided physicians with advice on the appropriate use of antimicrobial agents, as well as guidance in diagnosing common infections at the primary care level.

Study Design and Participants

A 6-month feedback intervention trial was conducted from April to September 2021. The trial employed a cluster randomized cross-over controlled trial approach, with the aim of investigating the impacts of the feedback intervention in reducing of IAPRs and APRs among primary care institutions. A random number table was employed to select 79 institutions out of the total of 132 primary care institutions. The participating institutions were then randomly assigned into groups A and B. Group A received the intervention for 3 months, while group B was assigned as the control group for 3 months (without any intervention). After a period of 3 months, the two groups underwent a transition, with group A transitioning to the control group and group B receiving the intervention for an additional 3 months. This transition allowed each group to serve as its own control, effectively reducing variation caused by individual differences. Furthermore, by administering the intervention at different times for each group, the potential confounding effect of time was mitigated. As a result, the evaluation of the intervention effect can be conducted more accurately. The trial spanned over a period of 6 months, and on June 30th, both groups reached the cross-over point. Ultimately, the trial ended on September 30th (Figure 1a).

Figure 1 Schematic diagram of the timeline adjustment for comparison between groups (A and B) at the equivalent stages. IAPR: inappropriate antimicrobial prescription rate, APR: antimicrobial prescription rate. (a) Diagram of the cluster randomized cross-over controlled intervention trial. (b) The variation tendency of IAPRs in groups (A and B) based on the original monthly timelines. (c) The variation tendency of IAPRs in groups (A and B) based on the adjusted equivalent intervention phase timelines. 0: Baseline for both groups, 1–3: Pre-intervention trial influence period, 4–6: Intervention period, 7–18: Follow-up period. (d) The variation tendency of APRs in groups (A and B) based on the original monthly timelines. (e) The variation tendency of APRs in groups (A and B) based on the adjusted equivalent intervention phase timelines. 0: Baseline for both groups, 1–3: Pre-intervention trial influence period, 4–6: Intervention period, 7–18: Follow-up period.

The 335 physicians who willingly participated in the feedback intervention during the trial were followed for a period of 12 months after the intervention stopped. Due to the nature of a randomized cross-over control trial, the intervention was implemented at different time points for group A and group B. Therefore, the follow-up started and stopped at different times: group A was followed up from July 2021 to June 2022, while group B was followed up from October 2021 to September 2022. During the follow-up period, 8 physicians in group A and 11 physicians in group B were lost to follow-up because of job transfers. The loss rate did not exceed 10%.

It is not easy to compare groups A and B because the baseline points started at different times. As shown in Figure 1b and d, the overall timeline of group A was advanced by 3 months compared to group B. To ensure comparability between the two groups, the baseline points were translated to the same starting points (Figure 1c and e). The interval between months was indicated using Arabic numerals ranging from 0 to 18. The utilization of this time tag method facilitates enhanced clarity in discerning the disparities between the two groups.

Data Collection

In this study, the antimicrobial prescription records are provided by LWTC, and we collected data from baseline to the end of the 12-month follow-up period. These data included: antimicrobial prescriptions, total prescription data and patient information in primary care institutions. A coding system developed by our team was employed to establish a connection between physicians’ and patients’ names on prescriptions. The demographic information of physicians was obtained from the personnel departments of primary care institutions. Physicians’ characteristics included: age, sex, professional title, education, and working years. All researchers signed strict confidentiality agreements in data collection process (Additional file 2).

ICD-10 is employed as the standard for disease classification in this study. According to the Essential Medicines List published by the World Health Organization (WHO) and the China’s clinical application guidelines for antimicrobial agents, the clinical application catalogue of antimicrobial agents was summarized (Additional file 3). In conjunction with the common types of antimicrobial agents used in primary care institutions, our study encompassed seven classes of antimicrobial agents: penicillins, cephalosporins, macrolides, quinolones, lincosamides, nitroimidazole, and aminoglycosides. This study exclusively focused on systemic antimicrobial agents while excluding topical antimicrobial prescriptions such as erythromycin ointment and levofloxacin eye drops.

Outcome Variables

In this study, the outcome variables are consistent with those we used in our previous trial.20 The primary outcome variable is the IAPR for each month while the secondary outcome variable is the APR for each month. The monthly IAPR is calculated by dividing the number of inappropriate antimicrobial prescriptions for each month by the total number of antimicrobial prescriptions. The monthly APR is defined as the number of antimicrobial prescriptions for each month divided by the total number of prescriptions. In this study, the average decline in IAPRs and APRs will be calculated by dividing the decline value by the number of months. The decline value is determined by subtracting the endpoint value from the starting point value of IAPR or APR for a certain period.

Data Analysis

The generalized estimation equation (GEE) method was employed to evaluate the impacts of the feedback intervention. The data analysis was conducted using the four representative time points: the baseline point, the 3 months (the end of trial’s influence before the intervention), the 6 months (the end of the intervention period), and the 18 months (the end of the follow-up period). The exchangeable correlation matrix was selected to construct the model of IAPRs based on the principle of minimizing quas-likelihood under the independence model criterion (QIC), while the unstructured correlation matrix was selected to construct the model of APRs. Additionally, robust standard errors were used to account for clustering within the same group and between time periods within the same group. The time-intervention interaction was utilized in the model to evaluate the change in trends between two groups during the corresponding period, that is, the respective difference of IAPRs and APRs in different periods: (1) the difference in change difference from the baseline point to the 3 months between group A and group B, (2) the difference in change difference from the 3 months to the 6 months between two groups, (3) the difference in change difference from the baseline point to the 6 months between two groups, (4) the difference between the difference in two groups at baseline point and the difference in two groups at 6 months, (5) the difference in change difference from the baseline point to the 18 months between two groups, (6) the difference in change difference from the 6 months to the 18 months between two groups (Figure 2). These analyses enabled us to comprehend the impacts of feedback intervention on outcomes from baseline to the 3 months, from baseline to the end of the intervention period, and from baseline to the end of the follow-up period. We applied the GEE with Gaussian errors and identity links to compute intervention effects for prescription outcomes, which is suitable for analyzing continuous outcome variables in most models.30

Figure 2 Flow chart for the evaluation of feedback intervention trial. The IAPRs and APRs at the baseline point were calculated from prescriptions during the month that remained entirely unaffected by the trial and was closest to the start of the trial (group (A) December 2020, group (B) March 2021). The IAPRs and APRs at 3 months were calculated from prescriptions during the last month of the trial’s influence before the intervention (group (A) March 2021, group (B) June 2021). The IAPRs and APRs at 6 months were calculated from prescriptions during the last month of the intervention period (group (A) June 2021, group (B) September 2021) and at 18 months were calculated from prescriptions during the last month of the follow-up period (group (A) June 2022, group (B) September 2022).

Abbreviations: IAPR, inappropriate antimicrobial prescription rate; APR, antimicrobial prescription rate.

In this study, we adjusted for covariates including sex, age, professional title, working years, and education of physicians. The covariate-adjusted results were considered as our primary outcome due to their potential to enhance statistical power and precision. Consistent with our previous trial analysis,20 the original intervention order for any clusters remained unchanged, and given the completeness of the study, we incorporated the data from the intervention period into our analysis. On the other hand, considering our primary focus on assessing the long-term impacts of the intervention and encountering instances where physicians were lost during the follow-up period, we chose to maintain consistency in terms of physicians’ number between the baseline and follow-up period. Consequently, data pertaining to physicians who were lost to follow-up were excluded from analysis during both time-frames.

The categorical variables were described as frequencies and percentages while the continuous variables were presented as means and standard deviations. Chi-square test was employed for comparing categorical variables. Continuous variables were compared using the t test. Data were visualised by calculating IAPR and APR for each month using WPS V5.5.1. The analysis was conducted with a double-sided alpha level of 0.05 to establish statistical significance. Data handling and statistical analysis were performed using R version 4.3.1.

Results

The groups A and B (group A switched to control after 3 months of intervention and group B switched to intervention after 3 months of control) were followed for a duration of 12 months (group A: from July 2021 to June 2022, group B: from October 2021 to September 2022) after the end of the intervention. In group A, the original count of 160 physicians was reduced to 152 during the course of the follow-up, while the original count of 168 physicians decreased to 157 in group B. The roster of baseline physicians refers to the list of physicians who have been excluded due to loss of follow-up, ensuring consistency with the designated follow-up period, while the intervention physicians remain consistent with our original trial analysis20 (Figure 2). Table 1 shows the demographic characteristics of the physicians during the baseline period, pre-intervention trial influence and intervention periods, as well as the 12-month follow-up period. The analysis revealed no statistically significant differences in sex, age, professional title, education, and working years across the four time periods. Additionally, our original trial analysis had confirmed that the characteristics of physicians were comparable between groups A and B at baseline.20

Table 1 Demographic Characteristics of the Physicians During the Baseline Period, the Pre-Intervention Trial Influence and Intervention Periods, as well as the 12-Month Follow-Up Period

Figure 3 depicts the trends of IAPRs in groups A and B. In group A, the IAPRs exhibited a certain range of rebound following the end of intervention (at 6 months, June 2021), which subsequently flattened after 2 months (at 8 months, August 2021). At 18 months (June 2022), the IAPR reached its highest point during the follow-up period at 67.33%, but remained below the baseline rate of 75.65%. Overall, the average decline in IAPRs for group A was 1.20% during the baseline point and 3-month period, 4.83% during the intervention period, −0.82% during the follow-up period, and 0.46% during the baseline point and 18-month period. As for group B, it terminated the intervention in September 2021 (at 6 months), resulting in a rebound of IAPRs, which subsequently flattened after 4 months (at 10 months, January 2022). Throughout the follow-up period, the IAPR reached its peak at 15 months (June 2022) with a value of 63.43%, yet remained below the baseline level of 74.25%. The average decline in IAPRs for group B was observed to be 2.31%, 4.59%, −0.20%, and 1.02% during the baseline point to 3-month period, during the intervention period, during the follow-up period, and during the baseline point to 18-month period, respectively. The above results demonstrated a gradual decline in IAPRs for both groups during the baseline point and 3-month period, followed by a rapid decline during the intervention period compared to the initial period. Subsequently, there was a gradual increase observed during the follow-up period in comparison to the intervention period. But there was still a slow decline from the baseline point to the end of the follow-up.

Figure 3 The temporal dynamics of IAPRs. The data points represent the values obtained by dividing the number of monthly inappropriate antimicrobial prescriptions by the overall number of monthly antimicrobial prescriptions. At baseline point: group A in December 2020, group B in March 2021. At 3 months: group A in March 2021, group B in June 2021. At 6 months: group A in June 2021, group B in September 2021. At 18 months: group A in June 2022, group B in September 2022.

Abbreviation: IAPR, inappropriate antimicrobial prescription rate.

The trends of APRs over time in the two groups are illustrated in Figure 4. It was observed that both groups experienced a certain degree of rebound after the end of the intervention (at 6 months, group A in June 2021 and group B in September 2021). In group A, the APRs rebounded to 27.89% in the sixth month after the stop of the intervention (at 12 months, December 2021), reaching its peak level of rebound, and subsequently displayed a declining trend. The average decline in APRs for group A was observed to be 2.46%, 2.88%, and 0.32% during the three periods: from baseline point to 3 months, during the intervention period, and during the follow-up period, respectively. In group B, the APR reached its peak rebound in the fourth month after the end of the intervention (at 10 months, January 2022), with an observed rate of 28.95%. Notably, neither group surpassed their respective baseline levels (the APR at baseline was 36.47% in group A and 35.11% in group B) at the highest point of the rebound. The average decline in APRs for group B was 1.12% during the baseline point and 3-month period, 3.42% during the intervention period, and 0.04% during the follow-up period. The above findings revealed a gradual decline in APRs for both groups during the baseline point and 3-month period, followed by a relatively rapid decline during the intervention period compared to the initial period. Subsequently, there was the slowest decline observed during the follow-up period among the three time periods.

Figure 4 The temporal dynamics of APRs. The data points represent the values obtained by dividing the number of monthly antimicrobial prescriptions by the overall number of monthly prescriptions. At baseline point: group A in December 2020, group B in March 2021. At 3 months: group A in March 2021, group B in June 2021. At 6 months: group A in June 2021, group B in September 2021. At 18 months: group A in June 2022, group B in September 2022.

Abbreviation: APR, antimicrobial prescription rate.

Analysis of Differences

The GEE analysis results presented in Table 2 demonstrate the impact of physicians’ demographic characteristics on IAPRs and APRs. Notably, physicians with a college-level education exhibited a statistically significant difference in the coefficient (−0.04) representing APRs compared to those with a technical secondary school degree, suggesting that higher levels of education may decrease the likelihood of prescribing antimicrobial agents. Apart from this coefficient, none of the correlation coefficients for physicians’ demographic characteristics showed statistical significance in either IAPRs or APRs.

Table 2 GEE Models Predicting the Impact of Demographic Characteristics on IAPRs and APRs

IAPRs

The results of covariate-adjusted IAPRs are presented in Table 3. The mean changes observed in the IAPRs showed statistically significant differences between the baseline and the intervention periods, which were consistent with our previous trial analysis.20 At 18 months (June 2022), group A exhibited a mean decrease in IAPR of 7% (95% CI: 0.90 to 0.97, P=0.0003) from the baseline point (December 2020) and 4% (95% CI: 0.92 to 1.00, P=0.0272) from the 3 months (March 2021). Furthermore, there was an observed increase of 11% (95% CI: 1.07 to 1.16, P<0.001) when comparing it with the 6 months (June 2021). The average decline in IAPRs for group A was 1.00% from the baseline point to the 3 months, 5.00% from the 3 months to the 6 months, −0.92% from the 6 months to the 18 months, and 0.39% from the baseline point to the 18 months. In group B, the mean IAPR at 18 months (September 2022) demonstrated a decrease of 13% (95% CI: 0.84 to 0.92, P<0.001) and 6% (95% CI: 0.90 to 0.98, P=0.0084) compared to the baseline point (March 2021) and the 3 months (June 2021), respectively. However, there was an increase of 5% (95% CI: 1.00 to 1.10, P=0.0420) compared to the 6 months (September 2021). The average decline in IAPRs for group B was observed to be 2.33%, 3.67%, −0.42%, and 0.72% from the baseline point to the 3 months, from the 3 months to the 6 months, from the 6 months to the 18 months, and from the baseline point to the 18 months, respectively. In the aforementioned results, the observed increases in IAPRs and the negative average decline in IAPRs for groups A and B indicated that once the intervention was stopped (group A in June 2021 and group B in September 2021), both groups experienced a certain degree of rebound in IAPRs. In addition, the IAPRs of groups A and B displayed the following dynamic changes: a gradual decline from the baseline point to the 3 months, a relatively rapid decline from the 3 months to the 6 months compared to the initial period. Subsequently, there was a gradual increase from the 6 months to the 18 months compared to the period between the 3 months and the 6 months. But there was a slow decline from the baseline point to the 18 months.

Table 3 The Covariate-Adjusted Analysis Results of IAPRs

APRs

Table 4 shows the differences in APRs between the two groups and within the same group at baseline point (group A: December 2020, group B: March 2021), at 3 months (group A: March 2021, group B: June 2021), at 6 months (group A: June 2021, group B: September 2021), and at 18 months (group A: June 2022, group B: September 2022). Based on the covariate-adjusted results, it was observed that both groups exhibited an overall decrease in APRs over time. In group A, the average decline in APRs was 1.33% from the baseline point to the 3 months, 3.67% from the 3 months to the 6 months, and 0.17% from the 6 months to the 18 months. During the corresponding three periods in group B, the average decline in APRs was 1.00%, 3.67%, and 0.08%, respectively. Both groups displayed a gradual decline in APRs from the baseline point to the 3 months, followed by a rapid decline from the 3 months to the 6 months compared to the period between the baseline point and the 3 months. Subsequently, there was the slowest decline observed from the 6 months to the 18 months among the three time periods.

Table 4 The Covariate-Adjusted Analysis Results of APRs

In alignment with our previous trial analysis,20 the observed mean changes in APRs exhibited statistically significant differences from the baseline point to the 3 months, from 3 months to the 6 months, and from the baseline point to the 6 months. At 18 months in group A, the mean APR decreased by 17% (95% CI: 0.82 to 0.86, P<0.001) compared to the baseline point, 13% (95% CI: 0.86 to 0.89, P<0.001) compared to the 3 months and 2% (95% CI: 0.96 to 1.00, P=0.0216) compared to the 6 months. Similarly, in group B, there was a decrease of 15% (95% CI: 0.84 to 0.89, P<0.001) and 12% (95% CI: 0.86 to 0.91, P<0.001) from the baseline point and the 3 months, respectively.

Discussion

The present study assessed the long-term impacts of the intervention trial on antimicrobial feedback in primary care institutions. The IAPRs and APRs in the two groups of physicians who received the feedback intervention exhibited a rebound within a specific range after the intervention stopped, followed by plateauing over time, and the IAPRs remained persistently high throughout the follow-up period. Compared to the baseline point, both groups of physicians demonstrated reductions in the IAPRs and APRs. However, the impacts of the feedback intervention in follow-up period could not maintain the significant effect like the intervention period. To ensure the integrity of our research, we incorporated the intervention phase into our analysis.

Based on Figures 3 and 4, there was a certain degree of rebound in IAPRs and APRs following the end of the feedback intervention. The negative average decline in IAPRs and the covariate-adjusted IAPRs exhibited similar rebound outcomes (group A: 11% (95% CI: 1.07 to 1.16, P<0.001), group B: 5% (95% CI: 1.00 to 1.10, P=0.0420)). Without further intervention inputs, we do not expect a persistence of intervention effects beyond this time. This was consistent with the study by Sangwan et al,31 which concluded that the impacts of a single-occasion antimicrobial stewardship education program diminished over time in terms of improving adherence to guidelines and appropriateness of antimicrobial prescription in general practice. Similarly, a trial conducted in rural primary care institutions of Vietnam introducing C-reactive protein testing resulted in a 14% reduction in APR. However, this positive impact disappeared shortly after study completion due to patient demand, incentives provided by pharmaceutical companies and pressure on these facilities to replenish drug stocks.32 If the aim is to sustain feedback intervention effects over the long term, these effective feedback interventions should be put into the real world. Our team will continue to carry out relevant research using implementation science in the future.

The comparison within the same group indicated that the IAPRs and APRs of both group A and group B were reduced with the statistical significance at 18 months (group A: June 2022, group B: September 2022) compared to the baseline points (group A: December 2020, group B: March 2021). This was consistent with findings reported in other published research studies.18,19,33–35 During the follow-up period, the average decline in APRs remained positive (group A: 0.32% with a covariate-adjusted value of 0.17%, group B: 0.04% with a covariate-adjusted value of 0.08%), indicating an ongoing overall decrease in APRs during this period. The above results suggested that the intervention exerts certain long-term impacts on the entirety. However, at 18 months, both groups A and B exhibited a significant rebound in IAPRs compared to the end of intervention (at 6 months: group A in June 2021, group B in September 2021). These findings indicated that once the intervention was discontinued, there was a certain range of rebound in IAPRs for both groups A and B, which persisted until the end of the follow-up (at 18 months) without any statistically significant decrease after reaching their peaks. Additionally, as depicted in Figure 3, despite the overall stability of the IAPRs during the follow-up period (6–18 months), it is noteworthy that the IAPRs still remained persistently high at 67% and 56% which means that more than half of the prescriptions were considered inappropriate.

The above findings suggested that intervention measures should be persistently implemented. Our feedback intervention demonstrated certain long-term impacts. However, the average decline in IAPRs during the follow-up period exhibited a negative value, indicating a rebound. During the same period, the average decline in APRs was much less than that observed during the intervention period. These demonstrated that the effectiveness of this influence may not be consistently evident like during the intervention period. This conclusion was consistent with the findings published by Wei X et al,19 stating that the effectiveness of the educational intervention on antimicrobial prescriptions would decline without further intervention inputs.

In the comparison between the two groups, after adjusting for covariates, there were no statistically significant differences in the between-group change difference (group B versus group A) on IAPRs and APRs during various time periods. The results revealed that the change trends observed in groups A and B are consistent with each other. On the other hand, at 3 months (group A: March 2021, group B: June 2021), we observed a 5% (95% CI: 0.91 to 0.99, P=0.0198) decrease in IAPR in group B compared to group A. And the average decline in IAPRs from baseline to 3 months was higher in group B (2.33%) than in group A (1.00%). These indicated that the decline was more rapid in group B than in group A. The potential reason could be the impact of feedback intervention conducted in group A from April to June in 2021 on the prescription behavior of physicians in group B.20 At 18 months, there was a 7% (95% CI: 0.88 to 0.99, P=0.0195) lower IAPR in group B compared to group A, and the rebound observed in both groups indicated that group B exhibited a slower rebound than group A. The average decline in IAPRs for group B from 6 to 18 months was −0.42%, whereas for group A it was −0.92%, further highlighting the comparatively slower rise observed in group B compared to group A. This could be attributed to a learning effect, as group B received intervention after being influenced by group A’s 3-month feedback intervention trial, leading to enhanced long-term impacts. The learning effect refers to a significant increase in cognitive test scores with an increasing number of repetitions until scores stabilize and no longer change.36

In Figure 4, it can be observed that the APRs began to exhibit a clear downward trend from February 2022 (group A: at 14 months, group B: at 11 months). This shift may be attributed to seasonal factors. According to the research, the winter season (typically from December to February) weakens the body’s antiviral defense mechanism due to cold temperatures and reduced daylight, thereby increasing susceptibility to respiratory viruses and raising the risk of respiratory infections such as influenza.37 Conversely, the change of seasons brings warmer temperatures and increased sunlight from March to August, resulting in a decrease in the incidence of respiratory tract infections and a corresponding decline in APRs. However, the IAPRs depicted in Figure 3 had exhibited a persistent trend of platform fluctuation and even gradual elevation since February 2022. This phenomenon may be attributed to the diminishing impacts of the intervention following its stop, leading physicians to revert back to their previous prescribing practices. Consequently, IAPRs did not experience a decrease while there was a decline in APRs possibly due to seasonal changes. In addition, the National Medical Quality and Safety Improvement Goals in 2022 was issued by the National Health Commission of China in March 2022. In the initial stage of policy implementation, physicians and medical institutions might overreact due to unfamiliarity or a lack of clear operational guidance, leading to excessive restrictions on necessary antimicrobial agents use in order to avoid violations. This might reduce APRs but not necessarily reduce inappropriate use in specific cases. Over time, as they became more accustomed to the new policy, the IAPRs were expected to decrease. This accounted for the decline in IAPR observed in group B at 18 months, while group A was still increasing within the same timepoint due to a shorter exposure of 3 months compared to group B’s 6-month duration.

In this study, we tried to modify physicians’ prescription behavior by implementing a feedback intervention. By comparing their own prescription practices against specific goals, physicians are more likely to align their prescription behavior with social norms and consequently adopt corrective prescription practices.20–22,38 The findings suggest that regular implementation of feedback intervention for inspection or supervision can be considered as a viable option for controlling antimicrobial prescriptions in primary care institutions. Furthermore, the implementation science framework has been widely employed by numerous domestic and international scholars to bridge the existing gap between effective evidence-based practice in clinical research and its practical application in real-world settings.39,40 The insights that derive from implementation science theory can offer novel perspectives for future control and intervention strategies concerning antimicrobial agents, thereby presenting possible resolutions to the issue of inappropriate antimicrobial use.

The primary strength of this study lies in its evaluation of the long-term impacts of antimicrobial agents feedback intervention, thereby providing empirical evidence to support the effectiveness and sustainability of this intervention, a critical aspect that has been overlooked in numerous previous studies. Additionally, the utilization of a randomized cross-over controlled trial design enables us to validate the reliability of our findings by analyzing trend consistency in groups A and B, without the need for re-running the intervention trial. In this study, we found no statistically significant differences in changes of both IAPR and APR between group A and group B from the baseline point to the 18 months, indicating consistent trends in the two groups. This enhances our confidence that the findings reflect the true impacts of the intervention itself. Thirdly, it is worth noting that this study was conducted in a low-resource environment. This suggests that the feedback intervention can be effective in similar low-resource healthcare settings. The findings, based on prescription data from primary care institutions in southwest China, may not be directly applicable to all primary care institutions nationwide due to regional variations but are practical and feasible for regions with comparable resources. And this study serves as a valuable model for facilitating the implementation of antimicrobial interventions in other healthcare settings facing resource constraints. These findings carry significant implications for enhancing antimicrobial stewardship and addressing resistance.

However, our study has several limitations. Firstly, the number of physicians involved in the baseline and follow-up periods was lower than initially anticipated due to work transfers, resulting in inconsistencies in physician numbers between the baseline, follow-up periods, and the intervention period. However, demographic analysis revealed comparable characteristics among physicians across all three periods. Secondly, the seasonal bias was not considered in this study. To ensure comparability between groups A and B, the comparison must be conducted at the equivalent stage. For instance, the intervention period in group A (from April to June in 2021) versus the intervention period in group B (from July to September in 2021). Therefore, we are unable to effectively evaluate the potential influence of seasonal bias on the research results. Thirdly, there might be some information bias in this study. Physicians were aware of the intervention during the trial, but their awareness was balanced between both groups. Additionally, prescription data during the follow-up period were directly obtained through HIS, and physicians were unaware of the follow-up study. Hence, our conclusions were minimally influenced by information bias. Fourthly, our study did not consider the impact of COVID-19 epidemics and changes in prevention and control measures on IAPRs and APRs. However, this had little impact on our conclusions as the treatment of COVID-19 cases was generally referred to tertiary hospitals or higher for comprehensive management, rather than being directly conducted in primary care institutions which were the focus of our study.

Conclusion

The long-term impacts of our feedback intervention were existent. Because neither the IAPRs nor APRs surpassed the baseline point during the follow-up period and the average decline in APRs exhibited a positive value. However, the impacts were limited. Although the IAPRs and APRs consistently remained below the baseline point, both rates demonstrated a certain degree of rebound in groups A and B after the intervention was stopped, with the IAPRs consistently remaining at elevated levels throughout the entire follow-up period. Therefore, the antimicrobial feedback intervention needs to be implemented continuously to maintain the desired effects. The future efforts should focus on finding a pathway that effectively bridges the gap between evidence-based practice and real-world application in the field of antimicrobial prescription intervention, facilitating the integration of high-quality intervention measures into daily management practices in primary care institutions.

Trial Registration

ISRCTN, ID: ISRCTN13817256. Registered on 9 January 2020.

Abbreviations

IAPR, Inappropriate antimicrobial prescription rate; APR, Antimicrobial prescription rate; HIS, Health Information System; LWTC, LianKe Weixin Co., LTD.; ICD-10, the 10th Edition of the International Classification of Diseases; WHO, World Health Organization; GEE, Generalized estimation equation; QIC, Quas-likelihood under the independence model criterion; SD, Standard deviation; CI, Confidence interval.

Data Sharing Statement

According to the ethical agreements, the data are owned by the primary care institutions in the study area (Guizhou province) and are available only for the research purposes of the study team. The research data, which contain sensitive information regarding the demographic details of both patients and physicians, cannot be posted or downloaded from a public data repository due to the National Privacy Regulation. The further inquiries regarding the datasets used in this study can be directed to the corresponding authors upon reasonable request.

Ethics Approval and Consent to Participate

Our study adheres to the ethical principles outlined in the Declaration of Helsinki for medical research. The trial was approved by the Human Trial Ethics Committee of Guizhou Medical University (Certificate No.: 2019 (148)) in Dec. 27, 2019). The physicians participating in the study were provided with written informed consent. The accessed data were in compliance with relevant regulations regarding data protection and privacy.

Acknowledgments

We express our sincere gratitude to all the participating institutions for their invaluable support and provision of information throughout the study, which facilitated our research. Additionally, the authors extend their heartfelt appreciation to all members of the survey team who diligently and meticulously collected the data, as their unwavering dedication laid a solid foundation for our findings. We would also like to acknowledge Edward McNeil, from Prince of Songkla University in Songkhla, Thailand, for his insightful feedback on improving this manuscript.

Author Contributions

All authors made substantial contributions to the work reported, whether in the conception, study design, execution, acquisition of data, analysis and interpretation, or in all these areas; took part in drafting, revising or critically reviewing the article; gave final approval of the version to be published; have agreed on the journal to which the article has been submitted; and agree to be accountable for all aspects of the work.

Funding

This research was supported by the National Natural Science Foundation of China Grant on “Research on Feedback Intervention Mode of Antibiotic Prescription Control in Primary Care Institutions Based on the Depth Graph Neural Network Technology” (71964009) and the Science and Technology Fund Project of Guizhou Provincial Health Commission Grant on “Application Research of Deep Learning Technology in Rational Evaluation and Intervention of Antibiotic Prescription” (gzwjkj2019-1-218). Corresponding author YC is the project leader. The funders played a role in the study which we should acknowledge. Specifically, the funders provided travel expenses incurred during the data collection process, as well as the experts’ fees for providing guidance on the study design, technological guidance, data analysis and interpretation.

Disclosure

The authors declare that they have no competing interests.

References

1. Aparicio-Blanco J, Vishwakarma N, Lehr CM, et al. Antibiotic resistance and tolerance: what can drug delivery do against this global threat? Drug Deliv Transl Res. 2024;14(6):1725–1734. doi:10.1007/s13346-023-01513-6

2. Ralhan K, Iyer KA, Diaz LL, Bird R, Maind A, Zhou QA. Navigating antibacterial frontiers: a panoramic exploration of antibacterial landscapes, resistance mechanisms, and emerging therapeutic strategies. ACS Infect Dis. 2024;10(5):1483–1519. doi:10.1021/acsinfecdis.4c00115

3. Rzymski P, Gwenzi W, Poniedziałek B, Mangul S, Fal A. Climate warming, environmental degradation and pollution as drivers of antibiotic resistance. Environ Pollut. 2024;346:123649. doi:10.1016/j.envpol.2024.123649

4. Hazra M, Watts JEM, Williams JB, Joshi H. An evaluation of conventional and nature-based technologies for controlling antibiotic-resistant bacteria and antibiotic-resistant genes in wastewater treatment plants. Sci Total Environ. 2024;917:170433. doi:10.1016/j.scitotenv.2024.170433

5. Jiang B, Lai Y, Xiao W, et al. Microbial extracellular vesicles contribute to antimicrobial resistance. PLoS Pathog. 2024;20(5):e1012143. doi:10.1371/journal.ppat.1012143

6. Saleem Z, Saeed H, Hassali MA, et al. Pattern of inappropriate antibiotic use among hospitalized patients in Pakistan: a longitudinal surveillance and implications. Antimicrob Resist Infect Control. 2019;8:188. doi:10.1186/s13756-019-0649-5

7. Li C, Cui Z, Wei D, et al. Trends and patterns of antibiotic prescriptions in primary care institutions in Southwest China, 2017-2022. Infect Drug Resist. 2023;16:5833–5854. doi:10.2147/IDR.S425787

8. Zhao H, Wang S, Meng R, et al. Appropriateness of antibiotic prescriptions in Chinese primary health care and the impact of the COVID-19 pandemic: a typically descriptive and longitudinal database study in Yinchuan City. Front Pharmacol. 2022;13:861782. doi:10.3389/fphar.2022.861782

9. Young EH, Panchal RM, Yap AG, Reveles KR. National trends in oral antibiotic prescribing in United States physician offices from 2009 to 2016. Pharmacotherapy. 2020;40(10):1012–1021. doi:10.1002/phar.2456

10. Pouwels KB, Hopkins S, Llewelyn MJ, Walker AS, McNulty CA, Robotham JV. Duration of antibiotic treatment for common infections in English primary care: cross sectional analysis and comparison with guidelines. BMJ. 2019;364:l440. doi:10.1136/bmj.l440

11. Haj Ali K B, Sekma A, Messous S, et al. Appropriateness of antibiotic treatment of acute respiratory tract infections in Tunisian primary care and emergency departments: a multi center cross-sectional study. BMC Prim Care. 2022;23(1):295. doi:10.1186/s12875-022-01904-7

12. Obakiro SB, Napyo A, Wilberforce MJ, et al. Are antibiotic prescription practices in Eastern Uganda concordant with the national standard treatment guidelines? A cross-sectional retrospective study. J Glob Antimicrob Resist. 2022;29:513–519. doi:10.1016/j.jgar.2021.11.006

13. Kuruvilla AV, Madhan R, Chandagal Puttaswamy M. Clinical pharmacist-initiated assessment and amelioration of appropriate antibiotic use in surgical units at a South Indian tertiary care hospital - A handshake approach. J Infect Dev Ctries. 2023;17(1):66–72. doi:10.3855/jidc.17032

14. Chang Y, Chusri S, Sangthong R, et al. Clinical pattern of antibiotic overuse and misuse in primary healthcare hospitals in the southwest of China. PLoS One. 2019;14(6):e0214779. doi:10.1371/journal.pone.0214779

15. Wushouer H, Du K, Chen S, et al. Outpatient antibiotic prescribing patterns and appropriateness for children in primary healthcare settings in Beijing City, China, 2017–2019. Antibiotics. 2021;10(10):1248. doi:10.3390/antibiotics10101248

16. Fu M, Gong Z, Zhu Y, et al. Inappropriate antibiotic prescribing in primary healthcare facilities in China: a nationwide survey, 2017-2019. Clin Microbiol Infect. 2023;29(5):602–609. doi:10.1016/j.cmi.2022.11.015

17. Molero JM, Moragas A, López-Valcárcel B G, Bjerrum L, Cots JM, Llor C. Reducing antibiotic prescribing for lower respiratory tract infections 6 years after a multifaceted intervention. Int J Clin Pract. 2019;73(5):e13312. doi:10.1111/ijcp.13312

18. Little P, Stuart B, Francis N, et al. Antibiotic prescribing for acute respiratory tract infections 12 months after communication and CRP training: a randomized trial. Ann Fam Med. 2019;17(2):125–132. doi:10.1370/afm.2356

19. Wei X, Zhang Z, Hicks JP, et al. Long-term outcomes of an educational intervention to reduce antibiotic prescribing for childhood upper respiratory tract infections in rural China: follow-up of a cluster-randomised controlled trial. PLoS Med. 2019;16(2):e1002733. doi:10.1371/journal.pmed.1002733

20. Yang J, Cui Z, Liao X, et al. Effects of a feedback intervention on antibiotic prescription control in primary care institutions based on a Health Information System: a cluster randomized cross-over controlled trial. J Glob Antimicrob Resist. 2023;33:51–60. doi:10.1016/j.jgar.2023.02.006

21. Ivers N, Jamtvedt G, Flottorp S, et al. Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2012;2012(6):CD000259. doi:10.1002/14651858.CD000259.pub3

22. Zeng Y, Shi L, Liu C, et al. Effects of social norm feedback on antibiotic prescribing and its characteristics in behaviour change techniques: a mixed-methods systematic review. Lancet Infect Dis. 2023;23(5):e175–e184. doi:10.1016/S1473-3099(22)00720-4

23. Hemkens LG, Saccilotto R, Reyes SL, et al. Personalized prescription feedback using routinely collected data to reduce antibiotic use in primary care: a randomized clinical trial. JAMA Intern Med. 2017;177(2):176–183. doi:10.1001/jamainternmed.2016.8040

24. Wattal C, Goel N, Khanna S, Byotra SP, Laxminarayan R, Easton A. Impact of informational feedback to clinicians on antibiotic-prescribing rates in a tertiary care hospital in Delhi. Indian J Med Microbiol. 2015;33(2):255–259. doi:10.4103/0255-0857.153582

25. Ratajczak M, Gold N, Hailstone S, Chadborn T. The effectiveness of repeating a social norm feedback intervention to high prescribers of antibiotics in general practice: a national regression discontinuity design. J Antimicrob Chemother. 2019;74(12):3603–3610. doi:10.1093/jac/dkz392

26. Soucy J-PR, Low M, Acharya KR, et al. Evaluation of an automated feedback intervention to improve antibiotic prescribing among primary care physicians (OPEN Stewardship): a multinational controlled interrupted time-series study. Microbiol Spectr. 2024;12(4):e0001724. doi:10.1128/spectrum.00017-24

27. Schwenk HT, Kruger JF, Sacks LD, Wood MS, Qureshi L, Bio LL. Use of prospective audit and feedback to reduce antibiotic exposure in a pediatric cardiac ICU. Pediatr Crit Care Med. 2021;22(3):e224–e232. doi:10.1097/PCC.0000000000002608

28. Chang Y, Yao Y, Cui Z, et al. Changing antibiotic prescribing practices in outpatient primary care settings in China: study protocol for a health information system-based cluster-randomised crossover controlled trial. PLoS One. 2022;17(1):e0259065. doi:10.1371/journal.pone.0259065

29. Wang Q, Chang Y, Cui Z, Yu S, Wang L, Fan X.Preparation of recommendation manual for diagnosis and evaluation of bacterial infectious diseases and rational use of antibiotics (respiratory system part) in primary institutions. Herald Med. 2022;41(5):733–742.

30. Chongsuviwatwong V, McNeil EB. Analysis of longitudinal data using R and epicalc / VirasakdI Chingsuvivatwong. Edward McNeil. 2018.

31. Sangwan R, Neels AJ, Gwini SM, Saha SK, Athan E. Is education alone enough to sustain improvements of antimicrobial stewardship in general practice in Australia? Results of an intervention follow-up study. Antibiotics. 2023;12(3):594. doi:10.3390/antibiotics12030594

32. NTT D, Nadjm B, Nguyen KV, van Doorn HR, Lewycka S. Reducing antibiotic overuse in rural China. Lancet Glob Health. 2018;6(4):e376. doi:10.1016/S2214-109X(18)30071-8

33. Regev-Yochay G, Raz M, Dagan R, et al. Reduction in antibiotic use following a cluster randomized controlled multifaceted intervention: the Israeli judicious antibiotic prescription study. Clin Infect Dis. 2011;53(1):33–41. doi:10.1093/cid/cir272

34. Cals JW, de Bock L, Beckers PJ, et al. Enhanced communication skills and C-reactive protein point-of-care testing for respiratory tract infection: 3.5-year follow-up of a cluster randomized trial. Ann Fam Med. 2013;11(2):157–164. doi:10.1370/afm.1477

35. Ferrat E, Le Breton J, Guéry E, et al. Effects 4.5 years after an interactive GP educational seminar on antibiotic therapy for respiratory tract infections: a randomized controlled trial. Fam Pract. 2016;33(2):192–199. doi:10.1093/fampra/cmv107

36. Tao M, Yang D, Liu W. Learning effect and its prediction for cognitive tests used in studies on indoor environmental quality. Energy Build 2019;197:87–98. doi:10.1016/j.enbuild.2019.05.044

37. Moriyama M, Hugentobler WJ, Iwasaki A. Seasonality of respiratory viral infections. Annu Rev Virol. 2020;7(1):83–101. doi:10.1146/annurev-virology-012420-022445

38. Sheeran P, Maki A, Montanaro E, et al. The impact of changing attitudes, norms, and self-efficacy on health-related intentions and behavior: a meta-analysis. Health Psychol. 2016;35(11):1178–1188. doi:10.1037/hea0000387

39. Sales JM, Anderson KM, Kokubun CW. Application of the consolidated framework for implement