We used qualitative methods in this embedded process evaluation to understand how and why the intervention worked (or did not work) as intended. We combined participants from both trials since both targeted antibiotic prescribing amongst family physicians and were delivered simultaneously and in contexts. This study received research ethics approval from the Women’s College Hospital Research Ethics Board. The reporting of this qualitative process evaluation adheres to the COnsolidated criteria for REporting Qualitative research (COREQ) reporting standards (Appendix 1).

The Clinical Performance Feedback Intervention Theory (CP-FIT) offers the most comprehensive theory on the conditions for optimal A&F [11]. It is a product of a qualitative synthesis of 65 studies that culminated in a healthcare-specific theory of A&F. The theory posits that the effects of A&F can be summarized by three propositions: (1) health care professionals and organisations have a limited capacity to engage with feedback, (2) these parties have strong beliefs about how patient care should be provided that influence their interactions with feedback, and (3) feedback that directly supports clinical behaviours is most effective. CP-FIT guided our thinking regarding the mechanism of action of A&F in clinical practice and factors that influence its effects. It was used to inform the development of the interview guide and the analysis [12].

CP-FIT states that effective feedback works in a cycle of sequential processes. We explored this process of feedback interaction, then recipient perception and acceptance of the feedback, followed by intention, and then behaviour change for clinical performance improvement. The theory stipulates that progress through the cycle will be weakened or halted entirely if any individual stage fails. CP-FIT highlights three types of variables that operate through common explanatory mechanisms to influence whether and how health professionals respond to A&F: the feedback intervention itself, characteristics of the feedback recipient, and contextual factors affecting the clinical environment.

Ontario has a population of over 15 million people where the majority of primary care is delivered by family physicians. A universal government-funded insurance plan without deductible or co-pay covers visits to family physicians. Medications, including antibiotic prescriptions, are covered for those on social assistance, those under 25 with no private (employer-funded) insurance, and all those above age 65.

The trials (Table 1) were conducted with Ontario Health and Public Health Ontario. Ontario Health— an agency created by the Government of Ontario with a mandate to connect and coordinate the province’s health care system to help ensure that Ontarians receive the best possible care—provides A&F to physicians who voluntarily sign up for their “MyPractice: Primary Care” reports. Approximately 4750 (of 9,500 eligible) Ontario family physicians signed up to receive these reports during this study. These are multi-topic reports with aggregated (physician-level) data, sent twice yearly via email using data collated from the Institute for Clinical Evaluative Sciences (ICES, a custodian to a data repository with patient and physician-level, coded and linkable health data sets in Ontario, Canada). ICES data includes publicly funded administrative health services records for the Ontario population eligible for universal health coverage (≈ 98.5%). However, dispensing data are complete only for patients 65 and older.

Public Health Ontario (PHO)—an agency of the provincial government responsible for providing scientific and technical advice on matters of public health concern— sent A&F specifically about antibiotic prescribing to family physicians who did not sign up for the MyPractice report from Ontario Health. The PHO A&F reports also used data held at ICES to link prescriber characteristics, including patient volume, and patient characteristics, including comorbidities, to antibiotic prescription data [10].

All A&F recipients were given a process evaluation survey that included an invitation to participate in an interview. Participants were asked to write their contact information so the study team could follow up. From the physicians who indicated interest in participating in an interview, participants were purposely sampled from defined strata to allow maximum variation across age, gender, experience (i.e., years worked as a family physician), and clinical context subgroups (i.e., walk-in physician, family health team, emergency). We also purposively sampled from each of the following groups: (i) PHO Trial, adjusted comparator, (ii) PHO Trial, unadjusted comparator, (iii) PHO Trial, harms emphasis, (iv) PHO Trial, no harms emphasis, and (v) OH Trial, mailed viral prescription pad/emphasis.’

An information letter introducing the research team and outlining the purpose of the study and an informed consent form were provided via email. All physicians who completed the interview were provided an honorarium of $100 as an electronic gift card, recognizing the time required to complete the interview. Participants were screened to confirm they had read the A&F letter before the interview over email or at the beginning of the interview if not previously answered. Recruitment ceased when data saturation was achieved, defined as the point in data collection and analysis when new incoming data produced little or no new information to address the research questions.

Brief demographic questions were asked at the beginning of the interview, including the type of A&F received, gender, years in practice, type of practice (Interprofessional practice, Community Practice, Walk-in clinic, Other), location of practice (urban, rural), and the average number of patients seen per day. Interviews were conducted between 1 February 2022 and 5 April 2022 by two non-clinician researchers (ML and JS) trained in qualitative methods. The interview guide explored CP-FIT theory constructs (Appendix 2) and the different aspects of the A&F letter (e.g., comparators, duration data, and harms information). All interviews were conducted on Zoom (Zoom Video Communications, San Jose, CA), recorded and transcribed verbatim and entered into NVivo (QSR International), a qualitative analysis software program. Only the researchers conducting the interview were present and the interviews were scheduled for 60 min.

We used reflexive thematic analysis that involved a constant comparative method, with our research questions guiding our analysis of transcripts [13]. In this process, we applied inductive open coding, involving a preliminary reading of full transcripts and generating initial descriptive codes- paraphrasing the text using participants’ own words. Transcripts were coded by four team members (JS, ML, MS, CR). All four team members coded the first four transcripts independently using open codes. The coders met to iteratively develop a mutually agreed upon analytical framework, which was applied to all transcripts using focused coding in NVivo. Our analysis considered physicians from both trials together.

We were interested in both high and low prescribers for data looking at how recipients responded to intervention factors (i.e., viral prescription pad, adjusted comparator). However, when gathering insights to inform future interventions, the response of A&F recipients without substantial room for improvement isless important from a public health perspective. Therefore, majority of our analysis focused on physicians who were described as “high prescribers” in their A&F report. “High prescribers” prescribed more than the target expressed in their A&F report (above the 25% target in the PHO trial and above 50% in the OH trial.

The team mapped codes onto the CP-FIT theory constructs, and broader themes were created by grouping codes based on the CP-FIT constructs. Subthemes and their relationships were reviewed, mapped, and discussed with the larger team. Finally, we examined the data in the context of the CP-FIT explanatory mechanisms that influence the different stages of the CP-FIT cycle. Specifically, we considered factors that affected progress through the CP-FIT stages for two different behavioural targets of the A&F reports: antibiotic prescription initiation and antibiotic prescription duration. Comparing factors influencing these behaviours could illuminate why some metrics seem more amenable to improvement via A&F than others. We also compared data, codes, and themes across important characteristics (e.g., type of clinic, rural vs. urban, age of physician).