Abstract

PURPOSE Meeting scholarly activity requirements continues to be a challenge in many family medicine (FM) residency programs. Studies comprehensively describing FM resident scholarship have been limited. We sought to identify institutional factors associated with increased scholarly output and meeting requirements of the Accreditation Council for Graduate Medical Education (ACGME).

OBJECTIVES Our goals were to: (1) describe scholarly activity experiences among FM residents compared with ACGME requirements; (2) classify experiences by Boyer’s domains of scholarship; and (3) associate experiences with residency program characteristics and scholarly activity infrastructure.

METHODS This was a cross-sectional survey. The survey questions were part of an omnibus survey to FM residency program directors conducted by the Council of Academic Family Medicine Educational Research Alliance (CERA). All ACGME-accredited US FM residency program directors, identified by the Association of Family Medicine Residency Directors, were sampled.

RESULTS Of the 691 eligible program directors, 298 (43%) completed the survey. The respondents reported that 25% or more residents exceeded ACGME minimum output, 17% reported that 25% or more residents published their work, and 50% reported that 25% or more residents delivered conference presentations. Programs exceeding ACGME scholarship requirements exhibit robust infrastructure characterized by access to faculty mentorship, scholarly activity curricula, Institutional Review Board, medical librarian, and statistician.

CONCLUSIONS These findings suggest the need for codified ACGME requirements for scholarly activity infrastructure to ensure access to resources in FM residency programs. By fostering FM resident engagement in scholarly activity, programs help to create a culture of inquiry, and address discrepancies in funding and output among FM residency programs.

INTRODUCTION

The production of scholarship is not a strength of family medicine (FM) as a specialty. Family medicine produces fewer publications and receives a disappointingly smaller amount of National Institute of Health funds than other medical specialties.1,2 Scholarship, though, is valued by family physicians. When surveyed, FM department chairs endorse interest in scholarship, but they also report barriers that thwart this interest.3 Exposure to scholarly activity and skill development during residency may be part of the solution. The current state of scholarly activity in FM residency programs is incompletely understood. Smaller studies have explored scholarly activity production along with explorations of cultural, structural, and educational changes to increase scholarship output, but the scope of FM resident scholarly activity has not been comprehensively described for more than a decade.4 Accurate description of current scholarly activity among FM programs requires the programs’ specific outputs to be quantified and detailed as the Accreditation Council for Graduate Medical Education’s (ACGME) scholarly activity requirements provide flexibility.5 The current ACGME program (last revised in 2019) requires FM residents to perform 2 scholarly projects; one must be a quality improvement project and second is at the program’s discretion. Describing the state of FM residency scholarly activity will provide a framework for continued advocacy by key stakeholders and national societies when creating strategic plans and accreditation policies to build a culture of inquiry within our specialty.

Scholarly activity as defined by Boyer includes 4 domains: discovery, integration, application, and teaching (Table 1).6 Inclusive in this characterization is the flexibility in the ACGME requirements that gives FM residency programs the final determination and approval of scholarly products. Much of the prior literature on FM resident scholarship has focused on a singular lens of research, which falls under Boyer’s domain of discovery. The other 3 domains (integration, application, and teaching) have yet to be systematically surveyed. In a 1999 survey of FM program directors, Neale showed that 48.6% of residency programs required resident research/scholarly projects.7 More than a decade later, Crawford and Seehusen found that only 46% of FM residency programs had over 50% of their residents engaged in research while 13% of programs reported that 25% or more of their residents had published a manuscript, and 30% of programs reported that 25% or more of their residents had given a poster or oral presentation.4

Table 1.

Purpose and Examples of the Domains in Boyer’s Paradigm of Scholarship

Our study sought to characterize the current state of scholarship in FM residency programs in the United States according to Boyer’s 4 domains. We sought to identify institutional factors associated with increased scholarly output and achievement of ACGME scholarly activity requirements. Understanding the current state of scholarship is necessary to determine an ideal future state of research within our specialty.

METHODSStudy Design

The questions were included in a Council of Academic Family Medicine Educational Research Alliance (CERA) national survey of FM residency program directors in 2023. The methodology of the CERA survey has previously been described.8 Survey questions were developed using the 7-step approach to survey design.9 The literature was reviewed for validated reporting of scholarship among FM residents. A validated survey was not identified so we consulted former FM residency program directors for their expert opinion on modifying previous survey items and constructing new questions. We developed survey items with the former program directors’ validation through the use of cognitive interviews and pilot testing.9 The CERA Steering Committee evaluated final drafts of questions for consistency with the subproject aim, readability, and existing evidence of reliability and validity. The questions were pretested on FM educators who were not part of the sample population. Final questions are publicly available on the CERA website and in Supplemental Appendix. Data was collected from April 18 through May 12, 2023. CERA surveys were approved by the American Academy of Family Physician Institutional Review Board (IRB) before dissemination in April 2023.

Data Collection

Purposeful sampling was employed as surveys were distributed to all ACGME-accredited US FM program directors as identified by the Association of Family Medicine Residency Directors. This group was chosen as they are knowledgeable regarding resident scholarly activity production as it relates to ACGME requirements. E-mail invitations to participate were delivered using SurveyMonkey (Symphony Technology Group). Two follow-up e-mails to encourage nonrespondents to participate were sent weekly after the initial e-mail invitation, and a third reminder was sent 2 days before the survey closed. There were 745 program directors at the time of the survey; 6 had previously opted out of surveys or their e-mails were undeliverable (Supplemental Figure). The survey was e-mailed to 739 individuals and contained a qualifying question to remove programs that had not yet graduated 3 resident classes. Forty-eight program directors indicated that they did not meet this criterion, so these responses were omitted, reducing the sample size to 691 and respondents to 309. Program directors completing the survey were not required to respond to all items, but 298 respondents answered some of the topic-specific questions. This response rate is consistent among CERA surveys of FM program directors and is similar to the most recently published membership data from the Association of Family Medicine Residency Directors.10

Variables and Definitions

Family medicine residency program characteristics (residency program type, program location, community size, total number of residents), infrastructure (faculty mentorship, scholarly activity curriculum, IRB, research librarian, statistician), and scholarly activity (presentation, publication, project type) were analyzed for associations. To remain consistent with previously published manuscripts describing similar concepts, we used 25% of residents as the threshold to dichotomize the data. Boyer’s expanded definition of scholarly activity was used to characterize resident production.6 We chose Boyer’s as each domain possesses a specific skillset that can be taught to residents and each domain includes a spectrum of expertise needed for different types of scholarly activity projects. Scholarship of discovery includes case reports, observational studies, and clinical trials; scholarship of integration could be letters to the editor, book chapters, working on the Family Physicians Inquiry Network (FPIN) HelpDesk, literature reviews, or meta-analyses; scholarship of application was defined as population health and quality improvement projects; and scholarship of teaching as curriculum projects (Table 1).

Statistical Analysis

To describe the quantity, type, and dissemination of scholarly activity projects produced by FM residents, descriptive statistics were used to characterize the data using basic statistical measures (percentage, 1-way frequency, and 2-way frequency). Data was analyzed using R statistics software version 4.2.2 (R Project for Statistical Computing). Univariate statistics were calculated for all variables, and bivariate statistics were computed to examine the associations between scholarly activity production, dissemination, achievement of ACGME requirements, infrastructure, and residency program characteristics. Chi-square tests of independence were used to evaluate bivariate relationships, employing a level of statistical significance set at α = 0.05. Post-hoc analysis was performed using the Holm method, and adjusted P values are reported throughout the manuscript.

RESULTSResidency Program Characteristics

The overall response rate for the omnibus survey was 44.7% (309/691). Table 2 shows the descriptive characteristics from the 298 FM residency program directors that completed some of our survey’s topic-specific questions. Most programs were community-based (81.2%), had 31 or fewer total residents (85.2%), and were in communities with less than 500,000 people (70.8%). Most FM residency programs had sufficient faculty mentorship (78.8%), scholarly activity curriculum (75.5%), access to an IRB (92.1%), a medical librarian (79.8%), and a statistician (66.5%). Additionally, most programs had at least 4 of these resources (69.0%).

Table 2.

Characteristics of US Family Medicine Residency Program Directors and Their Program (N = 298)

Less than one-half (43.1%) of FM residency programs had more than 25% of their total cohort of residents exceeding the minimum ACGME requirements for scholarly activity. The most common scholarly activity products were letters to the editor (33.2%), followed by case reports (19.7%), curriculum projects (13.1%), and population health projects (11.7%). Scholarship of integration was the most common domain of scholarly activity (48.9%), followed by scholarship of discovery (23.0%), application (15.0%), and teaching (13.1%). No characteristics (eg, as residency program type, infrastructure availability) were associated with residents exceeding the minimum ACGME requirements for scholarly activity.

A minority of FM residency programs (16.8%) reported 25% or more residents authored at least 1 peer-reviewed manuscript. Only one-half (49.5%) of FM programs report 25% or more residents presented poster or oral presentations at regional, national, or international conferences. When comparing the frequency of FM residents’ most common form of dissemination to either present or publish scholarly activity, FM residents were more likely to present (94.6%) than publish (5.4%). When describing the types of presentations, presentation within institutions (56.3%) and poster presentations (35.4%) were more common than oral presentations (2.9%).

Resident Scholarly Activity Production and Dissemination

Table 3 presents program characteristics associated with resident scholarly activity production and dissemination. Programs with 25% or more of their residents with at least 1 publication were more likely to be enrolled in the FPIN program (P <.001). No characteristics (eg, as residency program type, infrastructure availability) were found to be associated with programs having 25% or more of residents publishing or presenting their scholarly activity.

Table 3.

Program Characteristics Associated With Resident Scholarly Activity Production and Dissemination

Family medicine residency programs with fewer than 25% of their total cohort of residents exceeding the minimum ACGME requirements for scholarly activity were more likely to respond that they are struggling to meet ACGME requirements for scholarly activity (P <.001). Conversely, programs with 25% or more of their total cohort of residents exceeding the minimum ACGME requirements for scholarly activity were more likely to respond that they had higher amounts of dissemination of scholarly activity, including authorship on at least 1 peer-reviewed paper (P = .004) and presenting a poster or oral presentation (P <.001).

Current State and Future Direction of Resident Scholarly Activity

Table 4 presents information about residency programs based on their ability to meet ACGME requirements and future plans for increasing resident output. A minority of surveyed FM residency programs (9.4%) struggle to meet ACGME scholarly activity requirements, while 16.0% of surveyed programs are exceeding the requirements. Programs exceeding the requirements were more likely to be university based (P = .001). Programs that exceeded requirements were more likely to produce FPIN HelpDesk publications (P = .020). When grouped by scholarship domain, programs that exceed requirements were more likely to engage in the scholarship of integration (P = .02).

Table 4.

Program Characteristics Based on Ability to Meet ACGME Requirements and Future Improvement

Despite the lack of association between resident scholarly activity output and dissemination, there was association between programs exceeding ACGME requirements for scholarly activity and resident access to individual scholarly activity resources. These associations included faculty mentorship (P = .002), scholarly activity curriculum (P = .045), IRB (P = .007), medical librarian (P = .053), and statistician (P = .011). Collectively, programs exceeding ACGME requirements for scholarly activity were associated with having more scholarly activity resources (P <.001).

Almost one-half (45.6%) of programs that were meeting or exceeding ACGME scholarly activity requirements reported they are seeking to increase their residents’ scholarly activity output. There was no association between programs looking to increase their scholarly activity output and program characteristics such as type of program, size of program, or size of the community served.

DISCUSSION

We sought to describe the landscape of FM scholarly activity and identify institutional factors associated with increased scholarly output and achievement of ACGME requirements. We found that most programs had a minority of FM residents exceeding the minimum ACGME requirements for scholarly activity. Family medicine residents tend to disseminate scholarly activity through presentations rather than publications, but those that do publish are more likely to be enrolled in FPIN and involved in scholarship of integration more than other domains. Most programs reported meeting ACGME requirements, with a few exceeding them and a few reporting difficulty meeting them. Programs struggling to meet ACGME requirements are less likely to have adequate access to scholarly activity infrastructure such as faculty mentorship, scholarly activity curriculum, IRB, medical librarian, and statistician.

Programs that struggle to meet requirements lack sufficient infrastructure to support scholarly activity. Many programs are seeking to increase resident scholarly activity, and several characteristics of programs that exceed requirements were consistent with previous studies, including mentorship, scholarly activity curricula, and technical support.4,11,12 Participating in FPIN HelpDesk publications appears to be an effective strategy for programs to exceed requirements and increase resident scholarly activity output. This outcome may be a result of the FPIN program’s robust training and advising. Interestingly, scholarship of integration (defined as letters to the editor, book chapters, FPIN HelpDesk, literature reviews, and meta-analyses), not the traditional scholarship of discovery (defined as case reports, observational studies, and clinical trials), is most common for programs with strong scholarly activity output.

Programs have not significantly increased the quantity or the rigor of scholarly activity since 2011,4 despite calls for increased primary care research, evidence that scholarly activity promotes critical thinking skills, and evidence that scholarly activity may improve both patient care and future practice styles for residents.13-15 Literature describing FM residents’ perceptions have shown both a lack of interest in scholarly activity and uncertainty regarding how to engage in scholarly activity.16,17 Many programs have described curricula and systems that effectively increased scholarly activity in their programs in the past several years, indicating there may be both program- and specialty-level influences on the culture of inquiry.13,18,19 Not studied in this survey are the social drivers that increase the productivity of FM resident scholarly activity including scholarly activity point systems,20 Clinical Translational Science Awards,21 and Practice-Based Research Networks.22 The heterogenous experience with scholarly activity that FM residents experience parallels the diverse program-specific experience residents have in other disciplines. To advance FM as a specialty, all programs should foster a culture of inquiry and support scholarly activity.

As programs increasingly embrace population health due to ACGME requirements, projects within the domain of scholarship of application (population health and quality improvement projects) may become more prevalent and robust. Quality improvement and population health projects help advance system-based practice milestones. In this study, surveyed programs that exceeded requirements were engaged in scholarship of integration, such as letters to the editor and FPIN HelpDesk publications, rather than scholarship of discovery, such as case reports. Programs’ support of scholarship of integration may also serve to advance higher-level medical knowledge, system-based practice milestones, and problem-based learning and improvement, resulting in more meaningful and sustainable experiences.

While this study showed that few programs struggle to meet ACGME requirements, scholarly activity remains a common citation by the Review Committee for programs across specialties.2 The intuitive connection between scholarly activity infrastructure and scholarly activity production is a consistent trend in this and previous studies.7,8 The ACGME could meaningfully impact resident scholarly activity production by requiring a set number of scholarly activity products per resident and requiring programs to provide the infrastructural elements that fosters scholarly activity. These include adequate technical support; access to an IRB, statistician, and medical librarian; a formal scholarly activity curriculum; protected project time for faculty and residents; ongoing scholarly activity mentorship; and venues in which to share scholarly activity. These requirements not only increase the amount of scholarly activity but could also promote resident interest and confidence in completing it.

Limitations of this study include a relatively low response rate of 44%. This rate, however, is in line with previous CERA surveys of FM residency program directors and is similar to the membership data published by Association of Family Medicine Residency Directors. Respondents may have been more inclined to complete this survey if their program is strong in scholarly activity. Of the 309 respondents of the omnibus survey, 298 (96%) respondents completed some aspect of our survey’s topic-specific questions. This survey did not examine previously identified barriers to scholarly activity including protected time and funding. This survey also did not examine the impact of participation in scholarly activity collaboratives or practice-based research networks. Both of these facets have strong evidence supporting their impact on research production within FM and so they were excluded from the our survey’s questions.

Our survey study was able to assess program director perspectives on scholarly activity within their programs, but future studies are needed to assess resident knowledge, skills, and professional development in scholarship. Further studies should also elaborate on what types of mentorship and evidence-based scholarly activity curricula best support scholarly activity. Family medicine physicians, with the skill set to critically assess and respond to challenges, are well suited to care for patients in a variety of practice settings and an ever-changing health care environment. Ensuring that all residency programs have the necessary resources to support scholarly activity is essential to graduating the future family physicians our communities and nation needs.