Since the concept of entrustable professional activities (EPAs) was proposed 15 years ago,1 EPAs have been designed for many educational programs in health care. In a relatively short period, mainly after 2015, EPAs have been advocated for virtually all medical specialties, undergraduate medical education, and various other health professional training programs such as nursing, physical therapy, dentistry, oral hygiene, pharmacy, and veterinary medicine, including that of physician assistants (PAs). An EPA is a key task of a discipline (ie, specialty or subspecialty) that can be entrusted to an individual (a learner or a professional) to perform, in a given health care context, once sufficient competence has been demonstrated.1 Early reviews on the concept2,3 demonstrate that the literature is broad, recent, predominantly focused on the development of EPAs, and to a lesser extent on their implementation, and virtually lacks studies of follow-up, efficacy, and the use of EPAs after graduation. This gap in the literature is not surprising because hardly any program has graduated learners trained with the EPA concept. Considering EPAs as comprehensive “units of professional practice”4 that determine what learners should be trained for, they should arguably have significance after training. Graduates should recognize, confirm, and support the validity of these units of practice and feel they have been well-trained to perform these. So far, only a few studies have investigated whether EPAs of undergraduate medical education were practiced after graduation.5–8 The use of EPAs across the continuum of training and practice deserves attention in future studies.9,10 For graduates of EPA-based programs, many EPAs remain relevant during their practice; however, other EPAs may cease to be so, and there may be new activities that require additional training.5 EPAs play a significant role during training; however, little is known about the potential role of EPAs after graduation, such as in lifelong learning, when new tasks require new competencies, or when professionals transition to a new workplace setting. The EPA concept can potentially play a valuable role in PA licensing. The current licensing process is based on a broad, less well-defined scope of practice with limited specificity. The alignment with the ever-increasing specialization of health care professionals is unclear, and PAs' qualifications at graduation may not always reflect the evolution of individuals over time. A flexible and ongoing EPA-based recertification process may provide a better reassurance about each health professional's current knowledge and skills, thereby serving the needs of patients and credentialing organizations, as has been recommended for medicine.9 There is hardly any literature that addresses the use and significance of EPAs after training, let alone their use for personal development after training. Given their theoretical potential, we aimed to investigate the target group's opinion, that is, PAs after graduation, about their use of EPAs.

Dutch Master of Physician Assistant Program

The first Dutch MPA programs commenced around the years 2001 to 2002. As of the academic year 2005 to 2006, five MPA programs are operational and annually enroll 250 students on average. MPA students have a contract at a clinical workplace for the duration of their training; typically, they keep working at their institution and within their specialization after graduation. The programs are state-funded and have a length of 30 months. Characteristic of the Dutch MPA programs is that didactic and clinical learning is interwoven throughout the whole curriculum. All programs have theoretical in-school education, individualized workplace education, and clinical rotations. Students acquire both generalists and also specialist medical competencies, respectively, on-campus and during clinical practice.

Concerning the clinical practice learning trajectories design, two of the five MPA programs currently work with the EPA framework. To design a well-structured program for the workplace curriculum, the EPA format was introduced in the MPA program of the University of Applied Sciences (UAS) in Utrecht, the Netherlands, in 201011,12 and five years later at the MPA program of the Hanze UAS, the Netherlands. MPA students all have an individualized workplace curriculum with five to eight EPAs for which they are trained, determined jointly by a clinical workplace supervisor, the student, and the UAS. EPA-based assessment results in a summative entrustment decision by the clinical workplace supervisors to act under a specified level of supervision. Graduated PAs have permission to perform medical tasks unsupervised for which they have been qualified. Each entrusted task is described in an EPA. The sets of EPAs for which MPA students are trained may differ vastly between students and settings. For each student, the EPAs are well-described and must be approved by the school. Combined monitoring of progress happens by a local supervisor and the school, and adaptations in the initially intended set of EPAs for an individual student during the program are not uncommon. On average, at graduation, one in eight of the intended EPAs is being replaced, removed, or added, which reflects the flexibility of the MPA program and the EPA concept.12 The very first graduates of EPA-based MPA training programs were employed in 2012 (about 8 years before our data collection). Having graduates for several years, these two MPA programs in the Netherlands now allow reviewing the significance of EPAs after graduation.

After graduation, PAs may take on new professional tasks and add new EPAs, which are locally trained and acquired, assessed, and entrusted on the job. PAs may also switch workplaces in the same area of interest and training (ie, within a medical specialty) or even change to a different medical specialty. MPA programs offer a broad array of medical topics and experiences in their curriculum, enabling the possibility of changing workplaces after graduation. Such workplace mobility is not unique to the Netherlands. Changing specialty after graduation occurs among up to half of the PAs licensed in the United States.13,14

However, EPAs one is entrusted with initially may not match a new workplace. Changing workplaces would also require the possibility to add EPAs to one's scope of practice. There is currently no formal requirement for PAs in the Netherlands to document newly acquired competence and related entrustment for tasks. In addition, EPAs may have a potential role in specialty recertification as a tool to recognize and acknowledge maintenance of competence.9,15 This follows the idea that EPAs may have an expiry date if not practiced for a prolonged time.4

This exploratory and descriptive survey study aims to evaluate the use of EPAs after graduation among alumni of MPA programs in the Netherlands. We formulated the following three research questions:

How and for what purpose do PAs continue to use EPAs to guide their continuing professional development and skill acquisition after graduation? Do PAs envision and value a continued role for EPAs after graduation? Do PAs support the concept of formalized registration of competencies and/or skills attained after graduation? METHODS

From September 2020 to October 2020, we surveyed all graduated physician assistants (n = 1441) in the Netherlands with a voluntary, anonymous, 5 to 10-minute survey using QualtricsXM (Qualtrics, Provo, UT), a web-based application. Invitations to participate in the survey were distributed with the help of all five Dutch MPA programs and the Dutch Association of Physician Assistants (NAPA). We deliberately included alumni from the three programs not using EPAs to solicit their views on professional task acquisition after graduation. We collected data over a 6-week period, during which two email reminders were sent at weeks one and five. In addition, we used social media platforms LinkedIn and Twitter, managed by the NAPA and the authors, to draw attention and invite PAs to participate in this study.

We developed a survey based on a preliminary pilot study conducted in 2019 by F.W. and O.t.C. among UAS Utrecht alumni (not published). The 2019 survey items were scrutinized and discussed within the research team. Adaptations were made if needed on agreement within the research team. After multiple rounds of adaptations, a content validation process was conducted using cognitive interviews with graduated PAs to judge the understanding of the survey items.16 Based on these results, we further refined the survey. The final version was subsequently distributed to our study population.

The survey consisted of two parts, one created for respondent PAs trained with EPAs and one for PAs trained without EPAs (see survey in Appendix 1, Supplemental Digital Content, https://links.lww.com/JCEHP/A186). Respondents trained with EPAs were asked about the use and role of EPAs in lifelong learning, any transition of workplace contexts, and the use of EPAs when new tasks require new competencies. Respondents trained without EPAs were asked about their experience with lifelong learning, transitions of workplace contexts, and new tasks that require new competencies without reference to EPAs. The survey provided space for free-text responses to provide any relevant thoughts or explanations. From these responses, themes were identified using an inductive approach; that is, themes were obtained from the data according to participants' views, without using a pre-existing theory or framework.17 Themes were identified by TR and OtC using an iterative process. The data were coded individually, and provisional themes were first identified individually. Next, all themes were compared and discussed. In all cases of discrepancy, a consensus was reached.

The study protocol was reviewed and approved by the Netherlands Association for Medical Education's Ethical Review Board.

RESULTS

The questionnaire was made available to all certified PAs in the Netherlands (n = 1441) and completed by 421 PAs (response rate 29%). The mean age of the respondents was 43 years (range 22–75 years), and most of them were women (n = 308, 73%); 123 (29%) of the respondents were practicing a surgical specialty, and the other 298 respondents (71%) were practicing another medical specialty. Of the 421 respondents, 120 PAs (29%) were trained with EPAs (UAS Utrecht and Hanze). About half (55%, n = 66) of the PAs trained with EPAs reported having attained new skills after graduation; 24% (n = 102) of all PAs changed their professional workplace setting after graduation.

Documentation of Competencies Attained After Graduation: Qualitative Results

Of the respondents, 69% (n = 292) answered the open question of whether competence (to perform EPAs) attained after graduation should be formally documented and recognized. Almost all of them (90%; n = 262) affirmed this in one way or another. Thematic analysis revealed 19 themes and eight larger, overarching themes (Table 1). Among these eight overarching themes, five themes were classified under “Yes”: for clarity, for legal reasons, to document competence, for quality of care, and recognition of the profession. The other three themes classified under “No” were EPAs may hamper flexibility, fixed EPAs deprive of own responsibility, and financial and administrative burden. We present the two themes that were the most frequently identified as an affirmative response in the survey, and one theme that was most frequently identified as a negative response.

TABLE 1. - Themes That Emerged From the Survey to Justify or Discourage the Use of EPAs After Graduation Overarching Themes Themes Indicative Participants' Responses Yes  For clarity (coded 83 times) Colleagues This provides clarity in the workplace and for colleagues about roles and capacities of the PA within a speciality. Employer/departments It makes it clear to everyone within the organization what a PA is, can, and is not permitted to do. Patients Competencies must be clearly described for the [team and] patients. Diversity among PAs There are major differences in the skills and responsibilities among different Pas.  For legal reasons (coded 63 times) Legal requirements This justifies what you are authorized to do, and in case of problems/legal issues you have something rely on. Liability and insurance Ultimately in the situation of a complaint, this can provide clarity. Regulation privileges For accreditation and registration of competence.  To document competence (coded 53 times) Shifting work settings It is important when moving to a different work environment to be able to demonstrate what competencies you possess. Document personal competence To indicate what you are competent in and authorized to practice independently in case of changes in leadership. Acquiring new competency's It is important that additional or newly acquired tasks and skills are carefully documented and recognized by a medical specialist.  For quality of care (coded 22 times) Audit For external site reviews by specialty associations. Quality control This is a guarantee of quality.  For recognition of the profession (coded 12 times) Recognition It is important for recognition of the profession (how do we distinguish ourselves from non-PA colleagues) and to protect/strengthen the position of Pas. Visibility Recognition means that you have a right to exist—we are a relatively new profession. No  EPAs may hamper (coded 12 times) Local agreements Good working arrangements have already been made within my organization. Too unwieldy It is too much work. Verbal agreement of competence should suffice. Hampering EPAs are hampering [flexibility].  Deprives of own responsibility (coded 10 times) Autonomy It is your own responsibility. If you act incompetently, you should bear liability. For me it is important that everyone sets their own practice boundaries.  Financial and administrative burden (coded 6 times) Administrative Provides unnecessary administrative burdens to have to document all of this. Financial Not if it takes additional cost.

EPA, entrustable professional activity; PA, physician assistant.

Affirmative Responses

Formally recognizing and acknowledging competencies is valued by PAs because they want to avoid ambiguity with supervisors in clinical practice. PAs seem to appreciate a clear description of their competencies that also indicate the boundaries of their profession.

Yes, it is important that it is clear for each colleague which competencies and skills I have and [which activities I am] authorized to practice (Respondent 181)

Our profession has been set up based on substituting medical specialists in their tasks. Competences and proficiency indicate what the boundaries of our profession are, which often extend beyond other professional groups working within the task reorganization (Respondent 318)

Many respondents reported that a formalized documentation of which activities are entrusted could help with legal issues that might occur.

Yes, then you can also show, when incidents or legal issues arise, that my practice is based on a certain specified competence (Respondent 185)

Negative Responses

Some respondents reported that EPAs could hamper flexibility and autonomy in task execution, and they may lead to an administrative and financial burden because of the need for documentation. Other informal working arrangements or verbal agreement of competence were suggested to suffice, and the EPA framework could deprive PAs of their responsibility.

It is your own responsibility. If you act incompetently, you should bear liability. For me, it is important that everyone sets their own practice boundaries (Respondent 94)

Renewal of Entrustment

We asked PAs trained with EPAs (n = 120) about their perspectives regarding the use of EPAs as a framework to maintain and renew entrustment decisions from graduation to retirement. Most (80%; n = 97) responded, of whom 96% (n = 93) indicated that the EPA concept would be suitable to maintain and renew entrustment after graduation.

Yes, EPAs provide a great framework. Our profession needs to have a formalized transparent way of proving our competence (Respondent 7)

EPA Use After Graduation

We found it to be common practice among PAs to acquire new skills and competencies after graduation. Among all respondents, 60% (n = 252) reported having added new competencies and/or skills after graduation. Among the respondents trained with EPAs who acquired new skills and competencies after graduation (n = 66), 35% (n = 23) reported using EPAs to frame this expanded scope of practice. Of the respondents not initially trained with EPAs, 4% (n = 6) reported using EPAs after graduation to record their competence development. Narrative answers supported this.

We have developed new EPAs, in line with the new to-be-acquired skills. A training plan has been developed, and three staff members have signed statements of awarded competence (Respondent 57)

EPA-trained PAs who did not report using EPAs after graduation for continuous entrustment decisions provided a wide range of methods on how they acquired, formalized, and documented their new skills or competencies, for example, following certified training, patient case discussions, using clinical observations, on the job training, etc. The 96 free-text responses submitted by 120 EPA-trained PAs revealed that almost all (95%; n = 115) consider the EPA structure suitable to record their competence development after graduation.

EPAs clearly structure what the required competencies are, and these can be tested as such. Furthermore, by using EPAs, you get a clear job description (respondent 212)

The five respondents who did not think EPAs should be used argued that it would be too much work, and doubted whether their competence would be questioned, and believed there would be other (not specified) ways to document it.

Change of Workplace Environment

Change of workplace environment seemed to be common among PAs in the Netherlands, as 24% (n = 102) of the respondents reported changing their workplace environment after graduation. Respondents who indicated having changed their workplace environment were asked if and how they documented their competence in their new workplace. Many PAs (n = 78; 76%) provided a free-text response, 47% (n = 37) of which reported having no formal process or documentation of their competence in their new workplace, while 14% (n = 11) reported having used EPAs as the structure to document their new competencies in their new workplace.

DISCUSSION

This study shows that EPAs play an ongoing role after graduation, at least for Dutch PAs. Competency-based health professions education is increasingly viewed from a continuum perspective, in which continuing professional development is viewed as a logical consequence of lifelong learning.3,4 Medical professionals' training and education may move into individual, dynamic development and maintenance of competence in which EPAs can play a significant role. This study shows how EPAs can continue to be useful after graduation, at least for PAs. When professional tasks change because of advancement in professional practice, or when individuals decide to master different tasks or change their workplace environment, as PAs regularly do,13 new EPAs can be added to their portfolios, and other EPAs may cease to be relevant. Almost two-thirds of all respondents (60%) reported having attained new skills or competencies after graduation, 40% of them using an EPA framework to document their newly acquired competence. Almost all PAs familiar with the EPA framework support the use of EPAs after graduation. Therefore, the concept of EPAs may be well suited to play a role in the recertification processes to maintain high standards of practice.18 However, PAs who are not familiar with EPAs seldomly report using this framework. Health care professionals who are unfamiliar with the EPA concept may benefit from some additional information and training to start using the EPA framework after graduation. However, as EPAs are no more and no less than ‘units of professional practice to be entrusted to PAs’, the concept is not difficult to understand as a conceptualization of the profession. The concept is relevant for PAs to mark the boundaries of their autonomy. By the nature of this profession, PAs perform medical tasks and have autonomous responsibilities and liability for safe patient care. In the Netherlands, this autonomy was grounded in legislation in 2018. EPAs can serve to specify what individual PAs are entitled to do.

In health care, the legitimacy of specialty certification for a lifetime is in debate. The focus should be on meaningful recertification, and initial certification should not be considered a lifetime credential. For example, physicians who meet specialty continuing certification standards should also show their commitment to professionalism, lifelong learning, and high-quality patient care.19 This should apply to any health professional, and in the future, EPAs can play a role for the benefit of the individual and the health care institution, patients, and the public.10 Physician assistants constitute a profession that has only recently gained acceptance and legal status in the Netherlands. What PAs practice is highly variable because their scope of practice is highly individualized. We have shown that the EPA model would be welcomed to provide clarity for individuals, their professional community, health care settings, employers, and patients and consequently document a dynamic career development after initial training. One way to establish the continued certification of competence for PAs beyond the educational program is to have a national body (such as the NAPA) play a role in the formal implementation of the EPA structure for continuous entrustment decisions.

There are some limitations to this study. We do not know how many PAs received our mailings and social media messages, and the 29% response rate reported, based on all registered PAs in the country, is low but may be a conservative estimate. More importantly, we do not have a reason to assume that our findings diverge from a representative overview of how PAs in the Netherlands use and value EPAs after graduation. Furthermore, a survey method has limitations. To gain a deeper understanding of the concepts described and themes that emerged, interviews and focus groups could add to understanding why PAs switch their scope of practice, how much effort it takes to achieve a level of competence to be entrusted with new EPAs and to gain an understanding what is needed for trustworthy and convincing documentation. Because not all Master of Physician Assistant programs in the Netherlands use the EPA framework, some respondents were not familiar with the concept of EPAs. We excluded them from EPA-specific questions. However, they showed interest in similar concepts and themes as EPA-trained PAs regarding continuing certification. The extent to which our findings are generalizable to other health professionals remains to be seen. The unique character of PAs, with a highly individualized scope of practice, with at least some sort of collaboration with and supervision from physicians, and a large number of PAs changing specialty, makes the EPA structure highly suitable. How the EPA structure should be applied and optimized for PAs and other health professionals after graduation warrants evaluation in future studies.

CONCLUSIONS

We have demonstrated that EPAs play an ongoing role after graduation, at least for Dutch PAs. EPA-trained PAs indicate overwhelming support for the use of the EPA framework as a continuum from undergraduate education until retirement. Continued entrustment after graduation provides grounding for the qualities of PAs and other health professionals. This should serve transparency to inform colleagues, management, and the public about which medical tasks and activities employees, in this case, PAs, are trusted and privileged to perform. Furthermore, EPAs can be helpful with institutional accreditations, reregistration, lifelong learning, and, most importantly, patient safety.Lessons for Practice ■ EPAs, conceived until now to serve in educational programs, seem useful to support continued professional development after graduation. ■ Physician assistants (PAs) in the Netherlands constitute the first profession with graduates who have been trained with entrustable professional activities (EPAs). ■ EPAs are highly valued by the community of PAs in the Netherlands to define and legitimize their profession.

ACKNOWLEDGMENTS

The authors wish to thank the Dutch Association of Physician Assistants (NAPA) for inviting physician assistants in the Netherlands to participate in the survey.

REFERENCES 1. ten Cate O. Entrustability of professional activities and competency-based training. Med Educ. 2005;39:1176–1177. 2. O'Dowd E, Lydon S, O'Connor P, et al. A systematic review of 7 years of research on entrustable professional activities in graduate medical education, 2011-2018. Med Educ. 2019;53:234–249. 3. Shorey S, Lau TC, Lau ST, et al. Entrustable professional activities in health care education: a scoping review. Med Educ. 2019;53:766–777. 4. ten Cate O, Chen HC, Hoff RG, et al. Curriculum development for the workplace using entrustable professional activities (EPAs): AMEE guide No. 99. Med Teach. 2015;37:983–1002. 5. Ryan MS, Lockeman KS, Feldman M, et al. The gap between current and ideal approaches to the core EPAs: a mixed methods study of recent medical school graduates. Med Sci Educ. 2016;26:463–473. 6. Smith HL, Craig SR, Yost WJ. Examination of entering residents' self-reported confidence and supervision needs performing AAMC entrustable professional activities. J Grad Med Educ. 2018;10:474. 7. Holzhausen Y, Maaz A, Roa-Romero Y, et al. What can we expect from medical graduates? Empirical survey on the performance of core EPAs in the first days of residency. BMC Med Educ. 2020;20:452. 8. Lindeman BM, Sacks BC, Lipsett PA. Graduating students' and surgery program directors' views of the association of American medical colleges core entrustable professional activities for entering residency: where are the gaps? J Surg Educ. 2015;72:184–192. 9. ten Cate O, Carraccio C. Envisioning a true continuum of competency-based medical education, training, and practice. Acad Med. 2019;94:1283–1288. 10. ten Cate O, Balmer DF, Caretta-Weyer H, et al. Entrustable professional activities and entrustment decision making: a development and research agenda for the next decade. Acad Med. 2021;96:96–104. 11. Mulder H, ten Cate O, Daalder R, et al. Building a competency-based workplace curriculum around entrustable professional activities: the case of physician assistant training. Med Teach. 2010;32:e453–e459. 12. Wiersma F, Berkvens J, ten Cate O. Flexibility in individualized, competency-based workplace curricula with EPAs: analyzing four cohorts of physician assistants in training. Med Teach. 2017;39:535–539. 13. Hooker RS, Cawley JF, Leinweber W. Career flexibility of physician assistants and the potential for more primary care. Health Aff. 2010;29:880–886. 14. Quella AK, Hooker RS, Zobitz JM. Retention and change in PAs’ first years of employment. J Am Acad Phys Assist. 2021;34:40–43. 15. Touchie C, ten Cate O. The promise, perils, problems and progress of competency-based medical education. Med Educ. 2016;50:93–100. 16. Willis GB, Artino AR Jr. What do our respondents think we're asking? Using cognitive interviewing to improve medical education surveys. J Grad Med Educ. 2013;5:353–356. 17. Kiger ME, Varpio L. Thematic analysis of qualitative data: AMEE guide no. 131. Med Teach. 2020;42:846–854. 18. Norcini J. Is it time for a new model of education in the health professions? Med Educ. 2020;54:687–690. 19. Colenda C, Scalon W. Vision for the Future. Continuing Board Certification: Vision for the Future Commission—Final Report. Commissioned by the American Board of Medical Specialties; 2019. Available at: https://visioninitiative.org/commission/final-report/: American Board of Medical Specialities. Accessed March 3, 2021.