記住我
The Veterans Health Administration (VHA) Office of Nursing Services (ONS) is committed to the use of evidence to guide all types of nursing practice. The ONS prioritizes the implementation of evidence-based practice (EBP) as one of the most effective ways to improve the quality of patient care across all care settings.1-3 The ONS used the Advancing Research and Clinical Practice through Close Collaboration (ARCC) model4 for the education and implementation of EBP. The ARCC model is one of the most widely used EBP models, focusing on system-wide EBP implementation, culture, and organizational readiness. However, there is limited evidence that the any current EBP model results in EBP implementation. In fact, the most effective way to develop EBP skills that result in EBP implementation is not well developed.3 The VHA offered EBP courses, including the Ohio State University College of Nursing Helene Fuld Health Trust National Institute for Evidence-Based Practice in Nursing and Healthcare Immersion (hereafter referred to as EBP immersion), over the past 5 years. Leaders at the VHA identified that educational initiatives focused on attitudes, beliefs, and skills regarding EBP have not reliably translated into implementation initiatives.
The VHA provided direct care staff in 47 VHA healthcare systems an opportunity to attend a weeklong EBP immersion. The EBP immersion provides didactic and practical instruction for EBP, and staff worked on initiatives assigned by their facility leadership during the immersion. The EBP immersions teach why EBP is important; an overview of the use of statistics; how to create a search question with the Population, Intervention, Comparison, Outcome, and Time format; and how to conduct a literature search, use critical appraisal tools, and create evidence and summary tables. Breakout sessions for leaders and EBP mentors discussed implementation, dissemination, and sustainment of EBP initiatives. Material is presented during a 5-day period with the expectation that participants will implement the EBP initiative after the EBP immersion.
The EBP immersion offers follow-up for participants after the sessions, but the ONS elected to offer an optional VHA-specific follow-up course instead. This decision was made because the VHA is a unique healthcare environment in which EBP implementation is best addressed by a VHA team for specific application. The VHA EBP course was structured to reinforce the didactic instruction provided in the EBP immersion and provide support while practicing EBP skills. The course focused on translating the EBP knowledge gained through the EBP immersion into implementation of EBP initiatives in a VHA setting. Evidence-based practice subject matter experts taught the virtual EBP course to staff at multiple facilities. Attendance at the EBP immersion was a prerequisite for the VHA EBP course. The VHA EBP course provided four 4-hour sessions of knowledge review and implementation support over 4 months (Table 1). Participants in the VHA EBP course continued the EBP initiatives started during the EBP immersion and developed their initiatives through implementation and evaluation while supported by peers and instructors.
Table 1 - VHA EBP Course Outline Session 1 Relevant Clinical Question, PICOT, VA Evidence Sources Session 2 Evaluating and Synthesizing Evidence, Practice Recommendation, Implementation Plan Session 3 Implementation Support, Initiative Evaluation Session 4 EBP Initiative PresentationsAbbreviation: PICOT, Population, Intervention, Comparison, Outcome, and Time.
A literature review was conducted to determine the best method for EBP education that translated to implementation. Databases included Cochrane, PubMed, and CINAHL using the following search terms: EBP, readiness, competency, and knowledge transformation. Articles in English from 2017 to 2022 that described outcomes of EBP were included. Two of the 14 articles meeting the criteria3,5 addressed practice outcomes and recommend multimodal instruction as the most effective for promoting EBP implementation. Other articles described changes in knowledge and attitudes but did not address the ability of participants to implement EBP initiatives. Therefore, there is evidence to suggest that EBP instruction does not reliably translate to practice outcomes without additional intervention.
Objectives and Specific AimsThe objectives of this study were to measure the efficacy of the EBP immersion program alone as compared with the EBP immersion program plus the VHA EBP course in promoting EBP implementation. The research hypothesis was that the EBP immersion plus clinical support through the VHA EBP course would be more effective in facilitating EBP implementation than the EBP immersion alone. The research was guided by the question: What is the effect of offering the EBP immersion alone as compared with offering the EBP immersion plus the VHA EBP course on EBP implementation? Implementation was defined as using the best practice as identified by the EBP summary evaluation to make changes in a practice setting within 3 months of completing the course.
Specific aims of the study were the following:
Compare the effectiveness of staff who attended the EBP immersion with that of the staff who attended the EBP immersion plus the VHA EBP course as measured by 1) the number of EBP initiatives implemented; and 2) the impact of the EBP initiatives. Determine factors that influence EBP implementation. Compare the return on investment (ROI) for the EBP immersion with the EBP immersion plus the VHA EBP course through analysis of the cost of the program and the ROI for the EBP initiatives. Methods FrameworkTo answer the research question and address the specific aims, the researchers used Kirkpatrick's6 levels of evaluation, which are a standard for educational evaluation and a method for making a business case for educational activities (Figure 1).
Figure 1: Kirkpatrick model, developed from Kirkpatrick.6Levels
Level 1: Reaction
Participants were asked to complete a course evaluation, designed to measure the immediate response to the experience of the course, immediately after each offering.
Level 2: LearningParticipants were determined to have obtained EBP knowledge if they were able to implement an EBP initiative. All participants were queried to determine the percentage of participants who successfully implemented an EBP initiative.
Level 3: BehaviorParticipants were asked to provide feedback in virtual focus groups to describe EBP behaviors they were able to demonstrate as well as barriers and facilitators to EBP implementation.
Level 4: ResultsParticipants were asked to describe EBP initiative activity via email. Results were evaluated by determining the number of EBP initiatives implemented during the programs, estimated cost savings, cost avoidance where applicable, and changes in associated metrics such as pressure injury, falls, and so forth. These data were self-reported as an EBP initiative evaluation and presented in aggregate data.
Study Population and Strategy Target Population 1This population is composed of attendees of the EBP immersion cohorts from 2019 through 2020 who attended the EBP immersion. This group included 153 attendees from 17 facilities. Facilities worked on multiple initiatives.
Target Population 2This population is composed of attendees of the EBP immersion VHA cohorts from 2019 and 2020 who also attended the VHA EBP course. This group included 262 attendees from 30 facilities. Facilities worked on multiple initiatives.
Data Collection Methods Level 1Anonymous course evaluations were made available on electronic platforms. The Fuld independently evaluated the EBP immersion. Veterans Health Administration standard continuing education (CE) evaluations were used for the VHA EBP course.
Level 2Anonymous participant feedback queries were sent to each sample group separately via an electronic format.
Level 3Attendees from both groups were offered the opportunity to participate in virtual focus groups. Attendees in each group were offered 3 different dates and times for focus groups to allow for maximum participation.
Level 4All attendees were sent requests to submit EBP initiative outcome data as a routine part of the follow-up from the EBP immersion. Participants were instructed to use VHA data-specific sources, including facility-specific data, data from the VHA Support Service Center, and pyramid data cubes with nonclinical outcomes as appropriate to their EBP initiative. Participants were instructed to provide only aggregate data to the research team. The intent was to determine whether the EBP initiative had the desired effect to improve outcomes at the VHA.
SampleThe researchers were interested in facility-level EBP initiatives. Forty-seven facilities participated in the EBP immersion. Of those 47 facilities, 30 also participated in the VHA EBP course after the EBP immersion. The research team invited all the individuals who had attended the EBP immersion; however, data for EBP implementation are reported by each facility. Researchers attempted to contact all actively employed VHA attendees of all the EBP immersions held from 2019 through 2020. This sample was inclusive and allowed for the maximum data possible. Participants were separated into 2 comparison groups: 1 group who participated in the EBP immersion only and 1 group who participated in the EBP immersion plus the VHA EBP course.
ResultsData collection started after the VHA institutional review board (IRB) at the VHA facility where the primary investigator is employed granted approval as an exempt study. Per the IRB, participation served as consent. Participants were not deidentified to the researchers. Participants were assured that they would be anonymous on course evaluations, which were electronically and anonymously administered. Participants were not promised anonymity for focus groups or EBP initiative evaluation but were assured that responses would only be shared with VHA leadership and for publication in aggregate formats. Only the VHA researchers involved in this study have access to the VHA drive where the data were stored.
DataData collection response rates are summarized in Table 2, and response themes are presented hereinafter.
Table 2 - Summary of Data Collection Evaluation Level Unit of Analysis Sample Size (n) Measurement Results Level 1 Individuals' EBP immersion plus VHA EBP course 49 Course evaluation (Likert-type scale) 91.8% rated experience as 4 or 5 (5, highest) (immersion-only satisfaction published elsewhere) Level 2 FacilitiesBoth groups were invited to evaluate the EBP immersion. The ratings for the EBP immersion were conducted by the Fuld and are published separately.7 Staff were invited to evaluate the VHA EBP course through a VHA standard CE survey after every session. Questions evaluated learner perception of knowledge/skill, usefulness of learning activities, faculty attributes, and open text for comments. The VHA EBP course sessions were rated high4,5 on a Likert scale of 1 to 5. Feedback was used to further develop the course.
Qualitative ReactionsThe respondents in the EBP immersion group provided feedback to the Fuld, which is published separately.7Respondents in the EBP immersion plus the VHA EBP course provided additional comments to the research team. Two respondents wanted more assistance with gaining leadership support, and 2 respondents did not like the virtual platform. Prepandemic respondents stated that they had a difficult time with the virtual platform because many staff had not previously used virtual platforms. Participants appreciated the collaboration with other facilities as well as the faculty feedback on their initiatives. Participants found the presentation by the VHA national librarian to be particularly helpful because the librarian reviewed VHA-specific libraries and resources as well as general search parameters. Some expressed the need for more input from the course instructors between the scheduled sessions to assist with moving their initiatives forward. In response to the feedback, VHA EBP instructors provided additional support for participants during the study period.
Level 2: LearningParticipants from both groups were sent email queries to determine whether they had implemented their EBP initiative (Table 3). Questions were designed to determine whether the participant had implemented their EBP initiative within 90 days of completing either the EBP immersion or the EBP immersion plus the VHA EBP course. Participants were asked to give rationale for EBP initiatives that were not implemented.
Table 3 - Comparison of EBP Immersion Only and EBP Immersion Plus VHA EBP Course Group No. EBP Initiatives % Completed % in Process % Not Completed EBP immersion only 11 19 (n = 2) 0 (n = 0) 82 (n = 9) EBP immersion plus VHA EBP class 37 35 (n = 13) 22 (n = 8) 43 (n = 16)For the participants in the EBP immersion–only group, 2 groups stated that EBP implementation was put on hold because of COVID. Some groups were formed by the facility leadership and consisted of staff who did not regularly work together and who sometimes had interests that did not align with their assigned EBP initiative and therefore did not sustain cohesiveness and failed to complete the project. Representatives from 11 facilities felt they had sufficient knowledge to implement their project. When queried for what other assistance was needed for EBP implementation, the responses were as follows: leadership support (n = 2), at least bachelor's degree education (n = 1), collaboration with other participants (n = 1), and more knowledge of implementation methods (n = 1).
For the participants in the EBP immersion plus VHA EBP course, 1 group determined that there was insufficient evidence in the literature to formulate a best practice and therefore decided not to continue their initiative. Eleven groups were in the process of implementing one of the initiatives started during the EBP immersion, although with some delays for COVID. One group did not implement with no stated reason. Sixteen facilities stated that they felt they had sufficient EBP knowledge to implement their EBP initiative. When queried for what other assistance was needed for EBP implementation, the responses were as follows: continuing educational support especially with pauses for COVID (n = 4), dedicated time (n = 3), leadership support (n = 2), dedicated EBP position (n = 1), and VHA-specific information (n = 1).
Level 3: BehaviorDuring the focus groups, the participants who attended the EBP immersion reported that they attended because they wanted hands-on EBP experience. They found the EBP immersion tools helpful. The participants found the “5 days in a row” format very intense. During the EBP immersion, staff were assigned clinical problems by senior leadership at their facilities, sometimes based on the facility needs without consideration of the participant's interests. Participants stated they would rather work on initiatives based on their own interests. Finally, they recommended a more careful selection of the EBP immersion participants because many participants did not continue with EBP after immersion. Participants stated that implementation would be more feasible if they had participated with follow-up education and mentorship. Participants described a lack of nursing leadership support, staffing changes, and the pandemic as barriers to implementation. Whereas some participants were able to sustain their initiatives, groups that were composed of nurses enrolled in transition to practice programs, who work on EBP initiatives as part of their 1st year of practice, were not able to continue with the initiative. Participants shared EBP tools with staff in their facilities, and 1 facility formed a journal club.
During the focus groups, the participants who attended the EBP immersion and the VHA EBP course stated that they attended the EBP immersion because they were interested in personal growth and learning, and wanted to develop facility EBP structure. Some participants were assigned by their leaders to attend. They found the EBP immersion most helpful in providing tools and methods for evaluating evidence. The participants described the compressed time frame as very quick for the learning objectives. Participants stated they were overwhelmed with developing an initiative in such a short time frame and struggled to implement initiatives after the immersion ended. They also discussed that their facilities were assigned too many initiatives, which led to not enough staff assigned to each initiative.
Participants reported that they enrolled in the VHA EBP course to have follow-up mentoring on their initiatives and assistance to trouble shoot barriers, learn EBP at a more sustainable pace, connect with others conducting EBP in the VHA, and learn about VHA resources. They found the VHA EBP course helpful with more time to work on initiatives and easier to schedule with 4-hour sessions once a month instead of 5 full days in a row. The support they had from other facilities was appreciated. Participants stated they were supported to stop initiatives when no evidence to support a best practice was discovered, assisted with integrating their work with other facilities focused on similar initiatives, and provided a platform to evaluate and synthesize evidence. They also cited the need for ongoing assistance with the EBP process. Barriers they encountered were pandemic disruptions, lack of leadership support, staff assigned to an initiative that they did not fully understand and were not invested in, lack of protected time for EBP work, and the need for more help navigating the system, that is, developing policies. Participants used the EBP immersion tools to sustain their EBP initiatives. The aspect of the VHA EBP course reported to be most helpful were ongoing specific examples of EBP initiatives with a discussion of barriers and how to address them.
Level 4: ResultsParticipants in the EBP immersion–only group stated that successful initiatives supported current policy and practice and were important to their leadership. Reasons for not implementing included implementation not supported by evidence (n = 2), not enough time to implement (n = 4), needed more training (n = 1), and pandemic interruptions (n = 1).
Participants in the EBP immersion plus VHA EBP course who successfully implemented initiatives described aspects that supported EBP implementation including utilization of the EBP immersion tools, collaboration within their own team and with other facilities, access to VHA-specific information, CE and support by an EBP expert, and education at a sustainable pace. Two of the participants who did not implement were not in clinical roles and attended only for awareness. One participant did not attend the entire session and therefore did not implement. The participants who did not implement stated barriers of competing priorities, lack of dedicated time, data not supporting a practice change, lack of organizational support, need for more training, and being interrupted by the pandemic. The 2 most cited barriers to implementation were not enough dedicated time and being interrupted by the COVID pandemic. Participants were asked to provide aggregate data to evaluate the outcomes of their initiatives. Although the research team had anecdotal evidence that EBP initiatives were evaluated at the facility level, no participants responded to the researchers' query to provide outcome data for the purpose of the research study. Therefore, the research team was not able to obtain data from the participants that would have allowed for an evaluation of cost savings and cost avoidance as planned. Therefore, an analysis of ROI was not possible.
DiscussionThemes that describe the facilitators and barriers to EBP implementation are presented in Table 4. The most important factors to facilitate EBP implementation are a sustainable pace for education, ongoing support for staff as they work through the EBP initiative, and dedicated time to complete the initiative. The VHA EBP course participants were more able to overcome the barriers and implement their EBP initiatives, in part because of access to more facilitators of implementation than the EBP immersion–only group.
Table 4 - Facilitators and Barriers of EBP Implementation Group Facilitators of Implementation Barriers to Implementation EBP immersion only • Knowledge developmentThe ONS has used this evidence and is focusing on 2 sustainment activities: leadership development and ongoing education/mentorship. Veterans Health Administration leadership staff is provided with education designed to assist leaders to support and sustain EBP initiatives. The ONS currently provides monthly EBP open house sessions. The EBP open house sessions are open to all VHA staff and include EBP instruction on 1 selected EBP topic monthly, demonstration of EBP initiatives, and a question-and-answer period. The goal of the EBP open house sessions is to provide a platform for ongoing EBP education. The ONS EBP Field Advisory Committee provides ongoing support through consultation services. The ONS is currently updating the VHA EBP course as a stand-alone course. In addition, the VHA is piloting an EBP platform that allows for structured guidance through the EBP process as well as real-time monitoring of progress and ROI data storage. The course will incorporate participant feedback from this research study, provide facilitators, address barriers, and provide a more robust evaluation of EBP initiative ROI including data identification and monitoring.
RecommendationsAlthough both EBP immersion only and EBP immersion plus VHA EBP class participants implemented EBP initiatives, there are areas that need to be addressed to foster more EBP implementation and sustainability. The authors recommend that organizations should: 1) ensure educational program attendees are enthusiastic about EBP; 2) allow enough time for participants to meaningfully develop plans and start implementation; 3) provide ongoing EBP education as well as mentoring and support that include clear directions and tools; 4) target senior leadership to support EBP including budgeting protected time to fully engage in initiatives; 5) provide a platform for teamwork to work on EBP initiatives; and 6) provide a platform for ROI data on all EBP initiatives.
The research team identified a need to increase the focus on data and ROI evaluation for EBP work. More research needs to be conducted on how EBP knowledge is translated into practice changes including measurable improvements in outcomes. These actions would support the case for greater resources for EBP education and implementation.
References 1. Allbarqouni L, Hoffmann T, Straus S. Core competencies in evidence-based practice for health professionals: consensus statement based on a systematic review and Delphi survey. JAMA Netw Open. 2018;1(2):e180281. doi:10.1001/jamanetworkopen.2018.0281. 2. Melnyk BM, Fineout-Overholt E. Evidence-Based Practice in Nursing & Healthcare: A Guide to Best Practices. 3rd ed. Philadelphia, PA: Wolters Kluwer/Lippincott Williams & Wilkins; 2015. 3. Portela Dos Santos O, Melly P, Hilfiker R, et al. Effectiveness of educational interventions to increase skills in evidence-based practice among nurses: the EDITcare systematic review. Healthcare (Basel). 2022;10(11):2204 https://doi.org/10.3390/healthcare1011220. 4. Melnyk BM, Fineout-Overholt E, Giggleman M, Choy K. A test of the ARCC© model improves implementation of evidence-based practice, healthcare culture, and patient outcomes. Worldviews Evid Based Nurs. 2017;14(1):5–9. https://doi.org/10.1111/wvn.12188. 5. Mudderman J, Nelson-Brandley HV, Wilson-Sands CL, Brahn P, Graves KL. The effect of an evidence-based practice education and mentoring program on increasing knowledge, practice, and attitudes towards evidence-based practice in a rural critical access hospital. J Nurs Adm. 2020;50(5):281–286. 6. Kirkpatrick D. Great ideas revisited. Techniques for evaluating training programs. Revisiting Kirkpatrick's four-level model. In: Training and Development. Vol. 50. 1996:54–59. https://eric.ed.gov/?q=Great+Ideas+revisited+Techniques+for+evaluating+training+programs&ff1=subEvaluation+Methods&id=EJ515660. Accessed April 1, 2024. 7. Gallagher-Ford L, Koshy Thomas B, Connor L, Sinnott LT, Melnyk BM. The effects of an intensive evidence-based practice educational and skills building program on EBP competency and attributes. Worldviews Evid Based Nurs. 2020;17(1):71–81. https://doi.org/10.1111/wvn.12397.
留言 (0)