BACKGROUND

The field of emergency medicine involves many highly invasive procedures, such as resuscitative endovascular balloon occlusion of the aorta (REBOA). The REBOA has been recognized as effective in providing temporary hemostasis in patients exposed to fatal blood loss.1-3 However, correct knowledge and skills are essential because of the multitude of complications and the need to become proficient in the placement technique.4, 5

Thus, effective courses are essential for proficiency.3, 6-8 However, the coronavirus disease (COVID-19) pandemic has forced the postponement or cancelation of various educational courses due to transmission concerns.

Web-based education is effective not only for knowledge but also for procedures.9-11 However, these studies have shown effective education for minimally invasive procedures. Few studies have examined the effectiveness of Web-based education in highly invasive procedures, such as REBOA.

To address this knowledge gap, this study aimed to investigate whether knowledge acquisition and confidence after the Web-based course, which cannot provide a hands-on station, are different from those acquired after the on-site course, using pre- and postcourse test scores.

METHODS Setting and context

The REBOA working group of the Japanese Society of Diagnostic and Interventional Radiology in Emergency, Critical Care, and Trauma (DIRECT) conducted the REBOA course (DIRECT-REBOA course) in 2019. However, in 2020, we decided to conduct the course Web-based because of COVID-19. The DIRECT-REBOA course was created and piloted in Japan to address the critical need for a standardized handling of REBOA in an emergency and critical care setting. The course content was based on the standard handling of REBOA mainly for emergency and critical care residents and attendants. The on-site course material was modified after discussions with the course instructors. The instructors included emergency physicians, intensivists, interventional radiologists, and acute care surgeons to ensure the quality of the course.

The DIRECT-REBOA course comprises several components, including didactic lectures, skill stations using models, and simulation. The overall course objective is “to provide knowledge and technical skills for adequate and safe handling of REBOA.” The content of didactic lectures includes seven topics, including insertion, sheath and catheter removal, and complication. At the skill stations, learners actually apply the skills taught in the lectures to mannequins. As the Web-based course does not allow the hands-on practice of actual procedures, only basic procedure schemas or videos were presented. The instructors discussed the patients’ overall pre-, during, and postprocedural care during this course. Table 1 shows a detailed timetable of the on-site and Web-based course.

Table 1. Detailed timetable of the Web-based training course on resuscitative endovascular balloon occlusion of the aorta (REBOA) Length (min) Content 5 Opening remarks 10 Course description, pretest 35 Lecture 1: Basic knowledge of REBOA 35 Lecture 2: REBOA in non-trauma 10 Break 35 Lecture 3: Arterial access 35 Lecture 4: REBOA techniques 10 Break 35 Lecture 5: Complications of REBOA 35 Lecture 6: Resuscitative thoracotomy and REBOA 20 Posttest, Q&A session 5 Closing remarks Participants, data collection, and statistical analysis

The learners’ precourse test was distributed just before starting the first lecture, and the postcourse test was distributed immediately after completion of the course. The test questions were created by course instructors based on the objectives of each lecture. A series of questions were vetted to ensure that they reflected the course content, wording, and appropriateness. The test consisted of 28 true/false questionnaires. Both the pre- and postcourse test questions were written in Japanese and were the same. The questions reflected seven different topics in the course modules, including insertion, complication, indication, and management. Students were required to answer all questions, and each question received one point for each correct answer. Therefore, the maximum score of this test was 28 points. In addition to assessing knowledge, participants were asked about their self-rated confidence regarding insertion, sheath and catheter removal, and complication before and after the course. Self-rated confidence was measured using 10-point Likert scales (1 = least confident, 10 = most confident). Table 2 shows the actual test used before and after the course.

Table 2. Questionnaire taken before and after courses on resuscitative endovascular balloon occlusion of the aorta (REBOA) Question 1. Are the following statements about the position of the sheath true or false? Puncture the superficial femoral artery. Puncture just above the inguinal ligament. It is crucial to confirm the level of bifurcation by echocardiography due to the high femoral artery bifurcation. A real-time ultrasound-guided puncture does not improve safety. Question 2. Are the following statements about the REBOA true or false? REBOA has a high hemostatic effect. Some data suggest that REBOA could be harmful. REBOA is a safe device with no complications. REBOA has established itself as an effective resuscitation tool with solid evidence. Question 3. Are the following statements about REBOA insertion true or false? ZONE 2 is always the preferred position. The umbilicus marks the inferior border of ZONE 1. The pubic symphysis marks the inferior border of ZONE 3. Both the catheter and the sheath should be grasped before balloon dilation. Question 4. Are the following statements about the management of REBOA true or false? The target proximal arterial pressure is 60 mmHg. Compression hemostasis is not possible during the sheath and catheter removal of a 7Fr sheath. Do not remove only the REBOA catheter, leaving the sheath in place. Partial REBOA is a method to reduce organ ischemia. Question 5. Are the following statements about the complications of REBOA true or false? If the physician feels resistance when inserting a guidewire, try pushing it in. Organ damage by reperfusion is more likely to occur with intermittent dilation than continuous dilation. To reduce lower extremity ischemia, insert a large-bored sheath. If the vessel is tortuous, the Landmark method cannot work effectively. Question 6. Are the following statements about REBOA and aortic clamping true or false? The decision on which to perform REBOA or aortic clamping is easy to make. When the pulse is not palpable, it is better to choose REBOA. It is difficult to switch from aortic clamping to REBOA. Aortic clamping is highly invasive compared to REBOA. Question 7. Are the following statements about the difference between blunt trauma and nontrauma true or false? REBOA is contraindicated in AAA rupture. In postpartum hemorrhage, REBOA should be placed in ZONE 1. REBOA is not effective in upper gastrointestinal bleeding. REBOA is effective for high bleeding risk cesarean section. AAA, abdominal aortic aneurysm.

The institutional review board approved the protocol with waiver of informed consent because the pre- and postcourse tests were anonymized immediately after being paired. A Wilcoxon matched-pairs signed-rank test was carried out on the pre- and postcourse test scores. Stata/SE 16.0 (Stata Corp) was used for data analyses. The threshold for significance was set at p < 0.05.

RESULT

Thirty learners completed the on-site course in December 2019, and 21 learners completed the Web-based course in October 2020. Forty-seven learners completed both the precourse and the postcourse tests. The complete details of the sample characteristics are shown in Table 3. In summary, the median postgraduate years of the learner was 6 years (interquartile range [IQR], 4–9 years), and 43% (19/44) of the learners were board-certified emergency physicians.

Table 3. Basal characteristics of learners who participated in training courses on resuscitative endovascular balloon occlusion of the aorta (REBOA) Variable Overall (N = 47) On-site course (n = 26) Web-based course (n = 21) p-value PGY, median (IQR) 6 (4-9) 7 (5-10) 6 (4-9) 0.34 EM board-certified, n (%) 19 (40.4) 13 (50.0) 6 (28.6) 1.00 No previous experience of REBOA, n (%) 10 (21.2) 6 (23.1) 4 (19.0) 0.45 EM, emergency medicine; PGY, postgraduate year.

The median precourse test score was 25.4 (IQR, 24.4–26.4), compared with 27.2 (IQR, 26.8–27.6) for the postcourse test in the on-site course. In the Web-based course, the mean precourse test score was 25.7 (IQR, 24.7–26.7), compared with 27.3 (IQR, 26.7–27.8) for the postcourse test. In both courses, the difference between the precourse and postcourse test scores showed a statistically significant increase in knowledge among participants (Table 4). Thirty-two learners showed an improved score, while nine learners maintained the same score.

Table 4. Pre- and postcourse scores in on-site and Web-based courses on resuscitative endovascular balloon occlusion of the aorta Variable Precourse Postcourse Difference p-value On-site (N = 26) Test score 25.4 (24.4–26.4) 27.2 (26.8-27.6) 1.8 (0.8–2.8) <0.010 Self-rated confidence of insertion 4.5 (3.4–5.7) 6.8 (6.1–7.6) 2.3 (1.4–3.2) <0.010 Self-rated confidence of sheath and catheter removal 4.7 (3.6–5.7) 6.8 (6.0–7.6) 2.2 (1.4–2.9) <0.010 Self-rated confidence of complication management 3.6 (2.7–4.5) 5.9 (5.2–6.6) 2.3 (1.5–3.1) <0.010 Web-based (N = 21) Test score 25.7 (24.7–26.7) 27.3 (26.7–27.8) 1.6 (0.5–2.6) <0.010 Self-rated confidence of insertion 3.6 (2.5–4.7) 5.1 (4.2–6.0) 1.5 (0.0–2.9) 0.045 Self-rated confidence of sheath and catheter removal 4.1 (3.0–5.3) 5.3 (4.5–6.3) 1.2 (−0.3–2.7) 0.150 Self-rated confidence of complication management 3.1 (2.2–4.1) 5.0 (4.2–5.8) 1.9 (0.5–3.2) 0.010

The learners’ self-rated confidence scores are presented in Table 4. The increases in the self-rated confidence scores were statistically significant among learners in the on-site course. However, the difference was not statistically significant in the self-rated confidence scores about “sheath and catheter removal” among learners in the Web-based course.

DISCUSSION

There was a significant increase in scores between the pre- and postcourse tests for both the on-site and Web-based courses. In addition, the self-rated confidence levels in all the procedures significantly increased during the on-site course. In contrast, in the Web-based course, not all fields showed significantly increased self-rated confidence in the procedures.

The significant increase in test scores could be an indication that didactic lectures are an efficient method of transferring knowledge. The lecture content was the same for the Web-based and on-site courses; thus, the results suggest no difference in the educational effectiveness of on-site and Web-based courses in terms of knowledge transfer. Web-based resources have been shown to be practical educational resources. This result is consistent with previous studies.12-14

However, self-rated confidence about sheath and catheter removal was not significantly increased in the Web-based course. Self-rated confidence in a procedure is a concept that encompasses not only what one “knows” but also what one “can do.” Thus, the Web-based course, which consisted of didactic lectures that mentioned technical procedures, was insufficient for learners to gain self-rated confidence. A significant increase was observed in the on-site course, which included hands-on sessions, suggesting that a combination of lectures and hands-on sessions plays a significant role in gaining self-rated confidence.15

Emergency physicians are required to perform various procedures during shifts. “Just knowing” is insufficient for them. In this COVID-19 pandemic era, when it is difficult to undertake on-site training, Web-based training has become popular. However, educators need to be aware that there are still challenges in learning procedural skills through a Web-based course only. For students to gain confidence in the highly invasive procedure, taking the course while handling the actual devices or instructors peer-reviewing videos of students performing the actual procedures after the course could be possible solutions.

This study had several limitations. First, because knowledge and confidence were measured before and immediately after the course, it is unclear whether knowledge can be retained in the long term. It could be useful to reexamine knowledge and confidence after a year of the course with postcourse clinical experience. Although the differences in learning effects due to the learning methods were evident in this study, it is necessary to prove the scientifical and clinical usefulness of the course itself by measuring the long-term retention of knowledge so that this result can be informative to educators. Second, although the goal of education for a clinician is to enable them to “do” in a clinical setting, it is unclear whether students actually perform REBOA placement. Third, true/false tests could be insufficient to measure knowledge.16 The purpose of this study was to compare the scores before and after the course. Thus, the measurement method did not need to be rigorous. Finally, the number of students was limited.

CONCLUSION

Knowledge increased significantly in both the on-site and Web-based courses. However, the Web-based course might not be sufficient to give learners confidence in the procedures.

ACKNOWLEDGEMENTS

This research was supported in part by research grants from The General Insurance Association of Japan.

DISCLOSURE

Approval of the research protocol: The research was approved by the Ethics Committee of Tokyobay Urayasu Ichikawa Medical Center (reference no. 684).

Informed consent: The need for informed consent was waived because the pre- and postcourse tests were anonymized immediately after being paired.

Registry and registration number of the study: N/A.

Animal studies: N/A.

Conflict of interest: None.

APPENDIX 1

Investigators of the DIRECT-REBOA Course team (all based in Japan): Satomi Senoo (Saiseikai Yokohamashi Tobu Hospital, Yokohama), Yuri Kon (St. Marianna University School of Medicine, Kawasaki), Futoshi Nagashima (Saga University Hospital, Saga), Keisuke Tomita (Chiba University Graduate School of Medicine, Chiba), Takuya Sugiyama (Chiba Emergency Medical Center, Chiba), Makoto Aoki (Japan Red Cross Maebashi Hospital, Gumma), Yutaro Kurihara (Kitasato University School of Medicine, Sagamihara), Suguru Hitomi (Japan Red Cross Saitama Hospital, Saitama), Takayuki Irahara (Aichi Medical University Hospital, Aichi), Shinya Onishi (Osaka University Graduate School of Medicine, Osaka), Ryosuke Usui (Rinku General Medical Center, Osaka), Ken Shinozuka (Kyoto University Hospital, Kyoto), Junichi Matsumoto (St. Marianna University School of Medicine, Kawasaki), Hiroshi Kondo (Teikyo University School of Medicine, Tokyo).

REFERENCES

1Manzano Nunez R, Naranjo MP, Foianini E et al. A meta-analysis of resuscitative endovascular balloon occlusion of the aorta (REBOA) or open aortic cross-clamping by resuscitative thoracotomy in non-compressible torso hemorrhage patients. World J. Emerg. Surg. 2017; 12: 30. 2Long B, Hafen L, Koyfman A, Gottlieb M. Resuscitative endovascular balloon occlusion of the aorta: a review for emergency clinicians. J. Emerg. Med. 2019; 56: 687– 97. 3Samuels JM, Sun K, Moore EE et al. Resuscitative endovascular balloon occlusion of the aorta-Interest is widespread but need for training persists. J. Trauma Acute Care Surg. 2020; 89: e112– 6. 4Davidson AJ, Russo RM, Reva VA et al. The pitfalls of resuscitative endovascular balloon occlusion of the aorta: risk factors and mitigation strategies. J. Trauma Acute Care Surg. 2018; 84: 192– 202. 5Ribeiro Junior MAF, Feng CYD, Nguyen ATM et al. The complications associated with Resuscitative Endovascular Balloon Occlusion of the Aorta (REBOA). World journal of emergency surgery. WJES 2018; 13: 20. 6Engberg M, Taudorf M, Rasmussen NK, Russell L, Lönn L, Konge L. Training and assessment of competence in resuscitative endovascular balloon occlusion of the aorta (REBOA) - a systematic review. Injury 2020; 51: 147– 56. 7Chang YR, Park CY, Kim DH, Ma DS, Chang SW. A course on endovascular training for resuscitative endovascular balloon occlusion of the aorta: a pilot study for residents and specialists. Ann. Surg. Treat. Res. 2020; 99: 362– 9. 8Brede JR, Lafrenz T, Krüger AJ et al. Resuscitative endovascular balloon occlusion of the aorta (REBOA) in non-traumatic out-of-hospital cardiac arrest: evaluation of an educational programme. BMJ Open 2019; 9: e027980. 9Schell SR, Flynn TC. Web-based minimally invasive surgery training: competency assessment in PGY 1–2 surgical residents. Curr. Surg. 2004; 61: 120– 4. 10Chenkin J, Lee S, Huynh T, Bandiera G. Procedures can be learned on the Web: a randomized study of ultrasound-guided vascular access training. Acad. Emerg. Med. 2008; 15: 949– 54. 11Burnette K, Ramundo M, Stevenson M, Beeson MS. Evaluation of a web-based asynchronous pediatric emergency medicine learning tool for residents and medical students. Acad. Emerg. Med. 2009; 16(Suppl 2): S46– 50. 12Wolbrink TA, Burns JP. Internet-based learning and applications for critical care medicine. J. Intensive Care Med. 2012; 27: 322– 32. 13Kleinpell R, Ely EW, Williams G, Liolios A, Ward N, Tisherman SA. Web-based resources for critical care education. Crit. Care Med. 2011; 39: 541– 53. 14Pourmand A, Lucas R, Nouraie M. Asynchronous web-based learning, a practical method to enhance teaching in emergency medicine. Telemed. J. E Health 2013; 19: 169– 72. 15Knudsen L, Nawrotzki R, Schmiedl A, Mühlfeld C, Kruschinski C, Ochs M. Hands-on or no hands-on training in ultrasound imaging: a randomized trial to evaluate learning outcomes and speed of recall of topographic anatomy. Anat. Sci. Educ. 2018; 11: 575– 91. 16Chandratilake M, Davis M, Ponnamperuma G. Assessment of medical knowledge: the pros and cons of using true/false multiple choice questions. Natl. Med. J. India 2011; 24: 225– 8.