Setting

We conducted a prospective, 3-arm randomized trial to compare 3 different POCUS education methods (tele-education, conventional, and self-directed learning) in learners’ improvement in POCUS skill proficiency and training workload in an academic teaching institution (Mayo Clinic, MN, USA) from May through June, 2016. The Mayo Clinic Institutional Review Board approved the study [study ID: 15-007311].

Study participants

Ultrasound novices were recruited among the groups of physicians, nurse practitioners, physician assistants, and medical students in the Mayo Graduate School of Medicine and the Mayo Clinic Health System. Novices were defined as individuals who have not conducted POCUS in the actual patient care except for ultrasound-guided central line insertions. Individuals who previously attended a regional workshop were excluded from the study. Verbal consent for study participation and videotaping was obtained from all study participants.

Study design

The study design is summarized in Fig. 1. After verbal consent was obtained from study participants, they were randomized into one of the 3 education arms. A group allocation was concealed from participants until the start of the hands-on training session. One week prior to the scheduled hands-on training date, participants were given access to online didactic lectures on POCUS basics, consisting of mandatory (80 min) and optional lectures (140 min). These online materials were posted on the dedicated YouTube channel, and the participants were encouraged to review them as many times as they wished. Online lectures consisted of 14 video modules that aimed to establish the basics of ultrasound physics, knobology, image acquisition, and image interpretation. On a hands-on training date, participants were asked to attend a 30-minute orientation session in a tele-conference format. During the orientation session with the use of PowerPoint via Zoom teleconference (CA, USA), a remote instructor reviewed the ultrasound machine and training room settings, standardized cues for ultrasound transducer orientation and hand maneuver, specific goals of the training, and a brief explanation of National Aeronautics and Space Administration task load index (NASA-TLX) survey (Supplementary Material 1). All questions from participants were answered during this session. At the end of the session, the group allocation was announced to the participants. All participants underwent two 75-minute POCUS hands-on training on a healthy male standardized patient in an assigned training method. Knobology, vascular, lung, and abdominal POCUS examinations were covered in the first 75 min of the hands-on session, and the latter 75 min were dedicated to cardiac POCUS. This training session was followed by a 30-minute post-training skill test (Supplementary Material 2) [30] and the NSAS-TLX survey to assess participants’ perceived workload. Optional conventional instructor-led hands-on sessions were offered for tele-education and self-directed learning groups after the completion of the skill tests for their own education.

Fig. 1Three hands-on training methods; tele-education, conventional education, and self-directed learning

A hands-on examination room in each training method was equipped with anatomy models of the heart, the kidney, and the right upper quadrant abdominal organs, a computer with access to online didactic lectures, and an ultrasound machine (M-turbo, FUJIFILM SonoSite, Washington, USA) with 3 different transducers (linear, curvilinear, and phased array). A trainee: instructor ratio was kept 3:1 except a self-directed learning group, where no instructor was available.

Tele-education method

An instructor verbally guided learners’ machine operation and hand maneuver using a 2-way remote communication system attached to the ultrasound cart (Figs. 2 and 3). The remote coaching system consisted of 4 key components (1) a speakerphone (DUET PCS – MT202-PCS, California, USA) mounted directly to the ultrasound cart, (2) a Logitech c930e Webcam (California, USA) mounted to an adjustable modular hose to allow direct visualization of learners’ hand maneuvers, (3) INOGENI analog-to-digital video converter (Quebec, CAN) to enable real-time streaming of the ultrasound images to Vidyo video conference solution via (4) Vidyo Room HD40 codec (New Jersey, USA) to allow for Health Insurance Portability and Accountable Act (HIPAA) compliant system. The entire system was configured to auto-activate once the ultrasound machine was turned on. In addition to audio communication, dual video feeds on the Vidyo platform were available on the remote instructor’s computer. All the video/audio interactions occurred on the Vidyo platform and behind the secure firewall of Mayo Clinic Intranet through a hard-wired connection.

Fig. 2

System configuration of the 2-way communication system

Fig. 3

Appearance of the 2-way communication system. (Fig. 3. Legends) Left: An ultrasound cart placed in the learners’ room. The cart featured essential components, including a speakerphone, a webcam mounted on a flexible modular hose for direct visualization of learner’s hand maneuvers, and an analog-to-video converter, which enabled the real-time streaming of ultrasound images. This configuration allowed learners to receive immediate verbal feedback from the remote instructor via the speaker. Right: The instructor’s view. The real-time verbal feedback was provided to the learner based on dual video feeds: one from the ultrasound machine and the other from the webcam capturing the learner’s hand maneuvers

Conventional education method

An on-site instructor taught 3 novice leaners in the examination room. The instructor guided learners’ machine operation and hand maneuver. Anatomical models and video lectures were used as necessary to increase learners’ understanding.

Self-directed learning method

No instructor was available, and participants were encouraged to view pre-course materials online with a computer available in the room and help one another to facilitate their learning experience. The participants were also allowed to use the anatomic models available in the room.

Outcome measurements

Our primary outcome was POCUS skill test scores. We utilized the scenario-based skill assessment of our POCUS proficiency test that was used in our previous works [15, 16, 30]. The assessment consisted of 6 sections to evaluate the exam initiation, vascular POCUS, lung POCUS, abdominal POCUS, cardiac POCUS, and the exam completion. Each section was timed by a study facilitator. The participants were asked to save video clips of scenario-based tasks documented in the instruction sheets. The saved video clips were subsequently reviewed and scored by one of the investigators on a checklist developed by the study team. The checklist was utilized in our previous work which demonstrated good concordance between two score raters [30].

Our secondary outcomes were improvement in learners’ confidence levels and perceived workload associated with each training method. Pre-training survey collected the participants’ confidence levels in machine operation, image acquisition, and image interpretation (all out of 10), and time spent on pre-course learning. Post-training survey consisted of the same questions on confidence levels as in the pre-training survey and 4 additional questions on course improvement and satisfaction on each training method they were allocated. Tele-education group was asked to answer 2 further questions: one inquiring about personal preferences on remote tele-education vs. conventional on-site guidance on a scale of 1–10 “Do you think that the quality of your ultrasound training would have been better if the instructor was present in the room?”, and the other inquiring agreement to the sentence “I would recommend the remote ultrasound training to my colleagues” on the scale of 1–10. The post-training and the NASA-TLX surveys were completed immediately after the training session before the skill test conduction.

Sample size calculation and statistical analysis

All continuous data were presented with the median and interquartile ranges (IQRs). The time spent on pre-learning modules were presented with the number of participants in each category (% in each group), and the Fisher’s exact test was conducted with a web applet available at the following link at the time of protocol preparation: http://www.physics.csbsju.edu/stats/exact_NROW_NCOLUMN_form.html.

All continuous variables were tested with Kruskall-Wallis tests followed by pairwise comparisons among the 3 groups with 2-sample t-tests with a corrected alpha level of 0.017 (0.05 divided by 3). Otherwise, a p-value less than 0.05 (two-sided) was considered as statistically significant.

The target sample size was calculated as 18 in total, based on the results in our previous studies where POCUS proficiency of novice leaners was measured with the same assessment tools we utilized in this study. Assuming a standard deviation of 10% and an anticipated difference of 20% in terms of skill-test scores for a two-sample t-test, six participants per each group allocation was considered enough to achieve more than 80% power with an alpha level of 0.05. All the statistical analyses except for Fisher’s exact test were conducted with JMP 10 (SAS, Cary, USA).