This was a pilot study designed as a non-inferiority test of the hand-held ultrasound compared to a notebook ultrasound model currently in use by the Kenyan POCUS trainees. We recruited healthcare providers who had received prior POCUS training and were presenting for follow-up refresher training and testing. All participants were above 18 years of age and provided consent to participate. The study took place in April 2018.

The refresher session entails a full hands-on review of all the point-of-care ultrasound modalities that the trainees have been exposed to before. I.e., The E-FAST and focused first- and second/third-trimester obstetric ultrasound. These exams are carried out on healthy, pre-scanned volunteers who provide informed consent. At the end of the refresher program, trainees are evaluated on their ultrasound scanning skills using a Standardized Observed Structured Clinical Exam (OSCE) (Additional file 1: Appendix A). Specifically, the OSCE assesses the trainee’s clinical skills in detecting free fluid in the pleural, peritoneal, and pericardial cavities as well as ruling out a pneumothorax. Trainees are also assessed on their focused first- and second/third-trimester obstetric scanning skills. These include first-trimester dating using crown-rump length, detecting and measuring the fetal heart rate, identifying the presenting part, locating the placenta, and the measurement of the head circumference and biparietal diameter. Any deficits noted during the OSCE are usually addressed as part of re-training procedures.

During this session, trainees had a 2-h hands-on introduction to a hand-held ultrasound (Butterfly iQ, 2018 Butterfly Network, Inc.) that was connected to an iPhone 8. The trainees practiced using this ultrasound model to perform guided E-FAST and focused obstetric exams. Afterward, we performed the OSCE evaluation twice for each trainee: once using the current notebook ultrasound model (WED, WELLD) and once using the Butterfly iQ probe. The OSCE testing was carried out on a separate group of pre-scanned volunteers. The order of ultrasound use and ultrasound modality testing was randomized to minimize bias from the previous examination. This was achieved by creating a randomized order in which the trainees rotated through different testing stations. Each testing station had a different volunteer/model, half the stations were equipped with a hand-held ultrasound, and the other half were equipped with a notebook ultrasound.

Images were scored independently by three investigators who are credentialed in POCUS according to the American College of Emergency Physician guidelines. Two of these investigators had also completed fellowships in acute and critical care ultrasound. A standardized scoring system was used to grade image quality on a scale of 0 to 4 (0: no meaningful images; 1: poor, not sufficient for interpretation; 2: good, acceptable for interpretation; 3: excellent, minor suggestions for improvement; and 4: outstanding, no suggestions for improvement). This scoring system is currently in use in the Kenya rural ultrasound training program [9] as well as other similar training programs in the African region [14, 15] (Additional file 1: Appendix A).

Image interpretation was scored using a standardized scoring system from 0 to 3: (0: did not acquire images sufficient for interpretation to answer a point-of-care question; 1: correctly acquired images but incorrectly interpreted some of them; 2: correctly acquired images and able to interpret SOME but not all relevant point-of-care questions; and 3: correctly acquired and interpreted images to answer ALL relevant point-of-care questions) (Additional file 1: Appendix A).

Images obtained with the hand-held ultrasound were uploaded to a HIPAA compliant cloud account using a local 3G cell phone network. The median time required for image transmission was documented. Images obtained using the notebook model were saved on the ultrasound machine for review.

Descriptive analyses on image quality and image interpretation scores were conducted with median and interquartile range (IQR) reported. Wilcoxon tests were conducted to determine differences between the notebook and hand-held imaging systems on overall E-FAST and focused obstetric image quality and image interpretation. A series of Wilcoxon tests were also performed to determine differences between the two ultrasound systems on each separate E-FAST and focused obstetric view, e.g., identifying differences in image quality and image interpretation for Right Upper Quadrant (RUQ) and Left Upper Quadrant (LUQ) views separately. Within each view category (E-FAST and obstetric), the scores for the two internal reviewers were averaged. The score of the external reviewer was used as a validity check for the internal consistency of the two primary internal reviewers. Agreement rates between the two internal reviewers and the external reviewer were calculated for image quality. Agreement within 1 point between the internal and external reviewers was considered as agreement.

The study was approved by the Aga Khan University Research Ethics Committee (REC) and the National Commission for Science, Technology, and Innovation (NACOSTI) Kenya.