We directly compared four common handheld ultrasound devices and found Lumify™ was rated highest for overall image quality, while Vscan Air™ was rated highest for overall ease of use. The overall ranking, satisfaction, and recommendation for purchase was highest with Lumify™ followed by Vscan Air™, and Vscan Air™ was most often selected for use as a personal pocket device. No single handheld ultrasound device was perceived to have all desired characteristics. We explored the characteristics of handheld ultrasound devices that are considered most important from the perspective of expert POCUS users, and image quality was felt to be the most important.

Handheld ultrasound devices offer unique advantages, including greater portability and ease of disinfection [7, 15, 21, 22], but few studies have directly compared different handheld ultrasound devices. One study compared three different handheld devices with the same operator performing gynecological ultrasound exams [12]. In contrast, our study has important advantages and adds substantively to our understanding of handheld ultrasound devices. First, we had POCUS experts use the same four handheld ultrasound devices on the same standardized patients to control for potential patient, device, and operator variables that could confound results. Second, POCUS experts were asked to use a variety of presets using both high- and low-frequencies for common diagnostic and procedural views. Third, our study included 24 POCUS experts as operators from 5 different specialties. Thus, our data may be more generalizable across common POCUS applications and different specialties. Our study sought to quantitatively assess ease of use and image quality based on user experience and provide a global rating of overall satisfaction and recommendation for purchase. We believe our study better replicates the myriad of considerations that clinicians face when making a purchasing decision.

Ultimately, no single handheld ultrasound device was rated highest in all categories. A lack of consensus of a single handheld ultrasound device being superior to others was similar to the findings by Toscano et al. in which investigators sought one suitable handheld device to perform gynecologic ultrasound exams in a resource-poor setting and ultimately selected the Lumify™ based on its ease of use, battery-life, portability, cost, and ease with depth and gain adjustments [12]. Even though POCUS experts rated Lumify™ highest for “overall satisfaction” in our study, more experts selected Vscan Air™ as the device they would buy “today as a personal device to carry in my coat pocket”.

Our study sought to identify specific characteristics of handheld ultrasound devices that expert POCUS users considered important. Quantitative and qualitative data demonstrated image quality to be the most important characteristic. Comments from experts reflected that image quality is the most important characteristic because poor quality images require a repeat evaluation with a cart-based ultrasound machine, negating the benefits of having a handheld. After image quality, the most important characteristics of handheld devices were ease of use, portability, total costs, and availability of different probe types. Perhaps once image quality is adequate for clinical decision-making, these secondary characteristics, such as ease of use and software options, become more of a deciding factor in selecting a handheld ultrasound device. Some of the characteristics, such as portability and number probes, are inherently related. A deeper understanding of the characteristics that POCUS users consider important, as well as the variables that affect purchasing decisions, warrant further investigation and may guide manufacturers in product development.

We recognize our study has limitations. First, POCUS experts could not be blinded to the different handheld devices and completed their scanning of the 3 standardized patients in the same room. Despite the large size of the room, we could not prevent experts from sharing their thoughts about the devices which could have biased their ratings. Further, our POCUS experts were not provided with dedicated training on the different handheld ultrasound devices, and lack of training may have limited the experts’ ability to navigate the software and optimize image quality. Third, bias from prior experience with some of the devices may have been a factor in the experts’ evaluations; however, we did not find an association between experts’ prior experience and the overall satisfaction ratings for three devices (Additional file 2: Table S1). Although there was a weak association detected between the experts’ experience level and overall satisfaction for the Vscan Air™, the association was negative, indicating that familiarity with the device did not inflate the ratings from experts. Fourth, experts rated handheld devices against one another, but we did not assess whether each handheld device met a minimal acceptable standard for ease of use and image quality. Additionally, we did not evaluate all potential diagnostic applications, including lungs, first trimester pregnancy, bladder, and lower extremity deep venous thrombosis, and we did not evaluate the handheld devices for procedural applications. Finally, POCUS users will need to consider several factors that were not explored in our study, including total costs of device over its lifespan; compatibility with preferred smart phone or tablet; probe characteristics (ergonomics, overheating, wired versus wireless connectivity); battery life; quality assurance and image archiving options; and desire for remote teleguidance or artificial intelligence software. Thus, handheld ultrasound purchasing decisions are complex, involving individual user preferences and device features, as well as external factors, such as institutional device restrictions and purchasing contracts, which were not addressed in our study.

For clinicians, the implications of our findings are threefold. First, though Vscan Air™ was rated highest for overall ease-of-use and Lumify™ for overall image quality, a perfect handheld device that combines all desired features does not currently exist. Second, when rating the importance of sixteen characteristics of handheld ultrasound devices, image quality was the only characteristic rated by all twenty-four experts as being “very important,” and new users should consider giving image quality priority when evaluating devices. Third, we focused our comparison of handheld ultrasound devices using 2-dimensional imaging alone. However, handheld ultrasound technology is advancing rapidly with new features, including artificial intelligence to guide image acquisition and interpretation, image sharing capabilities for remote teaching and real-time image interpretation by off-site experts, and robotics to facilitate probe placement for diagnostic imaging and invasive procedures [16, 23]. Future studies will be needed to compare the advantages and disadvantages of these advanced technologies in handheld ultrasound devices.