In this study, we evaluated the ability of novice critical care trainees to quantify CO using both manual and automated VTI measurements. We then compared their measurements to those attained by our control group of expert echocardiographers. We found that the feasibility of using AI-augmented image acquisition was excellent and, as defined by the coefficient of variation, the precision of VTI capture was higher with automatic VTI tracing when compared to manual VTI tracing.

Our relatively small, prospective study addresses several important issues related to the use of POCUS in cardiac output assessment. First, our study demonstrates that the ability to obtain relatively adequate measurements required for CO estimation does not require absolute expertise but a short, structured course with opportunities for hands-on practice. All 28 trainees were able to obtain adequate PLAX and AP5c views, resulting in 100% feasibility for image acquisition despite minimal US training. However, our reported feasibility for VTI is significantly higher than those previously published [10,11,12] even when accounting for training level [10]. This may be because all examinations were obtained on a single healthy volunteer who was hemodynamically stable and adequately positioned by experienced ultrasonographers, thus facilitating image acquisition. However, it must be noted that medical professionals with minimal training can obtain adequate parasternal and apical views 88% of the time when assessing critically ill patients [13]. These data further solidify the need for including POCUS education in medical training [14, 15].

Our study also considered accuracy and precision separately when measuring cardiac output and VTI because their distinction is clinically relevant. For example, accuracy of cardiac output has diagnostic implications in determining etiologies for specific hemodynamic states such as cardiogenic or distributive shock. By contrast, precision of cardiac measurements carries therapeutic implications. As an example, knowing with statistical confidence that SV or CO changed in response to an intervention, independent of SV or CO accuracy, is crucial for determining treatment effectiveness. In our study, trainees had larger measurement variations for DLVOT (1.9 to 2.9 cm) when compared to expert echocardiographers. Interestingly, both VTImnl-train and VTIauto underestimated true VTI as determined by expert echocardiographers. As a result, both manual and automatic cardiac output measurements had poor accuracy compared to the standardized control. Measurement variation likely has multiple sources, including human factors [16, 17], ultrasound factors [18], and intrinsic fluctuation secondary to patient physiology (e.g., respiratory variation). However, ultrasound and patient factors were less likely to contribute to measurement variability in our study since trainees and expert echocardiographers used the same machines and obtained measurements from only one patient. Therefore, differences in the coefficient of variation are likely to be operator dependent. Other differences noted in our study include the number of beats sampled before and after an intervention to detect a 10% SV change with statistical confidence. Expert echocardiographers, automatic VTI tracing, or manual VTI tracing by trainees were 10, 27, and 34, respectively [19]. These beat numbers are greater than the recommended 3 in clinical practice [20].

Hemodynamic assessment in our study was based on velocity time integral, which has been criticized for its difficulty of attainment in critical care settings. For example, the reported feasibility of VTI in the intensive care unit (ICU) ranges from 37 to 90% [10, 11, 21], likely due to inter-rater variability [10]. Other downsides to using VTI to measure CO include its dependence on insonation angles, which presents their own set of challenges [22,23,24]. Nevertheless, VTI is commonly used in tracking changes in SV and CO in critically ill patients [22, 25, 26] and has also demonstrated the ability to predict outcomes in select populations [27]. Together with other Doppler-based measures [28], VTI has become an increasingly reliable metric to assess fluid responsiveness in shock [29, 30]. Indeed, our results showcase VTI’s reproducibility between trainees and expert operators. However, it should be noted that VTIauto was more reproducible across raters than VTImnl-train, demonstrating narrower measurement range, smaller standard deviation, and lower coefficient of variation. Taken together, these results show that automation could further solidify VTI as a reliable metric for cardiac output measurements and reduce operator-dependent variability in clinical practice.

AI-assisted programs not only augment user experience, but also redefine the capabilities of US in a critical care setting [31]. As an example, automated VTI measuring systems highlight the importance of real-time feedback by aiding fatigued users, who may be more prone to making errors, perform routine echocardiographic tasks such as spectral envelope tracing, chamber volume estimations, and tracing endomyocardial boundaries. Equally important, AI can help determine view quality and assist directly with US image acquisition by guiding users towards optimal viewing angles. In fact, Zhang et al. trained an algorithm through a convolutional neural network model to accurately identify 23 viewpoints and segment cardiac chambers across 5 different views. The model was able to perform these tasks correctly 96% of the time. Even more impressive was its ability to flag views with partially obscured cardiac chambers [32]. As demonstrated in our study, our US device was able to identify foreshortened, or otherwise inadequate, 5-chamber views with poor aortic outflow jets and guide the user to an acceptable view through a series of coded Doppler box colors. This live feedback can provide users with accurate measurements for clinical decision-making in addition to training users to improve their US skills through a positive feedback loop. Therefore, we believe that educating operators along with the development of AI will not only train operators to use ultrasound, but also train ultrasound itself to tolerate different operators.

There are numerous studies demonstrating the utility of AI in estimating left ventricular hypertrophy, spectral wave tracing, right and left ventricular ejection fraction, three-dimensional chamber volume analysis, and regional wall motion [33,34,35,36,37,38,39,40,41]. However, LVOT VTI automation remains understudied. To our knowledge, there is only one report assessing automated VTI (VTIauto) accuracy, which used an animal model for hemorrhagic shock [42]. In this study, however, feasibility was low (60%) and the correlation coefficient between VTIauto and PAC thermodilution was moderate at best (r = 0.66). As a result, our study will add to the somewhat sparse body of knowledge surrounding automated VTI and hopefully pave a path for larger, structured prospective studies that can further evaluate AI approaches in POCUS for hemodynamic assessments.

There are several limitations to our study. First, manual and automated measurements of CO are highly dependent upon DLVOT, so any inaccuracies in DLVOT are amplified by the modified Bernoulli equation. This was a large source of inaccuracy in CO measurement by the trainee group and could be improved by incorporating DLVOT measurement into the automation model. All exams were also done on one standardized volunteer to standardize comparison between participants and experts. While we minimized movement and stress on our study subject, stroke volume, and therefore VTI, can change during the study period, warranting the need for more frequent control measurements by expert echocardiographers. Additionally, our choice to only study novice sonographers inherently decrease endpoint measurement accuracy when compared to experienced echocardiographers. However, we chose this population for the study because there is no standardized US training in the ICU, and we sought to shed light on this subgroup of clinicians. In addition, novice US users are likely to benefit the most from AI assistance since they lack knowledge on US standard operating procedures. We also did not assess how long it took trainees to obtain appropriate LVOT VTI images as it was beyond the scope of our study. However, previous studies with participants of similar proficiency with US found the median acquisition time to be about 2 min [18, 19]. Further studies are needed to assess whether automated processes can help hasten image acquisition time. Finally, our study is small, with only 28 participants assessing VTI on one study patient who was healthy and hemodynamically stable. Therefore, future studies are needed to assess whether AI-assisted POCUS is a viable option in a clinical setting with patients with varying pathologies or experiencing active hemodynamic instability. Nevertheless, the variability in VTI measurement that we encountered between novice and expert sonographers remained significant despite a small sample size. Further evaluation of AI-augmented VTI to improve accessibility, reliability, and accuracy will be needed to fully appreciate the effects of AI on POCUS in critical care.