We have developed a consensus-based multisystem POCUS skills checklist to assess basic competency in image acquisition and anatomy identification. The checklist includes 153 items to evaluate skills to perform basic cardiac, lung, abdominal, and vascular ultrasound applications, including PIV insertion, that are commonly used in emergency medicine, critical care, and hospital medicine.

Our POCUS skills assessment checklist has noteworthy differences from other checklists. Most published POCUS skills checklists focus on assessing image acquisition skills of a single organ system, such as cardiac [5,6,7,8], thoracic [12,13,14], FAST exam [9,10,11], vascular [5], neuromuscular [15], musculoskeletal [16], or procedures [17]; or assessing skills of clinicians from a single specialty, such as emergency medicine [10], surgery [9], or critical care [5]. In contrast, our checklist was based on consensus from 14 POCUS experts from emergency (5), critical care (5), and hospital medicine (4), who practice at different medical centers across the United States. The value of our consolidated checklist is it establishes a common standard for assessing skills in image acquisition and anatomy identification for basic, common POCUS applications across specialties. Institutions seeking tools to assess POCUS skills prior to granting privileges to use POCUS for clinical decision-making can use our checklist to efficiently evaluate POCUS skills of physicians from different specialties.

Our multisystem POCUS skills checklist combines the use of both checklist items and global rating scales. Checklists use task-specific items that can provide both evaluative scoring with cutoff levels for “passing” as well as formative feedback. Checklists are perceived as being easier to use, especially for non-expert assessors, and having better interrater reliability [17]. However, checklists may focus more on thoroughness rather than overall competency and may not capture a summative assessment of one’s performance [18, 19]. One approach to overcome this limitation is increasing the point-value of critical checklist items, or identifying checklist items that result in immediate disqualification from competency if performed incorrectly [18, 20]. By comparison, global rating scales provide an overall assessment of a learner’s skills and can differentiate learner levels with high reliability and sensitivity, particularly when performed by content experts [21,22,23]. For these reasons, a final global rating question was included to determine whether the learner has demonstrated minimum skills to be considered competent in image acquisition and anatomy identification to perform the specified POCUS exam on patients.

A rigorous multi-step process was conducted to develop our checklist from 2017 to 2021. Initially, speed and efficiency of image acquisition were included in the checklist. However, after pilot testing the initial version of our checklist with novices, we noted substantial variability in interpretation and application of these checklist items among expert faculty and removed them, because consensus could not be achieved on the specific wording, anchoring, and scoring of these items. In the final phase of checklist development, a standardized set of recorded skills exams of novice, experienced, and expert learners were reviewed and scored by the expert panel members independently which led to insertion of additional anchors to clarify some checklist items.

Our consensus-based multisystem checklist has limitations. First, POCUS competency requires mastery of image acquisition and interpretation, and integration of findings into clinical decision-making, which include the cognitive, psychomotor, and affective domains of learning [24, 25]. Our POCUS checklist assesses image acquisition skills and identification of normal structures, while additional assessment is needed for the cognitive domain. Second, we were unable to assess interrater reliability of our checklist due to the cancellation of live in-person courses during the COVID-19 pandemic. We plan to validate our checklist with learners after resumption of live in-person POCUS courses in the future. Third, we had to balance completeness versus efficiency when selecting views to include in a multisystem POCUS skills checklist, and although important, certain views, such as the left upper quadrant, were not included based on group consensus. Finally, we have postponed weighting of critical checklist items until validation of our checklist prospectively. We anticipate greater weighting of the final global rating question on competency for granting privileges.