Available online 30 May 2023

Laparoscopic suturing (LS) is a challenging laparoscopic skill to teach. Its complexity and nuances are not modeled or measured in current simulation and assessment platforms.The script concordance test (SCT) is used to assess clinical reasoning.The purpose of this study is to provide evidence for validity of this novel SCT based online assessment for LS skills.

We designed a video-based online SCT for LS using a cognitive task analysis and expert panelists.The CTA yielded 4 LS domains: needle handling (NH), tissue handling (TH), knot tying techniques (KT) and operative ergonomics (OE). Five-point scales with anchoring descriptors from -2 to +2 were used. Scoring was based on a modified SCT methodology.

The test was administrated to 37 subjects (18 experts and 19 novices). There was no time limit given. A different expert group from the minimal invasive surgery (MIS) panelist were recruited. Experts were defined as surgeons and fellows with LS experience of >25 cases annually. Validity was assessed by comparing SCT scores of experienced and inexperienced surgeons. Cronbach's alpha was used to assess the internal consistency of the test.

The survey started off with 47 questions in each of the following domains: 13 NH, 4 TH, 20 KT and 10 OE. Thirty-seven surgeons (18 experts and 19 inexperienced surgeons). Questions that demonstrated a large discrepancy among experts and panelists with a weighted score difference more than 40 were discarded (n = 20). One question was discarded because it received a 100% score from all participants. This yielded 26 remaining questions in the following domains: 8 NH, 2 TH, 11 KT and 5 OE. The test reliability level (Cronbach a) was 0.80. The mean score was 72 ± 9% and 63 ± 15% (p = 0.02) for experts and inexperienced surgeons, respectively. The mean time to complete the test was 21 minutes.

This study provides validity evidence for a novel intraoperative LS assessment. The variability of responses between experts and panelists suggests that SCT may capture the clinical differences/surgeon preferences in performing LS intraoperatively.

Many surgical residents, even in senior years, consider LS to be challenging. Current surgical training for laparoscopic skills, including intracorporeal LS, is partly based on simulation training that focuses more on teaching technical skills rather than on the cognitive and judgment-related aspects of the skills. A previous qualitative investigation that studied the need for LS training revealed 6 unmet training needs for LS: task complexity in the clinical environment, training misalignment,

`The first stage of the study involved item selection and test development. The main goal of the SCT methodology was: (1) represent an actual operative scenario when performing LS with reasonable options, (2) use a Likert scale that reflect operative reasoning while performing LS, and (3) use a scoring system based on aggregate scoring methods that considers the variability of the experts’ response.

A modified cognitive task analysis (CTA) was performed to define the skill set. 13 MIS panelist

The CTA showed several domains of LS needed for intraoperative decision making while performing the task. Several rounds of data collection were conducted and continued until no new information or domains in LS are being generated and with adequate number of generated codes is achieved. LS domains identified were: needle handling, tissue handling, knot tying technique, and operative ergonomics. (Table 1)

Test optimization was performed by having 18 LS novice (6 MIS fellows, 3 PGY5, 3 PGY4, and 6

Historical teaching methods in surgical education depended solely on intraoperative training. This form of teaching is now limited by the recent residents’ work hour restriction, limited OR time, and the high cost to train residents in the OR.19,20 Skills acquisition in surgical training is also challenged by the continuous advancement in surgical procedures that requires an increase level of proficiency in laparoscopic skills, such as LS. Surgical simulation training provides more effective

In this study, we show that this novel video-based assessment may help assessing the cognitive aspects of operative skills for LS. This test appears to be internally consistent and have good test-rater reliability. The variability of responses between experts and panelists suggests that SCT may capture the clinical differences/surgeon preferences in performing LS intraoperatively. SCT can be used as a formative and summative tool to assess intraoperative decision-making skills may be feasible.

The authors declared no conflicts of interest.

© 2023 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.