Available online 27 October 2022

Background

Laparoscopic intracorporeal suturing is important to master and competence should be ensured using an optimal method in a simulated environment before proceeding to real operations. The objectives of this study were to gather validity evidence for two tools for assessing laparoscopic intracorporeal knot tying and compare the rater-based assessment of laparoscopic intracorporeal suturing with the assessment based on simulator metrics.

Methods

Twenty-eight novices and 19 experienced surgeons performed four laparoscopic sutures on a Simball Box simulator twice. Two surgeons used the Intracorporeal Suturing Assessment Tool (ISAT) for blinded video rating.

Results

Composite Simulator Score (CSS) had higher test-retest reliability than the ISAT. The correlation between the number performed procedures including suturing and ISAT score was 0.51, p<0.001, and 0.59 p<0.001 for CSS. We found an inter-rater reliability (0.72, p<0.001 for test 1 and 0.53 p<0.001 for test 2). The pass/fail rates for ISAT and CSS were similar.

Conclusion

CSS and ISAT provide similar results for assessing laparoscopic suturing but assess different aspects of performance. Using simulator metrics and raters’ assessments in combination should be considered for a more comprehensive evaluation of laparoscopic knot-tying competency.

Section snippetsMETHODSEthics

The study was exempt from ethical approval by the Danish National Ethics committee (ID: H-18047718). Participants were provided with written and verbal information before written consent was obtained.

Materials and equipment

We used a Simball Box® Trainer (Simball Box, version 2015, G-coder Goteborg, Sweden), which automatically recorded and stored: Task completion time, instrument movements, instrument force used, and high-definition videos. Two 5 mm laparoscopic needle holders were used as instruments. The training station was mounted on a height-adjustable table to ensure an ergonomically correct working position (Figure 1).

A 3D printed mould was designed and used to make a customized silicone pad with two

Participants

We recruited residents and surgeons from gynaecology, urology, and surgery departments in the eastern part of Denmark. Participants were categorized as either novice or experienced based on their experience with laparoscopic suturing, classified as experienced if they had performed 25 independent procedures with laparoscopic suturing as primary surgeon. Participants reported how many procedures they had completed. None of the participants had previously participated in studies involving

Participation

The intervention consisted of a warm-up session followed by a test session where participants completed the test twice. All participants were given a standardized introduction to the simulator and a test by an investigator. A demonstration video on how to laparoscopically do an intracorporeal interrupted suture as a standardized 3-throw square knot (a double-wrap throw followed by two single throws in alternate directions) was shown to participants.

Subsequently, all participants were allowed to

Simulator metrics

The laparoscopic instrument movements and time spent were recorded by Simball Box Trainer software. A compatible Simball Box Trainer force sensor was used to detect direct force in three directions. The simulator metrics are shown in Table 2. Instrument movements, video recording, and force used were automatically stored for each participant.

Video rating

We reviewed the literature and chose a rater-based assessment tool - Intracorporeal Suturing Assessment Tool (ISAT) (Supplementary Material) – which had

Statistical analysis

All analyses were performed using SPSS (IBM SPSS Statistics for Macintosh Version 26.0, IBM Corp., Armonk, NY, USA). We used log-transformed data on the numbers of laparoscopic procedures with independent suturing. The simulator metrics, which could discriminate between the novices and the experienced surgeons, were converted to z-scores using the mean of the experienced group because they were measured in different units, allowing them to be combined into on composite score reflecting the

Internal structure

The six significant simulator metrics (Table 3) strongly correlated with Pearson's r >0.7, giving a high test-retest reliability. The internal consistency reliability (Cronbach alpha) for simulator metrics converted to Z-scores was 0.53. Test-retest reliability for the CSS was 0.87, p<0.001 and 0.64, p<0.001 for the ISAT.

The good test-retest reliability shows the consistency of results when the same test is repeated at a different point of time.

Inter-rater reliability was analyzed by ICC

Consequences

The pass/fail was 23 for the ISAT and -0.8 for the CSS. Figure 3 shows how the pass/fail levels were defined. Seven novices and 15 experienced passed the ISAT pass/fail level, and eight novices and 20 experiences passed using the CSS. Pass/fail rates and sensitivity and specificity are shown in Table 4.

DISCUSSIONWe established validity evidence for both the ISAT and CSS and identified a pass/fail level for both. The ISAT had moderate inter-rater and intra-rater reliability; the simulator

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

References (36)M Tonutti et al.The role of technology in minimally invasive surgery: state of the art, recent developments and future directions

Postgraduate medical journal

(2017)

F Bjerrum et al.Randomized trial to examine procedure-to-procedure transfer in laparoscopic simulator training

British journal of surgery

(2016)

NO Kolozsvari et al.Mastery versus the standard proficiency target for basic laparoscopic skill training: effect on skill transfer and retention

Surgical Endoscopy

(2011)

S Lim et al.Laparoscopic Suturing as a Barrier to Broader Adoption of Laparoscopic Surgery

JSLS

(2017)

LL Lucian et al.What Practices Will Most Impove Safety? Evidence-Based Medicine Meets Patient Safety

The Journal of the American Medical Association

(2002)

RK Reznick et al.Teaching Surgical Skills — Changes in the Wind

New England Journal of Medicine

(2006)

E Bilgic et al.

Simulation platforms to assess laparoscopic suturing skills: a scoping review

(2019)

RA Agha et al.The role and validity of surgical simulation

International surgery

(2015)

View full text

© 2022 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.