Available online 4 April 2024

Author links open overlay panel, , , , , , , HIGHLIGHTS•

Observation based feedback is important for growth.

•

There are limited tools for giving residents feedback on their teaching.

•

It is difficult to implement frequent, low stakes feedback tools to give residents feedback on their teaching.

Objective

There are few assessments of the competence and growth of surgical residents as educators. We developed and piloted an observation-based feedback tool (FT) to provide residents direct feedback during a specific teaching session, as perceived by medical students (MS). We hypothesized that residents’ performance would improve with frequent, low stakes, observation-based feedback.

Setting

This prospective study took place at an academic general surgery program.

Participants

Focus groups of MS, surgical residents, and faculty informed FT development. MS completed the FT regarding resident teaching.

Design

The FT utilized 5 slider-bar ratings (0 to 100) about the teaching encounter and a checklist of 16 desirable teaching behaviors. QR codes and weekly email links were distributed for 12 months (6 clerkship blocks) to promote use. Residents were sent their results after each block. A survey after each block assessed motivation for use and gathered feedback on the FT. Descriptive statistics were used for analysis (medians, IQRs). Primary measures of performance were median of the slider-bar scores and the number of teaching behaviors.

Results

The FT was used 111 times; 37 of 46 residents were rated by up to 65 MS. The median rating on the slider-bars was 100 and the median number of desirable teaching behaviors was 12; there were no differences based on gender or PGY level. 10 residents had 5 or more FT observations during the year. Four residents had evaluations completed in 4 or more blocks and 19 residents had evaluations completed in at least 2 blocks. Over time, 13 residents had consistent slider-bar scores, 1 resident had higher scores, and 5 residents had lower scores (defined as a more than 5-point change from initial rating). Frequency of use of the FT decreased over time (38, 32, 9, 21, 7, 5 uses per block). The post-use survey was completed by 24 MS and 19 residents. Most common reasons for usage were interest in improving surgical learning environment, giving positive feedback (MS), and improving teaching skills (residents). Most common reasons for lack of usage from residents were “I did not think I taught enough to ask for feedback,” “I forgot it existed,” and “I did not know it existed.”

Conclusions

The FT did not lead to any meaningful improvement in resident scores over the course of the year. This may be due to overall high scores, suggesting that the components of the FT may require reevaluation. Additionally, decreased utilization of the instrument over time made it challenging to assess change in performance of specific residents, likely due to lack of awareness of the FT despite frequent reminders. Successful implementation of observation-based teaching assessments may require better integration with residency or clerkship objectives.

Section snippetsINTRODUCTION

Surgical trainees have a direct impact on the quality of the educational experiences of students, fellow trainees, and future colleagues in both their current and future roles.1 More effective educators theoretically contribute to improved learning and performance at all levels and this becomes both an opportunity to meet accreditation requirements as well as a practical reason to include teacher training in clinical training programs. Positive resident interactions are also linked to improved

METHODS

This study used a combination of qualitative methods to create and evaluate the tool as well as quantitative methods to assess the use of the tool and whether providing residents with feedback changed their ratings over time. This study was approved by the University Hospitals Institutional Review Board.

The Feedback Tool and all surveys were created in REDCap electronic database (Research Electronic Data Capture). REDCap is a secure, web-based software platform designed to support data capture

Creation of the Feedback Tool

Six focus groups consisting of 2-8 participants were held on Zoom over the course of a 2-month time period. There were 2 groups per participant level: general surgery faculty, general surgery residents, and medical students. The focus groups identified important characteristics of teachers on the wards, demonstrable behaviors, and also several potential pitfalls involved with the creation and implementation of the Feedback Tool (Table 1). When discussing characteristics of effective teachers,

DISCUSSION

Despite creating a Feedback Tool based on stakeholder suggestions about the attributes of an effective teaching encounter, use of this tool was limited in the context of a surgical residency. The majority of residents who did report using it (5 of 7) intended to make changes to their teaching based on the feedback they received. However, the number of observations per resident was highly variable and was not associated with improvement in teaching scores over time, however,

Our results highlight

ACKNOWLEDGMENTS

This study was completed in the context of the American College of Surgeons Certificate in Applied Surgical Education and Leadership (CASEL) Program. The authors would like to acknowledge and thank the faculty, staff, and colleagues from this program for their support and help in this work.

REFERENCES (15)

There are more references available in the full text version of this article.

View full text

© 2024 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.