Available online 21 October 2022

Background

The characteristics of quality feedback from the neurosurgery resident's perspective are not fully elucidated. The Surgical Autonomy Program is an intraoperative assessment tool based on Vygotsky's Zone of Proximal Development (ZPD). SAP facilitates assessment of a resident's operative performance accompanied by written feedback.

Objective

The goal of this study was twofold: to identify themes from the written feedback of SAP operative assessments and to examine if these themes influenced the neurosurgery residents’ perception of feedback quality.

Methods

In 2021, SAP data from 2019-2021 at two neurosurgery programs were reviewed. Feedback quality from the SAP was determined by the resident at the time of their assessment. Using a constant comparative technique, the feedback was coded using a thematic analysis. The quality of feedback within each code was analyzed.

Results

There were 2968 SAP entries evaluated. When the ZPD concept was fully used, residents reported high quality feedback 91.4% of the time compared to 58.6% when ZPD was not used (p < 0.001). Qualitative analysis of the written feedback revealed five themes: Non-Specific, Specific General Observations, Key Points, Next Steps, and Independent Practice. Feedback in the Specific General Observations, Key Points, and Independent Practice categories were associated with higher level feedback than leaving the space blank (p < 0.001) or writing Non-Specific comments (p < 0.001).

Conclusions

Providing comments that discuss the resident's specific performance in the case, key learning points, or their progress towards independence, results in high quality feedback. Utilizing a theory-based tool such as the SAP can provide meaningful feedback to neurosurgical residents.

Section snippetsINTRODUCTION

Feedback is a powerful educational strategy utilized across multiple clinical training platforms, including the operating room.1,2 Despite its recognized importance, feedback quality often goes under reported.1,3 While several studies acknowledge the role of feedback with intraoperative teaching, they also show that faculty and residents don't always agree on its value or if it was given.4, 5, 6, 7, 8 Feedback, with the intent to improve performance, is a critical component of competency-based

METHODS

The SAP allows neurosurgical cases to be evaluated in accordance with the ACGME Neurosurgery case log. Each case is broken down into four pre-defined zones. Within each zone, the resident is evaluated along the TAGS scale towards progressive independence Figure 1.20 This TAGS scale guides the faculty on the degree of involvement based on the resident's ability to perform that section of the case, moving across the spectrum of direct to indirect supervision. Using the TAGS scale, a resident

RESULTS

From academic years 2019-2021, there were 2968 SAP entries between the two institutions, with Duke entering 2607 assessments while the NCC had 361. Qualitative analysis of the written feedback revealed five independent themes. Thematic saturation was reached after 750 entries were reviewed. We continued coding all remaining entries for additional thematic content. Those evaluations with no written feedback (n = 904) in the comment section were removed from further qualitative thematic analysis

DISCUSSION

Using an embedded mixed methods design, this study was able to capture the complexity of feedback that is very difficult to fully appreciate with quantitative analysis alone. The actual words that are used in feedback are important and appeared to be noticed by the residents. Notably, our themes aligned with ZPD theory as well as the current literature on feedback.

Comparing themes within written feedback and evaluating them based on the neurosurgery residents’ perception of this feedback has

CONCLUSIONS

This study evaluated the resident's perception of written feedback from the operating room through the SAP system at two academic institutions. There were several key findings that influence how written feedback is perceived by neurosurgery residents. First and foremost, providing written comments that discuss the resident's specific performance in the case, key learning points or their trajectory towards independence results in higher quality feedback than leaving the written comment space

Conflict of Interest

RD, SPL, and MMH have financial interest in the Surgical Autonomy Program, Inc.

Funding Source

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Submission Declaration

This manuscript has not been previously published. It has been accepted as an oral Presentation Accepted at the American Association of Neurological Surgeons Meeting, Philadelphia PA, May 2, 2022.

Disclaimer

The views expressed in this article are those of the authors and do not reflect the official policy of the Department of the Army/Navy/Air Force, Department of Defense, or the US Government

REFERENCES (25)ES Holmboe et al.Reflections on the First 2 Years of Milestone Implementation

J Grad Med Educ

(2015)

S Ramani et al.Feedback Redefined: Principles and Practice

J Gen Intern Med

(2019)

View full text

Published by Elsevier Inc. on behalf of Association of Program Directors in Surgery.