This study protocol was approved by our institutional review board. Participants were voluntarily prospectively enrolled from a radiology residency, neuroradiology fellowship, and neuroradiology faculty at a single tertiary care academic medical center. Enrollment was open to radiology residents at all levels of training. Because the project was conducted in the spring, all participating residents had at least 9 months of residency training, including at least 6 weeks of neurologic CT and 1 week of night float call experience at the start of the project. Each participant agreed to participate in the pre-assessment, pre-survey, educational session, post-assessment, and post-survey. Each participant was assigned a unique numerical identification, and identifying information was kept on a separate spreadsheet so that data analyses could be performed blinded to identifying information.

Evaluators were voluntarily recruited from the same institution’s otolaryngology residency program. To qualify for participation, evaluators had to be in their second postgraduate year of training (PGY-2) or higher and had to have participated in facial trauma call and facial trauma surgery. Evaluators agreed to provide blinded assessments of participants’ reports based on a provided rubric and answer key.

Two neuroradiologists (KDH, WTM) collaborated to create a one-hour educational session on facial trauma, incorporating input from a plastic surgeon with facial trauma expertise (CMR). In keeping with common surgical characterization schemes, complex facial trauma was organized into upper, middle, and lower thirds with the midface comprised of 5 functional subunits (nasoseptal, NOE, ZMC, internal orbit, and occlusion-bearing maxillary). Upper face and midface trauma, including discussion of the midface subunits, were covered by the lecture. The lower face (mandible) was not included in this educational session, in part due to time constraints and also to serve as a control for the pre- and post-participation analyses discussed below.

The educational session was scheduled over the noon hour on two separate dates in a two-week period to maximize the potential for participation. The session consisted of an hour-long lecture utilizing PowerPoint (Microsoft Inc., Redmond, WA, USA) in a format typical for the regular noon hour educational sessions delivered at our institution. There were no competing lectures during these periods. No food or other incentives were provided. Nonparticipants were permitted to attend these sessions.

Pre- and post-surveys were designed to assess participants’ perceived competence and comfort with interpreting complex facial trauma on unenhanced CT overall and by individual facial subunits. The individual facial subunits included were: upper face, zygomaticomaxillary complex (ZMC), nasoorbitoethmoid complex (NOE), internal orbit, nasoseptal, and lower face, in keeping with those defined in the literature. Le Fort was also listed as a subunit, as this classification plays a role in defining the occlusion-bearing fragment of midface fractures and is useful in the clear communication of midface trauma [2, 5, 15]. Surveys included a reporting of participants’ familiarity with the referring surgeons’ expectations from radiology reporting of facial trauma and the perceived actionability of their current reporting styles. Participants were instructed to complete the pre-survey prior to starting the pre-assessment cases and to complete the post-survey upon completion of the post-assessment cases (Fig. 1).

Graphical chronology of the project design

For the pre- and post-assessments, 6 complex facial trauma cases were selected, comprising 3 pairs of similar cases. All cases were unenhanced face CTs including standard algorithm axial images, thin bone algorithm (0.625 mm slice thickness) axial images, and coronal and sagittal bone algorithm reformatted images. The first pair was selected specifically to include both multilevel Le Fort fractures and mandibular fractures, the second to include Le Fort fractures and NOE fractures, and the third to include internal orbital fractures. Representative 3D rendered CT images from the second pair cases are shown in Fig. 2. All cases were deidentified and made available in a shared folder on our institutional PACS (IntelliSpace PACS, Philips, Amsterdam, The Netherlands). Accessing cases in this environment allowed participants to interpret the case in the same manner as their normal daily workflow and enabled them to use the paired 3D rendering software (TeraRecon, Foster City, CA, USA) as needed to aid with interpretation.

3D rendered CT images from the second pair of assessment cases. Both cases demonstrate LeFort, NOE, and frontal sinus fractures with different degrees of comminution and levels of involvement. These cases were randomized to pre- and post-lecture assessments to evaluate for improved reporting of these subunits

Participants were assigned 1 case from each pair (3 total) as pre-assessment cases and the other case from each pair (3 total) as post-assessment cases. They were instructed to complete the pre-assessment cases within a 3-week period preceding the educational session and to complete the post-assessment cases within a 3-week period following the educational session (Fig. 1). Assignments were made such that each case was equally used as a pre- and post-assessment case across participants. For each case, participants were instructed to input their numerical ID, the case number, and a report in free text format into a standard data collection form made using Microsoft Forms (Microsoft Inc., Redmond, WA, USA).

For each of the 6 assessment cases, an answer key was created through the joint efforts of an attending neuroradiologist (KDH) and an attending plastic surgeon who specializes in facial reconstructive surgery (CMR). After collection of all pre- and post-assessment data, evaluators were each provided a set of cases anonymized to participant information and to whether the case was taken as part of a pre- or post-assessment. For each case, they were instructed to use the provided answer key to evaluate the accuracy and clarity of the description of each facial subunit and to assess the overall report accuracy, clarity, actionability, and organization (Table 1). Each metric was rated on a 5-point Likert scale. Evaluators compared the participant reports against the provided answer key and did not review the CT images.

Evaluators were asked to independently assess an interrater agreement set of 15 reports in addition to their assigned sets. This set included at least 2 reports for each of the 6 unknown cases and included 8 pre-assessment and 7 post-assessment reports submitted by participants distributed across the spectrum of levels of training. Grades assigned for each of these 15 reports were used to assess interrater agreement.

Pooled group responses on pre- and post-surveys and performance on pre- and post-assessments were compared using the Wilcoxon signed-rank test. Kendall’s W was selected to test interrater reliability. Statistical analyses were conducted using R, version 4.2.1 (R Project for Statistical Computing), with statistical significance defined as p < 0.05.