The hardware and software configuration, the training application itself and the evaluation of AETAR are described in this section.

AETAR is based on an anatomically correct male torso training phantom that is already used in medical training (Abdominal Examination Trainer by Limbs&Things). This is used for teaching the abdominal examination. It provides a flexible plug-in system for different organs allowing for the simulation of various diseases. A small fat pad and no organ variations were used to simulate a healthy human being. For AR representation, a high-resolution 3D model of the medical training phantom was created using tools based on structured-light imaging provided by the ProAV suite of ProjectionTools (domeprojections.com GmbH, Germany). The HoloLens 2nd generation (HL2, Microsoft Corporation, USA) was selected to create the AR environment as it provides freedom of movement and is widely used for AR in the medical context [18]. It was also used for its eye, hand and marker tracking abilities as well as optical see through design to facilitate usage in a clinical environment. Unity (Unity Technologies, USA) was used for the implementation. Interaction and interactive elements were implemented using the Mixed Reality Toolkit V2Footnote 2 in Unity. Image tracking for registration of the medical training phantom was implemented with Vuforia AR SDK (PTC Inc, USA) integrated in Unity.

Layout of AETAR in the preparation phase a: 1 - floating menu, 2 - medical training phantom and the AR phantom, 3 - registration marker (3). Additionally, two red indicators are displayed to access information. For auscultation b, the AR stethoscope had to be placed on the marker, upon which a circulating timer of 30 s was displayed. Guiding lines , subdividing the abdomen into quadrants, are displayed to visualize the examinations structure. These were also used for the percussion c. Here, the circular marker was intended to be pressed like a button with one hand, while performing a tapping movement with the other. For the palpation d, the abdomen was divided into nine regions, also represented by guiding lines. An orange circle visualized the center of pain. The circular marker in this case was intended to be pressed with both hands at once. The smooth virtual outline of the hands was added for visibility in the photos. Further impressions are available in the supplementary video.

AETAR was developed in cooperation with the University Medical Center of the Johannes Gutenberg University Mainz. First, a detailed flowchart of the abdominal examination was created based on medical textbooks [19,20,21] and instructional videos provided by the clinical partners. To ensure the completeness and correctness of the extracted information, the clinical workflow was then verified with clinicians teaching the examination. The content is consistent with the instructions provided at the University Medical Center of the Johannes Gutenberg University Mainz, so small differences may occur with other institutions.

Based on the documented workflow, areas where AR can support learners were identified. Together with teaching clinicians, AR concepts were defined and validated with regard to their conceptual suitability for medical teaching. A program flowchart was created from the concepts and the underlying clinical structure and this was verified again with the clinicians. Then, a first prototype was implemented. In a qualitative assessment, this was shown to three physicians and one medical student to obtain initial comments on usability and content appropriateness. Any errors and usability problems that were identified were then corrected, resulting in the application now presented.

AETAR is composed around a torso-only training phantom. To create context for symptoms on head and limbs and to generate a more realistic setting, the AR phantom, a simplified full body 3D model, is provided. A floating menu included a status display and interaction elements to navigate the application. It included forward and back buttons, access to the main menu and tracking controls. A registration marker was placed next to the medical training phantom. The layout is displayed in Fig. 1a. Buttons and indicators can be pressed directly or via airtap. A pinch gesture is used to grasp objects.

Teaching content was linked to body regions using pin indicators (see Fig. 1a). Indicators were presented in three ways: red, for important information, yellow, for more advanced knowledge not required by the curriculum, and semi-transparent for already selected pins. Triggering an indicator opened a popup (see Fig. 2), which included a short explanation text and, optionally, an image, audio sample or further information accessible by a button. Popups were placed in front of the user at a distance of about 60 cm and followed larger head movements. After selecting all indicators, an automatic query was made whether the next step should be started. Users could also navigate to the next step via the floating menu at all times.

Patient Preparation and Inspection The first step of the application was the Preparation. It taught how a patient should be positioned for a relaxed abdominal wall. The AR phantom was presented correctly positioned, with important aspects highlighted using indicators. This step was also used to familiarize the user with AETAR by introducing basic interaction elements and the handling of the application.

In the subsequent Inspection step, external signs of abdominal diseases were presented on the phantom. Due to the large variety of symptoms to consider, the exploration mode was subdivided into three clusters: deformations, cutaneous signs of diseases, and scars and hematomas.

Auscultation Training In the Auscultation step, an AR stethoscope was used to teach how to examine the patient’s bowel sounds. For this purpose, the abdomen was divided into quadrants for the examination, for which guiding lines were displayed (see Fig. 1b). To indicate directions for practical examinations, a circular marker was used (see Fig. 1b for more details). It was successively displayed in each quadrant to show where the AR stethoscope had to be placed. If a collision detection using Unity capsule colliders indicated correct positioning, a thirty-second timer with a radial loading bar was displayed and normal bowel sounds were played. After learning the process, possible pathological findings were taught and, if available, supported by audio examples.

Percussion Training In the Percussion step, the evaluation of tapping sounds for examining the abdomen was taught. The circular marker had to be pressed like a button, guiding through the quadrants marked by guiding lines (see Fig. 1c). In addition, liver size determination, ascites diagnosis and percussion of the flank lines were addressed. A new feature was introduced: the gaze target was used to teach users to maintain regular eye contact to check for pain caused by percussion. It required the user to look at the face of the AR phantom at least every ten seconds. To provide feedback and to motivate users to look at the head, a red balloon was displayed that grew larger while the user looked elsewhere (see Fig. 2). When the balloon was focused, it disappeared. If the user did not look at the face in time, a reminder was displayed (see Fig. 2).

Palpation Training In the Palpation step, palpation of organ changes and identification of pain points were taught. This was done in two steps: first, general palpation based on the nine abdominal regions (see Fig. 1d) was demonstrated. This was followed by more information on examination strategies for appendicitis and gallbladder evaluation. For the normal palpation, a distinction was made between superficial and deep palpation, in which different degrees of pressure are applied. Starting from the region farthest from the center of pain, users were guided through the examination twice in nine steps, using the circular marker. They were instructed to perform palpation movements on the circle. The movements were not taught or explained in detail, as our clinical partners stated in the workflow analysis that they were already known from other previous courses. The gaze target was also used here. The signs of appendicitis and the gallbladder examination were then presented in the next sub-step, using indicators. Pain points, such as McBurney, were displayed directly on the phantom. Due to the design of the medical training phantom, these were taught theoretically only and there was no practical exercise.

Concept and AR display of the gaze target. The concept on the left shows the AR phantom (black outline), the medical training phantom (blue) and the gaze target (red). The gaze target is intended to teach the user to focus a patients face regularly during percussion and palpation so as not to miss signs of pain. A red balloon was placed on the face, which grew larger over a time period of ten seconds. If the user looked at the face, the gaze target was reset. If the face was ignored for more than ten seconds, an info popup appeared

A mixed-methods exploratory user study was conducted to evaluate the suitability of the application for teaching the abdominal examination.

Sample Design Participants who had already covered the abdominal examination in their studies were invited. As AETAR is intended to be an addition to current teaching, students would be expected to be familiar with the matter. Participants were recruited at the University Medical Center of the Johannes Gutenberg University Mainz.

Twelve students (7f, 5 m) aged 23 to 31 (median = 27) participated in the study. They were in their fourth to sixth year of study (median = 4). Most had previous experience in performing the abdominal examination on patients beyond the lecture, but were rather inexperienced with the medical training phantom. Prior experience with AR was generally nonexistent.

Variables The usability of the application was quantitatively assessed using the system usability scale (SUS) [22]. It was applied after each examination step to evaluate the usability and potential for improvement at each stage. Thus, five subscores were collected: patient preparation (\(SUS_\)), inspection (\(SUS_\)), auscultation (\(SUS_\)), percussion (\(SUS_\)), and palpation (\(SUS_\)). The average value reflected the overall application: \(SUS_\).

A knowledge test was developed in advance of the study with teaching clinicians to assess the students’ familiarity with the abdominal examination. Six questions about the examination procedure, cutaneous signs of liver diseases, appendicitis signs, auscultation process, and patient preparation were included (see supplementary material). On a self-assessment questionnaire, participants were asked to rate their knowledge of the examination in total and of each step, as well as their confidence in performing the examination on the phantom and on a patient on 5-point Likert items ranging from 1 - ”very uncertain/ unconfident” to 5 - ”very certain/ confident”. Both questionnaires were completed twice: once before (pre) and once after (post) going through the application.

Additional qualitative feedback was gathered in a final semi-structured interview. Participants were asked for general comments and for feedback in three categories: interaction, application adequacy, and teaching perspectives.

Procedure First, the participants were informed about data protection and the purpose of the study. Written consent and demographic data were then collected. Next, the pre-iterations of the knowledge test and the self-assessment were completed. The participants were asked to put on and adjust the HL2. Eye calibration was performed and the HoloLens Tips app provided by Microsoft was launched to consistently teach interaction with AR content. Then, participants were asked to launch AETAR from the start menu. The basic setup of and interaction with the application was explained. Participants were asked to go through the patient preparation step at their own pace and to voice any comments or questions. After completing the patient preparation, the participants completed the SUS for this step. If there were no further questions about the study process, this procedure was repeated for each subsequent examination step. Any comments made during the study were recorded. Depending on the statement, technical assistance was provided, recommendations were given or further questions were asked. After completing the application, participants were asked to complete the post-iterations of the knowledge test and self-assessment questionnaire. Then, the semi-structured interview was conducted. On average, one session lasted one and a half hours.

Visualization of the SUS results (diamond = mean, error bars = standard deviation) with the adjective SUS rating scale based on Bangor et al. [23]

Complementary to the students’ perspectives, the correctness and appropriateness of the application for medical teaching were evaluated with expert interviews. Two physicians who were not directly involved in the development process participated. One of them already had previous experience with AR, the other none at all. Both perform the abdominal examination regularly and know the challenges of teaching it.

After a brief introduction to interaction with the HL2, they were then asked to complete AETAR at their own pace, making comments and suggestions as they went. Technical assistance was provided if necessary. Additionally, questions were asked about the correctness of the information presented, its use in education, and the presentation of the teaching content. Finally, a short semi-structured interview was conducted, with a focus on the suitability for teaching and the teaching setting.