Following University of Saskatchewan Research Ethics Board approval (BEH-2611, 6 May 2021), we conducted a prospective observational study assessing the inter-rater agreement of in-person airway evaluation performed by consultant anesthesiologists (consultant in-person) to VAEs performed by consultant anesthesiologists (consultant VAE), and VAEs performed by medical students (novice VAE).
Consultant in-person evaluations were performed by consultant anesthesiologists in the preoperative holding area as part of routine care. Consultant VAEs and novice VAEs were completed in an unspecified order (based on anesthesiologist availability) before or after consultant in-person evaluation (Figure). The consultant VAEs were completed by two anesthesiologists (W. M., P. H.) and novice VAEs by two medical students (J. M., M. Z.). Evaluators were blinded to each other’s findings. Data sheets were collected but not investigated until all evaluations for all patient participants were conducted. Data collection occurred between June and August 2021 in Saskatoon, Saskatchewan. Project protocols adhered to the Strengthening the Reporting of Observational Studies in Epidemiology guidelines.8
FigureConceptual diagram of methodology and participation results
Evaluator characteristicsEvaluators did not have a pre-existing relationship or known previous interaction with participants. The consultant anesthesiologists were all fellowship-trained and Royal College of Physicians and Surgeons of Canada-accredited in anesthesiology. Medical students (J. M., M. Z.) had no experience with airway evaluations prior to receiving a workshop taught by experienced consultant anesthesiologists (J. G., W. M., P. H.). Evaluators did not have prior experience with VAEs.
Sample size, recruitment, and participant populationWe targeted a convenience sample of 100 based on researcher availability during the study period. Sample size was a function of the bounded time frame of the study and the anticipated number of patients to be seen in hospital for anesthesiology consult. Eligible participants were all patients 17 yr of age and older booked for a preoperative anesthetic assessment having access to a device capable of using the Zoom for Healthcare videoconference software (Zoom Video Communications, San Jose, CA, USA).
Prospective participants scheduled for surgeries who required preanesthetic assessment and were not under care by coinvestigators (J. G., W. M., P. H.) were identified by reviewing the operating room schedules. Participants receiving consultations at the preanesthetic clinic were approached by medical students (J. M., M. Z.). A standardized consenting procedure was used. If needed, patients were given a brief standardized five-minute tutorial at the preanesthetic clinic by the medical students on how to download and use Zoom for Healthcare. We did not formally record the number of participants requiring a tutorial, nor did we assess participants’ technological familiarity. Virtual airway evaluations were conducted after the preanesthetic clinic encounter at an agreed time between the participant and evaluator. All VAEs were conducted remotely with the participants choosing a convenient location, most often their residence. Consultant and novice VAEs were conducted at different times before or after consultant in-person.
Data collectionThe airway evaluation scoring tool used was based on an airway evaluation publication,9 to which we added a single scorable item, the thyroid-mental distance, to reflect routine practice in our study centers. We evaluated ten components binarily by assigning 1 and 0 points for positive and negative findings, respectively, for a maximum total score of 10 points. These components were (1) facial trauma, (2) large incisors, (3) a beard or mustache, (4) mouth opening < three fingerbreadths, (5) thyro-mental distance < five fingerbreadths, (6) hyo-mental distance < three fingerbreadths, (7) thyro-hyoid distance < two fingerbreadths, (8) Mallampati class ≥ 3, (9) presence of an obstructed airway, and (10) poor neck mobility. Where fingerbreadth assessments were made, the patients’ fingers were used for VAEs. We considered an obstructed airway synonymous with a history of obstructive sleep apnea (OSA) or loud snoring, which is defined as louder than talking volume or loud enough to be heard through closed doors.10
All airway evaluations followed the same study protocol. Consultant in-person evaluators (anesthesiologists) were aware of any previously recorded airway management difficulties from patient participants’ medical charts; consultant VAE evaluators did not review participants’ medical charts. Each participant underwent three separate evaluations: a consultant VAE (anesthesiologists W. M., P. H.), novice VAE (medical students J. M., M. Z.), and a consultant in-person evaluation by a consultant anesthesiologist who was the participant patient’s attending anesthesiologist. Participants with incomplete assessments were excluded from data analysis.
Consultant in-person evaluationsConsenting participants were evaluated by consultant anesthesiologists as part of the preanesthetic evaluation during routine care immediately before their scheduled surgery. Prior to these in-person evaluations, the consultant anesthesiologists were contacted by email (the day prior) to introduce the data collection tool. Afterward, one of the student researchers met with the consultant anesthesiologists on the day of the scheduled procedure to review the data collection tool and answer any study-related questions. Consultant anesthesiologists did not have prior exposure to the study’s airway evaluation scoring tool. In addition to the standardized preanesthetic airway evaluation, the anesthesiologists reported if the intubation was difficult (as determined clinically by the anesthesiologist) in cases where endotracheal intubation was part of the intraoperative anesthetic management (Electronic Supplementary Material [ESM] eAppendix 1).
Consultant and novice VAEsConsultant and novice VAEs were conducted before or after consultant in-person evaluations in virtual meeting rooms hosted by Zoom for Healthcare.11 We evaluated the airway evaluation components 1 to 10 sequentially as ordered above. Directions given to the participants to optimize conditions for effective evaluation of the airway components (e.g., distance between face and screen, lighting, camera angle) were determined ad hoc by the evaluator during each evaluation. Evaluators recorded findings with visual and oral feedback from participants and collected field notes following evaluations. All evaluators used the same data collection form (ESM eAppendix 2).
Outcomes and data analysisWe tested our first hypothesis by assessing the inter-rater agreement of consultant in-person evaluations to consultant VAEs. We tested our second hypothesis by comparing the inter-rater agreement of consultant in-person evaluations to consultant VAEs against the inter-rater agreement of consultant in-person evaluations to novice VAEs (to elucidate the importance of clinical experience in airway evaluation). The inter-rater agreement for total airway scores, our primary outcome, was assessed using Cohen’s Kappa (CK). The secondary outcomes included the inter-rater agreement for each airway evaluation component of consultant in-person evaluations to consultant VAEs, and consultant in-person evaluations to novice VAEs. These secondary outcomes were assessed using prevalence-adjusted bias-adjusted Kappa (PABAK). Prevalence-adjusted bias-adjusted Kappa was used to account for disagreement between percent agreement and CK and low variability or prevalence in the data (see ESM eTable 1).12 Inter-rater agreement CK coefficients of consultant in-person total scores were compared with those of consultant and novice VAEs by calculating P values from CK 97.5% confidence intervals (CIs),13 with an alpha of 0.05. Analysis was performed with Microsoft Excel (Microsoft Corporation, Redmond, WA, USA) and IBM SPSS Statistics for Windows, version 26 (IBM Corp., Armonk, NY, USA).
留言 (0)