Cross-sectional evaluation of an asynchronous Multiple Mini Interview (MMI) in selection to health professions training programmes with ten principles for fairness built-in

Abstract

Objectives Ensuring equity, inclusivity, and diversity in health professions selection is an ethical and practical imperative. We have built the first known online asynchronous Multiple Mini-Interview (MMI). We aimed to explore psychometric properties for all users with sub-group analysis by key characteristics, acceptability, and usability. Design, setting, participants. Cross-discipline multi-method evaluation with applicants to Nursing, Midwifery and Paramedic Science under-graduate programmes from one UK university (2021/2022). Primary, secondary outcome measures Psychometric properties (internal consistency, construct validity, dimensionality) were assessed using Cronbachs alpha, parallel analysis (PA), Schmid-Leiman transformation and ordinal confirmatory factor analysis (CFA). Usability and acceptability were evaluated using descriptive statistics and conventional content analysis. Methods The system was configured in a seven question four-minute MMI. Applicants video-recorded their answers which were later assessed by interviewers and scores summed. Applicants and interviewers completed online evaluation questionnaires. Results Performance data from 712 applicants determined good-excellent reliability for the asynchronous MMI assessment (mean alpha; 0.72) with similar results across sub-groups (gender, age, disability/support needs, UK/non-UK). Parallel analysis and factor analysis results suggested that there were seven factors relating to the MMI questions with an underlying general factor that explained the variance in observed candidate responses. A confirmatory factor analysis testing a seven-factor hierarchical model showed an excellent fit to the data (Confirmatory Fit Index =0.99), Tucker Lewis Index =0.99, RMSE=0.034). Applicants (n=210) viewed the flexibility, relaxed environment, and cost savings advantageous. Interviewers (n=65) reported the system intuitive, flexible with >70% time saved compared to face-to-face interviews. Reduced personal communication was cited as the principle disadvantage. Conclusions Our findings suggest that the asynchronous MMI is reliable, time-efficient, fair, and acceptable. In the absence of any known precedent, these internationally applicable, cross discipline insights inform the future configuration of online interviews where building-in principles for fairness are relatively straight forward to implement.

Competing Interest Statement

Authors AC and LG are co-founders and Ach is an employee of Sammi-Select Lt, funded by UKRI as a spinout company from the University of Surrey to build the platform.

Funding Statement

This study was funded in part by Innovate UK

Author Declarations

I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.

Yes

The details of the IRB/oversight body that provided approval or exemption for the research described are given below:

The University of Surrey Research Ethics Committee gave ethical approval for this work (UEC/2017/111/FHMS). All participants sharing personal data gave informed consent.

I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.

Yes

I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).

Yes

I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.

Yes

Data Availability

All data produced in the present study are available upon reasonable request to the authors. Assessment criteria are withheld due to university interview test security.

留言 (0)

沒有登入
gif