Trial design

The present study is a mixed-methods, comparative interrupted-time series (CITS) study [40, 41] to compare the implementation metrics, drivers and outcomes, effectiveness, and cost-effectiveness of two bundled implementation strategies to deliver GlobalConsent to men attending six pair-matched universities in North, Central, and South Vietnam. The Framework for Reporting Adaptations and Modifications-Expanded (FRAME) [36] guides the aim 1 implementation assessments. The RE-AIM [24, 25, 42] and Proctor et al. [32] frameworks guide the aim 2 implementation drivers and outcomes and aim 3 implementation effectiveness assessments. A micro-costing approach guides the aim 4 cost-effectiveness assessment [43, 44].

Study setting

Table 1 provides details on the six universities taking part in this proposed study, including location, year founded, programs of study, and faculty/student population sizes. Partnering universities have experience collaborating on studies supported by major funders, including the National Institutes of Health (NIH), World Health Organization (WHO), United States Agency for International Development (USAID), and the President’s Emergency Plan for AIDS Relief (PEPFAR)/Substance Abuse and Mental Health Services (SAMHSA). Several universities are partners in an ongoing science leadership program in violence prevention [45]. Several also have partnered with the primary institutions (Emory University and the Center for Creative Initiatives on Health and Population [CCIHP]) on prior projects and, thus, have experience with the kind of collaboration proposed here. Lastly, across universities, there is substantial interest in sexual violence and intersecting health-related topics, including HIV prevention, sexual and reproductive health, general violence prevention, and preventive medicine. These interests and strong letters of support bode well for ensuring a strong commitment to the proposed project.

Table 1 Characteristics of participating universities in VietnamEligibility criteria University leaders (aim 1)

We will sample university leaders purposively to identify those most knowledgeable of and critical to the implementation of GlobalConsent [46, 47]. To the extent possible, leaders will be matched on position and rank across LIS and HIS groups (e.g., Dean/Vice Dean and Department Head/Deputy Head). If necessary to achieve saturation, we will use snowball sampling based on recommendations from interviewed stakeholders to identify additional leaders to serve as key informants [48].

Implementation team members (aims 1, 2, and 4)

Implementation team members will be sampled purposively to identify those with the most relevant expertise for the implementation of GlobalConsent. To the extent possible, implementation team members will be matched on position and rank (e.g., lecturer and/or staff by rank) across LIS and HIS groups.

University faculty (aim 2)

All fulltime, permanent lecturers who are currently working (not on extended leave) and not in the leader or implementation team samples at the time of the baseline faculty survey will be eligible to participate. The list of eligible lecturers will be refreshed before each survey wave to ensure that all eligible lecturers are included at each wave.

First-year male students (aims 1 and 3)

Eligible student participants will be male (sex assigned at birth), 18–24 years old at first contact, self-identified as heterosexual or bisexual (are attracted to women), and matriculating into the study universities in project year 2.

Who will take informed consent? Key informant interviews (KIIs) and focus group discussions (FGDs) and reflections (aim 1)

Researchers at CCIHP who will conduct the key informant interviews and who will facilitate the focus group discussions will obtain consent from each participant. Because written informed consent is not considered suitable for this setting, local interviewers and group facilitators will digitally record verbal informed consent with a witness before starting the qualitative interviews and group discussions. Informed consent for the qualitative research will require a clear understanding of the study’s purpose; voluntariness, nature, extent and duration of participation; procedures to ensure confidentiality; and right to not answer questions or to withdraw from the study at any time. Participants will be informed that, with permission, interviews will be digitally recorded, and interviewers will keep field diaries of their observations and experiences with participants. All qualitative data collection will be conducted in private settings at the study sites, where the interviewers and facilitators will provide more detail about the exact nature of the study, procedures, and any expected risks and benefits to eligible participants. If necessary, interviews and focus group discussions may be conducted via HIPAA-compliant video-conferencing software. Participants will be informed that (1) digital recordings and fully de-identified Vietnamese and English transcripts will be uploaded to separate folders on a HIPAA-compliant, secure network drive maintained by Emory University, with access limited to the study team for a specified duration before being destroyed; (2) that analysis of the transcripts will take place on a secure, password-protected computer in a private space that can be locked; and (3) that all participants will be compensated for their time and will be offered refreshments (for in-person interviews and group discussions). Participants also will be informed that the study team will recontact them at a later point in the study for follow-up interviews and/or focus group discussions (Appendix).

Online quantitative surveys (aims 2–3)

Eligible participants in the quantitative portions of the study will read an informed consent form provided in an online REDCap survey [49]. Eligible participants will indicate with check boxes that they have read each paragraph in the consent form and will be provided with a phone number to call if they have any questions that a non-study team member can answer. After all paragraphs are checked, the eligible participant will be invited to provide a response to confirm their consent or non-consent to participate in the study. Informed consent will be obtained before participants will be allowed to view and to participate in the online REDCap survey for which they are eligible (Appendix).

Costing surveys (aim 4)

CCIHP staff (including external facilitators and administrative staff involved in GlobalConsent) and university GlobalConsent implementation team members will complete costing forms regularly, either monthly or weekly, depending on the implementation phase. These individuals will receive a consent form to sign, indicating their agreement to participate. The consent form will include an information sheet, a statement confirming that they have read and understand the information sheet, a statement agreeing to participate in the survey, and contact information for a person who can answer any questions or concerns. The information sheet will cover the purpose of the costing surveys, how the data will be collected, used, and stored, the voluntary nature of participation, an assurance of confidentiality, risks and benefits of participation, and procedures to manage any risks (Appendix).

Additional consent provisions for collection and use of participant data and biological specimens

There are no additional consent provisions for the collection and use of participant data in this study. No biological specimens are being collected.

InterventionsExplanation for the choice of comparators

A local implementation team at each participating university will deliver the GlobalConsent intervention. This study will vary the implementation strategies that are delivered, with some universities using lower-intensity implementation strategies (LIS) and other universities using higher-intensity implementation strategies (HIS). Implementation strategies in the LIS group model standard approaches to deliver online sexual violence primary prevention programs in universities in the USA. Implementation strategies in the HIS group were selected from those used in the GlobalConsent efficacy trial [33, 50, 51], from the literature on learning collaboratives [34], and to address barriers and facilitators to program implementation that are common to university settings in Vietnam [35] (Table 2). Nomenclature for specific implementation strategies follows the Expert Recommendations for Implementing Change (ERIC) project [30]. The LIS and HIS implementation strategies are discussed by cluster and organizational level, where key stakeholders were identified to understand the implementation process (aim 1), drivers and outcomes (aim 2), and effectiveness (aim 3).

Table 2 “Lower” and “higher” intensity strategies to implement GlobalConsent with undergraduate men, six universities in Vietnam Develop interrelationships between stakeholders and engage university leaders

Given the demonstrated efficacy of GlobalConsent and the need for high-level institutional commitment for successful implementation, some strategies with university leaders [52] (Deans, Vice Deans, Department heads, Deputy heads) are common to both groups, and the HIS group will receive additional strategies (Table 2). Implementation strategies in both IS groups include prework to obtain formal commitments [30, 34], passive external web-support with educational materials [34], and pre-implementation educational outreach [34]. Prework, led by CCIHP, involves invitations to each university to take part, a written summary of the proposed project, site-specific dialogue, and written commitments to participate. External web-based support typically is available to US universities delivering online sexual violence prevention programs [53, 54]. Our Vietnam-based website will include links to the open-access GlobalConsent study protocol and impact assessments [33, 50, 51], short videos providing an overview of sexual violence and GlobalConsent in Vietnam and explaining findings in lay terms, one-page briefs with overviews of GlobalConsent and sexual violence among young people in Vietnam, one-page briefs about these findings and sexual violence among young people in Vietnam generally, and answers to frequently asked questions. The 2-h pre-implementation educational outreach led by CCIHP will address misinformation about sexual violence as uncommon, the normative climate for evidence-based prevention, and a description of the GlobalConsent program.

Engage potential consumers (university student users of GlobalConsent)

Internal facilitators in both IS groups will prepare students to become active consumers of GlobalConsent with an email introduction to the program, process and schedule for delivery, procedures for data collection, and consent to take part.

Train and support internal implementation facilitators and teams

Internal implementation facilitators and teams in both IS groups will receive passive external web-support with educational materials and 3 days of in-person pre-implementation manualized educational outreach on how to deliver GlobalConsent to eligible undergraduate men at their university (Table 3). The 3 days of in-person training will cover procedures about how to: maintain records on program adoption and penetration among students; identify and invite eligible students to complete an online informed consent and, if completed, a series of short surveys; and send text and email reminders to complete each module within two weeks.

Table 3 External training and support activities provided by CCIHP for implementation teams delivering GlobalConsentIntervention description

Implementation strategies in the HIS group include additional strategies (1) to develop inter-relationships between stakeholders and to engage university leadership and (2) to engage potential consumers (student users) of GlobalConsent (Table 2). CCIHP will provide additional training and support to the implementation team facilitators and members who are delivering GlobalConsent, in service of carrying out the intervention (Table 3).

Develop interrelationships between stakeholders and engage university leaders

Additional implementation strategies to develop interrelationships and to engage university leaders only in the HIS group will include internal-facilitator efforts to inform local opinion leaders [34] about implementation progress via regular emails to university leaders. Additional efforts to inform local opinion leaders will involve meetings with the university faculty (organized and led by university implementation teams) about sexual violence as a problem and prevention with GlobalConsent. Post-implementation educational outreach will entail a 1-h webinar jointly organized by CCIHP and internal implementation teams sharing anonymized findings by IS group and discussing plans to sustain GlobalConsent with future cohorts of university men.

Engage potential consumers (university student users of GlobalConsent)

Students only in the HIS group additionally will receive the following: educational outreach in a pre-implementation in-person orientation to GlobalConsent covering similar topics and three monthly 1-h learning sessions during implementation in which technical questions about program access or progression can be addressed, more intensive intervention to enhance adherence with more frequent email/SMS completion reminders (weekly versus every 2 weeks for 12 weeks), and demand generation with an option to enter a lottery to win prizes upon program completion.

Train and support internal implementation facilitators and teams

Internal implementation teams and facilitators only in the HIS group additionally will receive leadership training before implementation and external support and technical assistance (TA) during implementation. The 2-day leadership training will cover skills needed to champion GlobalConsent with diverse internal stakeholder groups. Topics will cover leadership styles, managing teams, influence without authority, managing conflict, emotional intelligence, negotiation, and leading institutional change. The leadership training also will cover effective ways to facilitate a student orientation to GlobalConsent, facilitate faculty/staff town halls (outreach sessions) about sexual violence, and send effective communications to leaders on implementation progress (Table 3). Ongoing external support and TA will involve six 1-h quality-improvement team consultations with CCIHP to provide refresher training, discuss implementation progress and modifications, build peer networks, and discuss anonymized data on implementation progress for shared problem-solving. Consultation sessions will be recorded and later coded as part of data collection activities.

Criteria for discontinuing or modifying allocated interventions

CCIHP will encourage adherence to the implementation plan of the LIS and HIS groups in response to any questions that are posted to the GlobalConsent website regarding deviations to the implementation plan. CCIHP also will encourage adherence to the implementation plan of the HIS group in its regular consultative meetings, when it provides technical support. The study team will conduct regular focus group discussions with implementation teams to monitor modifications to implementation strategies that LIS and HIS implementation teams may apply (aim 1). The decision to discontinue a student’s participation in GlobalConsent will be made based on an adverse event protocol that is described elsewhere in this study protocol.

Strategies to improve adherence to interventions

To improve implementation-team adherence to the LIS and HIS implementation strategies protocols, Emory and CCIHP will establish deliverable-based contracts with each university clarifying the terms of reference (TOR) and payment schedule for mutually agreed implementation activities. At the time of implementation team training, each team member at each participating university will assume specific roles and responsibilities, and a team supervisor will be responsible for overseeing the activities of all implementation team members. The completion of all activities will be assessed at the time that each university invoices CCIHP for its work, and payment of invoices will be based on the demonstrated completion of implementation activities in the TOR.

To support students’ adherence to the GlobalConsent program in the HIS group, the contracted IT company (with support from internal implementation team members) will send students weekly email and/or text reminders to complete each program module. In the LIS group, the contracted IT company, with support from internal implementation team members, will send students email and/or text reminders once every 2 weeks to complete each program module. Students will be offered to enter their unique ID into a lottery to win a small prize for completing each module. Student adherence to program participation will be monitored with a short survey after each program module and with passive monitoring by the IT company delivering the program (times module opened, time spent with module open).

Relevant concomitant care permitted or prohibited during the trial

There is no concomitant care that is specifically permitted or prohibited during the trial.

Provisions for post-trial care

A case management protocol for post-trial care is applied to all students who participate in the GlobalConsent program and each survey wave. First, near the end of each survey, all student participants are provided a comprehensive resource list of local fee-based and non-fee-based services. This resource list is provided to all participants. Second, after providing the resource list, all participants are asked to report their level of distress (1 = not at all distressed to 10 = extremely distressed) and the manageability of their reported distress (0 = manageable, 1 = manageable with resources, or 2 = not at all manageable). Any participant who reports extreme distress (= 10) or “not at all manageable” distress (= 2) regardless of the distress level reported will receive an emergency contact number in REDCap and will be offered the opportunity to follow-up with a professional counselor. The message will read: “Your wellness is important to us, and someone outside of the study can follow-up with you, if you wish. They will have no information about your answers. Please indicate how you are most comfortable seeking help.” If the participant reports that they want someone to follow-up with them, their ID will be shared with a non-study staff member at CCIHP. Within 3 days of the participant’s responses being submitted in REDCap, this staff member will introduce the case and his contact information to an expert (clinical psychologist) who is responsible for supporting participants in the relevant geographic region (North, Central, South). Within 1–3 days after the expert receives the participant’s contact information, the expert will contact the case via phone call to introduce themself and their professional background and to set an appointment for an online or in-person assessment. The expert will attempt to contact the participant over three days. If unsuccessful, the expert will re-confirm or correct the contact details and attempt contact again. If successful, the expert will schedule an appointment and complete an assessment, including whether the unmanageable distress was study-related. The expert will make recommendations regarding strategies for intervention, including fee-based and non-fee-based services. If the case refuses support at the time of the expert’s call, the expert will inform the case about available fee- and non-fee-based services. In all cases, the expert will report the general outcomes of their follow-up attempts and whether any unmanageable distress was attributable to the study or the intervention. The study team will report the findings of all adverse events (extreme distress and/or unmanageable distress, regardless of the distress level reported) to the responsible IRBs, independent data safety and monitoring board (DSMB), and study sponsor, in accordance with the timetable required by the National Institutes of Mental Health Reportable Events Policy [55]. In each case, determination about continuation or discontinuation of the program will be made by these parties independently.

Outcomes Implementation process outcomes (aim 1)

We will conduct multimethod qualitative research to document all implementation strategies done, to assess modifications to the implementation of GlobalConsent and to implementation plans, and to understand barriers/facilitators to implementation across HIS and LIS groups (Table 4). Twelve group reflections between CCIHP and the study team will be conducted to understand the GlobalConsent implementation process and modifications to the implementation plan in the LIS and HIS groups. Two key informant interviews (KIIs [46]) with each of 30 university leaders (five [56] leaders per university; 15 leaders per IS group; 60 KIIs total) will collect in-depth data from individuals who are knowledgeable about external factors (policies, regulations, funding) and organizational factors (resources, time constraints, institutional climate, leadership support) that may explain modifications to GlobalConsent and implementation plans [46]. A semi-structured interview guide, with open-ended questions and prompts, will guide the KIIs (Appendix). Focal topics before implementation will include (1) perceptions about sexual violence among university students and (2) perceptions about the feasibility, acceptability, and suitability of sexual violence prevention programs at universities. Focal topics after implementation will include these topics as well as (1) attitudes of university leaders, faculty, and students about continued implementation of GlobalConsent, (2) barriers and facilitators of future implementations with detailed probes, and (3) external contextual factors that may affect future implementation.

Table 4 Data collection methods for implementation assessment (aim 1)

Longitudinal focus group discussions (FGDs) will entail four discussions with each of six groups of 5–8 [57] implementation team members (one group per university; 24 FGDs total). These FGDs will provide detailed, near-real-time data on the project’s dynamic implementation context, including features of the implementation setting; modifications to GlobalConsent or implementation plans; changes in the university, local, regional, or national context that may affect implementation; and team sense-making and learning [58]. The FGD guide [46] includes open-ended questions aligned to FRAME; the rationale, timing, and guidance for each question; and probes about the use of core implementation strategies (Appendix). Some questions in the FGD guide are like those in the KII guide to facilitate triangulation of the data during analysis.

A brief checklist will be added at the end of every program module of GlobalConsent (Appendix). This checklist will be based on the Modifications and Adaptations Checklist [59, 60], a coding scheme for recording modifications to evidence-based interventions. The checklist will ask participants to self-report any modifications they made to planned use of GlobalConsent. As students cannot skip segments or modules in GlobalConsent, the checklist will focus on the following major modifications: (1) device used to view each module of GlobalConsent, (2) percentage of module watched, (3) number of sessions required to watch the module, (4) whether or not they watched part or all of each module mor than once, (5) “drift” by multi-tasking or doing other things while a module was open, ,6) “drift” by stepping away from the computer or mobile device while a module was open, (7) extent of satisfaction with the content of the module, and (8) any comments about the module.

Implementation drivers and outcomes (aim 2)

Table 5 summarizes the constructs to be measured, data sources, study samples, number of assessment points by focal population, and for variables measured in surveys (indicated with a superscript), the number of items per construct. The main constructs to be measured are drawn from implementation science and based on the hypothesized processes and outcomes in the present study. Scales that were created to assess general implementation of evidence-based practices in a medical or social-service setting are adapted to fit the current context of implementation by modifying item wording to focus generally on sexual violence prevention programming and specifically on GlobalConsent in a university context (Appendix).

Table 5 Implementation outcomes and potential drivers aligned with RE-AIM [24, 25] and Proctor et al. [32] for comparison across implementation strategies groups

Demographic implementation drivers that are measured in all focal populations include age in years, sex assigned at birth, gender identity, sexual orientation, and ethnicity. Questions on sex, gender, and sexual orientation are based on recommendations from the National Academies of Sciences, Engineering, and Medicine (NASEM) [61]. Normative implementation drivers include pre-implementation perceptions in all focal populations about sexual violence as a problem to address, perceptions of campus climate, legal knowledge of sexual violence, knowledge about active consent, and rape myth acceptance. Other implementation drivers include perceptions of implementation (1) leadership, (2) collaboration, and (3) climate.

Implementation outcomes include the (1) perceived acceptability, appropriateness, and feasibility of general sexual violence prevention programming in all focal populations, (2) perceived acceptability, appropriateness, and feasibility specifically of GlobalConsent in all focal populations, (3) implementation adoption among eligible students consenting to take part in GlobalConsent, and (4) implementation penetration among eligible students consenting to take part in GlobalConsent. Intervention adoption and penetration will be measured continuously using records from the collaborative IT company on the implementation of GlobalConsent by tracking the number of eligible male student participants who consent to take part in GlobalConsent (adoption) and who complete the intervention (penetration). Notably, the normative climate among leaders, faculty, and implementation teams at baseline also may change as a result of implementation, so these measures are listed as implementation outcomes.

Implementation effectiveness outcomes (aim 3)

Student-level primary outcomes (sexually violent behavior; prosocial intervening behavior) and student-level secondary outcomes related to cognition/knowledge, attitudes/beliefs, affect, and capacity are summarized in Table 6, and all question sets are provided in English and Vietnamese in the Appendix. Table 6 also includes the National Institutes of Health (NIH) common data elements to be collected.

Table 6 Primary and secondary implementation effectiveness outcomes among studentsIncremental cost-effectiveness outcomes (aim 4)

The primary effectiveness outcomes for the cost-effectiveness analysis will be the frequency of sexually violent behavior and prosocial intervening behavior. The costing items are organized sequentially based on the implementation phases: pre-implementation, implementation, and post-implementation. Each phase includes a detailed list of activities (Tables 2 and 3), with costs related to personnel, travel, space, and supplies/equipment collected for each activity. We will estimate the aggregate costs for each university and then divide this aggregate by the total number of participants at that university to determine the cost per participant.

Participant timeline

Figure 2 summarizes the start dates and end dates for enrolment (beginning March, 2024), interventions (implementation activities and delivery of GlobalConsent), and assessments (quantitative surveys, key informant interviews, focus group discussions, modification checklist, and costing forms) with each focal population in the “high intensity” implementation strategies group.

Fig. 2

Schedule of enrolment, interventions and assessments

Sample size

Focal populations for each of these assessments include university leaders who support the GlobalConsent implementation (n = 5 per university, total of 30), members of implementation teams (n ≈ 5 per university, total ≈ 30 members), fulltime permanent faculty at each university (estimated mean 589 per university, range 186–1038, total 3532; based on official figures for 2021), and male student participants in GlobalConsent (estimated mean n = 796 per university, range 449–1202, total = 4776 18–24 year-old, first-year undergraduate, heterosexual or bisexual men, who are attracted to women; based on official enrolment figures for 2021). We will assume 80% participation and 90% retention in each focal population.

Recruitment University leaders (aim 1)

University leaders will be identified purposively via discussions between CCIHP key personnel and implementation team focal persons. Eligible participants will be informed during the online informed consent process that they will be compensated $10 for completing each of two online surveys and $20 for completing each key informant interview (KII), for a total compensation of $60 for completing all assessments.

Implementation team members (aims 1, 2, and 4)

Implementation team supervisors and team members will be identified purposively via discussions between CCIHP key personnel and the identified focal implementation team member in each university and finally approved by the leaders of that university. Eligible implementation team members will be informed during the online informed consent process that they will be compensated $10 for completing each of four online surveys and $16 for participating in each focus group discussion (FGD), for a total compensation of $104 for completing all assessments.

University faculty (aim 2)

Each university will submit to non-study staff the list of all permanent, fulltime faculty (lecturers) at their university, with professional contact details. All eligible lecturers will receive an email invitation from staff within the university introducing the study and inviting their participation. Two days later, each faculty member will receive a secure REDCap link to the informed consent form. While completing the online consent form, the faculty will have an opportunity to call a non-study staff member to answer any questions about the study, as needed, before consent and participation. The faculty will be informed during the consent process that they will receive $10 for completing each online survey. If consent is confirmed, the faculty member will be directed immediately to the online REDCap survey. Any faculty member who has not completed the online survey will receive up to five automated reminders from REDCap at 2.5-day intervals over 2 weeks. In addition, on days 8 and 15 of the survey period, all faculty will receive a reminder invitation from within their university to complete the survey. On days 22 and 29 of the survey period, any faculty member who has not completed the survey will receive additional reminder invitations. During the 2 weeks following day 15, any faculty member who still has not completed the survey will receive up to two standard SMS text reminders with their unique survey link encouraging them to complete the survey. If, after 4 weeks, the faculty have not yet completed the faculty survey, they will receive a standard follow-up reminder and unique survey links via Zalo and a call via Zalo, an encrypted Vietnamese communication application similar to WhatsApp. Once this protocol is completed, implementation teams will send a general reminder through informal faculty networks at their universities for faculty to complete the survey.

First-year undergraduate male students (aims 1 and 3)

All eligible students will be invited to participate in six 6-monthly surveys and the GlobalConsent program. For the LIS and HIS groups, implementation team members will send a standard email invitation to all eligible men with a description of the GlobalConsent program, a description of the study, and an invitation to participate. They will separately receive a secure REDCap link to the online informed consent form and, if completed, the online survey (Table 6). For the HIS group, internal facilitators will organize an orientation session for all eligible male students to describe these elements in person and to answer any questions about participation. All eligible participants will be informed during the online informed consent process that they will be compensated $5.50 in average for completing each survey assessment, for a total compensation of $33 for completing all six assessments. All eligible participants also will be informed during the online informed consent process that they will be eligible to enroll in a lottery to receive a prize (e.g., smartphone) after completing three survey assessments and that they will be eligible to enroll in a second lottery to receive a prize (e.g., smartphone) after completing six survey assessments. All eligible participants in the HIS group will be informed about this compensation schedule during the recruitment orientation meeting.