Intervention fidelity: the FICUS trial

FICUS (Family support intervention in Intensive Care UnitS) is a hybrid effectiveness-implementation study (type 1) in 16 Swiss intensive care units (ICUs)Footnote 1. Its primary aim is to assess the clinical effectiveness of a multicomponent nurse-led family support intervention (FSI) compared to usual care using a cluster-randomized controlled trial [40] while also investigating the implementation process and outcomes using a mixed-methods multiple case study approach [41].

The FSI is led by trained ICU family nurses, a new role introduced to the ICU team in an interprofessional setting to support family members of critically ill patients during and after an ICU stay. It aims to improve family members’ service and (mental) health outcomes [40]. The FSI includes three core components, which are grounded in a family system nursing approach, based on evidence on systematic family interventions and existing guideline recommendations on ICU family care, and specified in a logic model (see Fig. 1 in [40]). (1) Engaging and Liaising (encounter) by relationally engaging families over time, connecting family with team, and coordinating and ensuring access to and involvement in care; (2) Supporting (therapeutic conversations) by assessing families’ situations and their preferences and needs, choosing and performing relationship-focused and psycho-educational interventions; and (3) Communicating (family meetings), including understanding values and priorities, informing about the patients’ conditions, discussing care plans and making joint decisions [40] Fig. 1.

Fig. 1

The FICUS family care pathway. This figure is unadapted from its original version and licensed under a Creative Commons Generic License (CC BY 4.0 OA). It is attributed to Naef et al. [40]

The core FSI components are standardized in the patient pathway but can be tailored to the patients’ courses of illness and families’ needs during and up to three months after patients’ ICU discharge. Therefore, FSI core activities are defined in a fixed and compulsory intervention dose. Family nurses adapt this dose, its frequency, and intervention components and activities to meet family members’ needs during the FSI pathway (Fig. 2).

Fig. 2

Adapted Conceptual Framework for Implementation Fidelity in the FICUS trial. This framework was adapted from the original framework by Carroll et al. 2007 [5, 30]

The aim of the FICUS trial is an adaptive intervention delivery to optimize FSI fit for every family member and family as a whole and to enhance the likelihood of reaching intended service and (mental) health outcomes. To understand how intervention fidelity, the trial’s main implementation outcome [41], influences intended FSI outcomes and explains its effects, we pre-defined the minimal FSI dose hypothesized to reach causal inferences. This minimal intervention dose limited the family nurses’ room for adapting FSI to individual family members’ needs. This highlights the instant dilemma in complex intervention research between a need for adaptive interventions on the one hand and minimum intervention standards on the other, among others to evaluate causal inferences between intervention (dose) and outcomes.

FSI fidelity

To measure intervention fidelity to the FSI pathway and core activities, a modified version of the Conceptual Framework for Implementation Fidelity developed by Carroll et al. [5] was used. The framework was supplemented with the concepts of the three fidelity domains outlined by Bellg et al. [42], i.e., intervention delivery (adherence), receipt, and enactment (Fig. 2). Delivery captures whether the intervention components were delivered as intended, i.e., consistent with content, frequency, duration, and coverage criteria. Receipt refers to the “degree to which the participant understands and demonstrates knowledge of and ability” to use an intervention (i.e., comprehension), and enactment pertains to “the degree to which the participant applies the skills learned” [42] (p.444), i.e., intervention activities [25, 42, 43].

In operationalizing FSI fidelity, moderators included in the Carroll et al. (2007) fidelity framework were considered and delineated as follows [5, 41, 42]:

Fidelity delivery was defined as family nurses’ adherence to core FSI activities and timeline, and the consistency of delivery across the eight intervention ICUs and between family nurses. Fidelity delivery included intervention consistency, frequency, dose, and availability (Table 1). It was assessed using an electronic structured intervention log completed by family nurses after each intervention session and by a self-reported capacity assessment measured three to six months after implementation start [41].

Table 1 Operationalization of fidelity delivery in the FICUS trial

Fidelity receipt was defined as the degree to which family nurses understand and perform the FSI, including required cognitive and behavioral skills. It was operationalized as family nurses’ participation in educational implementation strategies targeting FSI comprehension, e.g., attendance of intervention training and monthly case conferences used to enable them to perform the intervention [40, 44]. Additionally, the performance of cognitive and behavioral intervention skills was self-assessed with the psychometrically validated German version of the Family Nursing Practice Scale [45, 46] and in focus group interviews (Table 2). As such, fidelity receipt was measured for the intervention and one specific implementation strategy.

Table 2 Operationalization of fidelity receipt in the FICUS trial

Fidelity enactment was operationalized as family nurses’ engagement and performance of FSI core activities. For standardization purposes, a fidelity audit observation tool and a self-rating tool for family nurses were developed, both based on the FSI protocol [40]. The fidelity audit tool consists of four items on available structures, such as study information and 59 FSI activities alongside the three main FSI components engaging and liaising, supporting and communicating [41]. Observers rate family nurses’ performance during FSI activities as performed, partially performed, not performed, or not applicable. The family nurses’ self-rating tool includes three subscales, one for each FSI component (engaging and liaising: 15 items, supporting: 27 items, communicating: 16 items), all rated on a 5-point Likert scale. A lower score indicates lower, and a higher score higher, self-perceived FSI performance (Table 3).

Table 3 Operationalization of fidelity enactment in the FICUS trial

In the FICUS study, using a conceptual fidelity framework enabled a nuanced and comprehensive operationalization of fidelity beyond a simple “delivery as intended”, accounting for FSI having been designed as an adaptable intervention in dose and frequency. In addition to assessing fidelity at the individual participant level, i.e., family members, applying the receipt and enactment dimensions enabled fidelity assessment at the unit level.

Implementation fidelity: the REVERSE trial

REVERSE (“pREVention and management tools for rEducing antibiotic Resistance in high prevalence Settings”) is a stepped-wedge cluster-randomized hybrid effectiveness-implementation trial (type 2) aimed at assessing the effectiveness of infection prevention and control (IPC) and antibiotic stewardship (ABS) practice bundles on the health-acquired infections rate of 24 acute care hospitals in EuropeFootnote 2. REVERSE also involves assessing the effectiveness of tailoring as an implementation strategy, based on an additional randomization of participating sites to different implementation conditions. This required conceptualizing tailoring fidelity and operationalizing this concept for monitoring in the trial.

Tailoring fidelity

Tailoring has been characterized as a distinct implementation strategy [47] that involves prospectively identifying implementation barriers assumed to influence the implementation of an intervention. The goal is to inform the subsequent selection and design of implementation strategies assumed to address these pre-identified barriers and their planned and intentional application [48, 49]. While this implicit promise of tailor-made implementation appears appealing, the knowledge base for how tailoring can be systematically practiced, monitored, and evaluated remains limited. This gap has led to calls for more research in this area of implementation science [50, 51], including an in-depth exploration of tailoring fidelity, i.e., the actions or principles that constitute the core elements of tailoring which, when observed, confirm that such tailoring is taking place. This idea may appear counterintuitive. How can firm adherence be expected in the use of an implementation strategy that has context dependent flexibility as its core? Notwithstanding this tension, tailoring builds on generic principles – the prospective identification of determinants and the intentional determinant-informed selection and design of implementation strategies – making it possible to differentiate tailoring practice from less prudent attempts to select, develop, and apply implementation strategies. It is this conceptual challenge, the development of a tailoring fidelity framework, that is addressed in the REVERSE trial.

The REVERSE tailoring fidelity framework

A pre-existing conceptual model of tailoring [52] was used to define tailoring key principles and to integrate these into what was labeled the REVERSE Improvement Cycle (RIC, Fig. 3): (1) prospective determinant identification; (2) intentional strategy-determinant matching; (3) context-sensitive operationalization of implementation strategies; and (4) implementation strategy use and impact assessment. These principles were combined with further steps involved in implementing REVERSE practice bundles, including prioritizing concrete IPC and ABS practices for implementation and considering approaches to stakeholder engagement throughout implementation.

Fig. 3

The REVERSE Improvement Cycle (RIC)

The RIC represents a cyclical improvement logic that is widely used in quality improvement initiatives in healthcare [53,54,55], and which, as an implementation approach, was expected to resonate with study participants. It presents tailoring as an ongoing, potentially never-ending implementation strategy that requires paying continuous attention to changes in contextual factors relevant to implementation and to the needed adjustment of implementation strategies. This cyclical model forms the basis for teaching the principles of tailoring in workshops and online meetings with local implementation teams of participating hospitals working to implement the REVERSE practice bundles.

In a second step, the RIC was translated into the REVERSE implementation tool (RIT), a qualitative reporting instrument for participating sites to plan and document their implementation practice. Each step of the RIC is presented as one or multiple RIT questions for implementation teams to consider when preparing or documenting implementation actions. This includes thinking about the composition of the local implementation team, relevant stakeholders to engage, central determinants assumed to influence the implementation of a REVERSE practice, strategies for addressing these determinants, and indicators for monitoring own implementation practice. Sites are encouraged to use the RIT to plan their implementation activities, reflect on these activities, and document their implementation decision-making quarterly. Simultaneously, the RIT is a tool for the REVERSE implementation research team to follow sites’ tailoring efforts. It represents one element of the REVERSE tailoring fidelity assessment, which will be combined with RIT interviews with frontline hospital staff.

The assessment of RITs aims at understanding whether trial sites practice tailoring and is structured by the fidelity dimensions that are included in Demers et al.’s integrative conceptual fidelity framework [10]. Table 4 outlines how these dimensions were translated into questions, allowing the research team to assess local tailoring practice.

Table 4 The REVERSE tailoring fidelity framework

This table reflects that REVERSE tailoring fidelity will be explored at the level of tailoring implementers and tailoring receivers. Tailoring implementers are members of hospitals’ central IPC and ABS teams collaborating with frontline healthcare workers to facilitate the implementation of REVERSE practice bundles. These professionals function as internal implementation support practitioners [56, 57] and include IPC physicians, infectious diseases specialists, IPC nurses, microbiologists, pharmacists, and other healthcare workers specialized in preventing and controlling healthcare-acquired infections. They apply tailoring principles in their efforts to enable the local implementation of REVERSE practice bundles. Tailoring receivers are medical doctors, nurses, nurse assistants, cleaners, lab technicians, and other staff applying IPC or ABS quality standards in their daily work. They are the hospital stakeholders with whom IPC/ABS teams collaborate in their efforts to apply tailoring principles.

Hospitals’ IPC/ABS teams will share thirteen RITs throughout the REVERSE trial, leading to a maximum of 312 RITs documenting tailoring efforts. Each document will be assessed based on the questions in Table 4. Two REVERSE implementation research team members will review all RITs based on a list of indicators and attention points related to each targeting question. Of all RITS, 25% will be assessed in duplicate, and disagreements discussed in consensus meetings. The remaining RITs will be assessed by one team member only.

The RIT captures the implementation work of central IPC/ABS teams and, apart from the dimension reach, does not allow for a deeper examination of the degree to which tailoring receivers are exposed and respond to centrally initiated implementation efforts. In parallel with the administration of the RIT, healthcare frontline workers involved in REVERSE IPC/ABS implementation at each hospital will, therefore, be invited to interviews to explore to what degree exposure has taken place and has led to changed implementation practice at this level of service delivery. The targeting questions in Table 4 will be operationalized into guiding questions for semi-structured interviews to be held with a randomly selected sample of healthcare professionals. The findings from this assessment have the potential to help understand whether and how the use of tailoring in the REVERSE trial may influence trial results.

The experience from the REVERSE trial illustrates the complexity of conceptualizing fidelity to tailoring, a genuinely adaptive implementation strategy [21], and the considerable conceptual and empirical work required to assess this fidelity within the context of a hybrid type 2 trial with its inherent emphasis on implementation strategy fidelity.

The intricacies of balancing firmness with flexibility

Together, the above case studies illustrate that researchers are confronted with multiple complexities when working to balance needs for both firmness and flexibility in conceptualizing, operationalizing, and evaluating intervention and implementation fidelity.

First, when researching (novel) complex interventions and implementation strategies, often little knowledge exists on the active ingredients that may trigger changes in implementation or health outcomes. Therefore, a certain degree of subjective decision-making may be involved in defining minimum fidelity requirements and creating clarity around where firmness is required and flexibility allowed. This also applies when evidence exists for the intervention and the implementation strategies under study, as there may still be a lack of knowledge on their combined application. One way to address this challenge is to develop a clear theory of change for an intervention and its implementation based on, e.g., implementation research logic models (IRLMs) [13, 58, 59]. IRLMs depict anticipated relationships between intervention components, contextual determinants, implementation strategies and their presumed mechanisms, implementation, and clinical outcomes. By necessitating thinking about which elements of an intervention or an implementation strategy are essential to achieve intended outcomes and about how these elements may create meaningful differences in individual, organizational, and system behavior and patients’ health and wellbeing, IRLMs create transparency around the conceptual and empirical foundations of defined fidelity requirements and their assessment. As empirical knowledge about the intervention and implementation strategies in focus increases, these requirements can then be revised and refined, and the room for firmness and flexibility defined with greater certainty.

Second, the operationalization and measurement of fidelity are always intervention- and implementation strategy-specific and study context-dependent. While efforts have been made to develop generic fidelity measures usable across multiple clinical interventions, even these remain target population-specific in that they address similar interventions for similar conditions pursuing similar goals [60, 61]. In most cases, intervention as well as implementation fidelity requirements and their assessment, therefore, depend on a full conceptual development that is adjusted to the intervention and delivery setting in focus of a study. Hence, non-negotiable core intervention or implementation strategy elements viewed as requiring firmness when used with a particular target population in one setting may be adjustable and allow for flexibility when applied with another target population in a different setting. This possibility makes it critical for researchers to re-think fidelity concepts with every new study.

Furthermore, and of particular importance to implementation scientists, the design of such a study may represent a further contextual factor to consider when measuring fidelity. Depending on the type of hybrid study chosen [39], its primary focus will be either the intervention, the implementation strategy, or both, leading to different emphasis on measuring intervention and implementation fidelity. In a hybrid type 1 study, with the primary aim of determining intervention effectiveness, measuring intervention fidelity is essential, whereas implementation is primarily assessed from an exploratory perspective. There may, therefore, be limited room for flexibility and a focus on firmness when assessing intervention fidelity [44, 62]. In a hybrid type 2 or 3 trial, the assessment of implementation strategies is in focus. Consequently, the room for continuously adjusting these strategies may be limited to ensure a thorough evaluation of their effect.

To address this challenge of fidelity operationalizations being intervention- and implementation strategy-specific, some scholars have suggested moving away from intervention protocols that emphasize the often highly protocolled and detailed forms in which interventions should be delivered to instead focus on function descriptions as the key referent for firm fidelity requirements [63]. For an implementation strategy such as tailoring, prospective determinant identification and purposeful determinant-strategy matching represent such essential functions ensuring that implementation work considers and addresses the contextual conditions of a local setting. These functions could be pursued in various forms, e.g., through regular staff meetings, surveys, expert interviews, analysis of written materials, and similar approaches that implementers could select based on what is appropriate and feasible in their setting. In this way, the form-function model can enhance the flexibility with which interventions and implementation strategies are designed to fit a local, ever-changing context. Integrating this form-function thinking into the use of fidelity frameworks to develop fidelity measures can help make these measures more robust through nuanced thinking about, e.g., what appropriate timeliness, quality, or exposure for delivering an intervention or implementation strategy would mean.

Third, fidelity concepts continue to focus on individual or group behavior and actions. However, individuals and groups always operate in broader contexts that influence their implementation, an influence acknowledged in, e.g., the recently updated Carroll fidelity framework, now also including organizational and cultural context [64]. Here, context is viewed as a fidelity moderator, affecting the strength of adherence achieved in any given setting. If contexts are not supportive of fidelity efforts, individuals and groups striving to comply with intervention or implementation fidelity criteria may still be unsuccessful. While viewing contextual factors as moderators in this way clarifies implementation processes, it also represents a limitation in that context is treated as a given. Instead, it could be valuable to consider how organizational and system-level requirements could be better integrated into conceptualizations of fidelity. When viewing individuals as being embedded in teams, organizations, and systems, their ability to stay firm and be flexible when needed may depend on, e.g., leadership support, dedicated work hours, or formally recognized staff roles – factors that could be defined as additional organizational or system fidelity requirements, to indicate clearly that fidelity never occurs in a vacuum. In a recent publication on IPC implementation, this relationship was described as a cascading logic model [57] unfolding across three levels, combining intervention fidelity at the frontline healthcare worker level with implementation fidelity considerations at the level of direct superiors and that of hospital management, highlighting that the organization surrounding frontline implementers has levers to provide fidelity supports. To the degree such supports can be defined as vital functions for building fidelity, these could be integrated into multi-level fidelity concepts that go beyond the individual and assign the organization a degree of fidelity accountability. Implementation strategies developed to be applied at the organizational or community level to, for example, enhance implementation leadership, climate, or culture [65,66,67] allow for testing this relationship.

Finally, with contemporary conceptualizations of fidelity centered on function rather than form, thereby moving away from simple checklist-based assessments of firmness and instead creating room for intervention adaptation and implementation strategy tailoring, activities representing intervention or implementation fidelity often contain a substantial process component. In balancing firmness with flexibility, implementers increasingly require capacity building, process facilitation, or problem-solving skills. Fidelity is, therefore, not easily built and not easily assessed continually and objectively [68,69,70]. Its assessment will require regular training and exchange between researchers, implementation support practitioners, and frontline implementers, thereby depending on the use of considerable resources, as illustrated through the case examples shared above. While these resources may be available as part of a research trial, there is typically much less room for an ongoing fidelity assessment in routine healthcare settings unless pragmatic approaches can be found. This raises the question of how low one can go in finding this pragmatic level at which fidelity achievement is possible, data collection and feedback loops doable, and meaningful data can be generated, e.g., to inform decision-making on encouraging greater firmness or flexibility among implementers.

Intensifying contemporary fidelity research focusing on intervention and implementation fidelity is pivotal. Theoretically, there remains a need to understand better how an appropriate balance between firmness and flexibility in intervention delivery and implementation strategy use may add to improved implementation, service, and patient outcomes and how this balance can be obtained using the least possible resources and taking into account influences at the individual, organizational and system level. Empirically, there continues to be a need for complex fidelity studies examining the contribution of intervention and implementation fidelity, separately and combined, to different outcomes. Furthermore, greater knowledge is needed about practices for building, evaluating, an