Study design

This is an individually randomized, parallel, superiority, registry-based pragmatic trial with an embedded process and economic evaluation. We will target surveillance tests for cardiomyopathy, breast cancer, and/or colorectal cancer informed by the latest guidelines; a summary of surveillance guideline recommendations can be found in Table 1. This protocol complies with SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials) [19] (Additional file 1).

Table 1 Guideline recommendations for surveillance of late effects

The trial is considered to be pragmatic [20] for the following reasons: (1) it evaluates the effectiveness of a potentially sustainable intervention being introduced under usual conditions, across a jurisdiction; (2) eligible survivors are identified through the use of existing provincial registries and enrolled in this study through a waiver of consent; and (3) data collection utilizes routinely collected information by existing administrative databases and data will be analyzed using an intent-to-treat analysis. The checklist (PRECIS2 table) with explanation and elaboration can be found in Additional file 2. We received privacy and ethics approval from Clinical Trials Ontario (Project ID: 4152) and Ontario Health (Data Request #23–001). The trial is registered at clinicaltrials.gov (NCT05832138).

Study setting

In Ontario, visits to physicians are covered by the tax-funded, provincial health insurance plan, without a co-pay or deductible. Medically necessary tests, including echocardiography, colonoscopy, and breast imaging are also free of charge to patients if ordered by a licensed physician or nurse practitioner. Ontario has specialized pediatric cancer survivorship clinics, the Pediatric Oncology Group of Ontario AfterCare network. The AfterCare Program monitors survivors regularly for the long-term effects of cancer treatment. These are funded by the provincial government, but most adult childhood cancer survivors receive all of their medical care from a family physician [21]. However, most family physicians have few, if any, survivors in their practice [22, 23] and are unaware of surveillance guidelines [22]. Unfortunately, most childhood cancer survivors are also unaware of their surveillance requirements [24].

Population

As of February 28, 2021, there were just over 12,200 (18 +) childhood cancer survivors living in Ontario and 3284 of these survivors were at risk for one or more breast cancer, colorectal cancer, or cardiomyopathy based on their childhood cancer treatment. Our recent study found that many of these survivors were overdue for recommended surveillance: 92% of survivors at risk for breast cancer, 87% at risk for colorectal cancer, and 48% at risk for cardiomyopathy [unpublished data].

Eligible childhood cancer survivors will be recruited across Ontario. The ONLOOP Trial will be conducted in partnership with Ontario Health, the government agency responsible for health care planning and delivery across the province. Ontario Health holds multiple linked administrative datasets that allow it to measure health system performance. The research office at Ontario Health has established protocols to conduct studies involving patient contact. This study will leverage this infrastructure at Ontario Health to i) identify eligible survivors via analysis of linked administrative data and ii) invite these individuals to participate.

Research questionsPrimary question

Does the ONLOOP program, a multi-faceted intervention informed by behavioral theory, increase adherence to surveillance guidelines for cardiac disease, colorectal cancer, and/or breast cancer among at-risk adult childhood cancer survivors within 12 months, compared to usual care?

Secondary question

How can the implementation of ONLOOP be optimized in terms of fidelity, mechanism of action, sustainability, and efficiency?

Hypothesis

Providing a personalized health information kit with a surveillance reminder will improve adherence to recommended surveillance guidelines among childhood cancer survivors at-risk and overdue for surveillance of late effects at 6 months post-randomization.

Process evaluation

The process evaluation will examine why and how the intervention succeeded or failed. It will help us understand how the intervention generates effects, intended or unintended. It will also help us identify and understand barriers and facilitators of implementing and scaling up the intervention. Finally, it will help us understand whether the intervention will work in the current health system and for whom and in which context.

Economic evaluation

The economic assessment will assess the cost-effectiveness of the intervention.

Study cohort and participants

To identify eligible survivors, we will use Pediatric Oncology Group of Ontario’s cancer registry, POGONIS (Pediatric Oncology Group of Ontario Networked Information System). This registry holds cancer diagnosis and treatment data for all children (< 18 years at diagnosis) treated at one of Ontario’s five pediatric cancer centers since 1986. We will use POGONIS to identify a subset of living survivors at risk for a subsequent malignant neoplasm (SMN; breast/colorectal) and/or cardiomyopathy (Figs. 1 and 2). This POGONIS subset will be transferred to Ontario Health. Ontario Health has extensive expertise in conducting population-level outreach studies (under PHIPA 18(4) Reg.329, Sect. 44(6)(e)) [25] and refers to these as patient contact studies. Ontario Health will link the POGONIS subset to other administrative datasets using each survivor’s unique Ontario Health Insurance Program number. Ontario Health will then confirm survivor eligibility by determining if the survivor is overdue for the guideline-recommended tests by over 6 months [5] (Table 1) to create the study cohort. Ontario Health and the Pediatric Oncology Group of Ontario can hold identifiable personal health information and are able to undertake this work under Ontario’s privacy legislation [25].

Allocation

After the study cohort has been created, Ontario Health will randomize eligible survivors into either the intervention or delayed intervention (control) arm using a 1:1 allocation ratio (Fig. 3). An independent statistician at Ontario Health will use a computer-generated random sequence to allocate survivors stratified by sex and whether the survivor is overdue for breast cancer, colon cancer, or cardiac surveillance. The study cohort will be allocated simultaneously, minimizing risks of bias due to lack of allocation concealment. There is a small risk of contamination if a survivor from the intervention group speaks with a survivor from the delayed intervention group, which may bias results towards the null.

Fig. 2

Eligibility criteria. OHIP—Ontario Health Insurance Plan, SMN—Secondary Malignant Neoplasm

Fig. 3Blinding

Eligible survivors will be unaware if they are in the intervention group or the delayed intervention group (control). However, they will not be blinded to the intervention. The risks of bias from lack of blinding are minimal as they will not be made aware that they were randomized and allocated at the beginning of the trial. In addition, childhood cancer survivors randomized to the control arm will not receive any ONLOOP communications or materials until 13 months after childhood cancer survivors in the intervention arm receive their study invitation letter.

Recruitment

Intervention materials will be mailed out using Ontario Health’s established infrastructure for screening test reminders (Ontario Health sends reminders to eligible Ontarians who are due for certain cancer screening tests) and patient contact studies. Ontario Health can undertake this work under provincial privacy legislation [25]. The study cohort dataset will be held at Ontario Health. The dataset will include each survivor’s mailing address, which will be used to deliver the initial study invitation package. This initial contact will be made by Ontario Health’s Research Office. The study invitation package is part of the intervention; therefore, recruitment to the ONLOOP Trial is part of the intervention. Consistent with other Ontario Health patient contact studies, the ONLOOP Trial will be identified as a study conducted by The Hospital for Sick Children, Women’s College Hospital, and the Pediatric Oncology Group of Ontario, with support from Ontario Health. Ontario Health’s Research Office will address potential participants’ questions or concerns related to outreach or mailing errors, as well as concerns related to Ontario Health holding this sensitive data and conducting research studies with it.

Ontario Health will provide the research team with de-identified administrative data for both childhood cancer survivors who enrolled and those who did not; this will enable analyses that adhere to intention-to-treat principles. However, we will only send the personalized health toolkit and the 6-month screening reminder to those who actively opt-in by consenting to be part of the ONLOOP intervention.

Intervention arm

The intervention will include the following ONLOOP program materials (Fig. 3):

i.

Study invitation letter and 2 invitation reminders (5 weeks and 10 weeks after the study invitation letter).

ii.

For those who sign up: receipt of a personalized health toolkit and a screening reminder 6 months later.

iii.

For those who consent, an introductory letter to their primary care provider and a reminder letter sent 6 months later.

Step 1: survivor study invitation

A standard Ontario Health study invitation package will be mailed to survivors and will contain the following: (1) Ontario Health Initial Contact Letter (Additional file 3): this will note the Ontario Health research office’s contractual role in providing external researchers with their outreach infrastructure for population-level research. This letter will explain the purpose of the research study and why the individual is receiving an invitation package; (2) Study Invitation Letter (Additional file 4): this will contain generic information about the late effects of childhood cancer treatment and the importance of surveillance tests. Individuals who are interested in participating can register for the study; (3) Consent and Sign-Up Form (Additional file 5): Survivors can register for the study by mailing back an integrated Consent and Sign-Up form using a prepaid return envelope, or through our program website (www.onloop.ca, Additional file 6). For those who do not respond to the initial study invitation, we will mail a brief follow-up letter approximately 5 weeks and 10 weeks after the initial study invitation was sent.

In the delayed intervention group (control), survivors will receive usual care for 13 months after study invitation packages are mailed out to the intervention arm. At that point, survivors in this group will also receive the study invitation package. Comparing the intervention to usual care will maximize the external validity of the results. In keeping with the pragmatic design of the study, no attempts will be made to standardize usual care.

Step 2A: information kit (Additional file 7)

The study team will provide participants with a personalized health toolkit. Depending on the method of communication selected at enrollment, this will be delivered by post or email. This toolkit will provide information about the survivor’s childhood cancer treatment, recommended surveillance tests and completion intervals, and instructions on how to obtain the recommended tests.

Step 2B: physician letter (Additional file 8)

We will ask participants for consent to send a letter to their family physician or nurse practitioner. With consent, the study team will mail or fax a letter that includes the participant’s childhood cancer history, treatment exposures, and surveillance test recommendations.

Step 3: surveillance test reminder for survivors and physicians

Six months after mailing out the personalized health toolkit, the study team will send a personalized surveillance reminder. Participants will receive this reminder regardless of whether they have completed their screening tests during the follow-up interval. For participants who consent to the study team contacting their family physicians, a reminder physician letter will also be sent.

Delayed intervention (control) arm

These survivors will receive usual care (i.e., no initial contact regarding the study, no personalized materials regarding surveillance, and no contact with their primary care provider) for 12 months. In keeping with the pragmatic aims of the trial, no attempt will be made to standardize the care received by survivors during the trial period. After 12 months, these survivors will receive Intervention Step 1 and the survivor can opt-in to receive the additional intervention materials as described above or opt out of additional study contact and materials.

Outcome measuresPrimary outcome

Completion of one or more of the guideline-recommended cardiac, breast, or colon surveillance tests during the 12 months after study cohort randomization (binary outcome).

Secondary outcomes

(1) Completion of each type of surveillance test (among those eligible for the test); (2) being fully up-to-date with their surveillance tests; (3) number of outpatient visits to primary care professionals and cancer specialists; (4) number of emergency department visits and hospitalizations to understand the impact on health system resources.

Study outcomes in both study arms will be analyzed at 12 months and 24 months after study cohort randomization.

Data collection

This is a registry-based trial [26] with data collection from existing administrative databases held at Ontario Health. We will primarily rely on Ontario Health Insurance Program billing data to assess whether and when each childhood cancer survivor completed their mammogram/breast MRI, colonoscopy, and/or echocardiography. If applicable, the Ontario Breast Screening Program and Gastrointestinal Endoscopy datasets will also be used to assess the completion of these tests. See Table 2 for data sources for primary and secondary outcomes.

Table 2 Primary and secondary outcomes and data sourcesAnalysis

Primary outcome analyses at 12 months will be by intention-to-treat. Descriptive statistics will be used to summarize the baseline and clinical characteristics of participants in each arm. To obtain correct inferences and improve power and efficiency, we will use a logistic regression model to estimate intervention effects, adjusting for sex and number of tests overdue at baseline (the stratification factors) as well as the following pre-specified prognostic factors: rurality, age at diagnosis, and attachment to primary care. The intervention effect will be expressed as an adjusted odds ratio together with a 95% confidence interval. Results will be converted to relative risks and risk differences which will be used to calculate the number needed to treat [33]. The number needed to treat is the inverse of the absolute risk difference.

The primary outcome analysis will be the completion of at least one test, but secondary analyses will explore whether results differ according to the type of test. We will conduct separate analyses for each test among eligible childhood cancer survivors. The secondary outcome of being fully up-to-date according to guidelines at the time of outcome assessment will be analyzed as described for the primary outcome. Health services utilization outcomes (e.g., number of outpatient visits over the prior 12 months or 24 months) will be analyzed using negative binomial regression, adjusting for the stratification factors.

Secondary analyses will consider the subgroup of participants in the intervention arm who are exposed to the full intervention (i.e., those who opt-in to receive their personalized information kit). Subgroup analyses in both the intention-to-treat and per-protocol populations will also be carried out to examine effect modification for the primary outcome by sex, number of tests required (1 vs 1 +), type of test required, rurality, neighborhood-level socioeconomic status, age at diagnosis, current age, and attachment to primary care. Ontario Health will transfer the deidentified study administrative dataset to the Ottawa Hospital Research Institute for analysis.

Sample size

The sample size for this population-based study is fixed by the available number of eligible childhood cancer survivors. We previously identified 1724 childhood cancer survivors (53% of the POGONIS subset of 3268 childhood cancer survivors at risk for breast cancer, colorectal cancer, and/or cardiomyopathy) who were overdue for one or more of the recommended breast, colon, and/or cardiac tests as of 28 February 2021. Therefore, we anticipate allocating at least 862 individuals to each trial arm (intervention and control). Based on our prior analyses of administrative data [34], we estimate the uptake of recommended tests in a 12-month period will be 15% without any intervention (i.e., control arm proportion). Our anticipated sample size is sufficient to detect an absolute increase in the proportion of childhood cancer survivors with one or more tests completed as small as 6% (or a relative increase of 1.4) with 90% power in our intention-to-treat analyses. While a difference of 6% is considered relevant and important on a population level, it is expected that among those fully exposed to the intervention, the effect could be substantially higher. Loss-to-follow-up is anticipated to be minimal given the use of administrative data for outcomes.

Ethical considerations

Our approach meets ethical principles [35] as outlined in the Canadian Tri-Council Policy Statement: Ethical Conduct for Research Involving Humans–TCPS 2 (2022) and has been approved by Clinical Trials Ontario [####]. We used a similar approach in our recently published provincial registry-based trial of reminders for heart attack survivors [36].

A delayed intervention for the control group by 1 year will allow us to determine whether the intervention has an effect without putting the control group at undue risk by withholding critical health information (given current equipoise) and without creating substantial selection bias.

Ontario Health’s current approach to identifying and contacting patients for research purposes also involves using administrative datasets. Ontario Health is designated a “prescribed entity” in Sect. 45(1) of the PHIPA, 2004. As a prescribed entity, Ontario Health is authorized to collect personal health information from health information custodians without the consent of the patient, and to use such information for the purpose of analysis or to manage, evaluate, or monitor the allocation of resources. Ontario Health is also authorized to use personal health information for planning purposes for all or part of the health system, including the delivery of services. Participants cannot opt out of the analysis of these de-identified linked datasets.

Randomization and analysis of de-identified administrative data following an intent-to-treat protocol will occur without prior consent, but meets TCPS-2 criteria for this, based on (i) minimal burden or risk of harm from the intervention, (ii) infeasibility of answering the question at hand if prior consent was required, and (iii) provision of a debrief. Specifically, we will send the introductory information to both groups based on their randomized allocation (i.e., immediately or after a delay for the control group). Seeking prior consent from all those eligible for this health system intervention prior to sending them the introductory package would be redundant (because it would explain the reason for contact) and would involve substantial risks for performance bias, threatening the ability to accurately assess the effects of ONLOOP as it would be implemented, without reducing meaningfully any risk to the welfare or autonomy of participants. Upon receipt of the introductory letter, which denotes that ONLOOP is a research study, recipients could then choose to opt-in to receive (or not) the full intervention package and they would choose (or not) to have information sent to their primary care clinician. Finally, in our study invitation letter and on our study website, we offer the option to contact the study team for results as a debrief once the study is complete.

Process evaluation

Informed by guidance on process evaluations for complex interventions, we will assess intervention implementation fidelity (whether the intervention was delivered as intended), dose (the quantity of intervention implemented), and mechanisms of impact of the intervention [37, 38]. We will seek to understand: (1) how the intervention interacts with its context; (2) the underpinning program theory; (3) how diverse stakeholder perspectives can be included; (4) the key uncertainties; (5) how the intervention can be refined; and (6) whether the effects of the intervention justify its cost.

Fidelity and dose

The process evaluation will determine the fidelity of the intervention content (whether the intervention was delivered as intended) and dose (the quantity of intervention implemented). We will identify the proportion of all eligible intervention arm survivors who complete each intervention step: intervention sign-up (i.e., requesting their information toolkit), and whether they completed their surveillance test(s). We will assess factors (e.g., age, sex, location) that may explain their degree of engagement with ONLOOP. In the interviews, fidelity and dose will be further explored to understand the program theory, causal mechanisms, and contextual factors associated with variation of fidelity and dose. For example, we will assess understanding of the information kit, and why survivors may or may not have used their information kit to complete recommended surveillance tests.

Data collection

At 6 months post-randomization of the study cohort, we will carry out the process evaluation components below with survivors in the intervention arm and with their primary care providers. In the second study invitation reminder letter, we ask survivors who do not consent to the ONLOOP Trial for consent to send them information about the process evaluation interview. They can participate in this interview even if they do not participate in the ONLOOP Trial. We will use Canada Post to mail out process evaluation interview invitations. Survivors and primary care providers will be asked to contact the study team if they are interested in participating. Trained study team members from Women’s College Hospital will obtain informed consent from all individuals interested in participating in an interview.

Survivor interviews

We aim to recruit approximately 15 survivors who opted into ONLOOP and 15 who did not opt-in (i.e., non-engagers). All study interviews will last approximately 30–45 min and will be conducted by telephone or Zoom and will be recorded for transcription purposes.

Interviews with survivor’s primary care provider

The study team will mail an invitation to approximately 200 primary care providers who received a postal letter or fax regarding their patient’s screening recommendations. We anticipate a sample size of n = 15 and will stop recruitment once we reach thematic saturation (no new codes emerge from the analysis). We will purposively recruit physicians whose patients were recommended to receive a mix of cardiac, breast, and colorectal tests. We aim to recruit a diverse group of physicians based on age, location (urban vs rural), and type of practice (Family Health Team vs other).

Interviews with aftercare nurses

We will mail an invitation to AfterCare nurses to understand whether survivors were referred to an AfterCare clinic after receiving their ONLOOP Trial invitation. The study team will send an email invitation to each of the lead AfterCare nurses at the 7 sites across Ontario.

Physician fidelity check to confirm receipt of information letter by the childhood cancer survivor’s physician

We will assess whether physicians reviewed the physician’s letter. Approximately 6 months after intervention delivery, a research team member at WCH will call a subset of physicians whose patients were randomized to the intervention arm and provide consent to contacting their physicians. Physicians will be sorted by sex and area code, and we will randomly sample from each stratum until 78 physicians have been called. This sample size was calculated based on an anticipated proportion of 30% of physicians remembering receiving the letter, and a margin of error no greater than ± 10%. Each physician selected will be called up to 3 times over the course of one week. Non-respondents to phone calls will be tracked. Physicians who answer the call will be asked whether they received the letter from the study team; those who reviewed the letter will be asked if they can confirm what they thought the main message was, and whether they have any questions for the study team. Our research will not impact the participant’s employment, relationship with provincial organizations, or reputation in any way. It is meant to be informative for the research community and to contribute to the growing and relevant body of research.

Confidentiality

Interviews will be audio-recorded and then transcribed verbatim by an external third party. Any identifiable information (i.e., names of individuals or institutions) will be removed from transcripts to ensure that respondents remain anonymous. Identifiable information will not be used in any study records, except the consent form (which will be stored separately from the other study records). The interview transcripts will be assigned a unique identification code and will be referred to by this code during discussions and in documents. The audio recordings will be stored in a secure location and viewed only by members of the research team. The recordings will be kept until they have been transcribed, and then they will be destroyed. The transcripts will be kept in a secure location for 7 years and then destroyed.

Analysis of interviewsSurvivor interviews

We will employ a directed content analysis approach using the domains in the Theoretical Domains Framework (TDF) [16, 39] as deductive codes. The TDF is a comprehensive, theory-informed approach frequently used by implementation scientists to identify the determinants of behaviors in healthcare professionals and patients. Themes will then be developed inductively within and across TDF domains to describe barriers and enablers of key behaviors required as outlined in the program theory. In addition to the TDF, we will use the Social Determinants of Health framework to consider the broader determinants of health in a more comprehensive approach to addressing surveillance adherence [40, 41]. We will use cross-case comparisons, with cases defined at the level of the outcome (i.e., engagement at a given step and completion of surveillance tests) to understand the determinants of impact (or lack thereof). We will also explore how change is being brought about and how this may vary across different contexts and survivors. Analyses will help determine the key components of the intervention that need to be preserved in implementation to maximize the likelihood of achieving the effects found in the evaluation.

Physician interviews

We will explore their perceptions of ONLOOP and how they interacted with the intervention. Interviews will be informed by the TDF and will be used to explore barriers and enablers of the key behaviors required as outlined in the program theory [16]. Each behavior related to the desired intervention processes (e.g., reading the recommendation fax, discussing with their patient, ordering tests, setting reminders) will be specified [42] and explored to understand barriers and enablers and to inform intervention refinement [43]. The interview guide will be piloted with two physicians using the “think aloud method” [44] and will be revised as needed.

Aftercare nurse interviews

Interviews will explore how the intervention intersected with the survivor aftercare clinics. We will ask nurses about their experiences with survivors who engaged with the aftercare clinics after receiving the intervention materials. The interviews will explore from the aftercare nurses' perspectives how the intervention for survivors complements the aftercare clinics and the ideal way forward for the intervention to exist alongside the aftercare clinics. Transcripts will be analyzed by thematic analysis using inductive coding to describe the manifest and latent content [45]. Inductive coding will be considered on a case-by-case basis. Subsequent levels of coding will involve re-examining the content of the codes and narrowing in on more specific elements discovered in the data during coding. Initial themes will be reviewed and refined to ensure that the themes represent the dataset as a whole and that no themes are missed or over-represented.

Economic evaluation

We will conduct a cost-effectiveness analysis of ONLOOP compared to the delayed-intervention arm from the perspective of Ontario Health.

We will use a top-down and bottom-up approaches to estimate the costs of developing and delivering the intervention. We will obtain resource use data for developing and implementing the intervention, as well as their unit costs, from program financial records, service level agreements, and the program budget. We will track the time and costs required to deliver the intervention. Costs associated with health care utilization will be derived using cost macros developed at ICES [29]. Consistent with the trial, effectiveness will be measured as the proportion of survivors who complete one or more of the guideline-recommended cardiac, breast, or colon surveillance tests within 12 months after randomization (in accordance with the primary outcome). Analyses will conform to the most recent Canadian guidelines for economic evaluation [46] and current guidelines for such analyses in randomized control trials [47].

The incremental cost and outcome will be estimated using generalized estimating equations, a flexible multivariate regression framework that explicitly allows the modeling of non-normal distributional forms of repeated measures data. We will evaluate the uncertainty of the cost-effectiveness estimates using non-parametric bootstrapping. We will obtain 5000 estimates of costs and outcomes for each option. Results from the bootstrapping exercise will also be used to estimate 95% confidence intervals and depict cost-effectiveness acceptability curves, which show the probability of ONLOOP being cost-effective to a range of potential threshold values that the health system may be willing to pay for an additional unit of effect. We will also perform a budgetary impact analysis [48] to estimate the resources and financial implications of implementing the intervention in Ontario and other Canadian provinces over 5 years. As suggested by our policy stakeholders, we will conduct a scenario analysis by developing a simple simulation model and using data from the trial to project cost and adherence outcomes over a survivor’s lifetime. This scenario analysis could show the potential long-term consequences of the intervention beyond the duration of the trial.