Explanation for the choice of comparators

The SOUL study is a pragmatic comparison between two existing management strategies that are both widely considered routine standard clinical care. All potential participants are planning to undergo ureteroscopy with a urologist trained in ureteroscopy in accordance with American Urological Association (AUA) guidelines. The intervention in this trial is stent omission. Patients will either continue to receive a ureteral stent with the surgeon’s choice of stent, or undergo stent omission, both of which are widely accepted as standard clinical care. In the observational cohort, assignment to stent omission will be at the discretion of the operating urologist per routine clinical practice, which may or may not include patient preferences. In the randomization cohort, assignment to stent omission versus stent placement will be by 1:1 randomization. When a stent is placed, it may or may not be left on a string (tether), at the operating urologist’s discretion. The duration of stenting, and how it is removed, is left to the discretion of the urologist.

Intervention description

The only study-specific procedures representing anything more than routine clinical care are patient surveys. All participants in both the randomization and observational cohorts will receive PRO questionnaires via e-mail or alternative means of communication at several prespecified time points: pre-operatively, at 7–10 days after surgery, and at 4–6 weeks after surgery. If the patient is unable to complete the study questionnaires electronically, the patient may complete questionnaires over the telephone with assistance from a SOUL study team member or paper questionnaires upon request. Patients will also receive automated questionnaires (daily Ecological Momentary Assessments (EMA)) via text message, daily for 10 days after surgery and on day 30 after surgery. PRO and EMA responses will be recorded and secured.

We will also conduct semi-structured interviews with an initial subgroup of 20–30 study participants, striving for evenly balanced representation between the randomized and observational cohorts, to elicit prior knowledge, desires, preferences, and experiences regarding stent placement and stent omission. We will endeavor to keep patient interviews to less than 45 min. They will be conducted by phone or web-based conferencing service and recorded with the permission of the participant. Full transcripts will be made and anonymized for analysis, and the original recordings permanently deleted. Based upon the findings from these interviews, a survey will be developed and distributed to all participants in the study.

Similarly, we will conduct semi-structured interviews with urologists participating in the trial to evaluate perceptions, experience, preferences, and practice patterns regarding stent omission versus stent placement following uncomplicated ureteroscopy. We will also inquire about important factors and determinants in the urologist’s decisions to omit or place a ureteral stent. We will endeavor to interview all urologists participating in the trial or until we reach thematic saturation. We will strive to include representation from urologists in a variety of practice settings and include urologists both within and outside the state of Michigan. We will endeavor to keep urologist interviews to less than 30 min. They will be conducted by phone or web-based conferencing service and recorded with the permission of the participating urologist. Full transcripts will be made and anonymized for analysis, and the original recordings permanently deleted.

A CONSORT diagram of the detailed study design and workflow is provided in Fig. 2.

Fig. 2

CONSORT flow diagram showing study design with intersection of aims

Criteria for discontinuing or modifying allocated interventions

Stent omission (versus placement) is performed at a single time point intraoperatively and is not subject to discontinuation. However, a subject’s participation in patient surveys (PRO and EMA) may be discontinued at an investigator’s discretion for the following reasons:

Significant study non-compliance or inability to meaningfully complete study surveys

If the participant meets an exclusion criterion (either newly developed or not previously recognized) that precludes further study participation

If continued participation in the study would not be in the best interest of the participant

Participants are also free to withdraw from participation in the study at any time upon request.

Strategies to improve adherence to interventions

Adherence to the intervention (stent omission versus stent placement) occurs in the operating room at a single time point near the conclusion of surgery and is dependent upon proper understanding of the trial and workflow by the surgeon and staff. Therefore, each site’s participating urologists and staff receive an extensive site initiation visit and orientation with coordinating center staff and investigative team members prior to beginning their participation in the trial, and multiple lines of open communication including an Acute Patient Hotline are maintained to facilitate quick answers to any questions or issues that may arise. A dedicated public web page (https://www.stentornot.com/) has also been created, containing information on workflows, inclusion and exclusion criteria, study documents, consent forms, and survey instruments, to facilitate on-demand refresher training and allow rapid accrual into the study, especially for patients presenting acutely through the emergency department with minimal lead time to coordinate enrollment. The web page also hosts patient educational materials and a patient-facing video to help inform and educate patients about the study.

Relevant concomitant care permitted or prohibited during the trial

The pragmatic nature of the study enables continued standard postoperative care as directed by the treating urologist. There are no study-specific restrictions on allowable concomitant care or interventions.

Provisions for post-trial care

The trial does not specify any provisions or restrictions for post-trial care. Unplanned healthcare encounters and unplanned secondary procedures within the postoperative data collection period will be recorded and analyzed as part of the unplanned healthcare utilization outcome metric.

Outcomes

Outcomes will be assessed and analyzed similarly in both the randomized and observational cohorts within the study. SOUL has two co-primary outcomes:

Patient-Reported Outcomes Measurement Information System (PROMIS) Pain Interference (Short Form 6b) T-score, change at postoperative days 7–10 compared to preoperative score

Unplanned stone treatment-related healthcare utilization within 30 days of ureteroscopy; a hierarchical composite comprised of:

◦

Intensive care unit (ICU) care during hospitalization

◦

Unplanned hospitalization

◦

Unplanned additional procedure related to ureteroscopy: operating room or interventional radiology procedure

◦

Emergency department visit

◦

Unplanned clinic visit and/or diagnostic testing (urine testing and/or imaging)

◦

Number of ambulatory patient–provider interactions: phone calls/EMR messages

Secondary outcomes include the following:

ICU care within postoperative day 30

Unplanned hospital admission within postoperative day 30

Unplanned additional procedure within postoperative day 30

Emergency department visit within postoperative day 30

Unplanned ambulatory urology office visit and/or diagnostic testing (urine testing and/or imaging) within postoperative day 30

Number of ambulatory patient-provider interactions: phone calls/EMR messages within postoperative day 30

PROMIS Pain Interference (Short Form 6b) T-scores change at postoperative 4–6 weeks compared to preoperative score

PROMIS Pain Intensity (Short Form 3a) T-scores change at postoperative days 7–10 and 4–6 weeks compared to preoperative score

Symptoms of Lower Urinary Tract Dysfunction Research Network (LURN) Symptom Index (SI-10) change at postoperative days 7–10 and 4–6 weeks compared to preoperative scores

International Consultation on Incontinence Questionnaire Satisfaction (ICIQ-S) scores at postoperative days 7–10 and 4–6 weeks

Number of days taken off work by patients and caregivers during the first 7 days after ureteroscopy

The following exploratory outcomes will also be assessed:

Ecological momentary assessments (EMA) measured via daily text message on postoperative days 1–10

◦

Postoperative days 1–10 reported interference with the performance of usual work from PROMIS Ability to Participate in Social Roles & Activities

◦

Postoperative days 1–10 visual analog scale (VAS) pain scores

◦

Postoperative days 1–10 patient-reported pain medication utilization

Abnormal postoperative imaging findings within 60 days postoperative

Stone-free imaging outcomes within 60 days postoperative

Patient interviews to evaluate prior knowledge and preferences about ureteral stent omission versus stent placement during ureteroscopy to treat kidney stones

Surgeon interviews to evaluate preferences about ureteral stent omission versus stent placement during ureteroscopy to treat kidney stones

Patient survey to evaluate prior knowledge and preferences about ureteral stent omission versus stent placement during ureteroscopy to treat kidney stones

Participant timeline

As part of standard care, all patients will undergo a routine preoperative evaluation. Table 1 shows a full schedule of study-related activities and assessments in accordance with SPIRIT guidelines.

Table 1 SOUL study participant schedule of activitiesSample size

The sample size was derived by the sample needed to power each co-primary endpoint for the RCT cohort. The larger sample size among the co-primary endpoints was used to estimate the final RCT cohort size. Each power calculation used a two-sided significance level of 2.5% to allow for two primary endpoints for a trial-level type I error of 5% as described below.

Postoperative days 7–10 PROMIS® Pain Interference change from pre-surgery is a co-primary endpoint; 101 participants per treatment arm will provide the RCT cohort with at least 90% power to detect a mean difference between stent placement vs. omission treatment groups of 5 points with an assumed standard deviation (SD) of 10 points, based on a two-sample t test (SAS 9.4), with a two-sided significance level of 2.5%. Previous work estimated that a clinically meaningful difference in PROMIS® Pain Interference is 2–6 points [25, 26]. A pilot study at our institution found an initial difference between stent omission and placement of 7.2 points with a SD of 13 for change and a SD of 10 at each time point. We expect our pilot may overestimate the treatment effect. Thus, we have targeted a difference of 5 points to represent a clinically significant target. A 10% loss to follow-up (failure to respond to the 7–10-day questionnaire) is assumed. Thus, 112 patients are needed per arm (total 224 patients). After consent, we estimate that 15% of patients who plan to be randomized will be unable to do so owing to intra-surgical factors that deem the procedure as complicated requiring 264 patients consented to the randomization cohort to attain 224 randomized patients.

Unplanned healthcare utilization within 30 days of ureteroscopy is the second co-primary endpoint.

With a modest probability of improvement in the reduction of healthcare utilization in ureteroscopy with stent omission, the impact in morbidity reduction and cost reduction would be great. There are no current estimates in the literature that help inform a minimum important difference for the win ratio estimate of healthcare utilization, our co-primary endpoint. So, we reasoned that a marginal probabilistic index, which is the effect size measure corresponding to the nonparametric Mann–Whitney U statistic that is used to test the hierarchical composite “win ratio” endpoint [27], of at least 67%, would be impactful as the study could conclude that a randomly chosen participant with stent omission has a two-thirds or greater probability of lower healthcare utilization than a randomly chosen participant treated with a stent. In the RCT with a two-sided 2.5% alpha and requiring 90% power to detect a treatment effect of 67%, 72 patients would be required per treatment arm. The sample size is based on the Finkelstein and Schoenfeld methods [28] for analysis of hierarchical composite endpoints, later called the win ratio by Pocock et al. based on the nonparametric Mann–Whitney U statistic [29]; the formula is provided in Yosef et al. [30].

The observational cohort sample size was derived from an assumption that 1/3 of patients approached and consented would be willing to be randomized. Thus, the observational cohort will consist of the estimated 2/3 (528) of patients who are approached and decline the RCT. With those assumptions, we find the following power for each primary endpoint.

Approximately 85% (449) of the observational cohort consented patients are expected to undergo an eligible ureteroscopy. From our prior data, approximately 70% (314) of these patients will receive a stent during ureteroscopy, and 135 (30%) will have stent omission. Assuming a 10% loss-to-follow-up (days 7–10 pain interference questionnaire not completed), this will result in 283 stent placement patients and 121 stent omission patients evaluable for the primary analysis of the observational cohort. Assuming a two-sided 2.5% type I error for each primary endpoint, 283 patients in the stent placement arm and 121 patients in the stent omission arm will provide 99.1% power to detect a clinically meaningful difference of 5 points between treatment arms for the change from pre-surgery in PROMIS® pain interference measure at days 7–10, assuming a standard deviation of 10 using a two-sample t-test. Sensitivity to the stented proportion and proportion agreeing to randomization (Table 2) demonstrates at least 84% power to detect a clinically significant difference in pain interference.

Table 2 Observational cohort power with sensitivity to our assumptions

The power estimates presented assume that each observation is independent. However, due to surgeon preference in the observational cohort, there may be a potential for clustering which could impact the power of the study. We expect that surgeons within a site may behave more similarly and the variability between sites will be the correlation that will be important. The intraclass correlation (ICC) across multiple studies and therapeutic areas has been shown to likely be small, with a range between 0.01 and 0.05 [31]. Using our planned sample sizes, we can estimate the sample size needed under a range of ICCs using a method described by Klar and Donner [32] with an estimate of the sample size inflation factor (IF) based upon the ICC and the average sample size per cluster. Table 3 provides the power from using the effective sample size and the assumed proportions for each arm (stent vs stent omission using standard power calculations for the two-arm mean difference using a t-test with a two-sided type I error of 0.025 with the assumptions used for Pain Interference described above.

Table 3 Power for pain interference co-primary endpoint in observational cohort with clustering effect (ICC), assuming 14 participating sites

For the co-primary endpoint of healthcare utilization in the observation cohort, if we assume that the stented proportion is 50%, with 202 participants per treatment group and a two-sided 2.5% type I error (alpha), we have 90% power to detect a treatment effect, marginal probabilistic index, of 60.1%. If the stented proportion is as low as 25% or as great as 75%, then we can infer that with 90% power, we will be able to detect a treatment effect between 60.1% and 64.3% as the sample size will be at least 101 per stent group.

Recruitment

To date, ten practices in Michigan who are members of MUSIC are taking part in the SOUL trial. In aggregate, these practices perform 2878 ureteroscopies annually, with an estimated 1960 meeting study eligibility criteria. MUSIC is also currently undergoing an expansion of membership to include practices outside Michigan and thus far four such practices have indicated their intention to participate in the SOUL study. These “Outdoor MUSIC” sites perform an estimated additional 1300 ureteroscopies annually, of which at least half are estimated to meet SOUL eligibility criteria.