Figure 1 illustrates a summary of the data collection process, sources, different methodologies, and timeline throughout the CINDERELLA clinical trial.

Fig. 1

Timeline, methodologies, and sources of data collection throughout the CINDERELLA trial. eCRFs electronic case report forms, EQ-5D-5L EuroQol 5 dimensions 5 levels, HCP healthcare professionals, ICHOM International Consortium for Health Outcomes Measurement, MAUQ mHealth Apps Usability Questionnaire, TDABC time-driven activity-based costing, uMARS user version of the Mobile App Rating Scale

2.1 Economic Evaluation

The objective of the economic evaluation is to evaluate the costs and consequences of the CINDERELLA APProach versus the standard of care over the trial period through trial-based cost-consequence and cost-utility analyses, conducted in accordance with the Professional Society for Health Economics and Outcomes Research (ISPOR) Good Research Practices [28].

The patients will be followed from randomization to 12 months after the end of locoregional treatment (either surgery or radiotherapy, depending on the last treatment received). The total duration of the trial will be approximately 36 months, with recruitment spanning approximately 18 months. The expected sample size is 1030 patients (515 in each arm), calculated based on the two primary objectives (i.e., (i) differences in pretreatment expectations and the final aesthetic outcomes between the trial arms; (ii) agreement of the pretreatment AI evaluation and patients’ post therapy self-evaluation in treatment arm) and adjusted for potential dropouts [20].

The economic evaluation analyses will be carried out from a societal perspective and will cover the jurisdictions involved in the clinical trial (i.e., Portugal, Germany, Italy, Israel, and Poland). The Consolidated Health Economic Evaluation Reporting Standards for Interventions That Use Artificial Intelligence (CHEERS-AI) will be followed when reporting the health economic evaluation [29, 30].

2.1.1 Identification, Measurement, and Valuation of Costs

According to the societal perspective, all cost categories will be included in the analysis (direct healthcare costs, direct nonhealthcare costs, and indirect costs). Table 1 provides, for each resource use input, a summary of the sources and timing of measurement, as well as valuation sources for the different jurisdictions. Resource use data will be collected until 12 months follow-up through different sources: (i) trial electronic case report forms (eCRFs); (ii) online questionnaires filled in by the healthcare professionals (hereafter, HCP questionnaires) in presence of the patients; (iii) online questionnaires filled in directly by the patients (hereafter, patient questionnaires), which will be translated in the main language in use at the trial centers; (iv) interviews conducted with the staff and healthcare personnel in participating clinical centers, adopting the Time-Driven Activity-Based Costing (TDABC) methodology for estimating personnel cost; and (v) interviews with CINDERELLA platform developers (i.e., CANKADO).

Table 1 Measurement and valuation of resource use

All resource use will be valued in monetary terms using appropriate country-specific unit costs or participant valuations (e.g., for travel and accommodation expenses). Unit costs will be derived from national costing manuals (e.g., formularies, official national list pricing), the hospitals’ financial and administrative records (e.g., sourced from the management and control office and/or from the clinical pharmacy), publicly available datasets [e.g., Organisation for Economic Co-operation and Development (OECD) data] or sourced from the literature when not available from the previously listed sources. In case multiple sources are available for the same cost item, scenario analyses will be conducted using different cost sources. All costs will be expressed in Euros and referred to 2023, the year of trial start.

2.1.2 Identification, Measurement, and Valuation of Benefits

The benefits of the intervention will be estimated by considering patient-reported outcomes (PROs), that will be measured in the trial using questionnaires administered to patients in both arms at different time points (Table 2). For conducting the cost-consequence analysis, a variety of outcomes will be considered, namely expectations about the aesthetic outcome, Quality-Adjusted Life Years (QALYs) and patient satisfaction. For cost-utility analysis, only QALYs will be considered.

Table 2 Outcomes for economic evaluation

The patients’ match of expectations about the aesthetic outcome of locoregional treatment, before and after treatment, will be measured through the Expectations Questionnaire, developed ad hoc for the CINDERELLA trial. Patients’ expectations will be recorded using a five-point scale: much worse, worse, same, better, much better [20]. QALYs will be calculated from responses to the EQ-5D-5L questionnaire [31]. The EQ-5D-5L questionnaire is a generic quality of life instrument with questions covering five domains: mobility, self-care, usual activities, pain and discomfort, anxiety and depression. Each dimension is divided into five levels of severity: no problems, slight problems, moderate problems, severe problems, and extreme problems. First, the responses on the five domains of the EQ-5D-5L will be converted into a synthetic index using utility scores. This index is anchored between 0, corresponding to dead, and 1, corresponding to perfect health. Country-specific utility values will be used for Portugal, Germany, Italy, and Poland. If not available at the time of data analysis, a value set from a comparable country will be used for Israel. QALYs will be computed by multiplying the duration of time spent in a determined health state by the utility of that health state. Patient satisfaction will be recorded through the BREAST-Q International Consortium for Health Outcomes Measurement (ICHOM) [32, 33], a disease-specific PRO measure. In its complete version, it aims at measuring the satisfaction and HRQoL in patients undergoing breast surgery. In this trial, only the subdomain concerning patients’ satisfaction with breasts and breast cancer treatment will be considered. Patient satisfaction will be measured on a four-point scale: very dissatisfied, somewhat dissatisfied, somewhat satisfied, very satisfied. Responses will be converted into a synthetic score, ranging from 0 to 100 (the higher the score, the better the outcome), using questionnaire-specific conversion tables.

2.1.3 Statistical Analysis

The main analyses will be carried out considering the “intention to treat” (ITT) population to avoid affecting the randomized allocation. In the ITT analysis, all randomized patients will be included, regardless of actual intervention received or adherence to the intervention protocol [34]. This includes patients who did not use the CINDERELLA APProach in the intervention arm and those who dropped out from either arm. This is a realistic approach that reflects the clinical scenario in the real-world, where noncompliance, dropouts and protocol deviations may frequently happen. A per-protocol analysis will also be conducted [35]. This analysis will include only a subset of the ITT population who completed the study without any major protocol violations (i.e., patients who did not adhere to treatment, switched groups, missed measurements, or dropped out will be excluded).

Descriptive statistics will be presented along with statistical tests of difference, either parametric (for normally distributed data) or nonparametric (for nonnormally distributed data) as appropriate.

The use of resources will be estimated for both arms. Differences in the consumption of resources will be qualitatively described but not compared statistically. Mean and median total costs, also disaggregated by cost category, will be computed and appropriate measures of dispersions (e.g., 95% confidence intervals) will be reported. Multivariable regression analyses will be performed to assess the impact on costs and outcomes of different baseline variables that, according to the literature, are expected to have an impact on SDM, patient outcomes, and/or costs. The set of variables includes age, education level, marital status, comorbidity profile (Charlson or Elixhauser index), employment status, digital health literacy, and trial country. As an example, marital status is used as a proxy of family support, which has been documented as significantly associated with patients’ HRQoL in breast cancer patients [36]. Employment status is considered given that recent works [37] showed that, during oncology consultations, employed patients with a recent diagnosis of breast cancer asked significantly more questions than nonemployed and retired patients, which may suggest a more pronounced attitude toward SDM. The threshold for statistical significance will be set at 0.05.

Trial data will be examined for any missing data. The appropriate method for dealing with missing data will depend on the proportion of missing data and likely mechanism of missingness [i.e., missing completely at random (MCAR); missing at random (MAR); missing not at random (MNAR)] [38]. Multiple imputation will be undertaken to account for MAR and MCAR.

Analyses will be conducted using StataSE, R and Excel (latest versions available).

2.1.4 Cost-Consequence and Cost-Utility Analyses

The mean differences in costs and outcomes for the two arms will be provided. The results of the cost-consequence analysis will be presented in a disaggregated format, with costs and outcomes (i.e., expectations about the aesthetic outcome, QALYs and patients’ satisfaction) reported separately. Estimates (mean and measure of dispersion) will be provided for each arm, along with the incremental differences between arms. Trade-offs between costs and effects will not be explicitly provided. For the cost-utility analysis, the incremental cost–utility ratio (ICUR) will be estimated, dividing the difference in total mean costs between the CINDERELLA APProach and the standard of care arm by the differences in QALYs. The ICUR will be interpreted using appropriate cost-effectiveness thresholds. As none of the jurisdictions involved in the clinical trial provides an explicit, discrete value for cost-effectiveness threshold, the cost-utility analysis will use two generic cost-effectiveness thresholds of 50,000 €/QALY and 100,000 €/QALY. Discounting will not be applied to either costs or benefits as the economic evaluation will be performed over a time horizon of 12 months.

Sensitivity analyses will be conducted to explore uncertainty surrounding key parameters and determine the reliability of the results from the base-case analysis [28]. Univariate sensitivity analysis will be performed by varying one key parameter at a time across its plausible range, while the remaining values are held at their baseline values. Results will be presented in table format and/or graphically through a tornado plot. Probabilistic sensitivity analysis will be performed by assigning to each key input parameter a specified distribution and by drawing randomly from this distribution. The nonparametric bootstrapping approach will be used to determine the level of sampling uncertainty surrounding the mean costs, outcomes, and cost-effectiveness summary measure (ICUR) by generating 10,000 estimates of (incremental) costs and benefits. To represent decision uncertainty in the cost-utility analysis, cost-effectiveness acceptability curves (CEACs) will be used. CEACs display the proportion of the estimates produced by bootstrapping that would be considered acceptable over a range of willingness-to-pay (WTP) thresholds. Results will be presented in table format and/or graphically through a cost-effectiveness plane when appropriate (e.g., for cost-utility analysis).

Subgroup analyses will be conducted to investigate how costs and outcomes vary between different patient subgroups. Following the same rationale explained in the previous section, several baseline variables will be used for conducting these analyses, including age, education level, marital status, comorbidity profile, employment status, digital health literacy, trial country.

2.1.5 Budget Impact Analysis

The economic evaluation will be complemented by a budget impact analysis to assess the financial impact of the CINDERELLA APProach implementation. This analysis will be conducted from the healthcare provider and the healthcare system perspectives (i.e., the budget holders’ perspectives) over two different time horizons: 1 year (short term) and 5 years (medium term), in accordance with the ISPOR Good Research Practices [39].

The cost analysis will be drawn from that conducted for economic evaluation, although only hospital-related costs will be included when considering the healthcare provider perspective, and direct healthcare costs will be included when considering the healthcare system perspective. Discounting will not be applied as the budget holder is interested in the expected financial streams at each point in time rather than their net present values.

To conduct the budget impact analysis, which compares the status quo (without the CINDERELLA APProach) with a future scenario entailing the gradual uptake of the CINDERELLA APProach, additional parameters besides resource use and costs will be estimated, namely the eligible population and the uptake of the new intervention. The eligible population consists in all patients eligible for the new intervention, i.e., the CINDERELLA APProach, during the time horizon of interest, given any access restrictions. The definition of the eligible population will start with an analysis of country-specific epidemiological data, namely disease prevalence and incidence, which will be sourced from either European-based databases (e.g., European Cancer Information System [40]) or national cancer registries (e.g., AIRTUM–Italian Network of Cancer Registries [41]). This will be complemented with an estimate of the number of patients covered by the locally approved indications for the new technology (i.e., breast cancer patients eligible for locoregional treatment). The uptake of the CINDERELLA APProach is, by definition, not known at the time of analysis, therefore some hypotheses on its diffusion need to be made and included in the future scenario. More specifically, two different changes will be included in the analysis: (i) the CINDERELLA APProach is added to the standard of care (combination) and (ii) the CINDERELLA APProach replaces the standard of care (substitution). Expert opinions will be collected to hypothesize credible levels of uptake for the new intervention.

The total costs of the status quo and the future scenario will be estimated and compared to estimate the financial impact of the uptake of the CINDERELLA APProach.

2.2 Implementability

The effectiveness of an intervention is critically influenced by its implementation in a given context [42]. Implementation science has emerged as the scientific study of methods to promote the systematic dissemination of research results in clinical routines, with the aim of improving the quality and effectiveness of health services [43]. By embedding an implementation research component into the CINDERELLA trial, we aim to foster the successful diffusion of the CINDERELLA APProach via a thorough understanding of the factors that affect its translation into real-world settings and an evaluation of its usability, acceptability, organizational impact, and overall feasibility.

2.2.1 Usability

Usability can be defined as “the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use” [44]. User evaluation is one of the most commonly used method for assessing the usability of mobile applications [45], with several rating scales developed to evaluate the usability of mHealth apps [46, 47]. The usability of the CINDERELLA mobile application will be evaluated through an online questionnaire administered to patients in the intervention arm at 6 and 12 months follow-up. The questionnaire was developed starting from two previously validated questionnaires, namely the standalone version of the mHealth Apps Usability Questionnaire (MAUQ) for patients [48] and the user version of the Mobile App Rating Scale (uMARS) [49]. The MAUQ standalone version for patients contains 18 items, referred to three dimensions of app usability (i.e., ease of use, interface and satisfaction, usefulness). For each item, patients will be asked to express their degree of accordance on a seven-point scale (strongly agree, agree, somewhat agree, neither agree nor disagree, somewhat disagree, disagree, strongly disagree). From the uMARS questionnaire, we extracted only the four questions concerning the information quality subscale, whose answers are on a five-point scale. The questionnaires were translated into the main language in use at the trial centers.

Data on CINDERELLA usability derived from the questionnaire will be analyzed quantitatively. First, a principal components analysis (PCA) will be performed [50] to evaluate the psychometric properties of the questionnaire obtained from the integration of MAUQ and uMARS questionnaires, separately for each trial country to accommodate differences in questionnaire language, and the resulting subscales will be tested for internal consistency (Cronbach’s alpha). Then, a descriptive analysis will be conducted on the collected data. The means and standard deviations (SD) for individual statements, subscales and the entire questionnaire will be computed. Based on data distribution, parametric (e.g., t-test) or nonparametric tests (e.g., Kruskal–Wallis test) will be used to investigate differences among patients according to different baselines characteristics (e.g., age, education level, digital health literacy).

2.2.2 Acceptability

Acceptability is a multifaceted concept and, to date, there is still no consensus around its definition [51]. Sekhon and colleagues (2017) defined acceptability of healthcare interventions as “the extent to which people delivering or receiving a healthcare intervention consider it to be appropriate, based on anticipated or experienced cognitive and emotional responses to the intervention” [52], and Weiner and colleagues (2017) as “the perception among implementation stakeholders that a given treatment, service, practice, or innovation is agreeable, palatable, or satisfactory” [53]. Acceptability can be related to several concepts, including users’ attitude (e.g., how they feel about the intervention), usage intentions (e.g., willingness to engage), actual usage (e.g., number of interactions), and satisfaction after having engaged with the intervention [54]. The users’ perception of acceptability can be measured at different points in time, namely before (prospective acceptability), whilst (concurrent acceptability), and after (retrospective acceptability) engaging with the intervention [52]. For the purpose of this study, the acceptability of the CINDERELLA APProach will be measured according to two different user perspectives, i.e., the patients’ and the healthcare professionals’ perspective, as the successful implementation of a healthcare intervention depends on its acceptability to both intervention deliverers and recipients [55]. Moreover, data collection will be performed at different time points to capture both concurrent and retrospective acceptability.

Acceptability for patients will be referred strictly to the CINDERELLA mobile application and will be measured alongside different dimensions and using different data collection methods. First, the app’s actual usage will be evaluated using data collected electronically by the CANKADO system (e.g., number of log ins, app retention rates) throughout the clinical trial (concurrent acceptability). Second, patient satisfaction will be derived from the related section in the MAUQ questionnaire [48] at 6 and 12 months follow-up (concurrent and retrospective acceptability).

Quantitative data on CINDERELLA acceptability for patients (i.e., app’s usage and patients’ satisfaction) will be summarized using medians, interquartile ranges, means, standard deviations, counts, and percentages as appropriate. Based on data distribution, parametric (e.g., t-test) or nonparametric tests (e.g., Kruskal–Wallis test) will be used to investigate differences among patients according to different baselines characteristics (e.g., age, education level, digital health literacy).

Third, semi-structured interviews will be conducted with a sample of patients to measure retrospective acceptability at the end of the trial period (12 months follow-up). The final sample size for this study phase is not determined a priori but will be based on the data saturation principle [56]. We will set an initial sample equal to ten patients; after the first ten interviews, three further interviews will be conducted to verify whether there are any emerging themes [57]. The interview process will continue until no new insights emerge from the interviews. The enrolment will be done by the healthcare professionals at each clinical center, and the interviews will be conducted remotely with commonly used teleconferencing tools (e.g., Microsoft Teams) to accommodate the geographical dispersion of the interviewees. Every interview will last approximately 40 minutes. The interviews will be conducted making efforts to ensure that the interviewers speak the language of the interviewees. All interviews will be audio-recorded and transcribed. When needed, the interview transcripts will be translated into English. Back translation will be done to ensure the quality and accuracy of the translated text.

For healthcare professionals, acceptability of the overall CINDERELLA APProach will be evaluated based on qualitative data from focus groups. As the care pathway of breast cancer patients usually requires the coordination among different professional figures, focus groups are deemed an appropriate method to capture interaction, stimulate discussions and explore different experiences and points of view [58]. Two types of stakeholders will be primarily involved in the focus groups at each trial site: (i) breast surgeons/oncologists and (ii) nurses. The focus groups will be conducted either face-to-face or remotely (with commonly used teleconferencing tools, e.g., Microsoft Teams), according to the healthcare professionals’ availability. Focus groups will be conducted at two different time points, namely at 12 months and 24 months after trial start to perform a longitudinal analysis of experienced acceptability. Focus groups will be audio-recorded after obtaining verbal consent from each participant. The recordings will be verbatim transcribed. The outlines of the interview (Supplementary Material 1) and focus group (Supplementary Material 2) will be pilot tested and, if needed, modified iteratively.

The transcripts of the interviews and focus groups will be analyzed using thematic analysis [59], and the themes will be driven by a theoretical framework of acceptability [52]. Findings for the qualitative data from the interviews and focus groups will be reported using the Consolidated Criteria for Reporting Qualitative Research (COREQ) checklist [60].

2.2.3 Organizational Impact

The assessment of the organizational impact is critical for complex interventions, whose adoption in clinical practice frequently requires substantial organizational investments, multidisciplinary teams, need for supervision, change in the relationships between different organizations, and training for patients and professionals [61]. The evaluation of the organizational impact of the CINDERELLA APProach will be performed by conducting focus groups at each clinical center at different time points. One focus group will be held at the beginning of the trial with the aim of describing the current organizational model and in particular the delivery and service model (“as is” scenario). Two additional focus groups will be conducted at 12 months and 24 months after the trial start to understand how the organizational model has changed with the introduction of CINDERELLA APProach (“to be” scenario). Specifically, we aim to investigate whether and how work processes, human resources availability and skills, relationships between healthcare professionals, resource need, and involvement of patients have changed following the implementation of the intervention. Focus groups will be audio-recorded, and the recordings will be verbatim transcribed. The outline of the focus groups (Supplementary Material 3) will be pilot tested and, if needed, modified iteratively. In analyzing the data using thematic analysis [59], we will rely on the HTA Core Model proposed by the European Network for Health Technology Assessment (EUnetHTA) [61]. Findings will be reported using the COREQ checklist [60].

2.2.4 Feasibility

Feasibility of an intervention can be related to intervention’s content and delivery, acceptability, adherence, likelihood of cost-effectiveness, or capacity of providers to deliver the intervention [62]. Evaluating the feasibility of an intervention allows to determine whether the intervention is appropriate, and to identify what (if anything) in the research methods or protocols needs modification and how changes might occur [63]. The feasibility of the CINDERELLA APProach will be evaluated by integrating the evidence collected in previous phases, namely economic evaluation, usability, acceptability, and organizational impact. Although a comprehensive and unique framework for assessing feasibility does not exist, our analyses will be driven by a recent conceptual framework developed by Klaic and colleagues (2022) [64].

2.3 Environmental Impact

The environmental impact assessment of the CINDERELLA APProach will employ the framework proposed by MacNeill and colleagues (2021) [65]. The framework has several strengths, including a clear delineation of the dimensions addressed, a comprehensive perspective on the opportunities for the healthcare system to mitigate its diverse and heterogeneous environmental impacts, and its linkage with dimensions that that can be monitored with well-known indicators in ongoing studies. Grounded in three fundamental principles, the framework focuses on (i) reducing demand for health services; (ii) matching the supply of health services with demand, ensuring appropriate care delivery; and (iii) reducing emissions associated with the provision of health services. Consequently, the environmental impact of the CINDERELLA APProach will undergo quantitative evaluation across three key dimensions: quantity, appropriateness, and emissions.

The quantity dimension requires an estimation of the amount of care received by patients in the CINDERELLA versus the standard of care arm. To provide a comprehensive estimate of the different healthcare services received, their costs will also be considered. Hence, data on the healthcare services consumed together with their costs will be drawn from same sources employed for conducting the economic evaluation and analyzed accordingly for comparisons.

To assess appropriateness, a set of key performance indicators (KPIs) (Table 3) will be calculated for the two arms starting from the data collected through the eCRFs and compared using statistical tests of difference with 95% confidence intervals. The indicators have been recently developed [66, 67] and focus specifically on avoiding inappropriate use of healthcare services after breast cancer interventions.

Table 3 Description of KPIs for the “appropriateness” dimension of the environmental impact analysis

Emissions resulting from patients’ travel to access healthcare services will be estimated using data collected from HCPs and patient questionnaires at the designated time points for economic evaluation. Data will be derived from close-ended questions investigating average travel distances to reach healthcare facilities (e.g., for hospital visits) and modes of transport (e.g., car, public transportation). The total distance traveled will be calculated by summing the declared kilometers for each healthcare service, considering the mean of the range reported by the patient. The emissions associated to different travel modes will be based on the most recent literature [68]. A multivariable linear regression model, with kilometers traveled as the dependent variable, will be fitted to identify variables with the greatest impact on costs, using baseline covariates.

Given the absence of a unified framework for environmental impact analysis, the quantitative assessments will be supplemented by qualitative insights from healthcare professionals and managers. A focus group with healthcare professionals at participating clinical centers at 24 months after trial start will investigate their perceived commitment in terms of practices adopted and information shared with patients. Additionally, interviews will be conducted with healthcare managers at participating clinical centers to address issues related to the healthcare sector's awareness of its responsibility in reducing environmental impact and increasing awareness among healthcare workforce and patients. The outline of the interviews and focus groups are provided in Supplementary Material 4 and 5, respectively. The qualitative data pertaining to environmental impact will be analyzed using the same methodology previously described for assessing acceptability and organizational impact.

In light of the ongoing debate on how to incorporate environmental impact within health technology assessment, we opted for a parallel evaluation [69], a flexible approach that entails analyzing and presenting environmental data alongside established health economic analyses.