In total, 59 NICE TAs in IO indications were identified. Of these, 34 included discussions of TEW. After screening for IO agent clinical trial stopping rules, 18 NICE TAs were included, with publication dates between January 2017 and September 2022. A total of, 13 were recommended by NICE, 3 were not recommended and 2 were entered into the Cancer Drugs Fund (CDF). The 18 NICE TAs are summarised in Table 1.

The application of TEW assumptions varied between submissions. In 11 of the 18 TAs, TEW was not applied in the company’s base-case. In 10 of these 11 TAs, independent survival models were used, so it was not clear whether the base case modelled treatment effect was increasing, decreasing or approximately remaining constant. In these instances, the External Assessment Group (EAG) and/or the appraisal committee often deemed scenario analyses that included TEW to be more appropriate for decision making.

Various methods for applying TEW were used across the 18 TAs reviewed. These included: immediate waning, gradual waning, conditional waning, waning involving cure assumptions and waning based on independently fitted survival models which converged in the long term. Table 2 provides a description of each of these approaches, outlining their underlying assumptions. Figure 1 illustrates the three most commonly used approaches—immediate waning, gradual waning and waning based on independent survival models.

A shows a treatment effect expressed as a hazard ratio from a (hypothetical) trial including a modelled treatment effect using independent survival models, a constant treatment effect and different types of treatment effect waning. B and C include plots showing hazard functions (converging hazards [B] and diverging hazards [C]) from a (hypothetical) trial (0–2-year trial period to the left of the vertical dashed lines) with extrapolation over a 30-year time horizon using independent survival models

Independent survival models were used in 17 of the TAs. Dependent models were used in 1 TA, more specifically a time-dependent treatment effect was incorporated. When independent survival models are used, assumptions around the treatment effect are implicit rather than explicit—i.e. the implied treatment effect over time depends upon the ratio of the hazards of the survival models fitted to each treatment arm, which may imply a treatment effect that is increasing, decreasing or constant. This implicit nature of the treatment effect when independent survival models are used is acknowledged in technical support document (TSD) 21, published by NICE’s Decision Support Unit (DSU). This document recommends that plots of the hazards and HRs predicted by independently fitted survival models should always be presented [8].

The use of independent models was not mentioned as a ‘waning approach’ in any of the TAs but is included as these models could (and often do) imply waning. Where hazards gradually converge (i.e. the treatment effect of an intervention is diminishing at a faster rate than the control), then this would imply that any TEW is to some extent already accounted for in the model. A crude illustration of this is shown in Fig. 1B. Figure 1C shows an illustration of hazards diverging and underlines the importance of examining these plots, as outlined in TSD 21 [8].

It is possible to gain some understanding as to whether independently fitted models predicted converging or diverging hazards over time by analysing the results of TEW scenarios compared with base-case scenarios (presented in the final column of Table 1). For example, if waning scenarios had only a minor impact on the incremental cost-effectiveness ratio (ICER), it is likely that the independently fitted survival models already predicted hazards that converged at a timepoint close to that used in the TEW scenario. In contrast, when TEW scenarios have a substantial impact on the ICER this may be because the independently fitted survival models predict diverging hazards or because the models predict hazards that converge at a timepoint substantially after that used in the TEW scenario. In some instances, TEW scenarios substantially increased the value of the ICER (e.g. comparing base-case to immediate at 3 years in TA428 [9]), lifting it above commonly accepted thresholds, therefore making TEW an important driver of uncertainty for decision making.

Few NICE TAs reviewed included a justification for the TEW methods used. It broadly appears companies generally held the view that there was a lack of evidence to justify not applying TEW, whereas EAGs and committees were of the opinion that a lack of evidence was not the same as evidence of absence. The often-missing justification and different perspectives from companies, EAGs and committees potentially led to TEW scenarios being explored without specific rationale for the type and timing of the waning modelled.

The reasons for the TEW assumptions used in different NICE TAs varied. They included referring to expert clinical opinion [10,11,12,13], results from certain clinical trials [14,15,16,17,18] and methods from previous NICE TAs for IOs [19,20,21,22]. In the appraisals which cited previous trials on the same treatment, the key reasoning used for this was that these trials had longer follow-up data and, therefore, could be used to either confirm or challenge the pre-existing TEW assumptions made by the company. When TAs cited previous TAs to justify their TEW assumption, the TEW assumption made in the previous TA was typically retained by the NICE committee, with this commonly involving immediate waning around 3–5 years. It is possible that this timeframe was used arbitrarily as it was effectively a precedent that had been set for IOs. Few appraisals assessed the clinical reasoning behind these waning timepoints. There was some evidence of precedence by disease type. For example, TA531 (ref. [23]), TA683 (ref. [19]) and TA770 (ref. [24]) for pembrolizumab in first line non-small cell lung cancer indications all explored immediate TEW at 3 years.

When searching the other English language HTA agencies, we encountered several instances where TEW approaches differed between HTA agencies. The most notable difference in broad approach taken was the substantially shorter time horizons often used for the PBAC appraisals (often 7 years versus lifetime). This is relevant in the context of TEW, because TEW assumptions are generally made in the relatively long-term; thus, if shorter time horizons are considered (such as for PBAC submissions or generally for more aggressive cancer types), assumptions around TEW may have a relatively lower impact on cost-effectiveness estimates.

Generally, any other differences were typically the year at which waning began or ended. For example, for PC0250-000/TA737 (pembrolizumab for oesophageal cancer), in the NICE TA, a gradual TEW scenario was explored between years 5 and 7. In the CADTH submission, the company applied the gradual waning between 2 and 5 years as a scenario, which was subsequently used by CADTH as their base-case. Generally, any differences in handling TEW between NICE and CADTH were because of differences in EAG/economic guidance panel for CADTH submissions or committee preference. It was common for CADTH to use TEW scenarios in their base-case, even when the company did not include them in their base-case.

There were 11 ISPOR presentations of relevance identified (Supplementary Material [Supplementary Fig. 1]). Two studies investigated the accuracy of different TEW methods used in past NICE appraisals in lung cancer using subsequently published later data cuts from the relevant trial (pembrolizumab in KEYNOTE-024 and nivolumab in CheckMate-057). Conclusions were similar in that gradually equalising hazards of death (gradual waning, described in Table 2) demonstrated improved predictive accuracy versus immediate TEW [27, 28] for observed and predicted longer-term survival. It is however notable that neither presentation provided information on what their independently fitted survival models implied about the treatment effect over time when waning was not applied.

One review uncovered inconsistencies in assumptions made by companies, EAGs and NICE appraisal committees around TEW and highlighted the need for further guidance for consistent incorporation of TEW methods in HTA submissions [6]. The authors reviewed ten nivolumab NICE TAs and found that TEW was not incorporated into the company’s base case in any of the original submissions. However, TEW was later included in the company's accepted base case in three of these TAs following requests from the EAG or committee. Although, details were lacking due to the review being in poster presentation format and not solely being focussed on TEW.

Kamgar et al. [4] created smooth hazard ratio (HR) plots based on pseudo-individual patient level data from longer-term follow up data (~5 years) of the pivotal trials from TA428 (ref. [9]), TA531 (ref. [23]), TA578 (ref. [11]) and TA692 (ref. [16]). The plots illustrate the ratio of hazards between treatment groups, demonstrating whether the treatment effect remains approximately the same over time, or whether the effect appears to increase or decrease. In the examples, the HR trended towards one in the longer term suggestive of a waning effect. However, when smoothed HRs are fitted to randomised controlled trials (RCTs) with low numbers at risk in the long-term, results become uncertain as has happened in the Kamgar et al example [4].

Finally, other presentations that were included used TEW in analyses, discussed model structures with relevance for TEW or just mentioned TEW in any capacity meaning that the presentations were eligible for inclusion but did not provide information useful for this review. Further information on each of the presentations is provided in Supplementary Material (Supplementary Table 1).

The initial search identified 880 articles. A total of 799 articles were removed at the abstract screening stage with most failing to mention treatment effects over time following discontinuation of treatment. We subsequently searched the full texts of the remaining articles and provide a breakdown of reasons for exclusion in Supplementary Fig. 2. After screening, there were 30 articles included that mentioned or discussed treatment effects following discontinuation.

Of the 30 papers included, 14 presented clinical trial data, e.g. clinical trials with long-term follow-up or pooled analyses of clinical trials. The articles based on clinical trial data provide longer-term survival outcomes from clinical trials exploring effects of IOs on a range of different cancers (Supplementary Table 2). A total of 10 out of 14 were for nivolumab [29,30,31,32,33,34,35,36,37,38], 3 for pembrolizumab [39,40,41] and 1 for avelumab [42]. Kaplan–Meier plots of the survival outcomes we discuss are presented in main article text for each reference.

Also identified were 11 real-world evidence (RWE) observational analyses (summarised in Supplementary Table 3) [43,44,45,46,47,48,49,50,51,52,53], and 5 were review papers (including systematic reviews, more general reviews and short communications), summarised in Supplementary Table 4 [54,55,56,57,58].

Despite the terminology none of the trial papers identified presented a specific analysis of the relative treatment effect over time. For instance, hazard plots akin to those illustrated in Fig. 1 were not included in any of the trials to examine this aspect explicitly. Therefore, while there is clearly evidence available on the long-term outcomes of IO treatments for overall survival (OS), progression-free survival (PFS) and duration of response (DOR), this evidence does not provide information that directly addresses the potential for TEW as it is applied in HTA.

Similarly, it is difficult to derive direct estimates of the effectiveness of IO treatments beyond treatment discontinuation from the available evidence. Although several studies report information on treatment duration alongside survival estimates, it is not straightforward to determine whether patients that discontinue treatment continue to benefit.