NICE invited Daiichi Sankyo UK to submit evidence for the clinical and cost effectiveness of quizartinib (Vanflyta®) for the treatment of newly diagnosed FLT3-ITD+ AML. The NICE scope outlined two comparators to be considered as part of the decision problem: standard chemotherapy (SC) and midostaurin plus SC. The midostaurin regimen represents the current standard of care in the National Health Service (NHS), so was the main comparator. Quizartinib clinical effectiveness evidence was based primarily on QuANTUM-First [8], a phase III, double-blind, randomised controlled trial of quizartinib plus SC versus placebo plus SC in patients with newly diagnosed FLT3-ITD+ AML.

In the absence of direct comparative effectiveness evidence for quizartinib versus midostaurin, the company presented ITCs for key outcomes including overall survival. The ITCs included QuANTUM-First and the FLT3-ITD+ subgroup of the RATIFY trial [9], a phase III, double-blind, randomised controlled trial of midostaurin plus SC versus placebo plus SC in patients with newly diagnosed FLT3+ AML (Fig. 1).

Indirect treatment comparison (ITC) of the QuANTUM-First and RATIFY trials. AD aggregate data, FLT3-ITD+ FMS-like tyrosine kinase 3 internal tandem duplication positive, IPD individual participant data, MAIC matching adjusted indirect comparison, ML-NMR multi-level network meta-regression, SC standard chemotherapy

Due to differences in patient characteristics across the QuANTUM-First and RATIFY trials, the company performed MAICs that were used to inform their base-case economic analysis (Fig. 1; Table 1). Unadjusted ITCs without population adjustment were also presented for reference.

QuANTUM-First and RATIFY differed substantially in terms of design, setting and eligibility criteria. Among the most important differences were eligibility criteria relating to participants’ age. In QuANTUM-First, patients aged 18–75 years were eligible, while RATIFY was limited to patients aged 18–59 years. As is common in NICE STAs, the company only had access to IPD from their own QuANTUM-First trial. Therefore, to implement a MAIC, the company matched the QuANTUM-First trial population to the published AD of the RATIFY FLT3-ITD+ subgroup. The resulting relative effect estimates thus reflected a ‘RATIFY-like’ population.

The EAG raised concerns, supported by clinical experts, regarding the generalisability of the RATIFY trial to the NHS population eligible to receive quizartinib. Changes in clinical practice have improved treatment outcomes since the conduct of the RATIFY trial between 2008 and 2016, and the age of the RATIFY population is younger than the current NHS population. Therefore, the EAG concluded that the results of the MAICs, reflecting a ‘RATIFY-like’ population, were not suitable for decision making. To overcome this fundamental limitation of MAICs, the EAG requested that the company also perform ML-NMR [10] and generate results reflecting a ‘QuANTUM-First-like’ population (Table 1), which were included in a scenario economic analysis.

In the two-study scenario typically encountered in NICE STAs; data are insufficient to estimate independent effect-modifier interaction terms for each treatment. Therefore, without additional assumptions, all population-adjustment methods can only produce estimates in the AD trial population. With ML-NMR, a simplifying assumption of shared effect-modification may be made, where effect-modifier interactions are equal for quizartinib and midostaurin so that estimates can be produced for any target population [4, 5]; ML-NMR can apply this assumption for any data type, whereas for MAICs, this assumption is only applicable with continuous outcomes so could not be applied [6]. Clinical experts supported this assumption as both treatments are tyrosine kinase inhibitors targeting FLT3 mutations. ML-NMR estimates were produced both for a ‘QuANTUM-First-like’ population and a ‘RATIFY-like’ population. The NICE committee considered that the results of the ML-MNRs were more applicable to the target NHS population and suitable for decision making [7].

In the context of decision making, it is important to note that the different methods estimate different types of population-average treatment effects. MAICs produce marginal treatment effects, whereas ML-NMR can produce either marginal or population-average conditional treatment effects. Typically these do not align where non-collapsible effect measures such as hazard ratios (HRs, for survival outcomes) and odds ratios (for dichotomous outcomes) are used [6, 11], which can lead to different decisions regarding comparative clinical or cost-effectiveness.

Marginal treatment effects are specific to a target population with given distributions of prognostic and treatment-effect modifier characteristics and, for survival outcomes, shape of the baseline hazard [6, 10]. Marginal HRs additionally depend on the length of follow-up and the censoring distribution [11]. For decision making, the marginal treatment effect represents the effect of moving everyone within the target population from treatment with midostaurin to treatment with quizartinib. The existence of covariate effects (prognostic or effect modifying) means that hazards are mathematically non-proportional at a marginal level [6, 10]; i.e. the marginal HR is time varying and a single constant HR cannot be applied.

Population-average conditional treatment effects are specific to a target population with a given distribution of treatment-effect modifier characteristics and are interpreted as the average of the individual-level treatment effects in the population, i.e. the average effect of moving each individual within the target population from treatment with midostaurin to treatment with quizartinib. Unlike marginal treatment effects, population-average conditional treatment effects do not depend on the distribution of prognostic characteristics or the baseline hazard and may be constant over time.

Careful interpretation of population-average conditional and marginal treatment effects in the context of the decision problem is required [6]. Crucially, the different interpretations of population-average marginal and conditional effects mean that care must be taken to incorporate them appropriately into an economic model, where the primary focus is on obtaining an appropriate population-average survival curve for each treatment.

The company’s approach to modelling relative treatment effects was to apply population-average conditional HRs estimated from the ML-NMR to extrapolated survival evidence from the quizartinib arm from QuANTUM-First [7]. However, directly applying constant, population-average conditional HRs from the ML-NMR to a baseline marginal survival curve, which is unadjusted for covariates, combines incompatible conditional and marginal quantities, resulting in aggregation bias [10]. The effect of this bias on cost-effectiveness estimates is, however, unknown [7].

A correct approach to a cost-effectiveness analysis when covariate effects are present would be to use the population-average survival curves for each treatment in the target population predicted from the ML-NMR directly within the economic model. Baseline survival data for an external target population may be included within the ML-NMR model to produce coherent predictions for populations other than those of the included studies [10, 12].

However, incorporating population-average survival curves is not trivial because of the need to account for changing hazards across the model time horizon and the computational burden of accounting for uncertainty in estimates. Analysts considering the use of ML-NMR to inform economic evaluations must carefully consider how estimates should be incorporated in advance of economic model development. Similar considerations apply to other population-adjustment methods: the common approach of applying a constant marginal HR derived from a MAIC to a baseline survival curve contradicts the assumed existence of covariate effects, which are the motivation for population adjustment. Attention should also be given to how survival data are extrapolated. NICE committees and EAGs will usually expect to see the exploration of a variety of extrapolation approaches. Development of further guidance on how ML-NMR estimates can be integrated into economic analyses is an important research priority.