Introduction

Many jurisdictions now request economic evaluations as part of their decision-making procedures for the pricing and/or reimbursement of pharmaceuticals and other health technologies. In most cases, these requests are supported by national guidelines on the conduct of economic evaluation. As more and more jurisdictions request economic studies, the burden on health technology manufacturers and researchers increases, particularly when the various national guidelines insist on the presentation of local data, or the use of specific methods.

The data requirements for a full economic evaluation of two or more treatment alternatives are quite diverse, including information on baseline risk, treatment effect, resource utilization, health state preference values (utilities), and costs. Therefore, it is common for some of the data to be from outside the jurisdiction for which the evaluation is being performed. This is especially true of the clinical data, which may be drawn from large controlled trials conducted in a wide range of settings, which may or may not include patients from the jurisdiction of interest.

There are a number of reasons why clinical outcomes, health state preference values, resource utilization or costs might vary from location to location 1. Thus, it is to be expected that decision-makers will prefer economic evaluations that are applicable in their own jurisdiction. As far as individual categories of data may not be relevant to other jurisdictions, this will affect the generalizability, or transferability of the study results as a whole. Studies may be considered generalizable if they can be applied to a range of jurisdictions without any adjustment needed for interpretation. In addition, some studies may be transferable if they can be adapted to apply to other settings. Finally, some may be so specific to a given jurisdiction that they are simply not transferable to any other jurisdiction.

The objectives of this article were to assess positions of the various national guidelines on transferability (or lack of transferability) of clinical and economic data, to review the methods suggested in the guidelines for addressing issues of transferability of data, and to discuss these in the context of the recommendations of the recent ISPOR Good Practices Task Force on Economic Data Transferability 2.

Methods

A review of all the existing national pharmacoeconomic guidelines was conducted to assess which recommendations are given on the transferability of clinical and economic data, whether there are important differences between countries, or whether common methodologies have been suggested to address key transferability issues.

Pharmacoeconomic guidelines were identified through the ISPOR Web site http://www.ispor.org/PEguidelines/index.asp. We considered this the most valuable and comprehensive source because guidelines are selected for inclusion in the database on the basis of contacts with experts from approximately 60 countries from around the world and annually updated, or whenever new guidelines are issued. Guidelines available in English, or in languages that could be understood by at least one author of this article (e.g., Spanish, French, Portuguese, Italian, Polish, Dutch, German), were included in the analysis. (This resulted in the exclusion of two sets of Asian guidelines that were only available in the local language.) Each guideline was reviewed in detail to extract recommendations on the need for local data, or comments on the degree of data transferability, for the following five main categories of data:

It is probably best to think of the transferability of data as being represented by a spectrum, with “generalizable” being at one end and “not transferable” at the other. Within the spectrum, hard-and-fast judgments about transferability are hard to make and the views taken in individual countries' guidelines are often tempered by views about the ease of finding relevant local data in situations where data from outside the jurisdiction of interest are deemed to be unacceptable. Nevertheless, to characterize the similarities, or differences, between different countries' guidelines, recommendations on transferability/lack of transferability of data and/or study results were summarized as having high or low transferability:

1

High transferability. Elements of economic evaluations from other locations are considered to have high transferability when they can be used in local analyses. For example, this could be the case for an estimate of absolute treatment effect obtained from a multinational randomized controlled trial (RCT) conducted in patient populations considered similar to those in the jurisdiction of interest. An estimate of relative treatment effect could also be considered to have high transferability even if derived from an RCT conducted in a patient population potentially different from the population in the jurisdiction of interest, as long as it is applied to a local estimate of baseline risk to make it relevant to the new jurisdiction. More generally, this category includes all cases where the guidelines accept the use of data elements from other countries without major restrictions.

2

Low transferability. This category includes all cases where data from other jurisdictions are not accepted, or cases in which the guidelines are very restrictive in the use of those data. For example, it includes situations where data from other locations are not accepted under any circumstances (e.g., estimates of unit costs from a jurisdiction known to differ in price levels from the jurisdiction of interest). It also includes situations where there is a strong preference for local data, but where the guidelines state that estimates from other locations could be assessed for relevance to the local context. This could be the case for resource-use data obtained from studies conducted in other jurisdictions with different treatment patterns, where an expert panel might review the data, with a view to providing estimates more relevant to the local context should this be considered necessary.

This classification can be considered in relation to the algorithm described in the report of the ISPOR Task Force on Economic Data Transferability 3. In this algorithm, after it has been determined that the data in an existing study are sound and relevant to the current decision problem, a decision has to be made about whether the patient populations and resource-use practice patterns in the study are similar to those in the jurisdiction of interest. If not, further analysis will be required, either through the statistical analysis of individual patient data, or through decision-analytic modeling (See Fig. 1).

Steps for determining appropriate methods for adjusting cost-effectiveness (CE) information.

A secondary aim of our review was to identify which methods, if any, existing pharmacoeconomic guidelines suggest for addressing the issue of transferability of data elements between locations. Consistent with those explored in the Task Force report, we considered whether the guidelines recommended any analytical methods to overcome the problems of dealing with data obtained from other locations in two different analytic contexts:

Results

Thirty-seven guidelines for economic evaluation were identified and reviewed. These were issued in the following 27 countries: Australia, Austria, the Baltic countries (Latvia, Lithuania and Estonia), Belgium, Brazil, Canada, Cuba, England and Wales, Finland, France, Germany, Hungary, Ireland, Israel, Italy, Mexico, The Netherlands, New Zealand, Norway, Poland, Portugal, Russian Federation, Scotland, Spain, Sweden, Switzerland, and the United States. All these guidelines were obtained from the ISPOR Web site, except for Poland, for which an updated version of the guidelines was known to one of the authors. Twenty-three guidelines were available in English versions, 3 in Spanish (Spain, Cuba, Mexico) and one in Portuguese (Brazil).

There were seven countries (Belgium, Canada, England and Wales, Germany, Poland, Spain and the US) where more than one set of guidelines was identified. In these cases, we selected and extracted information from the set of guidelines that was considered to be the “official version” for manufacturer submissions (i.e., guidelines for the Drug Reimbursement Committee in Belgium, National Institute for Health and Clinical Excellence (NICE) for England and Wales, Institute for Quality and Efficiency in Health Care (IQWiG) for Germany and Guidelines for Conducting Health Quality Assessment (HTA) in Poland), or the most recent (i.e., Lopez-Bastida et al. for Spain). In the case of Canada, the guidelines issued by the Common Drug Review (CDR) and Canadian Agency for Drugs and Technologies in Health (CADTH) were assessed together because the official guidelines produced by the CDR often refer to those of CADTH for methodological issues. Finally, for the United States, we selected the AMCP Format because, although it is not in any sense “official,” it is currently the most widely followed set of guidelines. Although we restricted the analysis to one set of guidelines per country, we also checked any other sets of guidelines for these countries to identify whether any alternative method to deal with the issue of transferability of data was recommended.

Appendix 1 (found at: http://www.ispor.org/Publications/value/ViHsupplementary/ViH13i8_Drummond.asp) provides a full list of guidelines, together with the most recent year of publication in those cases where updated versions of previous guidelines had been produced. Based on the inclusion and exclusion criteria outlined above, our analysis was based on 27 sets of guidelines.

Transferability of Key Data Inputs

The results on the degree of transferability of the five key data inputs (baseline risk, treatment effect, health state utilities, resource use, and unit costs) are presented in Figure 2. Table 1 gives the same results stratified by country. A more detailed presentation of the recommendations made by each guideline on each data input is available in Appendix 2 at: http://www.ispor.org/Publications/value/ViHsupplementary/ViH13i8_Drummond.asp.

Level of Transferability of Data Elements (As reflected in National Guidelines). Note: The “not stated” category includes both cases in which no details were given and cases in which the item of interest was not discussed separately from another item. For baseline risk and treatment effect no details were given in 4 cases, while in 12 cases they were discussed together. For health utilities, no information was given in 16 cases. Resource use and unit costs were discussed together in 5 cases, while no details were given in 2 cases for unit costs and in 6 cases for resource use.

Table 1. Degree of transferability of data between locations stratified by country Country Baseline risk Treatment effect Health utilities Resource use Unit costs Australia Not transferable Transferable with adaptation Not stated Not transferable Not transferable Austria Transferable under extreme circumstances Generalizable Not stated Not transferable Not transferable Baltic (Latvia, Lithuania, Estonia) Transferable with adaptation Not stated Transferable with adaptation Belgium Transferable under extreme circumstances Transferable with adaptation Transferable under extreme circumstances Transferable under extreme circumstances Not transferable Brazil Transferable with adaptation Generalizable Transferable under extreme circumstances Not transferable Canada Transferable with adaptation Transferable under extreme circumstances Transferable under extreme circumstances Cuba Not stated Not stated Not stated Not stated Not stated England & Wales (NICE) Not transferable Transferable with adaptation Not transferable Not transferable Not transferable Finland Not stated Not stated Not stated Not stated Not transferable France Not transferable Generalizable Transferable with adaptation Not transferable Not transferable Germany Not transferable Not stated Not stated Transferable under extreme circumstances Not transferable Hungary Transferable with adaptation Transferable with adaptation Transferable with adaptation Not transferable Ireland Transferable with adaptation Not stated Transferable with adaptation Israel Not transferable Transferable with adaptation Not stated Transferable with adaptation Not transferable Italy Not transferable Generalizable Not stated Not transferable Mexico Transferable with adaptation Not transferable Transferable with adaptation Not transferable The Netherlands Transferable with adaptation Not stated Transferable with adaptation Not transferable New Zealand Not transferable Generalizable Not transferable Transferable under extreme circumstances Not transferable Norway Transferable with adaptation Transferable under extreme circumstances Transferable under extreme circumstances Poland Transferable with adaptation Transferable with adaptation Transferable with adaptation Not transferable Portugal Not transferable Generalizable Not stated Transferable under extreme circumstances Not transferable Russian Federation Transferable with adaptation Not stated Not stated Not transferable Scotland Transferable with adaptation Not stated Transferable with adaptation Not transferable Spain Not clearly stated Generalizable Not stated Not stated Not transferable Sweden Transferable with adaptation Generalizable Not stated Not transferable Switzerland Not stated Not stated Not stated Not stated Not stated USA (AMCP) Not transferable Generalizable Not stated Not transferable Not transferable Generalizable: when data from other locations are accepted and can be taken without need for adjustments. Transferable with adaptation: when data from other locations are an acceptable basis for the economic evaluations, but potential adjustments are needed (high transferability). Transferable only under extreme circumstances: when guidelines stated that there is a strong preference for local data (low transferability). Not transferable: when data from other locations are not accepted.

Although there was great variation in the level of detail and significance given to the issue of transferability of data, it appears that there is general agreement among guidelines on which data are more or less likely to be transferable between jurisdictions.

Clinical data: baseline risk and relative treatment effect. The ISPOR Good Practices Task Force Report made a distinction between estimates of relative treatment effect, which are often considered to be more generalizable, and estimates of baseline event rates, which are often considered to be more jurisdiction-specific. Approximately half of the guidelines make an explicit distinction between baseline risk and treatment effect for clinical data, while the other half do not clearly delineate between these two components. Four sets of guidelines do not make any statement on transferability of clinical data.

In all cases where baseline risk and treatment effect are discussed separately, the answer to the question of whether these data are transferable among locations is clear: baseline risk is considered to have low transferability (11 cases), while treatment effect is generally considered to have high transferability (11 cases). Even in cases of high transferability, the guidelines often suggest giving attention to the applicability of the trials to the local context (e.g., differences between trial patients and local population, adequacy of comparators, etc.). Indeed, it is possible that, even within a given jurisdiction, the treatment effect from a trial may not be generalizable because of differences between the selected nature of the patients enrolled in studies for drug licensing and the patients treated in routine practice. This, however, is an issue that goes well beyond that of transferability of studies across jurisdictions.

The main reason for using country-specific data for baseline measures of clinical events is the potential difference in epidemiological data between locations where the use of estimates from other settings could be misleading. In a few cases (New Zealand and Australia), the pharmacoeconomic guidelines suggest that only when these estimates are not available for the country of interest, the absolute risk of clinical events or other epidemiological estimates could be taken from similar locations, as long as these can be made relevant to the country where the analysis is performed and validated (on the basis, for example, of expert opinion).

The guidelines often suggest that treatment effect (e.g., relative risk of an intervention with respect to a comparator) is transferable between locations. In this case, the use of high quality data (for example, from systematic reviews or large, randomized clinical trials) is recognized as a key factor, and more important than the use of local data. Therefore, a recommendation made by some guidelines (New Zealand and AMCP in the United States) is to obtain treatment effect from multinational clinical trials and then to superimpose this on estimates of baseline probabilities of survival, or other endpoints obtained from local population-based sources.

Nevertheless, in some cases, the guidelines highlight the need for potential adjustment of the results obtained from a clinical trial (even when extracted as a relative risk reduction) to the local context, given the potential differences between settings (mainly in terms of population characteristics). In fact, in some circumstances, differences in patient characteristics might have an impact not only on baseline risk but also on treatment effect (e.g., treatment effect might depend on duration of disease, severity of patient population, number of previous drugs received, etc.). All of these factors could potentially differ between locations, between trials and regular practice, and within a given trial. In the latter case, this would suggest the use of methods to deal with heterogeneity, rather than being an issue of data transferability. The key issue is whether the trial shows that there is a difference in treatment effect between patient subgroups.

Those guidelines that do not make an explicit split between baseline risk and treatment effect generally suggest that clinical data are transferable, but with adaptation to the local context for differences in some factors. Among these factors, epidemiological data, patient characteristics, comparators used, and clinical practice are often mentioned.

Health state preference values (health state utilities). More than 60% of the guidelines (16 out of 27) do not make any recommendation on the transferability between locations of utility estimates. In the remaining cases, there is great variability, with six guidelines suggesting that utility estimates are highly transferable and five suggesting that they have low transferability.

It appears, however, that these differences are mainly caused by different degrees of flexibility in the various national guidelines for accepting data from other locations, based on the availability of these data in the country of interest. For example, the Canadian guidelines state that utility values obtained from other countries are, in general, not transferable to the Canadian setting because of cultural differences. Also, in England and Wales, the NICE guidelines state that utility values should be measured in patients using a generic and validated classification system for which reliable UK population preference values are available, although they will consider estimates from other sources. A similar recommendation is made in New Zealand.

In many cases, utility weights are not readily available for the country of interest; it is often recognized that utilities need to be obtained from other countries. This is particularly true for small countries (e.g., Hungary, Ireland, and Belgium) or countries where the use of economic evaluation might be less developed (e.g., Brazil). In these cases, great attention is given to the need for transparency, in that the data sources and instruments used should be stated, together with a discussion of their applicability to the country of interest. Finally, sensitivity analyses are often recommended when utility values are obtained from studies performed in other settings.

Resource use and unit costs. Resource use and unit costs were addressed separately in 22 of the 27 guidelines, but 6 of these 22 guidelines did not provide any explicit information on the degree of transferability of resource use. The majority of the remaining guidelines recommend obtaining resource use from the local setting, arguing that estimates from elsewhere low transferability has. Differences in clinical practices, payment systems, incentives, and the opportunity to redeploy resources are often mentioned as the main reasons for variability in resource use between locations. These guidelines suggest that it is fundamental to use local data for resource consumption and estimates obtained from other locations are often not considered as an appropriate and valid source.

As for utility weights, however, small countries appear more flexible in accepting data from other settings and in six cases estimates of resource use are seen as having high transferability. This happens, for example, in the case of Hungary, Poland, or Israel, where there might be greater difficulties in collecting local data or obtaining these data from local published studies. In general, these guidelines state that if resource utilization estimates are obtained from published studies performed in other locations, these should be adapted to the country of interest, for example by means of expert opinion. Also, they suggest that any method used to adjust resource use from other locations to the country of interest should be explained and justified.

As regards unit costs, all the 22 guidelines that analyzed these data separately from resource use agree that they must be jurisdiction-specific because of differences in relative and absolute prices between countries. Some guidelines also provide sources for unit costs (e.g., an official list). Most guidelines recommend presenting quantities of resource use separately from unit costs to increase the transparency of the analysis.

Transferability of Key Data Inputs: Additional Analyses

In addition, it was hypothesized that the degree of flexibility in accepting data from other jurisdictions might be dependent on two factors: 1) the year in which a guideline was issued, and 2) the level of development of economic evaluation methods in the country of interest.

To assess whether these two factors have an impact on the results of the study, we conducted two additional analyses. First, we divided the 27 guidelines into two groups based on publication date. It was found that 14 guidelines were issued before 2005, while 13 guidelines were published from 2005 to date. Thus, 2005 was chosen as reference year because it split the guidelines in two almost equal groups. Second, we divided the guidelines into two groups based on the degree of development of economic evaluation in the jurisdiction concerned. To identify countries that were mature in their experience of economic evaluation, two of the authors (MB and MD) independently made the assessment and then discussed the findings to reach a consensus.

Transferability of data by year of publication of the guidelines. Although the sample of guidelines was relatively small to draw firm conclusions, two important differences were found between guidelines published before and after year 2005. First, “older” guidelines provide much fewer details on transferability issues. For example, nothing was stated on transferability of utility weights in 10 cases out of 14 for older guidelines versus 6 of 13 for the newer guidelines. Similarly, no information was given on transferability of resource use in five cases for guidelines published before 2005, while this happened only in one case for newer guidelines. In addition, baseline risk/treatment effect or resource use/unit costs were discussed separately more often in newer guidelines as compared with the older guidelines. Specifically, baseline risk and treatment effect were not presented separately in seven cases for older guidelines versus five cases for newer guidelines, while resource use and unit costs were not presented separately in four cases for older guidelines versus only one case for newer guidelines.

The second element that distinguishes the two sets of guidelines is the extent to which data from other jurisdictions are accepted in the country of interest. In general, the older guidelines are more flexible in accepting data from other countries. In fact, while for some data inputs the two sets of guidelines provide very clear and similar results (baseline risk and unit costs are generally considered to be not transferable, while treatment effect is considered to have a high degree of transferability), different results were found for resource use and health utilities. In the case of resources used (when presented separately from unit costs), the older guidelines suggest that they are highly transferable in 3 cases out of 5, while only in 3 cases out of 11 for the newer guidelines. Similarly, utility weights are viewed as being highly transferable in three cases out of four for older guidelines, while only in two cases out of nine in newer guidelines (in all the remaining cases nothing was stated). These findings suggest that more attention has been given to transferability issues in recent years and that updated versions of old guidelines might be more restrictive in accepting data from other jurisdictions.

Transferability by degree of development of economic evaluation. On the basis of the independent assessment of two authors of the article, the following two groups of countries were as follows:

1

more mature countries (Australia, Belgium, Canada, England/Wales, Finland, France, Germany, Italy, the Netherlands, New Zealand, Portugal, Scotland, Spain, Sweden, USA)

2

less mature countries (Austria, Baltic countries, Brazil, Cuba, Hungary, Ireland, Israel, Mexico, Norway, Poland, Russian Federation, Switzerland).

The results of this analysis are quite similar to those of the comparison between older and newer guidelines First, less mature countries appear to provide fewer details on specific data items, although this difference is not as great as in the case of older versus newer guidelines. For example, baseline risk and treatment effect are presented separately in 11 cases out of 15 in more mature countries, while only in 4 cases of 12 in less mature countries. On the other hand, the number of “not stated” items is similar between the two groups.

Second, as in the case of older guidelines, less mature countries appear to be more flexible in accepting data from other jurisdictions. This is evident in the case of resource use, which was considered not transferable in 8 cases out of 10 in more the mature countries, while considered to be highly transferable in four out of six cases in the less mature countries. This might reflect the fact in mature countries the availability of local economic data are likely to be higher than in less mature countries. Nevertheless, we did not find clear differences for all the other data items (e.g., baseline risk, treatment effect, utilities, and unit costs).

The Application of Guidelines in Practice

Although pharmacoeconomic guidelines often make clear statements about transferability (or lack of transferability) of data, it is interesting to assess whether published economic evaluations and industry submissions to specific agencies follow the recommendations in the jurisdiction concerned. It is clearly unrealistic to analyse all studies or reports produced in the 27 countries considered, so we focused on the specific case of NICE for England and Wales. NICE was chosen because of the high transparency of the appraisal process and the wide availability of documentation on the NICE Web site. All published technology appraisals on the NICE Web site from July 2008 to October 2009 were considered because this time period followed the introduction of the latest methods guidelines in June 2008. Two data inputs were considered on the grounds that there is still some debate about whether they are transferable or not:

1

baseline risk;

2

health utilities.

NICE's methods guidelines recommend that baseline risk has “to be relevant to UK practice and patients, and to compare all relevant treatment options for the relevant patient groups.” Nevertheless, it is added that “evidence on effectiveness might come from outside the UK health care system . . . . Despite such weaknesses in the evidence base, decisions still have to be made about the use of technologies. Therefore, analyses should use the best evidence available, be explicit about data limitations and any attempts to overcome these, and quantify as fully as possible how the limitations of the data are reflected in the uncertainty in the results of the analysis.”

With respect to utility values, the NICE guidelines state that: “The valuation of changes in HRQL reported by patients should be based on public preferences, elicited using a choice-based method in a representative sample of the UK population” and “The EQ-5D is the preferred measure of HRQL in adults. When EQ-5D data are not available or are inappropriate for the condition or effects of treatment, the valuation methods should be fully described and comparable to those used for the EQ-5D.” For example, “methods can be used to estimate EQ-5D utility data by mapping (also known as “cross-walking”) EQ-5D utility data from other HRQL measures included in the relevant clinical trial(s).” Finally, “[t]he EQ-5D may not be an appropriate measure of health-related utility in all circumstances. When an alternative measure is preferred, those submitting analysis should provide reasons, supported by empirical data on the properties of the instrument used.”

The results of the analysis are presented in Table 2. It should be noted that some of these results are based on a single technology assessment submitted by the manufacturer and reviewed by an expert review group (ERG) that makes comments to the Appraisal Committee. In other cases, appraisals are based both on a manufacturer submission and a technology assessment produced by the independent group. In this section, we focus on economic submissions prepared by manufacturers, plus any concerns raised by the Committee, or the ERG. More detailed results on all submissions are presented in Appendix 2 at: http://www.ispor.org/Publications/value/ViHsupplementary/ViH13i8_Drummond.asp.

Table 2. Source of data for baseline risk and health utilities in NICE technology assessments (July 2008–October 2009) TA no Baseline risk Utility values TA151 From UK database EQ-5D from UK study TA152 From UK data audit SF-36, SF-12 and EQ-5D from UK patients TA153 From literature: uncertainty on its application to UK population Standard gamble (SG) from a published study using UK tariffs TA154 From a multinational trial (including UK). It was considered applicable to UK (clinical specialists) From published studies that used the EQ-5D generally conducted in the UK TA155 From literature: uncertainty on its application to UK population Different instruments and methods (HUI-3, time trade-off (TTO), SG). UK values generally used TA156 From UK epidemiological studies Canadian study that used the HUI-2 TA157 From 2 multinational trials that appear applicable to the UK EQ-5D and SG from UK populations and studies conducted in other countries TA158 Both UK data and published meta-analyses HUI-2 and 10-point Likert scale, not only from UK patients TA159 From a multinational trial that appears applicable to the UK EQ-5D and SF-36 (UK tariffs) TA160 Both UK data and epidemiological studies in several countries From a Swedish study that probably used the EQ-5D TA161 Both UK data and epidemiological studies in several countries From a Swedish study that probably used the EQ-5D TA162 From a trial not conducted in the UK: concerns on its applicability to the UK populations EQ-5D and VAS (UK tariffs) TA163 From a trial conducted in France: values were validated by a panel of UK specialists EQ-5D from UK patients TA164 Both UK studies and a trial not conducted in the UK with concerns on its applicability. EQ-5D from a study conducted in France, Germany and the UK. TA165 From a UK registry From a UK study that used the EQ-5D TA166 From UK databases and studies conducted in the UK HUI-3 from US and Canadian patients TA167 From UK databases and studies conducted in the UK EQ-5D and SF-36 from UK patients TA168 From studies conducted in the UK and UK registries EQ-5D and Harvard utility scores from UK and Dutch patients TA169 From a multinational RCT, not clear if applicable to the UK population EQ-5D from a multinational trial using UK tariffs TA170 From a trial relevant to the UK From published studies (EQ-5D preferred) TA171 From trials with very few UK patients: concerns about its applicability to the UK EQ-5D from UK population TA172 From a trial which appears relevant to the UK QLQ-30 converted to EQ-5D through an algorithm (UK tariffs) TA173 Both UK data and other studies carried out in several countries SG and VAS from UK patients TA174 From a trial which appears relevant to the UK Based on experts' opinions TA176 From 2 multinational trials validated by specialists EQ-5D and HUI from UK population TA177 From a RCT which appears representative of the UK population DLQI values converted to EQ-5D scores (UK tariffs) TA178 From a RCT relevant to the UK population EQ-5D from a RCT (UK tariffs) TA179 From a multinational RCT, not clear if relevant to the UK EQ-5D from a UK study TA180 From pooled values of 3 RCTs that appear relevant to the UK PASI and DLQI values mapped to EQ-5D scores TA181 From a RCT with some differences compared to UK patients VAS and SG from UK population TA182 From a RCT with comparable population to a UK registry EQ-5D from US population adjusted to UK values TA183 From a multinational RCT, not clear if relevant to the UK FACT-G values from a trial

With respect to utility values, it is clear that the vast majority of submissions followed the NICE guidelines, either using the EQ-5D instrument in UK patients, using the EQ-5D in patients from other countries but then applying UK tariffs, or using disease-specific instruments that were then mapped to EQ-5D scores. There were only few cases (e.g., TA156, TA158, TA166, and TA174) where different instruments and populations were used, but this was generally justified. For example, in the case of TA166, which considered cochlear implants for children and adults with severe to profound deafness, it was stated that the Health Utilities Index (HUI-3) was used as main source in all the analyses because it was the only standard instrument which includes items relating to functional limitations caused by impaired hearing or speech. Only in the case of TA183, which considered topotecan for the treatment of recurrent and stage IVB cervical cancer, the use of Functional Assessment of Cancer Therapy-General (FACT-G) appeared not to be justified and was criticized by the ERG and the Committee.

More contrasting results were found for baseline risk. Although the majority of submissions used local data, there were several cases where epidemiological data, patients' characteristics, and/or disease progression were estimated on the basis of populations from outside the UK. While in some circumstances the Appraisal Committee and /or the ERG considered these sources applicable and similar to the UK populations or they were validated by UK experts (e.g., TA154, TA157, TA159, TA163, TA170, TA174, TA178, TA182), there were some cases where concern