This study offers a starting point towards further development and application of MUDMs in the form of a clear definition, a list of potential applications, and an overview of related issues and challenges of MUDMs using input from a large group (N = 54) of international HTA modelling experts. Results were validated using an independent advisory board of academic experts and HTA agency representatives.

MUDMs can be defined as a health economic decision model that can be repeatedly used for a certain disease condition, to accommodate the evaluation of a range of health care interventions over several disease stages. While a number of challenges and issues remain to be solved, such models have many promising potential applications. The most important of these challenges were transparency and stakeholder involvement, while high priority recommendations on handling these issues included again transparency, but also regular updates and the model’s ability to account for time trends. Model ownership organization and proper choice of the level of complexity were other relevant issues highlighted by the panel as well as the project’s advisory board.

Several previous studies have addressed the definition and terminology related to MUDMs [2, 3, 5,6,7, 10]. Tappenden et al.’s [2] definition of a ‘whole disease model’ differed from the current one by its explicitly very wide scope and by the requirement of consistency throughout. That is, whole disease models should be suitable for the health economic evaluation of interventions for prevention, diagnosis and treatment across the whole disease pathway. As such, a ‘whole disease model’ is the most complete implementation of the idea of a MUDM. The disadvantage may be lack of feasibility, due to its very stringent requirements. Afzali et al. [5, 6] defined ‘reference model’, or ‘disease-specific reference model’ as a model that should represent “the knowledge and uncertainty about states/events relating to the disease progression on the basis of the best available evidence.” It is to be applied to a wide set of interventions for a specific disease. Compared with our current definition, reference models seem to require a certain ‘gold standard’ status, which is left open for MUDM. That is, a reference model is seen as the best possible model; in contrast, MUDMs do not necessarily claim this. As an example, in diabetes, more than 10 different MUDMs exist [26].

While the panel indicated explicitly that they also wanted to include models that did not cover the complete disease pathway, many of their recommendations and choices and especially suggested applications would require a model that covers a large part, if not all, of the disease pathway. The highest scores for applications of MUDMs were given to “comparing alternative policies in prevention and treatment” and “resource allocation over the entire disease pathway of interest”. This points at the most important challenge for developers of an MUDM, namely to choose a scope for the model that balances feasibility with applicability.

Insight and scientific evidence on diseases develop over time, while epidemiology and other model inputs may change. Therefore, regular maintenance is crucial for MUDMs, as also indicated by panel results. Maintenance and options for updates have to be integrated into the model development right from the start, for instance by using a modular model structure. This has been previously underlined, among others, by the ISPOR-SMDM Modeling Good Research Practices Task Force [27].

In contrast to the publications on reference models [6, 7], the panel did not clearly advocate using a single model as the gold standard, and some panel members warned of a potential lack of insight into structural uncertainty that may come with such a reference model. However, in itself, a reference model does not prohibit use of alternative model structures alongside the reference model analysis [7].

Finally, most previous studies on MUDMs [2, 3, 6] paid little attention to more practical and organizational issues like model ownership, maintenance and access for external users, while the current study shows that about half of the issues identified fall into these categories. To enable useful policy advice regarding the application of MUDMs, possible solutions for the methodological and organizational issues that were identified should be investigated.

It may appear that MUDMs bring a lot of challenges from this final overview. However, a very important advantage is the reduction of the inefficiency involved in repeated development and validation of new single-use models. When properly implemented, MUDMs could benefit several stakeholders. HTA agencies and assessors would benefit from improved consistency and transparency of model-based economic evaluations, while model developers could also benefit, when they do not need to develop a model from scratch, but could start with an existing MUDM and tailor it to their needs.

In addition to showing these advantages, the implementation of MUDMs will reduce the amount of model review time needed by HTA agencies and external reviewers during assessments of specific treatments. Additionally, MUDMs are typically available to HTA agencies, in contrast to single purpose models which are often built under commission of applicants. Therefore, usage of MUDMs could broaden the scope of treatments that may be evaluated by HTAs by allowing more opportunities to the HTA agency to perform independent evaluations. Furthermore, MUDMs will enhance consistency in the model-based evaluations to support (a broader range of) decisions within disease areas and potentially improve validity of model results, with models being more elaborately tested and used.

The current study focused on the definition and potential applications of MUDMs as well as the identification of challenges and recommendations. How to practically develop MUDMs was not addressed here. Several of the challenges identified by the panel indeed concerned topics that concern these practical matters, for instance the funding of model development, their ownership and their maintenance (see Fig. 5). In our project, the research team used the findings from the panel surveys to develop five business cases based on different choices regarding model ownership as potential blueprints for the implementation of MUDMs [28]. This, however, was not part of the current study, which only reported on the panel consultation.

Our study has several limitations. First, the panel was asked to reflect on largely theoretical issues. Not having to actually implement or develop an MUDM, or being confronted with an actual application, implies that some panel recommendations may lack practical relevance. We tried to reduce this risk by using a second round of feedback to the panel, and by having ZIN experts and an advisory board of three independent experts comment on the results. Nevertheless, the results serve as a starting point rather than a final blueprint for the application of multi-use models.

Second, we were able to recruit only a limited number of participants from industry to participate in the panel. More participants were approached, but these often declined based on a stated lack of experience in health economic decision modelling. Industry often hires consultancy firms to develop health economic decision models and the panel did contain a number of participants affiliated with a consultancy.

Third, our findings should be interpreted with care, in that we cannot claim that our panel was representative of all stakeholders in health economic decision modelling. The group of active participants may over-represent people with a prior interest in the topic of MUDMs. This may have an impact on the priority setting in the Round 2 survey, for example the priority scores for applications, issues and recommendations.

Fourth, the research team and ZIN were both from the Netherlands. In particular, the final prioritization of issues was influenced by input from ZIN experts. Also, the transferability of MUDMs to other countries was not extensively assessed and discussed. That is, the typical MUDM foreseen in the current project was a locally adapted model, using local input data where appropriate. Yet, the findings of the current study concerning MUDMs would also be relevant outside of the Netherlands. The expert panel was international, and the results presented in the current study therefore reflect the opinions of these international experts. Our external advisors were international as well. Furthermore, similar work on MUDMs is ongoing in UK and Canada, which indicates the relevance of MUDMs outside of the Dutch setting.

Strengths of the current research are that we did use a broad panel of experts, so that the findings reflect the insights of people from academia, decision makers and industry representatives, including consultancy. Another strength is the systematic approach, using expert panel input for a clear definition of concepts and terminology, as well as a two-step Delphi-like procedure to derive an inventory of challenges and recommendations. The expert panel members were asked to also reflect on practical and organizational aspects, which were further commented on by employees from the Dutch HTA agency.

This study was commissioned by ZIN, the Dutch HTA agency, as part of a larger project investigating the possibilities, advantages and disadvantages of MUDM for ZIN. The research team subscribed to a competitive tender with a project proposal. During the resulting project, the research team had regular contact with ZIN experts to keep them informed on project progress. Discussion of Round 1 panel results took place in a consensus meeting of the research team and ZIN experts together. However, Round 2 results were first processed by the research team, after which the ZIN experts provided their input, which led to a combined final set of results. The latter allows for better insights into the role of the HTA agency experts (Supplement 4, see ESM). Results of this project were intended to support future steps by the HTA agency. Therefore, it was important to closely collaborate with the ZIN experts throughout the entire project and one of them is a co-author of this publication. However, the current paper reflects the opinion of the co-authors and not necessarily the official policy of ZIN.

In our view, MUDMs may have varying levels of comprehensiveness. For some applications, it may be desirable and feasible to develop an extensive model with multiple decision points, a broad range of outcomes and costs (health and other), and all parameter estimates available (perhaps even linked to patient registries for regular updates). For other applications, a less extensive model may be suitable, or even a set of mandatory model components (for example concerning costing parameters, the core risk engine for a patient level model, or the most important model states and their care as usual transitions for a state-transition model). Such partial solutions may help to gradually introduce multi-use disease modelling in an efficient and feasible way and overcome issues with inconsistencies across assessments and with technical validation. Developing model code in a structured and modular way will further support this [29].

Ultimately, MUDMs will improve consistency among coverage decisions for various treatments in the same disease area. In addition, they will enable engagement in an overall evaluation of several treatments for one disease. Examples of the latter can already be found in the multiple technology appraisals at NICE, for instance when comparing different modalities for glucose monitoring in type 1 diabetes [30]. Other agencies may apply MUDMs in support of policy making concerning management of infectious diseases, public health policy aiming at prevention through a healthy lifestyle, and population screening programs. Examples are the vaccination policy advice documents by the Dutch health council which apply MUDMs of infectious diseases [31, 32]; several COVID-19 epidemiology models which can also be considered MUDMs [33, 34]; evaluations of tobacco control [35, 36], alcohol misuse [37, 38] and overweight policy [39] using health impact models, and evaluations of cost effectiveness of colon cancer screening using extensive colon cancer models [40,41,42,43].

As a logical extension, the support of clinical guideline development is a very attractive application of MUDMs. Several authors have argued for better integration of cost-effectiveness information in clinical guidelines [44, 45]. In some jurisdictions such as in England, cost effectiveness is integrated into clinical guideline development. In that case, MUDMs have a clear role to support clinical guideline development. Also, in settings that do not consider cost effectiveness as an explicit criterion in clinical guidelines, MUDMs have a potential role in guideline development, since they offer a consistent framework to evaluate implications for capacity requirements or resource use of choices advised by clinical guidelines. While this was not an explicit element in our panel survey, the potential application ranked 4th in our panel and future research could pay more attention to the specific requirements that application for guideline development may bring.

Finally, MUDMs can be applied in budget impact analyses, but this brings an extra requirement for the model that its model population reflects the total patient population under consideration, that is, sufficient information on disease incidence and prevalence, and on population characteristics is included. This is another area for future research, with close links to existing public health models.