Background

The positive impacts of using patient-reported outcomes (PROs) in monitoring quality of care and improving health outcomes have generated increasing interest in their wide-ranging application. Several Health Technology Assessment (HTA) agencies require evidence on PROs to assess the effectiveness of health interventions.1–4 PROs are defined as a subjective assessment of health and/or disability provided directly by the patient without any interpretation by a clinician or other health professional.5 6 Patient-reported outcome measures (PROMs) are instruments developed to assess PROs that comprise a series of structured questions that ask patients for example about their health status, quality of life (QOL) or well-being from their point of view at a single point in time.7 8

Typology of PROMs include instruments that are non-disease specific, that is, considering a broad range of generic dimensions of QOL that, in principle, could be impacted by any disease; or disease-specific, that is, concentrating on dimensions impacted by a particular disease.9 Since disease-specific instruments have a limited use to comparisons between studies within the same disease area only, generic measures are preferred for (economic) evaluative purposes.10 Both disease-specific and generic instruments can be accompanied by utilities/value sets to facilitate the estimation of quality-adjusted life years (QALYs), which can be used in economic evaluations to assess the value of medical interventions.11

QOL is ‘a broad construct reflecting subjective or objective judgement concerning all aspect of individual’s existence, including health, economic, political, cultural, environmental, aesthetic and spiritual aspects’.12 Health-related quality of life (HRQOL) is part of QOL assessment that focuses on the impact of health on the person’s overall well-being.13 14 The scope of some commonly used HRQOL PROMs (eg, EuroQoL Five Dimensions (EQ-5D)) might not reflect all important consequences of health-related interventions. In recent years, new measures have been developed with the focus on well-being broader than HRQOL, such as, social care-related QOL (Adult Social Care Outcomes Toolkit (ASCOT)),15 process of recovery for users of mental health services (ReQoL)16 or capabilities (eg, ICEpop CAPability measure (ICECAP), Oxford CAPabilities questionnaire-Mental Health (OxCAP-MH)).17 18 Moreover, for disease areas with major impact on the health and well-being of carers and families (eg, mental health problems or chronic diseases), multiperson outcomes may have to be measured and valued jointly, preferably within the same framework and with the same method.

Outcome measurement within the area of mental health deserves special attention due to the increasing disease burden of mental health disorders,19 20 paired with a proportionally low expenditure on mental healthcare worldwide.21 The cumulative global economic impact of mental, neurological and substance use disorders was estimated at US$16.3 trillion between 2011 and 2030, exceeding cardiovascular disease, chronic respiratory disease, cancer and diabetes in its contribution to global burden of disease.21 However, standard methods can only be applied with some caution.22 A general issue is that mental health service users often have more than one diagnosis, particularly those with severe and complex problems.23 Furthermore, QALYs, the most commonly used outcome measure for economic evaluations, have limited ability to capture effects beyond direct health impacts (eg, educational attainment, work productivity, self-realisation, safety, discrimination, freedom of choice, social networks, attachment, control), may be insensitive to the impact of social care interventions, and likely underestimate the full welfare impact of mental health interventions.22 24 This resulted in the recent development of several PROMs that could potentially be used for broader well-being measurement in the economic evaluation of mental health interventions. However, the choice between these instruments and their practical application in particular contexts lack a systematic approach.25 26

Objective

The ProgrammE in Costing, resource use measurement and outcome valuation for Use in multisectoral National and International health economic evaluAtions (PECUNIA) project aimed to tackle the healthcare challenges of an ever-growing and rapidly ageing population in the European Union by developing new standardised, harmonised and validated methods and tools for the assessment of costs and outcomes in European healthcare systems (https://pecunia-project.eu/). The project used selected mental disorders (schizophrenia, depression, post-traumatic stress disorder) as illustrative examples to address methodological issues at their maximum in dealing with complex services, major intersectoral economic and social consequences and broader well-being impacts in all ages, and to contribute to major public health challenges with highly needed applied evidence. One work stream of the project focused on improving the methods and tools for comparable outcome assessment in economic evaluations of mental health interventions. To achieve this goal, one of the tasks was to harmonise methods for broader health and well-being impact measurement across sectors, countries and age groups using PROMs. Therefore, this study aimed to systematically collate and synthesise existing, publicly available generic and mental health-specific QOL PROM information and compile their meta-data in the form of an electronic compendium, the PECUNIA PROM-MH Compendium. The goal was to develop a compendium that provides comparable, easily searchable information about PROMs potentially suitable for QOL/well-being measurement for mental health economic evaluations. In addition, the specific applicability of PROMs in multisectoral, multinational, multiperson economic evaluations was assessed.

Study selection and analysisSystematic literature review

The aim of the systematic literature review was to identify self-reported, non-mental health disease-specific (ie, not specific to a single mental disorder) PROMs used in mental health research for the measurement of QOL/well-being outcomes (including recovery) potentially suitable for economic evaluations. The protocol for the literature review has been registered in the PROSPERO database (CRD42018117800), and the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklist is shown in the online Ssupplementalry material. Five bibliographic databases (Embase, Medline, PsycINFO, Social Science Citation Index and CINAHL) were searched in 2018 for the past 10 years (2008 onwards) with an updated search in February 2020. This restriction on time-period was deemed appropriate in order to capture instruments actively in use for outcome measurement over the past 10 years, and was in line with the period for significantly increased annual number of internationally published economic evaluations as reported by the NHS EED.27 Therefore, the potential marginal benefit of bibliographic searches for instruments in use only prior to 2008 was not in line with the estimated additional workload and the expected relevance of the findings to the study’s overall objective. Instead, we used reference searches of existing reviews and PROM databases as part of the secondary searches reported below for this purpose. No language restrictions were applied in the search, but eventually articles in the languages represented in the PECUNIA Group were included (Dutch, English, German, Hungarian, Polish, Spanish). Search strategy exported from Medline (searched via Ovid) is presented in online supplemental table 1.

Two independent researchers (AŁ and TMH) screened titles and abstracts of the identified de-duplicated records, while the third reviewer (JS) decided on the inclusion/exclusion of articles if there was no agreement between the two independent researchers. At this stage, studies were excluded using the following exclusion criteria: (i) not relevant for mental health outcome measurement (ie, the study was not concerned with a mental health condition), (ii) no standardised PROM was used, (iii) outcome other than QOL/well-being or recovery was measured, (iv) only disease-specific outcome was measured (ie, neither generic PROMs, nor those designed for mental health overall), (v) study was a systematic literature review/editorial/commentary, (vi) only abstract available. The full texts of potentially eligible studies were retrieved and independently assessed for eligibility by two review team members (AŁ and TMH).

Additional grey literature search was conducted to identify instruments that have not been captured in the systematic search of bibliographic databases. The search included the COSMIN database of systematic reviews of outcomes measurement instruments (https://database.cosmin.nl/), previous systematic literature reviews of PROMs searched in Medline by searching the keywords describing a ‘review’ and ‘PROM’. Finally, existing online PROM databases (eg, https://scireproject.com/outcome-measures/) were searched to identify relevant PROM instruments. The final list of 21 systematic reviews and 7 databases from which information was later extracted is provided in online supplemental appendix table 2.

Inclusion of PROMs

Individual PROMs were identified and extracted from all eligible studies from the bibliographic database search. PROMs were included if they measured QOL/well-being (including recovery) and were generic or designed for use in a broad field of mental health diseases without being disease-specific.

According to existing definitions, the term QOL encompasses a broad range of aspects, and boundaries between QOL and well-being are increasingly blurred.12 28 29 Therefore, in this work, QOL and well-being are presented as overlapping concepts. Moreover, according to several existing definitions, concepts such as happiness, spirituality and life satisfaction are part of QOL.30 Therefore, all instruments measuring and valuing QOL (including HRQOL), well-being as well as happiness, spirituality and life satisfaction were considered with multisectoral referring to those measures that are able to capture outcomes beyond individual health (eg, capabilities, social outcomes). Since recovery was identified in the scoping review as a related concept with particular importance in the context of mental disorders,31 and personal recovery is increasingly emphasised as key in evaluation of mental health interventions,37 instruments measuring recovery in mental diseases were also included. Other concepts such as resilience, which focus particularly on the ‘ability to withstand, adapt to and recover from adversity and stress’32 rather than overall QOL/well-being, were not identified or included.

In addition, different versions of identified PROMs and additional PROMs extracted from the scoping review search were included. PROMs from all sources were compiled into one list.

Secondary searches and extraction of meta-data

Secondary searches were conducted with the aim to locate most up-to-date PROM meta-data information. Meta-data were searched in Google, Google Scholar, Google Books, PubMed, instrument manuals, review articles, other available peer-reviewed publications and other web-based sources. The secondary search strategy is presented in online supplemental appendix table 3. Extraction of the meta-data information was conducted between April and July 2020.

The extracted meta-data of included instruments were compiled in Microsoft Excel. A specific extraction template was developed by the authors of this study and reviewed by outcome research experts within the PECUNIA project to present the most crucial PROM characteristics in a concise, transparent and replicable way. Elements were organised into the following sections: (1) basic characteristics; (2) scoring information; (3) availability of validation studies in mental health; (4) application in multisectoral, multinational, multiperson economic evaluations and (5) other information. Six criteria were applied to judge the suitability of PROMs for the application in multisectoral, multinational and multiperson evaluations. They are outlined in table 1.

Table 1

Assessment criteria applied to the identified PROMs

FindingsSystematic literature review

The PRISMA chart (figure 1) presents the process of identifying the relevant PROMs. Bibliographic database and grey literature searches initially resulted in 313 unique instruments, and a further 29 questionnaires that were different versions of them (eg, 17D was added as a version of 15D, Control, Autonomy, Self-realization, and Pleasure scale (CASP)-12 was added as a version of CASP-19). Overall, 204 unique instruments were included. Of these, 88 were individual PROMs that had no other related versions available (eg, brief version or adolescent/child version). The other 116 instruments were eventually grouped into 46 PROM families meaning that at the time of the data extraction there were two or more related questionnaires belonging to the same ‘family’ of instruments (eg, questionnaires 15D, 16D, 17D grouped into 15D family).

Figure 1

Preferred Reporting Items for Systematic Reviews and Meta-Analyses flow chart. PROM, patient-reported outcome measure; SLR, systematic literature review.

Characteristics of the included PROMs

A total of 204 individual instruments were included for meta-data extraction along 47 meta-data categories. When the relevant information could not be identified, ‘not known’ was entered in the respective data field. A summary of the main characteristics of PROMs is provided in table 2. PROMs were mostly designed for adults (n=152; 74%), the majority was developed in the 1990s (n=73; 36%) and 2000s (n=70; 34%), most frequently in Europe (n=91; 45%) or North America (n=88; 43%). The vast majority (n=165; 81%) of the extracted instruments were generic, whereas 19% (n=39) were designed for use in a broad field of mental health diseases and thereby allow comparison across the area of mental health without being disease-specific. Regarding the characteristics of individual PROMs or PROM families relevant to multisectoral, multinational, multiperson economic evaluations, the total number of scales equalled 134 with each PROM family counted as one. The list of PROMs (n=204) included in the review is provided in online supplemental appendix table 4.

Table 2

Summary characteristics of individual PROMs

Applicability in economic evaluations

Among the eligible (families of) PROMs, 16 (12%) had both adult and child/adolescent versions, and 27 (20%) had a proxy completion option (table 2). A total of 11 (8%) measured capabilities/family outcomes considered a criterion for broader well-being, beyond HRQOL, measurement. Furthermore, 15 (11%) had value set, of which 9 (7%) were available for more than one country. A total of 97 (72%) of (families of) PROMs had at least two language versions available. None of the (families of) instruments met all six proposed criteria for multisectoral, multinational and multiperson economic evaluations. At least five assessment criteria were met by the 15D family and the EQ-5D family of instruments, and four criteria were met by the Assessment of Quality of Life Scale family, ASCOT family, Care-related Quality of Life instrument, Child Health Utility 9D, Chateau Sante Base, Health Utilities Index Mark family and ICECAP family of instruments.

Applicability in mental health research

For the completeness of data, studies on the development and validation of the PROMs in populations suffering from any mental disorders were retrieved and compiled in the electronic compendium. Online supplemental appendix table 5 presents the PROMs for which validation studies in the mental health field were found. Section on the availability of the validation studies in mental health in the compendium consists of information whether any relevant studies were found in the secondary searches and relevant links to identified studies were provided. Specific assessment of the different aspects of validity of the PROMs based on these studies was beyond the scope of the current study. Altogether, 97 (48%) instruments had available validation studies in at least one mental disease category. Frequently, the validation studies of the instruments were available for depression, schizophrenia or mental health in general (without specifying the disease). Of all PROMs, EQ-5D-3L, Q-LES-Q, SF-36 and WEMWBS were the measures with most validation studies in the mental disease area.

The PECUNIA PROM-MH Compendium

The PECUNIA PROM-MH Compendium compiles PROM meta-data information that is either generic or designed for use in a broad field of mental health diseases, and thereby allows comparison across the area of mental health without being disease-specific. It includes dedicated sections on their validity in mental health outcome measurement focusing on QOL/well-being (including recovery) and their applicability in multisectoral, multinational and multiperson mental health economic evaluations across 220 Excel worksheets.

Information about each individual PROM is presented along 36 items that are organised into five main sections: (1) basic characteristics; (2) scoring information; (3) availability of validation studies in mental health; (4) application in multisectoral, multinational, multiperson economic evaluations and (5) other information. Furthermore, several links to websites, sample questionnaires, instrument manuals and scientific articles relevant to a specific PROM are provided to allow the user to easily identify relevant sources of information. Moreover, PROMs belonging to the same family of instruments are linked with each other so the user has a brief overview of all linked questionnaires.

Table 3 presents the comparison of two families of PROMs that fulfilled most of the proposed criteria for multisectoral, multinational and multiperson economic evaluations: the EQ-5D (consisting of EQ-5D-3L, EQ-5D-5L and EQ-5D-Y instruments) and 15D (consisting of 15D, 16D and 17D instruments) families.

Table 3

Comparison of selected PROMs based on main meta-information extracted from the PECUNIA PROM-MH Compendium

Conclusions and clinical implications

This paper describes the process that led to the synthesis of existing, publicly available PROM information suitable for QOL/well-being (including recovery) measurement in mental health economic evaluations with specific focus on their applicability in multisectoral, multinational, multiperson economic evaluations and the resulting electronic PROM library, the PECUNIA PROM-MH Compendium. Overall, we found >200 non-disease-specific PROMs that have been used or are potentially suitable for outcome measurement in mental health economic evaluations. However, no single instrument fulfilled all the criteria necessary for a fully multisectoral, multinational and multiperson economic evaluation as potentially relevant for European-level HTAs.

The PECUNIA PROM-MH Compendium is a useful Microsoft Excel-based resource when designing comparative mental health economic evaluations as it compiles an overview of all relevant instruments (generic or designed for use in mental health) with browsable features. It is a unique and unprecedented collection of PROMs in the area of mental health, and of information on their important specifics for economic evaluations. The library also addresses the increasing need for overview of methods and tools that are available, standardised, harmonised and validated across sectors, countries and demographic groups for this purpose. Main strengths of this study are the scientifically rigorous systematic review methods with which PROMs were collated, and the comprehensive meta-information that will simplify the work of future researchers who wish to identify a suitable instrument with specific characteristics. For instance, it is now easy to look for an instrument designed for children with a proxy-completion option available in multiple languages with validation studies in mental health. For this purpose, the instruments listed in PECUNIA PROM-MH Compendium are classified according to different criteria (eg, target age group) to allow more efficient searches. Furthermore, the well-documented systematic review methods enable future update of the information on a regular basis, thereby also keeping the compendium up to date.

The development process of the PECUNIA PROM-MH Compendium identified some challenges. One such challenge is the distinction between fully generic and non-disease-specific but mental health oriented instruments, for instance, the OxCAP-MH questionnaire. While the latter ones are expected to be more sensitive for mental health-specific research, they may have comparative limitations beyond the mental health field. Such comparative assessments, however, are scarce. The cut-off point between disease-specific instruments and those designed for broader mental health outcome measurement was also not clear in some instances. The categorisation of PROMs as measurements of QOL/well-being, as experienced for example for recovery measurements, is also not without controversy. A further challenge relates to the linking of multiple instruments that belong to the same PROM family because some PROMs have been renamed or further developed since their initial publication, therefore, requiring very resource-intensive secondary searches. Finally, the availability of translations, mapping studies, value sets and validation in mental health would require further quality assessment in terms of adherence to given guidelines and standards. Although, these steps were beyond the scope of the current study, based on the meta-information now collated and linked for each identified instrument in the compendium, such assessments should be much easier and systematic to do in the future.

The main limitation of the current study mainly stem from the compromises that had to be made during the design of the systematic literature review. The PECUNIA PROM-MH Compendium includes no information about instruments in development, and the information about the included PROMs is time stamped to the date of the review which should be updated on a regular basis to provide fully reliable information for its users. For example, the systematic review in this study might have missed some protocol papers and instruments under development in this rapidly changing field. The search was limited to certain languages, thereby excluding potential instruments of interests and existing non-peer-reviewed translations. The PECUNIA PROM-MH Compendium currently only includes officially recognised translations, but further translations might be potentially available. Furthermore, the synthesis excluded disease-specific measures that could be relevant for specific cost-effectiveness analyses.

The PECUNIA PROM-MH Compendium can be accessed free of charge for non-commercial research, healthcare and academic teaching activities following registration.33 A sample version of the PECUNIA PROM-MH Compendium can be accessed under the following link: https://www.pecunia-project.eu/tools/prom-mh-compendium.

Data availability statement

Data are available in a public, open access repository. The PECUNIA PROM-MH Compendium can be accessed free of charge for non-commercial research, healthcare and academic teaching activities following registration: Łaszewska A, Helter H, Perić N, Simon J: PECUNIA PROM-MH Compendium. Version 1.0; 20 Nov 2020. DOI: 10.5281/zenodo.5037082.

Ethics statementsPatient consent for publication

Not applicable.

Ethics approval

This study does not involve human participants.

Acknowledgments

We thank the PECUNIA consortium and members of the Scientific Advisory Board of the PECUNIA project for their feedback on the meta-data information template and the earlier version of the electronic PROM compendium.