For this study, we used cross-sectional data obtained in the fourth survey wave of the European COvid Survey (ECOS) project, which is described in detail elsewhere [11]. Generally, this online survey examined support for COVID-19 containment policies, including vaccinations, worries about COVID-19, and trust in different information sources. The data in the fourth wave of this survey was obtained between 5 and 16 November 2020. Respondents (n = 7115) were recruited from the general public in seven European countries (Denmark, France, Germany, Italy, Portugal, the Netherlands, and the United Kingdom) by the market research company Dynata using multisource online panels. To ensure that the sampling frame was representative of the population in each country, the company used various recruiting procedures for different subgroups of the population in each country. It used for example advertised/open recruitment, loyalty programs, affiliate networks and mobile apps [11]. Quotas based on age category, regional distribution and gender were implemented by the authors using the Qualtrics research suite to ensure and control the representativeness based on the country specific census data. Dynata ensured representativeness with regard to educational categories based on their expertise in the differences in educational degrees for each country. The authors proceeded by excluding incompletes answers and speeders (faster than 1/3 of the median time in each country), both of which were replaced by Dynata to ensure the representativeness of the sample. The resulting sample of respondents from each country (with n ~ 1000) was representative of its adult population in terms of region, gender, age group and education level.

The questionnaire was available in the seven languages of the included countries. The MHQoL had existing official versions in Dutch, English and German, which were used in this survey. For the other four countries and languages, the MHQoL instrument was translated by native speakers with a background in health economics.

Respondents completed the MHQoL instrument along with the EuroQol (EQ-5D-5L and EQ-VAS), ICECAP-A (the ICEpop CAPability measure for Adults), and PHQ-4 (Patient Health Questionnaire for Depression and Anxiety) instruments. For these instruments, official translations available from the developers of these instruments were used. Respondents also were asked to answer questions about their demographic characteristics including gender, age, relationship status, and level of education, next to COVID-related questions.

The MHQoL is an instrument intended to be used to describe and value respondents’ current mental health-related QoL [9]. In the descriptive part, respondents are asked to describe their mental health state using seven specific dimensions of mental health-related quality of life and four answering levels per dimension. The seven dimensions are: self-image, independence, mood, relationships, daily activities, physical health and hope. Levels for self-image for example range from ‘I think very positively about myself’ to ‘I think very negatively about myself’. Preference-based tariffs, allowing scores on the different levels to be converted into utility scores anchored on 0 (dead) and 1 (full mental health-related QoL), are not yet available for the MHQoL. In the absence of tariffs, the MHQoL sum score is used as an alternative, which ranges from 0 (lowest level on all seven dimensions) to 21 (highest level on all seven dimensions), with higher scores indicating better mental health-related quality of life. Next to the descriptive part, the MHQoL instrument also has a direct valuation part in which respondents are asked to rate their psychological wellbeing using a horizontal visual analogue scale (MHQoL-VAS) ranging from 0 (representing ‘worst imaginable psychological wellbeing’) to 10 (representing ‘best imaginable psychological wellbeing’).

The five-level EQ-5D (EQ-5D-5L) is a generic instrument to measure and value respondents’ current health-related QoL [4]. Within the questionnaire, respondents are asked to describe their health using five dimensions and five answering levels per dimension. These five dimensions are: Mobility, Self-Care, Usual Activities, Pain/Discomfort, and Anxiety/Depression. The five answering levels range from having no problems to having extreme problems. Using country-specific, preference-based tariffs, answer scores can be converted into utility scores, with 0 as the equivalent of the state ‘dead’ and 1 as the equivalent of the state ‘perfect health’. In addition to scoring the five dimensions, respondents are asked to rate their current overall health using a vertical visual analogue scale (EuroQol Visual Analogue Scale; EQ-VAS) ranging from 0 (‘worst imaginable health state’) to 100 (‘best imaginable health state’).

The ICE-CAP-A (ICEpop CAPability measure for Adults) is an instrument to measure and value respondents’ overall capability wellbeing and is grounded in Sen’s capability approach [12, 13]. Within the questionnaire, respondents are asked to describe their capabilities in relation to five important life domains: stability, attachment, autonomy, achievement and enjoyment. Each domain is scored using four levels, ranging from full capability to no capability. Using preference-based tariffs, answer scores can be converted to standardised index scores, ranging from 0 (no capability) to 1 (full capability). Currently, tariffs for the United Kingdom [12] and the Netherlands [14] are available.

Finally, the PHQ-4 is a four-item self-complete screening instrument that measures respondents’ likeliness of an anxiety disorder and/or depression [15]. The PHQ-4 is based on the Patient Health Questionnaire (PHQ), a more extensive instrument used by care providers to diagnose patients with mental health disorders. Dimensions are “Feeling nervous, anxious or on edge”, “Not being able to stop or control worrying”, “Feeling down, depressed or hopeless” and “Little interest or pleasure in doing things”. The PHQ-4 has four levels which range from “Not at all” to “Nearly every day”. A sum score can be calculated, which can be used to categorise the respondent’s psychological distress level as none (0–2), mild (3–5), moderate (6–8), or severe (9–12).

Using descriptive statistics, the basic characteristics of the respondents were summarized. Mean MHQoL sum score and mean MHQoL-VAS score were compared to scores known for the Dutch general population [9]. For subgroups based on country, gender, and age group, MHQoL sum scores were calculated, also to provide a reference for future studies. In doing so, we do acknowledge and stress the exceptional situation due to the COVID-19 pandemic.

We additionally investigated the MHQoL dimension scores, comparing the youngest and oldest age groups. A lower and upper MHQoL quartile were distinguished using cut-off values (respectively < 12 and > 16) for the MHQoL sum score. We investigated the membership of these quartiles focusing on background characteristics of the respondents including income, mean, minimum and maximum values of the four other outcome measures. EQ-VAS mean scores per country were compared to population norm scores, also given the fact that the survey was conducted during the COVID-19 pandemic.

To examine the internal consistency of the MHQoL, Cronbach’s alpha of the seven dimensions was calculated, both overall and by country. Cronbach’s alpha is a coefficient that represents the extent to which items of a measure are correlated, indicating the extent to which these items measure the same construct, here mental health-related quality of life. Cronbach’s alpha is expressed as a number between 0 (no correlation) and 1 (full correlation), and a score of > 0.7 is seen as indicating good internal consistency [16]. In addition, Spearman’s rank correlation coefficients of the MHQoL sum score and the MHQoL-VAS were calculated, again overall and by country. This correlation was expected to be high and positive since both aim to measure mental health-related QoL. When interpreting results, a correlation coefficient is seen as trivial when < 0.1, as small when 0.1–0.3, as moderate when 0.3–0.5, as high when 0.5–0.7, as very high when 0.7–0.9, and as nearly perfect when > 0.9, following previous evaluation studies like Hoefman et al. [17]. Additionally, the association between the MHQoL-VAS and the MHQoL dimension scores was investigated using a linear regression model (ordinary least squares regression; OLS). A moderate to high positive correlation and a moderate to high adjusted R2 were expected since these dimensions levels represent mental health states which determine overall mental health-related QoL. An adjusted R2 of > 0.2 was expected, based on previous models which modelled EQ-VAS as the dependent variable and the levels of the EQ-5D dimensions as independent variables [18, 19].

To examine convergent validity, Spearman’s rank correlation coefficients between the MHQoL sum score and respectively: EQ-5D index score, EQ-VAS, EQ-5D anxiety/depression dimension score, ICECAP-A index score, and PHQ-4 sum score were calculated, overall and by country. The resulting coefficients inform whether these measures of theoretically interrelated constructs are correlated, which is an indication of construct validity. We expected the MHQoL sum score to have a moderate to strong positive correlation (0.3–0.7) with the EQ-5D index score, the EQ-VAS, and the ICECAP-A index score. This was based on the assumption that having a better mental health-related QoL is associated with both a better health-related QoL and a better wellbeing. It is acknowledged that these are complex associations, e.g. mental health has direct and indirect effects (e.g.by life style choices) on physical health and vice versa [20].

Furthermore, we expected the MHQoL sum score to have a strong negative correlation (0.5–0.7) with the EQ-5D anxiety/depression dimension and with the PHQ-4 sum score, since a better mental health-related QoL is strongly related to the absence of mental health problems. Tariffs to compute EQ-5D-5L index scores were obtained from the EuroQol website [21]. Tariffs to compute ICECAP-A index scores were obtained from the website of the University of Birmingham [12]. United Kingdom (UK) tariffs were used for EQ-5D-5L and ICECAP-A for all countries. This was done given that tariffs were not available for all countries. Since the current UK tariffs for EQ-5D-5L have been disputed [22] the analyses were also performed using Dutch EQ-5D-5L tariffs [23] to check whether this would influence results.

Beforehand, we did not formulate expectations regarding mean MHQoL sum scores in different countries although they are expected to differ, e.g. based on country differences in depression stigma [24].

All analyses were performed using STATA version 16.1 (StataCorp, 2019. Stata Statistical Software: Release 16. College Station, TX).