1 INTRODUCTION

Research on mental health, including epidemiological and intervention studies, typically involves collection and management of sensitive personal data. This may be done either anonymously, where researchers do not know the identity of the respondents, or confidentially, where respondents are given information that no traceable record of their responses will be disclosed. Self-report questionnaires are a common method for data collection in epidemiological studies, and may in some cases be the only feasible way to measure certain phenomena. In collecting self-reported data, perceived invasion of personal integrity, or privacy, may affect validity of data in terms of biased responses and completion rates. Such bias may be due to social desirability, meaning that people convey a particular impression of themselves to others to minimize their own discomfort (Krumpal, 2013), or that participants alter their responses in what they believe to be the direction of the researchers' hypotheses, sometimes known as demand characteristics (Orne, 1962). While anonymity offers participants the opportunity to provide information without revealing their personal identity, and by doing so hypothetically reducing the risk of social desirability bias, confidential data are required for any study that requires record linkage, longitudinal follow-ups, and for allocation to interventions with longitudinal follow-ups (D’Orazio, 2015).

It has been hypothesized that anonymously collected surveys should generate higher rates of accessing the consent materials and higher reporting of items with potential social desirability bias (Nederhof, 1985). However, previous findings on whether different levels of personal integrity invasion affect response and completion rates, or data validity are inconclusive, and few studies cover the range of different conditions that mental health research encompasses. Across different research areas, several studies have shown that anonymous questionnaires result in higher quality data (Beatty et al., 2014; Beebe et al., 2006; Chase et al., 2013; Durant et al., 2002; Futrell et al., 1978; Malvin & Moskowitz, 1983; Murdoch et al., 2014; Olson et al., 2004; Richman et al., 1999; Rolnick et al., 1989; Singer et al., 1993; Stander et al., 2002; Werch, 1990; Zagumny et al., 1996), while other studies have shown that quality is not affected by a confidential approach (Albaum, 1987; Bjarnason & Adalbjarnardottir, 2000; Campbell & Waters, 1990; Esposito et al., 1984; Fear et al., 2012; Kundig et al., 2011; McKee, 1992; O’Malley et al., 2000; Ong & Weiss, 2000; van de Looij-Jansen et al., 2006). Few of these studies have investigated online surveys. In addition, research is limited on whether quality of data is affected by requesting consent to collect register data in addition to a confidential survey, although a review has shown that research participants are generally positive about participating in register studies (Da Silva et al., 2012).

1.1 Study aim

The aim of the present study was to evaluate whether initiation rates, consent rates, completion rates, and response patterns differed, depending on whether an online mental health self-report questionnaire, screening for eight psychiatric conditions, was distributed (1) anonymously; (2) confidentially, or (3) as a confidential survey where the respondents gave their consent for the collection of register data.

2 METHODS 2.1 Design and setting

A three-arm, parallel-group, single-blind randomized controlled trial was used to address the study aims. The study was conducted among students from two Swedish universities: a large university in the capital area from which educational programs were strategically selected; and a middle size university situated in a metropolitan area in the southern part of the country, where all registered students were included. The data collection took place as an initial pilot study for Swedish participation in the World Health Organization (WHO) World Mental Health-International College Student (WMH-ICS) initiative. This is a global initiative designed to generate epidemiological data on mental health issues and treatment needs in university students by using a validated web-based survey providing estimates of a wide range of mental disorders (Cuijpers et al., 2019).

2.2 Ethics and procedure

In accordance with the Swedish Act concerning the Ethical Review of Research Involving Humans (2003:460), all procedures and information, including sending invitation emails to participants and informed consent materials, were vetted by the Swedish Ethical Review Authority (Ref. No. 2020-01465, approved May 12, 2020). The informed consent materials followed a pre-specified format supplied by the Swedish Ethical Review Authority. Participants' email addresses were obtained via a national student register containing student level information on course registrations and results. Included students were randomized by a 1:1:1 ratio, resulting in three groups: an anonymous arm informing participants that survey responses were anonymous (Group 1); a confidential arm informing them that survey responses were linked to personal information via a so-called pseudonymization code (Group 2); and a confidential register arm, informing participants that survey responses were linked to personal information and register data on study activity and results (Group 3).

Students were sent one email and up to two reminders. Each invitation email provided a brief text about this research project on students' mental health and a hyperlink to a webpage with the informed consent materials. As shown in Figure 1, both the invitation email and study information included group specific information concerning the handling of personal integrity based on randomization. For students assigned to any of the three groups, participation did not differ in any way other than the content of this information. No information was given that participants had been randomized, or that the study was intended to investigate the effects of level of personal integrity invasion on levels of consenting, survey completion, or prevalence of mental health issues among respondents.

Group-specific information provided in invitation email and study information

2.3 Survey

The web-based WMH-ICS survey provides estimates of a wide range of mental disorders. Such data concerning individual health is considered sensitive information according to the European Union (EU) General Data Protection Regulation (GDPR). The survey is divided into 11 sections: background, current health, attention and concentration, emotional problems, alcohol and drugs, self-harm, seeking treatment, childhood background, recent experiences, sexuality, and concept of self. The survey screens for the following mental disorders: depression, anxiety, bipolar disorder, panic attacks, post-traumatic stress disorder, self-harm, alcohol use disorder, and drug use (Auerbach et al., 2018).

Definition of positive screens for mental disorders: Depression and anxiety are defined as ever having experienced symptoms of any of these conditions sometime in one's lifetime; bipolar disorder is defined as ever having experienced an episode lasting two days or longer; panic attacks are defined as ever having experienced three or more panic attacks; post-traumatic stress disorder is defined as ever having experienced an episode of trauma-related stress lasting one month or longer; self-harm is defined as ever having experienced suicidal ideation, thoughts on suicide, or deliberate self-harm; alcohol use disorder is defined as problematic alcohol use consisting of at minimum drinking monthly, and having more than 2 standard drinks at each drinking occasion; drug use disorder is defined as ever having taken drugs.

2.4 Analysis

The analysis plan was registered at the Open Science Framework (OSF; Andersson et al., 2020), before downloading the data from the survey platform but after the data collection had begun; however, purpose, method and outcomes were specified in the ethical application which was approved before data collection was initiated. Figure 2 shows the four basic steps of the analyses. First, we conducted group comparisons of the proportion of respondents who, after receiving the invitation email, pressed a hyperlink to continue to the webpage providing study information. Second, we carried out group comparisons of the proportion of respondents who, after having taken part of the study information, provided informed consent. Third, we conducted group comparisons on the cumulative proportion of consenters who abandoned the questionnaire per question presented. The fourth group comparison concerned proportions of positive screens concerning the following eight psychiatric disorders: depression; anxiety; bipolar disorder; panic attacks; post-traumatic stress disorder; self-harm; alcohol use disorder; drug use. See Figure 2 for an overview.

Four levels of group comparisons

All analyses, except where noted, were conducted according to intention to treat principles, analyzing participants within the groups to which they were randomly allocated. Analyses were conducted using available data, assuming that any missing data were missing at random. Imputation was not considered an option, since no baseline data were available prior to randomization and causal mechanisms underlying missingness in this context are not well understood due to the varied outcomes shown in currently published empirical research. Outcomes were analyzed using Chi-2 and regression models comparing the three groups pairwise: logistic for proportions and proportional hazard for cumulative proportions. Models were estimated by using Bayesian interference with standard (half-) normal priors. Marginal posterior probability of estimates being greater (or less) than the null are reported, and posterior medians are used as estimates of effects alongside 50% and 95% compatibility intervals. The Bayesian estimates are complemented with null hypothesis testing of maximum likelihood estimates of model covariates (at the 0.05 significant level). All models, except proportions of participants who pressed the hyperlink provided in invitation emails, and proportions of participants providing informed consent, were adjusted for gender identity and age. Effect-modification analyses were performed for all outcomes, except for proportions of participants who pressed the hyperlink in invitation emails and proportions of participants providing informed consent. These analyses were conducted by adding interaction terms between group allocation and the following potential effect modifiers: gender identity, age, international student, response language, current student status, and university.

3 RESULTS 3.1 Participation overview

Figure 3 shows the participant flow for university students randomized into the three groups: anonymous (Group 1); confidential (Group 2); and confidential plus register arm (Group 3). Invitation emails including group specific content were sent to 16,152 university students. A total of 374 (2.3%) of the email addresses failed or bounced, resulting in a total of 15,778 receiving an invitation. Of those receiving an invitation, 2635 (16.7%) pressed a hyperlink in the email to continue to a webpage providing group-specific study information. Of these, a total of 2005 (76.1%) gave their consent to participation. Participants were then directed to the questionnaire (the same for all groups), which was completed by 1224 (61.0%) of the consenters. Relative to the total number of students who received the invitation, 12.7% agreed to participate and 7.8% completed the questionnaire.

Participant flow. a = Percent of those who received email. b = Percent of those who proceeded to study information. c = Percent of those who provided consent. Italicized numbers indicate participants who did not proceed to the next step, thereby abandoning the survey

3.2 Study information and consent

Table 1 presents group comparisons of proportions of participants who initiated study participation by pressing a hyperlink in the invitation email (upper section), followed by group comparisons of proportions who consented participation (lower section). Analysis of those who pressed the link for study information showed that participants in the anonymous arm (Group 1) provided their informed consent to a larger extent than participants in both confidential arms (Groups 2 and 3; 79.3% vs. 73.4% and 75.5%).

TABLE 1. Group comparisons of proportions of participants who pressed a hyperlink provided in emails to reach a webpage providing group-specific study information, and proportions of participants providing informed consent Bayesian marginal posterior distribution Maximum Likelihood estimates and null hypothesis testing Median (2.5%; 97.5%) OR < 1 OR estimate (95% CI) p-value Study information Group 2 vs. Group 1 0.98 (0.89; 1.09) 61.9% 0.98 (0.89; 1.09) 0.76 Group 3 vs. Group 1 0.98 (0.89; 1.09) 63.8% 0.98 (0.89; 1.09) 0.73 Group 3 vs. Group 2 1.00 (0.90; 1.11) 52.8% 1.00 (0.90; 1.11) 0.97 Informed consent Group 2 vs. Group 1 0.73 (0.58; 0.90) 99.8% 0.72 (0.58; 0.90) <0.01 Group 3 vs. Group 1 0.81 (0.65; 1.02) 96.3% 0.81 (0.65; 1.01) 0.063 Group 3 vs. Group 2 1.12 (0.90; 1.38) 15.2% 1.12 (0.90; 1.39) 0.31 Note: Group 1 = anonymous response; Group 2 = confidential arm with personally identified data; Group 3 = confidential arm with personally identified data and linkage to register data. Second group is reference group. 3.3 Abandoning the questionnaire

Table 2 compares the cumulative proportion of consenters abandoning the questionnaire per question presented, converted to a progress rate between 0 and 100 percent. Proportional hazard regression with adjustments for gender identity and age was conducted using two subsets: the left column includes participants who consented and initiated the survey; the right column excludes completers, since these are a strong majority skewing the distribution towards 100% progress. No marked between-group differences were found.

TABLE 2. Cumulative proportions abandoning the questionnaire per question presented Bayesian marginal posterior distribution Subset 1: Consented and initiated survey (n = 1912) Subset 2: Completers excluded (n = 694) Median (2.5%; 97.5%) HR > 1 Median (2.5%; 97.5%) HR > 1 Group 1 Baseline hazard 0.018 (0.014; 0.022) - 0.072 (0.049; 0.105) - Group 2 Hazard ratio 0.94 (0.84; 1.05) 12.3% 0.98 (0.82; 1.17) 41.3% Group 3 Hazard ratio 0.95 (0.85; 1.06) 18.2% 1.05 (0.88; 1.26) 71.5% Note: Proportional hazard regression with adjustments for gender identity and age. Group 1 = anonymous response; Group 2 = confidential arm with personally identified data; Group 3 = confidential arm with personally identified data and linkage to register data. 3.4 Positive screens for psychiatric conditions

Table 3 presents group distributions of participants screening positive for any of the following eight psychiatric conditions: depression; anxiety; bipolar disorder; panic attacks; post-traumatic stress disorder; self-harm; alcohol use disorder; drug use. In each cell the percentage of positive screens is followed by the frequency of positive screens in relation to the total number of respondents.

TABLE 3. Proportions of positive screens and group-comparisons of proportions screening positive for eight psychiatric disorders Positive screens Group-comparison % (n/N) Chi-2 Bayesian marginal posterior distribution Maximum likelihood estimates and null hypothesis testing p value Median (2.5%; 97.5%) OR < 1 OR estimate (95% CI) p value Depression Total 56.3 (937/1664) Group 2 vs. Group 1 53.7 (286/533) 59.7 (339/568) 0.044 0.78 (0.61; 0.99) 97.8% 0.78 (0.61; 0.99) 0.045 Group 3 vs. Group 1 55.4 (312/563) 59.7 (339/568) 0.15 0.84 (0.61; 1.07) 92.1% 0.84 (0.66; 1.06) 0.15 Group 3 vs. Group 2 55.4 (312/563) 53.7 (286/533) 0.56 1.07 (0.84; 1.35) 27.8% 1.07 (0.84; 1.36) 0.57 Anxiety Total 76.3 (1277/1673) Group 2 vs. Group 1 76.6 (410/535) 76.4 (438/573) 0.94 1.03 (0.78; 1.36) 42.7% 1.03 (0.77; 1.36) 0.86 Group 3 vs. Group 1 75.9 (429/565) 76.4 (438/573) 0.84 0.99 (0.75; 1.29) 53.9% 0.99 (0.75; 1.30) 0.92 Group 3 vs. Group 2 75.9 (429/565) 76.6 (410/535) 0.79 0.95 (0.72; 1.26) 63.5% 0.95 (0.71; 1.26) 0.72 Bipolar disorder Total 21.9 (277/1262) Group 2 vs. Group 1 22.7 (92/431) 21.3 (92/431) 0.65 1.07 (0.77; 1.48) 33.4% 1.07 (0.77; 1.49) 0.67 Group 3 vs. Group 1 21.9 (93/425) 21.3 (92/431) 0.85 1.01 (0.73; 1.41) 47.1% 1.01 (0.73; 1.41) 0.93 Group 3 vs. Group 2 21.9 (93/425) 22.7 (92/431) 0.79 0.95 (0.69; 1.31) 62.0% 0.95 (0.68; 1.32) 0.75 Panic attacks Total 50.6 (678/1341) Group 2 vs. Group 1 48.7 (209/429) 50.8 (235/463) 0.54 0.94 (0.73; 1.23) 66.6% 0.94 (0.72; 1.23) 0.67 Group 3 vs. Group 1 51.1 (234/449) 50.8 (235/463) 0.68 1.07 (0.83; 1.39) 29.1% 1.07 (0.82; 1.40) 0.60 Group 3 vs. Group 2 51.1 (234/449) 48.7 (209/429) 0.31 1.15 (0.88; 1.49) 16.2% 1.15 (0.88; 1.50) 0.32 Post-traumatic stress disorder Total 61.7 (819/1327) Group 2 vs. Group 1 62.1 (267/430) 61.4 (277/451) 0.84 1.04 (0.79; 1.36) 37.9% 1.04 (0.79; 1.37) 0.76 Group 3 vs. Group 1 61.7 (275/446) 61.4 (277/451) 0.94 1.04 (0.80; 1.36) 38.0% 1.04 (0.79; 1.37) 0.77 Group 3 vs. Group 2 61.7 (275/446) 62.1 (267/430) 0.89 0.99 (0.75; 1.30) 53.0% 0.99 (0.75; 1.30) 0.93 Self-harm Total 61.5 (783/1274) Group 2 vs. Group 1 60.0 (252/420) 62.1 (265/427) 0.54 0.92 (0.70; 1.22) 72.0% 0.92 (0.70; 1.22) 0.56 Group 3 vs. Group 1 62.3 (266/427) 62.1 (265/427) 0.94 0.99 (0.75; 1.31) 52.2% 0.99 (0.75; 1.31) 0.94 Group 3 vs. Group 2 62.3 (266/427) 60.0 (252/420) 0.49 1.08 (0.82; 1.43) 28.9% 1.08 (0.82; 1.43) 0.59 Alcohol use disorder Total 71.7 (929/1295) Group 2 vs. Group 1 70.0 (297/424) 72.1 (316/438) 0.50 0.91 (0.68; 1.22) 72.4% 0.92 (0.68; 1.23) 0.56 Group 3 vs. Group 1 73.0 (316/433) 72.1 (316/438) 0.78 1.05 (0.78; 1.41) 37.4% 1.06 (0.78; 1.43) 0.73 Group 3 vs. Group 2 73.0 (316/433) 70.0 (297/424) 0.34 1.17 (0.87; 1.57) 15.0% 1.18 (0.87; 1.59) 0.29 Drug use Total 34.8 (447/1286) Group 2 vs. Group 1 34.1 (144/422) 36.2 (158/436) 0.52 0.90 (0.68; 1.19) 77.9% 0.89 (0.67; 1.18) 0.43 Group 3 vs. Group 1 33.9 (145/428) 36.2 (158/436) 0.47 0.87 (0.66; 1.16) 82.6% 0.87 (0.65; 1.15) 0.33 Group 3 vs. Group 2 33.9 (145/428) 34.1 (144/422) 0.94 0.98 (0.74; 1.31) 55.5% 0.98 (0.73; 1.31) 0.90 Note: Logistic regression with adjustments for gender identity and age. Group 1 = anonymous response; Group 2 = confidential arm with personally identified data; Group 3 = confidential arm with personally identified data and linkage to register data. Second group is reference group.

Table 3 also presents the analyses of comparative proportions of positive screens on the eight psychiatric disorders in the three groups, expressed in odds ratios from logistic reg