Background

Obsessive-compulsive disorder (OCD) is characterised by persistent thoughts, images and/or sensory perceptions that cause distress and repetitive behaviours performed in order to temporarily reduce distress. Its global lifetime prevalence is 2%.1 Clinical presentation is heterogeneous, but without treatment OCD may profoundly impair quality of life and social functioning. Selective serotonin reuptake inhibitors (SSRIs), cognitive–behavioural therapy with exposure and response prevention, or a combination of both is currently recommended for management of OCD.2 Clomipramine was the first effective pharmacotherapeutic intervention, but its side effect profile renders it a secondary option to SSRIs.3

In the context of major depressive disorder (MDD), concerns have been raised regarding overestimation of SSRI efficacy due to flaws in trial quality.4–6 For instance, a recent reanalysis of a meta-analysis of randomised SSRI trials in MDD showed all studies to be of high or unclear risk of bias. Moreover, another meta-analysis found a small, clinically irrelevant symptom reduction by SSRIs.7 8 However, these results are derived from average treatment effects and did not take modifiers of effect into account. Furthermore, a decreasing effect size might be the result of increasing placebo effect or increasing treatment resistance of the trial population.9 10 Whether overestimation of efficacy pervades in studies examining SSRIs in OCD as well remains currently unanswered.

Besides quality, study-level characteristics such as sponsor status, number of treatment arms and trial population have been found to influence trial success.11 12 For OCD, there is currently a lack of information on the influence of randomised controlled trial (RCT) methodology on efficacy, even though this information could contribute to optimising future trial design.

Recent evidence suggests that publication bias exaggerates pharmacotherapy efficacy in MDD, with more pronounced effect size inflation in older trials.13 For OCD, a study examining publication bias compared efficacy measures of published trials with original Food and Drug Administration (FDA) data. They found a pooled effect size of 0.39 Hedges’ G according to the US FDA, compared with 0.45 according to the published scientific literature.14 This increase was non-significant; however, this analysis only included SSRI trials submitted for FDA approval, potentially skewing the representation of reporting bias for OCD pharmacotherapy. Also, publication bias is often identified using funnel plot models that focus on small-study effects, while selection models that focus on biases in publishing of significant studies might be more sensitive to publication bias.15 ,16 In addition, Bayesian approaches to selection models are especially suited for smaller meta-analyses.17 18

Objective

In this study, we examined the effect of study-level characteristics on the efficacy of pharmacotherapy in placebo-controlled OCD trials. Furthermore, we analysed the influence of risk of bias on effect size and investigated publication bias employing a Bayesian approach.

We hypothesised an inverse correlation between risk of bias and effect size and that publication bias could exaggerate efficacy. We predicted larger effect sizes in industry-sponsored studies, older studies, studies with multiple treatment arms, with placebo run-ins and with fixed-dose regimens. We also hypothesised that studies with participants having shorter illness duration, higher baseline severity and younger age show higher efficacy.

Study selection and analysisSearch strategy

We searched Embase, Medline, PsycINFO and the Web of Science Conference Proceedings Index on 22 February 2023. Additionally, we searched the WHO International Clinical Trial Registry Platform search portal for registered studies, did a scoping search on Cochrane CENTRAL and used websites of several major conferences to search for unpublished literature or conference proceedings. We checked the included articles for references and conducted citation screening. For a detailed account of our search strategy, see online supplemental material.

Screening and inclusion

Two investigators (SEC and BW) independently screened studies using Rayyan.19 We included double-blind RCTs for monotherapy with an SSRI or clomipramine in adult patients (18 years and older) with OCD. We included a non-selective patient population suffering from OCD with all subtypes. We included short-term studies with a primary endpoint up to 16 weeks using the Yale-Brown Obsessive-Compulsive Scale (YBOCS).

If studies did not publish quantified data or if we were unable to retrieve the full text, we contacted the authors to request the information necessary for our analysis. If we were unable to retrieve this information, we excluded the study. Inclusion or exclusion conflicts were resolved by consensus, or if necessary through a consensus meeting with the coauthors. We prespecified the methods in the PROSPERO database for systematic reviews (registration number CRD42023394924).

Data extraction

Two authors (SEC and BWCS) extracted the following data from the included studies: mean age and gender, number of participants in active or placebo group, exclusion criteria, intervention and dosing regimen, washout period, time to primary endpoint and difference in YBOCS response for placebo and intervention. If a study used a fixed-dosing regimen with multiple doses, we subdivided each dosing group and compared them with a proportionally reduced placebo group. Also, we extracted publication year, sponsor status and sponsor name, country or countries of trial site(s), use of a placebo run-in, and number of trial site(s). To assess risk of bias, we used the Cochrane Risk of Bias V.2.0 tool.20 Risk of bias, subdivided into low, some concerns and high risk of bias, was assessed by SEC and BWCS. Discrepancies were discussed in the entire research team in order to reach consensus.

Meta-analytic method

As the primary outcome, we used the mean change in YBOCS at primary study endpoint compared with baseline. If mean change scores were not reported, we used difference in YBOCS at study endpoint after ensuring baseline symptom severity to be balanced across intervention arms. For effect size, we used Hedges’ G, with a standard CI of 95% using Knapp-Hartung adjustments.21 22 Assuming between-study variability, we used a random-effects model for pooling effect sizes, with a restricted maximum likelihood estimator to calculate the estimated SD of the true mean difference (τ).23 We chose to pool clomipramine and SSRIs into one group and carried out a meta-regression for potential moderating effect of intervention type (clomipramine or SSRI). We analysed SSRIs as one group as they share their primary mechanism of action even though they differ in chemical structure.24 25

We performed meta-regression for mean age, gender distribution, mean duration of illness, mean baseline severity, use of two or more trial arms, sponsor status, use of a placebo run-in phase, risk of bias (high risk yes/no) and publication year. Since we use a multitude of single regressions, we adjusted the significance threshold by adjusting halfway p=0.05 and the Bonferroni adjustment, that is, p=0.010. If moderators accounted for heterogeneity, we included them in a multiple meta-regression analysis. In order to avoid multicollinearity, we tested for prediction correlators. If highly correlated (r>0.8), we included the moderator causing the highest amount of heterogeneity. For sponsorship status, we distinguished between studies that were fully sponsored by pharmaceutical companies, publicly funded studies, privately funded studies by any other institution than a pharmaceutical company and studies in which only medication was reimbursed by a pharmaceutical company. Additionally, we performed a separate meta-analysis for the SSRI group only, using the same methodology.

Effect of study quality was evaluated in a separate analysis by excluding studies that were at high risk of bias. For the moderating effect of trial quality, we included a sensitivity analysis excluding the randomisation process from the risk of bias assessment, since randomisation processes are currently defined more strictly compared with when the original studies were published.26

Publication bias

For publication bias analysis, we performed a fully Bayesian Copas analysis using set, weakly informative, priors. Thus, we were be able to quantify the magnitude of publication bias in our selected studies using a measure D, and calculate the difference between corrected and non-corrected effect sizes, using a Bayesian meta-analysis. Additionally, we performed a corrected Egger’s test and used funnel plot for visual inspection of publication bias.

For meta-analytic estimation of pooled effect sizes, as well as for meta-regression and conventional publication bias quantification, we used the metafor and dmetar packages in R.27 28 For Bayesian analysis for publication bias, we used the Robustbayesiancopas package in R.18

FindingsSearch results and study description

Our search yielded 3729 articles, of which we excluded 3646 after screening of titles and abstracts. Of the 83 articles we included for full-text screening, we included 20 papers, one of which reported on two separate studies, for a total of 21 studies.29 For a full account of our inclusion and exclusion, please refer to figure 1 and online supplemental material. The 21 included studies contained a total of 4102 participants, with 739 participants in the clomipramine RCTs and 3363 in the SSRI RCTs. 51 per cent of the participants were male. We included four studies focused exclusively on clomipramine, two of which were combined in a single manuscript. One three-armed study compared paroxetine with clomipramine and placebo. Of the included studies, all except four used a placebo run-in phase. Most studies were fully sponsored by a pharmaceutical company. Nine studies were supported by a grant independent from the producing company, most of which were granted free use of medication. The total number of included patients in independent studies was 232, against 3870 patients in sponsored studies. Nine studies used a fixed-dose regimen, one of which was a clomipramine study. All variables for meta-regression were available, except for mean duration of illness which was not mentioned in eight studies, and information on placebo run-in, which was unavailable in one article. For a full overview of study characteristics, see table 1 and online supplemental material.

Figure 1

Flow chart of inclusion and exclusion of studies.

Table 1

Study characteristics

Risk of bias

In total, 4 out of 21 included studies were judged as having a low risk of bias (see figure 2). Seven studies were at high risk of bias and 10 had some concerns. Concerns on randomisation resulted mostly from absence of reporting on the allocation sequence generation process and allocation concealment. Regarding assignment of intervention, four studies were at high risk of bias for not performing an intent-to-treat analysis, with dropouts possibly leading to attrition bias. Four studies excluded more than 10% of patients from the efficacy analysis. Risk of bias in measurement of outcome was low in all studies. In reporting of results, all but three studies did not report on the use of a prespecified analysis plan. For a full account of the risk of bias assessment, see online supplemental material. Our predefined sensitivity analysis excluding the randomisation process from the risk of bias assessment did not change overall bias results.

Figure 2

Risk of bias assessment for all studies using the Cochrane Risk of Bias V.2.0 tool. CSG, Clomipramine Study Group.

General meta-analysis

A random-effects meta-analysis with Knapp-Hartung adjustments resulted in a standardised mean difference (SMD) of −0.59 (Hedges’ G, 95% CI −0.73 to −0.46; see figure 3), which is equal to a mean difference of 4.2 points on the YBOCS. The test for heterogeneity demonstrated a significant level of variability across the samples (Q=114, p<0.0001), with an estimated tau of 0.30, suggesting considerable between-study variance and an I2 value of 73%, indicating a high proportion of the observed variance reflects real differences in effect sizes. Heterogeneity was further demonstrated by the 95% prediction interval, which ranged from −1.22 to 0.03. Random-effects meta-analysis for SSRIs specifically resulted in an effect size of −0.47 SMD (Hedges’ G, 95% CI −0.56 to −0.39) with low heterogeneity across studies (I2=16.0%, tau<0.0001) (see online supplemental figure S2).

Figure 3

Forest plot of random-effects meta-analysis of included studies. c, clomipramine; CSG, Clomipramine Study Group; e, escitalopram; f, fluvoxamine; p, paroxetine; SMD, standardised mean difference.

Meta-regression

After controlling for baseline differences, studies using clomipramine had a higher efficacy than SSRI studies (β=−0.55, 95% CI −0.24 to −0.9686, p=0.0011), as were studies at high risk of bias (β=−0.51, 95% CI −0.19 to −0.82, p=0.0029). Publication year was negatively associated with YBOCS change (suggesting that RCT efficacy decreases over time; β=0.028, 95% CI 0.0053 to 0.050, p=0.017). Furthermore, studies that were fully sponsored by a pharmaceutical company had a smaller effect size than non-sponsored or partially sponsored studies (β=0.46, 95% CI 0.092 to 083, p=0.016). Studies using two intervention arms had a higher efficacy compared with studies using one intervention arm (β=−0.33, 95% CI −0.061 to 0.60, p=0.016). Using our corrected significance threshold (p=0.010), publication year, sponsor status and number of study arms did not reach significance level. We performed a separate meta-regression in which dosing arms of fixed-dose studies were combined in one intervention arm, which led to a non-significant influence of the number of study arms on efficacy (see online supplemental material). No other regression results differed in these sensitivity analyses. Regarding baseline characteristics, increased mean age at baseline was associated with a decrease in efficacy (β=0.075, 95% CI 0.0074 to 0.14, p=0.031), which was insignificant after correcting for multiple analyses. Use of a placebo run-in, use of fixed or flexible dose, illness duration at baseline, baseline YBOCS and gender were not associated with changes in efficacy (see table 2 for the results of single meta-regression models).

Table 2

Regression coefficients of single regressions

Subsequently, we performed a multiple meta-regression analysis for significant predictors. Testing for multicollinearity showed no redundant variables (see online supplemental table S2 for the multicollinearity table). We included risk of bias, clomipramine, sponsor status, number of intervention arms, mean age and publication year in our multiple meta-regression model using a mixed effect of maximum likelihood. Using our best-performing meta-regression model, we found that when correcting for high risk of bias clomipramine remained significantly correlated with a higher effect size compared with SSRI (β=−0.43, 95% CI −0.74 to −0.12, p=0.0085; table 3). For additional information regarding model performance and selection, see online supplemental material.

Table 3

Regression coefficients of multilevel meta-regression for the model, including studies with high risk of bias and clomipramine use

Publication bias

For the full sample, visual inspection of the funnel plot (figure 4), as well as the Egger’s linear regression test of funnel plot asymmetry, as implemented using the method by Pustejovsky, shows no indication of publication bias (t=0.23, p=0.82). In contrast, using a Bayesian Copas selection model, a moderate amount of publication bias was found (D=0.48). After adjusting for publication bias, efficacy was reduced with an SMD of 0.11, from −0.53 (95% credible interval −0.64 to −0.42) to −0.42 (95% credible interval −0.60 to −0.22) using Bayesian analysis for determining efficacy.

Figure 4

Funnel plot for visual inspection of publication bias.

GRADE assessment

We used the Grading Recommendations Assessment Development Evaluation (GRADE) to evaluate the certainty of evidence. 30 Due to risk of publication bias and the large percentage of studies at high risk of bias or with some concerns, we graded the certainty of evidence as low, meaning that confidence in the effect estimate is limited and the true effect may substantially differ from the estimated effect (online supplemental table S7).30

Conclusions and clinical implications

In our meta-analysis of RCTs, we found that pharmacotherapy for OCD has a medium effect size in favour of intervention compared with placebo, amounting to 4.2 points on the YBOCS. The results were heterogeneous, with studies at high risk of bias being more likely to lead to a larger effect than studies that were at low risk of bias or had some concerns. Clomipramine was more efficacious than SSRIs, even after correcting for studies at high risk of bias. Furthermore, using a Bayesian analysis, we found a moderate risk of publication bias, with a small decrease in estimated efficacy after correcting for publication bias.

Our efficacy findings were comparable with those of an earlier meta-analysis and a recent network meta-analysis.31 32 The advantage of clomipramine over SSRIs is also in line with earlier evidence.33 However, in contrast to our analysis, Skapinakis et al 31 found that clomipramine was not significantly more efficacious than SSRIs and that any non-significant differences further dissipated when considering studies with low risk of bias. This result might be explained by the fact that the authors conducted a network analysis, also including head-to-head clinical trials. Furthermore, including only studies with low risk of bias is based on the stern assumption that studies with some methodological shortcomings do not have any added empirical value and it depletes power for subgroup analysis. Alternative explanations for clomipramine’s higher efficacy include lower quality and older studies conducted before SSRIs, suggesting the population was medication-naïve with fewer non-responders.33 34 Although studies did not register the degree of earlier non-response, clomipramine remained a significant predictor of treatment success after correcting for publication year.

After correction for multiple tests, we did not find an effect of participant characteristics on trial efficacy, in line with earlier literature.35 Furthermore, study characteristics such as publication year and amount of intervention arms were not significant after adjusting for multiple testing. We expected that pharmaceutical-sponsored studies would demonstrate higher efficacy than public or non-industry-funded studies, as found in a recent Cochrane review.36 However, we saw a negative influence of industry sponsorship, even though this effect was not significant after correcting for multiple tests. This contrasts the argument of investigator bias inflating effect size. We hypothesise that this rather reflects the paucity of non-sponsored studies as besides being older they are relatively small, single-centre and therefore more likely to be homogeneous and non-generalisable. Although a placebo-run in phase has been found to increase efficacy by decreasing placebo response in depression trials, we were unable to demonstrate an effect of placebo run-in, possibly as almost all studies actually used a run-in phase.37 Notably, the European Medicines Agency guideline on clinical investigation of medicinal products for the treatment of OCD recommends against using a run-in phase as it might impair generalisation of the study results.38

Our meta-analysis is the first study on OCD pharmacotherapy to demonstrate an effect of publication bias on efficacy, using a Copas selection model with a Bayesian approach. Selection models, which assume that publication bias arises from selective publishing of statistically significant studies, are thought to be preferable over funnel plot methods for detection of publication bias.15 39 40 Moreover, Bayesian selection models circumvent the assumption of a normal distribution of effect sizes and is thought to enhance sensitivity in smaller meta-analyses.18 In the case of OCD, we conclude that there is clear evidence of effect size inflation due to publication bias in the literature and treatment benefit is diminished somewhat after correction.

We found only four studies with a low risk of bias and seven studies with a high risk. The remainder of the studies were deemed at some concern for risk of bias. Most studies did not have a predefined analysis plan or protocol, and most studies did not disclose the randomisation process or allocation procedure. Both might be explained by shifting publication and quality standards for RCTs, more specifically because almost all included studies were published before the 2004 requirement by journals for prospective registration in a public trial registry and before the FDA mandated preregistration in 2007.41 42 In fact, the most recent article we were able to include was published in 2007. In addition to critically appraising current evidence, we took into account the influence of risk of bias on effect size in OCD pharmacotherapy. To our knowledge, this is the first meta-analysis to do so and we found that studies at high risk of bias are correlated with an increased effect size. This finding raises concerns regarding effect size inflation in pivotal OCD pharmacotherapy trials and emphasises the need for (novel) high-quality evidence on SSRI and clomipramine efficacy in OCD.

Limitations

Our meta-analysis does have some limitations. Although we did perform meta-regression for participant-level characteristics such as age, gender and illness severity, using aggregate data meta-analysis for these interactions is problematic, since results might be falsely positive due to ecological fallacy, in which treatment effects are confounded by a third, unknown factor.43 On the other hand, potential modifiers on the participant level might also be missed if studies did not show variation in aggregate measures. Specifically for inspection of patient-level modifiers, the designated research method would be an individual participant data meta-analysis (IPDMA), which to the best of our knowledge has not been performed yet, specifically for OCD pharmacotherapy.44 Furthermore, for the meta-regression, we relied on how the original manuscripts defined variables, which might vary significantly. Additionally, although we did search grey literature and found evidence for publication bias, we did not find any unpublished studies. This might be due to selective reporting by sponsors, where negative findings not only remain unpublished in full, but are not reported on any public domain. Finally, as we only included placebo-controlled trials, our results pertain to a clinical trial population which might decrease representativeness of real-world population, for instance, regarding comorbidity and treatment resistance.45

Clinical implications and recommendations for future research

Although our findings extend current literature by supporting evidence on the efficacy of OCD pharmacotherapy, the effect size is diminished after correcting for methodological issues and publication bias.33 46 Furthermore, the fact that most placebo-controlled trials for OCD, based on registered medications, fall short on contemporary quality standards is a concerning matter.

When interpreting these results, we should also differentiate between statistical and clinical relevance. It is questionable whether a mean active medication versus placebo separation of 4.2 points on the YBOCS and less for SSRIs amounts to a noticeable change in illness severity. This is unknown as no generally accepted ‘minimal important difference’ exists for OCD, describing the smallest clinically noticeable decrease in symptoms. However, these are group-level effects, and within-group effects may vary significantly.

The advantage in the efficacy of clomipramine over SSRIs persisted even after correcting for risk of bias. However, direct head-to-head trials have not shown a clear increase in the efficacy of clomipramine over SSRIs, and clomipramine is known to have a more severe side effect profile compared with SSRIs, especially anticholinergic, cardiac and metabolic side effects.34 As such, we are not able to recommend concrete changes in clinical practice based on this article. Yet it is noteworthy that clomipramine, the inaugural approved pharmacological treatment for OCD in 1989, may remain one of our most potent treatment options.

Given that study effects are modest, susceptible to publication bias and influenced by the risk of bias, a new RCT reassessing the efficacy of OCD pharmacotherapy would hold significant clinical and regulatory importance. This trial should be conducted in a representative population, adhere to the latest quality standards and prioritise social functioning and quality of life, alongside symptom reduction. As the included RCTs focused on general efficacy rather than on predictors of treatment success, future research on the effect of symptom dimensions, the presence of sensory phenomena or metacognition on pharmacological treatment outcome would be valuable. However, we acknowledge the financial burdens associated with conducting such an RCT and the ethical concerns surrounding the inclusion of a placebo arm, thereby potentially depriving participants of an established first-line treatment for OCD.20 In light of these considerations, undertaking an individual IPDMA of OCD RCTs to first assess patient-level characteristics predicting improved treatment response would be both economically and ethically more favourable. Such subgroup analyses could refine interventions, identifying specific patient populations with enhanced treatment success likelihoods. Consequently, insights from an IPDMA could inform targeted cohorts for upcoming drug trials.

In summary, our findings suggest a pronounced efficacy for clomipramine over SSRIs, even after adjusting for risk of bias. While pharmacotherapy remains a viable therapeutic option for OCD, it is essential to recognise that reported effect sizes may be attenuated when considering publication bias and stringent methodological standards.