Introduction

Disease activity in rheumatoid arthritis (RA) was initially defined by a number of core set variables, agreed on by the American College of Rheumatology (ACR) and EULAR in the 1990s.1 2 These variables comprised tender joint count (TJC) and swollen joint count (SJC), patient assessment of global disease activity (PtGA) and of pain, evaluator/physician global assessment (EGA), a measure of function such as the Health Assessment Questionnaire (HAQ) and an acute-phase reactant such as C reactive protein (CRP) level.

At the time of defining the core set variables, remission was more aspirational than a realistic goal.3 Today, however, remission can be obtained in a sizeable portion of patients and is seen as a major therapeutic target.4–6 A clinical definition of remission for RA should reflect no or only minimal disease activity, and patients attaining this state should have a low risk of both structural progression and functional impairment.6

ACR and EULAR endorsed provisional remission criteria over 10 years ago.7 Their publication served the purpose of providing a common definition for this prime treatment target.8 Two types of remission definitions were agreed on by the ACR/EULAR committee after extensive data analyses and consensus-based deliberations. The Boolean definition required that, to attain remission, each of 4 core set variables (TJC, SJC, PtGA, CRP) must have a value of ≤1. (PtGA is scored on a 0–10 points or 0–10 cm scale, CRP in mg/dL). The index-based definition used the remission cut-off point of the Simplified Disease Activity Index (SDAI).9 The committee also endorsed remission criteria that did not include CRP level, namely a Boolean definition that comprised SJC, TJC and PtGA and an index definition based on the remission threshold of the Clinical Disease Activity Index (CDAI).10

Since their publication, arguments have been made claiming that remission definitions may, on the one hand, be too stringent, with the risk of overtreatment if used as treatment targets, or, on the other hand, too lenient, proposing addition of imaging confirmation of remission. A particular matter of debate was the requirement of achieving a PtGA score of ≤1; the stringent threshold for the PtGA has been criticised, because some patients do not achieve it despite the absence of tender and swollen joints and an elevated CRP level.11 Moreover, the agreement between the Boolean and index definitions was only moderate, primarily due to the PtGA threshold.12 However, the PtGA is the core set measure most sensitive to change in RA trials,1 13–15 best differentiating between patients receiving active treatment and those receiving placebo. Thus, PtGA is an important measure of disease activity. Consequently, the PtGA was included in the ACR core set, composite activity scores and remission definitions. However, PtGA may also be influenced by other factors related to RA. For example, patients with pain from irreversible joint damage may have elevations in PtGA even if their RA is in clinical remission.16 17

To circumvent the strictness of the 1.0 rule for PtGA and to increase the agreement with SDAI-defined remission, a higher PtGA threshold has been proposed.18 19 Furthermore, since the index-based criteria can be used instead of Boolean criteria, both criteria should identify the same patients as having disease in remission. However, remission rates based on SDAI are higher than those using the Boolean criteria, because summing several components permits one component, such as the PtGA, to be slightly elevated if compensated by a lower score in others.20 A study evaluating alternative Boolean definitions of remission, with PtGA thresholds ranging 1.0–2.5, found that using a threshold of 2 cm (Boolean2.0) led to a higher agreement with the index-based definition without jeopardising the strong association between remission and subsequent good functional and radiographic outcomes, a key criterion in the development of the provisional definition of remission.12 The purpose of the present study is to externally validate the performance characteristics of this revision of the Boolean criteria12 and provide external validation of the provisionally endorsed SDAI and CDAI remission definitions. This provides the evidence base for ACR and EULAR to fully endorse the remission criteria, changing their status from the current ‘provisional’ to a ‘definite’ status.

Patients and methodsPatients

RA patient data were retrieved from four clinical trials testing the efficacy of biological disease-modifying antirheumatic drugs (bDMARDs) against placebo or placebo with methotrexate (MTX), with an available observation period between 1 and 2 years. The GO-AFTER trial tested golimumab as an active compound, the FUNCTION and LITHE trials tested tocilizumab, and the SERENE trial tested rituximab. GO-AFTER evaluated patients who were insufficient responders to TNF inhibitors (TNFi), LITHE and SERENE included patients with an insufficient response to MTX, and FUNCTION included MTX-naive patients with early RA. Results and detailed patient characteristics of the individual trials have been previously reported.21–24 These trials included patients with RA with varying disease durations and treatment histories. In all four trials, the PtGA was evaluated using a 100 mm Visual Analogue Scale (VAS).

Definitions of remission and their modifications

The Boolean definition includes SJC, TJC, PtGA (cm) and CRP levels (mg/dL); for a patient to meet remission criteria, all components must have scores of ≤1 (in the case of a 100 mm VAS, this translates to a score of ≤10). A version without CRP was also approved by the ACR/EULAR committee (three-variable Boolean (3vBoolean)). The SDAI-based definition of remission sums the scores for the components used in the Boolean definition in addition to EGA, and patients meet criteria if the score is ≤3.3. The CDAI-based remission definition consists of the same components, excluding CRP level and remission is fulfilled at a score of ≤2.8.7

Similar to a previous study,12 we increased the threshold of the PtGA criterion by steps of 0.5 cm from 1 cm up to 2.5 cm, and labelled these as Boolean1.0, Boolean1.5, Boolean2.0 and Boolean2.5. The Boolean definition that does not include the PtGA criterion was labelled as BooleanX; in this definition, only CRP, TJC and SJC needed a score of ≤1 to attain remission, regardless of PtGA value.25

Statistical analysis

We performed descriptive analyses and tested the revised Boolean2.0 criteria against the provisional Boolean1.0 criteria for convergent and predictive validity. Finally, we investigated the impact of the exclusion of the PtGA from the definition of remission (BooleanX). Analyses were performed on 6-month and 12-month data using SPSS Statistics V.25 and Stata V.15. An experienced patient research partner (MdW) was involved throughout the study. He took part in all meetings, reviewed data at different time points and provided written as well as oral feedback. His contribution focused on a critical review of the PtGA as part of the RA definition of remission.

Descriptive analysis

We analysed how the rates of remission at 6 and 12 months after treatment initiation in the trials were affected by the different modifications described above. For the Boolean modifications, we also studied which components prevented achievement of full remission by identifying participants who fulfilled three of four required criteria but not all four of them.11

Convergent validity

We tested the agreement of different Boolean criteria with the index-based remission definitions. We cross-tabulated remission fulfilment for Boolean remission versions with the SDAI and CDAI definitions and analysed their agreement using McNemar’s test for agreement with kappa statistics. In addition, the well-established concordance between SDAI-defined and CDAI-defined remission was tested to confirm the interchangeability of these definitions.

We examined the optimal PtGA threshold to achieve concordance with SDAI-defined remission by carrying out classification and regression trees (CART) analyses (R rpart package; https://cran.r-project.org/web/packages/rpart/index.html), in which, after assuming that CRP, TJC and SJC were all in remission (BooleanX), we asked what threshold of PtGA would provide the best prediction of SDAI-defined remission.

Predictive validity

As a next step, we explored the impact of using the modified Boolean-based and index-based remission definitions assessed at 6 months after treatment initiation on outcomes at 1 year. Differences in mean radiographic progression (based on the change in modified total Sharp and van der Heijde score (mTSS) between baseline and 1 year) and the proportions of patients without progression (change in score ≤0) and with good function at 1 year (HAQ≤0.5) were assessed. Attaining an HAQ of≤0.5 without radiographic progression at 1 year of treatment was defined as a good combined outcome, similar to the procedure used to develop the provisional ACR/EULAR remission definition.7 These analyses were repeated separately for early and late RA participants. Positive and negative likelihood ratios (LRs) were calculated separately for each remission definition to assess predictive validity for good functional and structural outcomes.

Impact of PtGA score and PtGA exclusion from the remission definition

In addition to the comparison of Boolean2.0 to Boolean 1.0, we analysed the effect of excluding PtGA from remission criteria (BooleanX) in the context of each of the above analyses.

ResultsPatients, remission rates and components limiting achievement of remission

Data from 2048 clinical trial participants, 1101 with early RA (mean±SD disease duration 0.8±0.5 years) and 947 with established RA (mean±SD disease duration 7.1±5.4 years) were included. As expected, using Boolean2.0 yielded higher remission rates compared with Boolean1.0 at 6 months: 20.6% (n=227) compared with 14.8% (n=163) in early RA; 6.0% (n=57) vs 4.2% (n=40) in established RA (figure 1). These correspond to a relative increase in remission rates of 39% and 42%, in early and established RA, respectively. This trend was consistent at 1 year, although remission rates were generally higher (online supplemental figure 1). Omitting the PtGA criterion using the BooleanX definition further increased remission rates over Boolean2.0, in early RA (from 227 patients (20.6%) to 297 patients (27%)) and in established RA (from 57 patients (6%) to 95 patients (10%) patients at 6 months), with relative increases of 31% and 66%, respectively.

Figure 1

Rates of rheumatoid arthritis (RA) disease remission according to modified Boolean classifications, using a patient global assessment (PtGA) threshold of 1.0 (‘Boolean’), 1.5, 2.0, 2.5 or omitting the PtGA completely (BooleanX), as well as according to the Simplified Disease Activity Index (SDAI), Clinical Disease Activity Index (CDAI) and Disease Activity Score in 28 joints (DAS28) definitions. Rates at 6 months are shown for patients with early RA and those with established RA.

Within the total study population, 311 participants (15.2%) achieved ‘near misses’ of Boolean remission, meaning that they fulfilled 3 of the 4 criteria. In 60% of these participants, this was due to not meeting the criterion of PtGA ≤1 cm. By using Boolean2.0, this proportion was reduced to 47% of all near misses. Consequently, among all participants, 14% were classified as having Boolean2.0-defined remission, 5% missed achieving remission only because of the PtGA criterion and 3% missed achieving remission only because of the SJC criterion (online supplemental figure 2).

Convergent validity

Increasing the PtGA cut-off from 1.0 to 2.0 cm for participants with early RA yielded higher concordance rates between Boolean-defined and SDAI-defined criteria for remission. This led to more participants contemporaneously fulfilling the SDAI and respective Boolean remission definition (increase from 71% to 92% of participants when using Boolean1.0 vs Boolean2.0) (table 1). Rates of concordantly classified participants with respect to remission increased from 93.4% to 95.9% at 6 months. A similar increase in concordantly classified participants was observed for the agreement between the corresponding Boolean and CDAI definitions (online supplemental table 1). In patients with established RA, the percentage classified as having disease in remission by Boolean in addition to CDAI or SDAI definitions likewise increased from 74% to 94% for SDAI and from 70% to 83% for CDAI, when using Boolean1.0 vs Boolean 2.0; and from 78% to 96% when using 3vBoolean1.0 versus 3vBoolean2.0 to assess agreement with CDAI. The proportion of participants concordantly classified as having disease in remission remained similar in established RA.

Table 1

Agreement rates between different modified Boolean remission definitions and the SDAI remission definition in RA patients at 6 months

Kappa analyses showed higher agreement between SDAI-defined remission and Boolean2.0-defined than with Boolean1.0-defined remission at 6 months (figure 2). The 12-month data showed similar results and are depicted in online supplemental figure 3. Kappa estimates and 95% CIs of agreement with SDAI-defined and CDAI-defined remission at 6 months increased when using Boolean2.0 compared with Boolean1.0 definitions (0.86 (95% CI 0.83–0.89) vs 0.77 (95% CI 0.74–0.82) for SDAI at 6 months and 0.81 (95% CI 0.77–0.84) vs 0.76 (95% CI 0.72–0.81) for CDAI at 6 months) (kappa curves for CDAI are shown in online supplemental figure 4).

Figure 2

Kappa values and 95% CIs representing agreement between modified Boolean remission definitions and SDAI-defined remission, for patients with early RA (red line), those with established RA (green line) and all RA patients (blue line) at 6 months. Kappa estimates and 95% CIs are provided in the accompanying table. See figure 1 for definitions. RA, rheumatoid arthritis; SDAI, Simplified Disease Activity Index.

A further increase in the PtGA threshold beyond 2 cm led to a decrease in concordance. Reduced concordance was particularly seen when omitting the PtGA (BooleanX) both in terms of percentage agreement and according to kappa estimates (table 1 and figure 2).

Additionally, CART analyses confirmed the percent agreement and kappa results: in participants with SJC, TJC and CRP values of ≤1, and PtGA values of ≤2.3 cm at 6 months and ≤1.8 cm at 12 months showed the highest likelihood of concurrent SDAI-defined remission. The same analyses stratified by early or established RA yielded a PtGA threshold value of ≤2.3 cm in early RA and ≤1.4 cm in established RA at 6 months (≤1.5 cm in early RA and≤1.9 cm in established RA at 12 months). Generally, all agreement estimates point to 2.0 cm as the optimal threshold.

Predictive validity

We studied rates of participants achieving a good functional outcome (HAQ≤0.5) and no radiographic progression (ΔmTSS) at 1 year for participants classified by the different Boolean definitions at 6 months.

Similar results were found for Boolean2.0 and index-based definitions when predicting good functional outcome. HAQ scores at 12 months were as follows: mean±SD 0.24±0.40 for Boolean1.0, 0.31±0.45 for Boolean2.0, 0.41±0.53 for BooleanX, 0.27±0.42 for SDAI and 0.26±0.42 for CDAI. Fewer participants scored an HAQ of ≤0.5 when the PtGA was omitted (70% in BooleanX vs 78% in Boolean2.0) (table 2). Increasing the PtGA threshold for Boolean-based remission was associated with a linear increase in HAQ scores. While there was a drop in positive LR from 6.1 to 4.4 when using the Boolean2.0, this was similar to the positive LR predicting a good functional outcome for SDAI-based and CDAI-based remission, which ranged from 4.3 to 4.9.

Table 2

Rates and LRs of RA patients achieving a good functional outcome (HAQ ≤0.5) and/or no radiographic progression (ΔmTSS) at 1 year, according to different remission criteria*

Table 2 outlines the similarity of LRs for predicting lack of radiographic progression during the first year when the different remission definitions were fulfilled at 6 months of treatment. The radiographic outcomes were similar regardless of the PtGA threshold or whether PtGA was included in the Boolean criteria, and scores were similar between different definitions (mean±SD ΔmTSS 0.29±2.08 for Boolean1.0, 0.25±1.81 for Boolean2.0, 0.21±1.9 for BooleanX, 0.27±1.86 for SDAI and 0.27±1.9 for CDAI). This observation is consistent with previous findings that PtGA is not associated with radiographic progression.12 26 Using the different Boolean definitions and index-based definitions led to similar proportions of participants with disease in remission who had radiographic progression (defined as ΔmTSS >0) during the first year (29.6% for Boolean1.0, 28.5% for Boolean2.0, 28.6% for BooleanX, 28.2% for SDAI and 28.6% for CDAI).

The proportion of participants achieving both good radiographic and functional outcomes were similar for all remission definitions, from 57% to 60% (58.6% for Boolean1.0, 57.3% for Boolean 2.0, 59.2% for SDAI and 60.4% for CDAI), except for BooleanX (50.8%). Again, index-based remission definitions performed similarly to Boolean1.0 and Boolean2.0 definitions with respect to their predictive ability (positive LR between 3.8 and 4.3). This pattern could also be seen when analysing data on early RA and established RA separately (table 2). However, good functional outcomes when using BooleanX were even less frequent in established RA compared with early RA (HAQ ≤0.5 in established RA was 57% compared with 74% in early RA). Of note, no differences in radiographic progression in patients with established RA were observed between the remission definitions fulfilled. Overall, more than two-thirds of patients with established RA showed radiographic progression throughout the first year.

Discussion

This study provides evidence of external validation of the previously proposed modification of the Boolean ACR/EULAR remission criteria, to include a threshold of 2 cm rather than 1 cm for the PtGA criterion, and of the provisionally endorsed index-based remission definitions. The study was performed using independent clinical trial data sets not included in any of the previous studies (eg, the data sets generating the provisional definition of remission and the recent analyses on raising the PtGA threshold (7, 12) in which the revised threshold was derived). Our study assessed different aspects of validity for the revised definition of remission. The composition of this patient population was heterogeneous in terms of disease duration and previous DMARD treatments, and therefore, our results are applicable to a broad spectrum of patients with RA.21–24

The remission validation outlined here builds on work done 10 years ago when the selection of components was undertaken by a large ACR/EULAR consortium.7 Due to criticism around the stringently low threshold of the PtGA component within the Boolean remission definition11 25 27 and concerns that the two approaches (Boolean-based vs index-based) to remission were not concordant, alternative thresholds for the PtGA were explored using multiple clinical trial data sets.12 Our analyses support the notion of a slight increase of the PtGA threshold since it provides better agreement with the SDAI remission definition and higher rates of Boolean-defined remission, without jeopardising the prediction of good long-term functional and radiographic outcomes.

Our results replicate previous findings that a Boolean definition using 2 cm as threshold for PtGA (Boolean2.0) yields better agreement with both index-based remission definitions than Boolean1.0.12 Furthermore, patients who attain Boolean2.0, CDAI and SDAI remission thresholds at 6 months have a higher likelihood of good functional and radiographic outcomes after 12 months of treatment than those attaining Boolean-based disease remission without PtGA (BooleanX). We have also shown the agreement between the three-variable Boolean approach definition and the CDAI definition, which can be applied during a clinic visit, without knowledge of current acute-phase reactant levels.

The PtGA threshold within the remission criteria does not influence the prediction of radiographic non-progression, as all tested definitions yielded the same positive LRs for nonprogression of ~1.7 and the same proportions of patients not progressing (~79%). This is consistent with findings from a recent meta-analyses including data from 11 clinical trials showing that people fulfilling the SJC, TJC and CRP criteria but not the PtGA criterion demonstrate better radiographic outcomes than those not in any Boolean remission category.26 We note that successful management of RA is not only defined by the prevention of joint damage, but, ideally, attaining remission should also prevent residual symptoms that matter to patients, such as pain, fatigue and anxiety.

The PtGA has not only been criticised for its stringent threshold in the remission definition. The Outcome Measures in Rheumatology (OMERACT) Working Group focusing on ‘Remission in RA: Patient Perspective’ questioned whether the PtGA is the best instrument to reflect the perspective of patients in the current Boolean remission definition. They explored the effect of replacing PtGA with three patient-assessed domains identified by patients as most important: pain, fatigue and independence. Their search for a better incorporation of the patient perspective has not yet resulted in a promising set of validated patient-reported outcome measures that can replace the PtGA. In their most recent working group report, they concluded that there is currently insufficient evidence to propose a change to the existing ACR/EULAR remission criteria.28 This report also discussed the concept of a ‘dual-target’ approach, trying to decouple the assessment of disease activity from disease impact in defining remission.25 29 At this stage, no data are available about the effectiveness and feasibility of such a dual-target approach.

Concerns have been expressed that the ACR/EULAR remission criteria allow few patients to achieve disease remission. Within our validation work, we additionally provide data on the shift in remission frequencies and the distribution of patients that miss Boolean-defined remission due to fulfilling only 3 of 4 criteria. By using a threshold of 2 cm rather than 1 cm in the revised Boolean definition, 40% more participants in our data sets achieved disease remission (14% instead of 10%). Importantly, when applying the Boolean2.0 definition, the SJC criterion threshold of 1 seems to be nearly as prominent in limiting participants attaining full remission as the PtGA criterion (3% due to high SJC and 5% to high PtGA when fulfilling the other three criteria). The revised PtGA threshold of 2 cm has been proposed as one item in a set of seven criteria that defined minimal disease activity of RA by OMERACT in 2005.30 Notably, the definition of remission should remain strict and ensure beneficial long-term outcomes for patients with RA and prevent unnecessary treatment escalation at the same time. Furthermore, it appears that changes in the overall approach to treating RA before patients enter clinical trials or trends over time have led to much higher provisional ACR/EULAR remission rates in more recent clinical trials than in earlier ones, with recent rates reaching ~30% in early disease, 20% in patients with insufficient response to MTX,and 15%–20% in patients with insufficient response to bDMARDs.31–35

A preferable approach for more patients to achieve remission is to foster a collaborative relationship between patients and clinicians, to initiate treatment early, and to use a treat-to-target approach,8 rather than omitting potentially problematic items such as the PtGA.36 37 Studies have shown that a treat-to-target approach is not yet fully implemented in clinical practice; in one-third of instances where treatment was not increased, this was influenced by factors unrelated to RA and in another third it was the patient’s preference to continue receiving the current treatment.38 39 All measurements and their interpretations need, in any case, to be complemented by the discussion between the patient and rheumatology clinician to reflect and decide on the appropriate steps in a shared decision.40 41

Remission has become a key target for the management of patients with RA.42 The ACR/EULAR 2011 initiative on remission criteria was undertaken to harmonise the definition of the term ‘remission’ and thus to facilitate the fair assessment and comparison of remission rates in clinical trials and clinical practice (eg, for different healthcare settings or providers). It will be helpful to further study the performance of the revised criteria in trials using other antirheumatic drugs, such as JAK inhibitors, and in other countries and ethnic groups, since RA severity and the interpretation of the PtGA may vary across ethnicities. We validated the results of the performance of the Boolean2.0 and the provisionally endorsed index-based remission definitions. With the validation of the threshold of 2 cm for the PtGA, we propose that these revised ACR/EULAR remission criteria be adopted both for future clinical trials and as a target in clinical practice.

Ethics statementsPatient consent for publication

Not applicable.

Acknowledgments

We would like to thank the companies that kindly provided the patient-level data from their trials (Hoffmann-La Roche, Janssen Biotech).