The 168 studies in the final evaluation sample were conducted in a total of 35 countries (~ 30% of studies conducted in the United States). Almost half (78/168) of studies were retrospective, i.e. medical chart reviews. As such, in terms of study quality, 90% of articles stated the research question, but the majority of articles did not include a sample size justification (93%), a control group (92%), or any masking of the treatment effect (76%). Further, only 46% of articles reported on efficacy of a standardized treatment regimen. Further detail on study characteristics can be found in Supplemental Materials.

Patients included in the studies comprised all subtypes, with craniofacial scleroderma and generalized morphea being the most commonly reported (Table 2). The majority of treatments evaluated in the studies were systemic medications, followed by phototherapy/UV therapy, topical creams, or reconstructive surgery (Table 2). More than half of studies (56%) reported on multiple treatments.

The most common outcome reported across all articles was “disease activity” (111/168), although the operationalization of this term varied across studies (Table 3). Other outcomes names by authors to determine treatment efficacy included “disease damage”, “skin thickness”, “lesion size”, “surgical outcomes”, and “clinician satisfaction”. Outcomes utilized to evaluate other aspects of treatment from the patient perspective commonly included “health-related quality of life” and “patient satisfaction/success” (Table 3).

The most common way treatment outcomes were measured were via clinical assessments that were both informal and formal/standardized (59%) along with other measures of health (40%; blood tests, imaging; Table 1). Patient- or observer-reported measures (like those completed by a parent) were used more infrequently (26%).

Clinical assessment of treatment outcomes. Almost 59% percent of articles (99/168) included some type of clinical assessment of treatment benefit that was commonly conceptualized as improvement, progression, and/or change in disease status (e.g. activity or damage). Disease progression typically included the appearance of new lesions or the expansion of pre-existing lesions over a specific period of time. Disease improvement typically was defined as lesions getting smaller or disappearing. Change in disease status was often conceptualized as a “halt” in disease activity or progression.

The clinical assessments within this category included clinical evaluations that both did and did not explicitly utilize a COA. Some clinical assessments were described as being less formal in nature and did not utilize specific criteria or time frame when reporting on the clinical evaluation.

Other clinical assessments for disease activity were more formal and did include specific criteria that was utilized to categorize patients. For example, one study [14] evaluated defined clinical improvement in activity as the absence of the criteria suggested by Careta and Romiti: appearance of new lesions in the last 3 months, expansion of the pre-existing lesion in the last 3 months, moderate or severe erythema or skin lesions with erythematous borders, violaceous lesion or lesion border, increased induration of the lesion border, and worsening of the hair loss on the scalp, eyebrows or eyelashes. Other clinical assessments described in the articles included the evaluation of surgical outcomes or general clinician satisfaction/success by assessing facial symmetry, complications, and need for surgical revisions.

Clinician-reported outcome measures. The most frequently named ClinRO(s) across studies was the Localized Scleroderma Assessment Tool (LoSCAT) [15, 16] and/or its components (18.5%; 31/168), the modified Localized Skin Severity Index (mLoSSI), which measures disease activity, and the Localized Scleroderma Damage Index (LoSDI), which measures disease damage. All studies that utilized the LoSCAT or its components cited validity and/or reliability information in the article, including general PRO measure development, content validity, reliability, and responsiveness. The LoSCAT was most often used in association with disease activity.

Physician-global assessments (PGAs) [15, 16] were utilized in 10.7% of studies (18/168), and often together with the LoSCAT, focused on disease activity. Other named ClinROs included skin scores (7.1%; 12/168), such as the modified Rodnan Skin Score (mRSS), most often utilized to assess skin thickness, and other clinical activity scores (e.g. ‘CAS’ or LS Cutaneous Activity Measure; 1.8%; 3/168). Other named ClinROs that were utilized infrequently were the Quantitative facial symmetry score and the Derriford Appearance Scale (both n = 1) which were linked to surgical outcomes.

Patient-reported/Observer-reported outcome measures. Patient-reported outcomes (PROs) and/or observer-reported outcomes (ObsROs; typically a child’s caregiver) were included in about 26% of the manuscripts (43/168). PROs were most often used to evaluate treatment outcomes described as ‘health-related quality of life’ (HRQOL) and ‘patient satisfaction/success’. Some of the PROs/ObsROs utilized to evaluate HRQOL were listed as the World Health Organization Quality of Life (WHOQOL-110) [17], the Children’s Dermatology Life Quality Index (CDLQI), [18] the Scleroderma Health Assessment Questionnaire (SHAQ), [19] Child Health Assessment Questionnaire (CHAQ), [20] Pediatric Quality of Life (family impact, generic, rheumatology module), [21, 22] and visual analog scales (VAS) for symptoms.

Many of the patient satisfaction outcomes were linked to patient satisfaction with surgery and did not utilize specific PROs, but instead noted if the patient was satisfied. For example, one study had patients rate their satisfaction using [1] very good/very satisfied, [2] good/satisfied, [3] bad/not satisfied, and [4] very bad/very dissatisfied [23]. Some studies used unnamed ObsROs and PROs to assess if the patient (or caregiver) noticed an improvement, no change, or worsening of the disease.

Performance-based outcome measures. Performance-based outcome measures (PerfOs) were infrequently utilized to evaluate treatment benefit (5.4%; 9/168), and when included, typically were related to measuring joint mobility/range of motion before and after treatment.

Other measures of health were included in about 40% of articles (67/168). The most frequent external equipment used to assess outcomes were magnetic resonance imaging (MRI), ultrasound, thermography, and photographs. In one study [24], MRI was used before and after 6 months of treatment to evaluate the depth and thickness of the soft-tissue structures and the degree of inflammation and edema. The MRI images were graded from 0 to 10 by an experienced and blinded radiologist. The MRI scores were also compared and scored from 0 (no improvement) to 10 (total healing) by investigators. These outcome measures were often used as a supplement to other COAs. Additionally, authors commonly reported using photographs when evaluating surgical outcomes.