Septoplasty: defining a desirable clinical outcome according to baseline symptom scores

Introduction

Outcomes of septoplasty are usually assessed by testing the statistical significance of symptom improvements from baseline (1–3). However, statistically significant improvements may be difficult to interpret clinically and are not necessarily clinically important (4, 5). One strategy for addressing this limitation of statistical significance is to establish the minimum improvement that is clinically important. This is called the “minimal clinically important difference” (MCID) and is often used to assess the clinical meaningfulness of septoplasty outcomes (6–8). In contrast, Ziai and Bonaparte (9) used the rating “surgical success” as clinically meaningful. They found that this rating corresponded to a 41.1% improvement in symptom scores from baseline. We have used a similar rating of “much improved” to distinguish these results from lower levels of improvement (5) and named this measure a “desirable clinically important difference” (DCID).

In septoplasty outcomes, the MCID and DCID have only been applied to whole cohorts. Patients with nasal obstruction have different baseline ratings and different grades of improvement after septoplasty. The outcomes of cohort studies are not helpful in assessing individual results. The aim of our study was therefore to assess whether the MCID and DCID differ according to baseline levels of nasal obstruction. We divided the cohort into subgroups according to baseline levels of nasal obstruction. The results from the study should help assess the improvement of obstruction ratings in individual patients and may also provide insights for preoperative planning of surgery for individual patients.

Material and methods

Patients undergoing endonasal septoplasty, with or without turbinate surgery, are included in the Septoplasty Quality Register (SQR) at the Department of Otorhinolaryngology at Lovisenberg Diaconal Hospital, Oslo, Norway. Written informed consent is not required for inclusion of patients in the SQR in accordance with national legislation and internal requirements. We have audited anonymous data from the SQR for this study. Patients respond to the Nasal Surgical Questionnaire (NSQ) (10) preoperatively and 6 months postoperatively as part of our quality assurance policy. Between April 2014 and September 2019, 1,260 patients were entered into the register. Of these, 325 patients (26%) did not return the second questionnaire, and 1 patient omitted some ratings. The remaining 934 patients were included in this analysis. The included patients were at least 17 years of age and did not have any other nasal or sinus disease except allergy.

The questionnaire contains separate visual analog scales (VASs) for nasal obstruction during the day and at night. Each VAS consists of a 10 cm line, with the left end representing no obstruction and the right end representing complete obstruction. The patients were asked to rate their sense of nasal obstruction on each scale with a vertical line. Each VAS score is measured in mm from the left-hand side of the scale, with a range of 0–100. Patients were asked to respond to the items based on how they felt on a normal day without any infection.

The questionnaire was also mailed to each patient 6 months postoperatively, along with a cover letter signed by a surgeon from the department and a prepaid return envelope. Three weeks later, a reminder containing the same questionnaire was mailed to those who had not returned the first questionnaire. The postoperative version of the questionnaire included an additional question about the patient's retrospective sense of change in nasal obstruction following surgery: “Is your breathing now completely, much, or somewhat improved, unchanged or worse?” These global outcome ratings were assigned the following status: status 1 = completely improved, status 2 = much improved, status 3 = somewhat improved, status 4 = unchanged, and status 5 = worse.

We used the anchor-based method to establish the clinically important difference as recommended by the US Food and Drug Administration (11). Of the different statistical strategies to establish the DCID, we chose the receiver operating characteristics (ROC) approach, which establishes the borderline, maximizing specificity and sensitivity (described in the Statistical analysis section).

The minimum improvement in the VAS score corresponding to the “much improved” rating was defined as the VAS score at the borderline between status 2 (much improved) and status 3 (somewhat improved). The DCID is calculated as the difference between this borderline and the baseline VAS score.

The study cohort was large enough to be divided into three subgroups according to the patients’ baseline VAS scores of nasal obstruction: subgroup 1 (moderate obstruction) had VAS scores of 0–70, subgroup 2 (severe obstruction) had VAS scores of 71–85, and subgroup 3 (very severe obstruction) had VAS scores of 86–100.

VAS ratings of nasal obstruction are higher at night than during the day (5). As the night values better reflect the global rating of improvement, we used the night ratings for comparison with the global ratings.

Statistical analysis

Statistical analyses were performed using IBM SPSS version 28.0 (IBM Corp, Armonk, NY, USA) and Stata statistical software (StataCorp, College Station, TX, USA). Descriptive statistics were expressed as numbers (%) and mean (SD) for patient demographics, VAS scores of nasal obstruction (both preoperative and postoperative), and outcome status groups. All of the analyses were two-sided, and statistical significance was defined as p < 0.05. Paired t-tests were used to evaluate the differences between preoperative and postoperative VAS scores for each status and nasal obstruction subgroup. p-values <0.05 were considered statistically significant.

The statistical plan was to determine the MCID and DCID for each baseline level of nasal obstruction. However, in calculating the MCID of the different subgroups, we found that the MCID value for the subgroup with “moderate” nasal obstruction was less than the statistical error of measurement (SEM) of our instrument. We were therefore unable to present results for the MCID. However, the results of the DCID did not have this issue and are therefore still helpful for assessing individual results.

The DCID for each baseline level of nasal obstruction was determined using ROC curves and the user-developed Stata command cutpt, which empirically estimates the cut point for a diagnostic test. By default, the cutpt uses the Liu method (12), which maximizes the product of the test's sensitivity and specificity. The Youden and nearest methods were also evaluated, but since they yielded similar results, we opted to use the default Liu method. Both the sensitivity and specificity estimates for the DCID of each baseline level of obstruction are reported. Relative DCIDs were calculated by dividing the absolute DCID by the baseline level of nasal obstruction to determine the percentage improvement needed for each obstruction severity subgroup.

Results

Nine hundred and thirty-four patients (663 men and 271 women) with a mean age of 37.7 years underwent endonasal septoplasty with or without turbinate surgery between 2014 and 2019 and completed both the preoperative and postoperative questionnaires.

The mean preoperative VAS score of the cohort was 75.1, with variations across baseline nasal obstruction subgroups: 55.5 for moderate, 78.7 for severe, and 94.0 for very severe (Table 1).

Table 1. Preoperative VAS scores, number of patients in each subgroup, and number of patients with status 1–5 (global rating).

The numbers of patients in each obstruction severity subgroup with each outcome status (1–5) are also listed in Table 1.

Table 2 presents the preoperative, postoperative and change in VAS scores according to obstruction severity subgroup and outcome status (2–3). For those with outcome status 2 (much improved), the mean improvement in VAS scores varied from 32.7 to 67.2 and increased with increasing baseline obstruction severity. The postoperative scores for each obstruction severity group differed only slightly but increased from 22.8 to 26.2 with increasing baseline nasal obstruction. For outcome status 3 (somewhat improved), the mean improvement in VAS scores increased from 14.0 to 34.1 with increasing baseline obstruction severity, leading to a postoperative result varying from 40.8 to 60.0.

Table 2. Preoperative, postoperative, and change in VAS scores in each subgroup for statuses 2 and 3.

Minimal clinically important difference

As described in the Statistical analysis section, the MCID value for the subgroup with “moderate” nasal obstruction was found to be less than the SEM of our instrument. We are therefore unable to present valid results for the MCID.

Desirable clinically important difference

The VAS scores at the border between outcome statuses 2 and 3 were used to establish the DCID. This was done with the ROC, which gives the specificity and sensitivity of the values. In Table 3, we present the baseline and borderline (between statuses 2 and 3) scores together with the DCID (change in the VAS score from baseline) and the relative DCID (percentage change from baseline). The borderline VAS score increased from 28.0 to 38.0 with increasing baseline obstruction scores, resulting in the DCID scores ranging from 27.5 to 56.0 according to the severity of baseline obstruction.

Table 3. Preoperative (baseline) VAS score, borderline VAS score between statuses 2 and 3, specificity and sensitivity of the borderline score, DCID, and relative DCID of each subgroup.

ROC curves showing the results according to sensitivity and specificity for each VAS obstruction subgroup (moderate, severe, and very severe) are presented in Figure 1. The relative DCIDs varied from 49.6% to 61.3% with increasing severity of baseline obstruction.

Figure 1. ROC curves of DCID for the VAS obstruction subgroups. Ratings are plotted according to sensitivity and specificity: (a) moderate (0–70), (b) severe (71–85), and (c) very severe (86–100).

Discussion

The DCID varied across the three subgroups of baseline nasal obstruction severity. The DCID after septoplasty increased with increasing baseline ratings of nasal obstruction. The relative DCID also increased according to baseline obstruction severity, from 49.6% for moderate, 56.8% to severe, to 61.3% for very severe obstruction. Unfortunately, results could not be presented for the MCID due to the SEM of our instrument exceeding the MCID for one of the nasal obstruction subgroups. This measurement limitation highlights the importance of accuracy in symptom assessment. Nonetheless, as the MCID represents the “minimal” level of improvement considered clinically important, the larger DCID is likely a more useful standard for determining “surgical success” from the patient's perspective.

Several studies have evaluated septoplasty outcomes based on baseline ratings of obstruction. Hong et al. (13) in 2015 studied surgical outcomes after 3 months in 49 patients who underwent septoplasty and/or turbinoplasty using the nasal obstruction symptom evaluation (NOSE) scale. They concluded that “the baseline NOSE score was the sole factor that represented the subjective septoplasty outcome. Patients with more severe preoperative nasal obstruction may achieve a higher degree of satisfaction after septoplasty.”

Alakärppä et al. (14) in 2018 studied the result of septoplasty after 12 months in 76 patients using the Sinonasal Outcome Test-22 (SNOT-22). They found that the odds ratio of having a beneficial result was 10 when comparing patients with a baseline SNOT-22 score of 20 or more to those with lower scores. A baseline score of 30 was associated with the best postoperative improvement.

Pedersen et al. (15) in 2019 studied the results of septoplasty after 12 months in 888 patients. They subdivided the cohort into four categories according to the following baseline grading: none, mild, moderate, and severe obstruction. They defined postoperative improvement as a change in the grading of at least one step. The percentage of patients that experienced improvement increased according to the severity of the baseline rating: 31% among those with mild, 57% among those with moderate, and 81% among those with severe obstruction.

These three studies (13–15) are consistent with ours in finding that the higher the baseline ratings of nasal obstruction, the larger the improvement after septoplasty. As all of these studies used different instruments to rate obstruction severity, this finding seems independent of the instrument used. The main difference between these prior studies and ours was that the prior studies primarily used baseline obstruction severity as a predictor of septoplasty outcome and did not report the degree of improvement observed in each severity subgroup. In contrast, our study was primarily intended to serve as a guideline for assessing individual septoplasty results.

Weaknesses

The primary weakness of this study was that it was based entirely on subjective data. Although patient-reported outcomes are important, we have no objective data to support our findings. An additional weakness is that the instrument we used to assess nasal symptoms had a degree of measurement error that prevented us from presenting valid MCID results for each level of nasal obstruction. Future studies using a different instrument may yield better results. The patients were asked to respond as on a normal day to avoid a temporary illness influencing the assessment. However, as this study aimed to compare VAS and global ratings, we did not account for the possible effects of smoking, allergy, use of medication, or quality of surgery on the ratings. The VAS scores were recorded prospectively, while the global outcome rating was done retrospectively. Retrospective ratings seem to correlate better to postoperative than to improvement scores. However, we previously found there to be a good correlation between the change in VAS scores and global ratings (5), so we believe that recall bias did not substantially influence the results.

Conclusion

For assessing septoplasty results, it is important to find the level of improvement that the patient considers a surgical success. We have called this level the DCID. The DCID and the relative DCID increased with increasing baseline obstruction scores. Percentages of improvement as provided by the relative DCID can be used as guidelines to assess individual results of septoplasty. They can also be used in planning surgery for individual patients. Future studies are needed to determine the MCID for different levels of baseline nasal obstruction, as this study was unable to estimate this lower standard of clinically important symptom improvement.

Data availability statement

The original contributions presented in the study are included in the article, further inquiries can be directed to the corresponding author.

Ethics statement

This study is an audit of the Septoplasty Quality Register. It has been approved by the Ethics Committee of Lovisenberg Diaconal Hospital, Oslo, Norway in accordance with national legislation and institutional requitements. Written informed consent is not required for inclusion of patients in the Septoplasty Qality Register (SQR) in accordance with national legislation and internal requirements.

Author contributions

RH: Writing – original draft. LD: Writing – review & editing. MT: Writing – review & editing. CG: Writing – review & editing. AP: Writing – review & editing. OS: Writing – review & editing.

Funding

The author(s) declare that no financial support was received for the research, authorship, and/or publication of this article.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

1. Rhee JS, Sullivan CD, Frank DO, Kimbell JS, Garcia GJM. A systematic review of patient reported nasal obstruction scores: defining normative and symptomatic ranges in surgical patients. JAMA Facial Plast Surg. (2014) 16(3):219–32. doi: 10.1001/jamafacial.2013.2473

PubMed Abstract | Crossref Full Text | Google Scholar

2. Alessandri-Bonetti M, Constantino A, Cottone G, Carbonaro R, Cardone F, Amendola F, et al. Efficacy of septoplasty in patients with nasal obstruction: a systematic review and meta-analysis. Laryngoscope. (2023) 133(12):3237–46. doi: 10.1002/lary.30684

PubMed Abstract | Crossref Full Text | Google Scholar

3. Fearington FW, Awadallah AS, Grant BS, Hamilton S, Olson MD, Dey JK. Long-Term outcomes of septoplasty with or without turbinoplasty: a systematic review. Laryngoscope. (2024) 134(6):2525–37. doi: 10.1002/lary.31193

PubMed Abstract | Crossref Full Text | Google Scholar

4. Mouehli Y, Jouve E, Castelli C, Gentile S. How is the minimal clinically important difference established in health related quality of life instruments? Review of anchors and methods. Health Qual Life Outcomes. (2020) 18(1):136. doi: 10.1186/s12955-020-01344-w

PubMed Abstract | Crossref Full Text | Google Scholar

5. Haye R, Døsen LK, TarAngen M, Gay C, Pripp AH, Shiryaeva O. Clinically important estimates of improvement after septoplasty. J Laryngol Otol. (2023) 137(11):1285–8. doi: 10.1017/S0022215123000993

PubMed Abstract | Crossref Full Text | Google Scholar

6. Stewart MG, Smith TL, Weaver EM, Witsell DL, Yueh B, Hannley MT, et al. Outcomes after nasal septoplasty: results from the Nasal Obstruction Septoplasty Effectiveness (NOSE) study. Otolaryngol Head Neck Surg. (2004) 130(3):283–90. doi: 10.1016/j.otohns.2003.12.004