Primary ciliary dyskinesia (PCD) is a genetic respiratory disease characterized by chronic cough, recurrent respiratory infections, and rhinosinusitis. The measurement of nasal nitric oxide (nNO) against resistance has been suggested as a sensitive screening method. However, current recommendations argue for the use of expensive, chemiluminescence devices to measure nNO. This study aimed to compare nNO measurement using three different devices in distinguishing PCD patients from healthy controls and cystic fibrosis (CF) patients and to evaluate their diagnostic precision. The study included 16 controls, 16 PCD patients, and 12 CF patients matched for age and sex. nNO measurements were performed using a chemiluminescence device (Eco Medics CLD 88sp), and two devices based on electrochemical sensors (Medisoft FeNO+ and NIOX Vero) following standardized guidelines. Correlation estimation, Bland–Altman, ROC curve, and one-way ANOVA were used to assess device differences and diagnostic performance. Significantly lower nNO output values were observed in PCD and CF patients compared to controls during exhalation against resistance. The correlation analysis showed high agreement among the three devices. ROC curve analysis demonstrated 100% sensitivity and specificity at different cut-off values for all devices in distinguishing PCD patients from controls (optimal cut-offs: EcoMedics 73, Medisoft 92 and NIOX 87 (nl min−1 )). Higher nNO output values were obtained with the Medisoft and NIOX devices as compared to the EcoMedics device, with a bias of−19 nl min−1 (95% CI: −73–35) and −21 nl min−1 (−73–31) accordingly. These findings indicate that all three tested devices can potentially serve as diagnostic tools for PCD if device specific cut-off values are used. This last-mentioned aspect warrants further studies and consideration in defining optimal cut-offs for individual device.

Primary ciliary dyskinesia (PCD) is a genetic respiratory disease that causes upper and lower respiratory symptoms, including chronic wet cough, recurrent respiratory infections, chronic rhinitis and sinusitis.

Diagnosis of PCD typically involves a combination of tests, including measurement of nasal nitric oxide (nNO) levels, analysis of ciliary beat frequency and pattern by high-speed video-microscopy analysis (HSVMA), transmission electron microscopy (TEM), genotyping and immunofluorescence [1]. The benefit of using nNO for PCD screening is that it is non-invasive, fast, and inexpensive compared to other methods. It also has high sensitivity and specificity [2]. Both European Respiratory Society (ERS) and American Thoracic Society (ATS) task forces have concluded that there is no stand-alone test or combination of tests that can exclude PCD with 100% certainty and agree that there is a place for nNO testing in PCD diagnosis as it is quick and simple to perform and has good sensitivity and specificity when conducted according to standardized protocols [3]. There are several hypotheses why nasal nitric nNO levels are low in PCD. Increased breakdown, reduced biosynthesis, reduced production and trapping of NO in the paranasal sinuses are all plausible and have thoroughly been discussed elsewhere [4]. The ATS guideline states that PCD diagnosis can be confirmed with two low nNO values, whereas the ERS guideline states that low nNO only in combination with abnormal HSVMA makes the diagnosis highly likely (but not confirmed) [3]. Based on a systematic review on the diagnostic accuracy of nNO testing in PCD diagnosis [5], in individuals 5 years of age or older, with an appropriate clinical phenotype for PCD, and when CF is excluded, the diagnostic accuracy of nNO measurement (performed with chemiluminescence devices using established, standardized protocols) is comparable to that of TEM and/or genetic testing. Both ERS and ATS guidelines recommend using a stationary chemiluminescence analyzer (during a velum closure, such as breath hold or oral exhalation against resistance) and state that there is currently no consensus over what threshold constitutes a positive or negative cut-off nor is there any recommendations about any specific devices [1, 6]. While chemiluminescence analyzers are recommended by guidelines, electrochemical devices are more affordable, but lack fully standardized operating procedures, nor have they been tested against established diagnostic criteria [7]. The ATS practical guidelines also highlight the need for further research examining the diagnostic accuracy of portable electrochemical nNO devices that could potentially validate less expensive alternatives for PCD diagnosis [8].

This note focuses on comparing nNO levels measured with three different devices against resistance in controls, PCD and cystic fibrosis (CF) patients, as well as evaluating the sensitivity and specificity of different cut-offs for these devices in distinguishing between the groups.

2.1. Study population

44 participants aged 11–48 years were recruited, including 16 controls, 16 with PCD and 12 participants with CF. 5 children were included (2 controls aged 11 and 14 years; 2 PCD patients aged 13 and 15; and one 16 year-old CF patient). The controls were recruited from staff, students, and their social network at Uppsala University Hospital. PCD and CF patients were recruited from an outpatient clinic specialized in CF and PCD at Uppsala University Hospital. PCD diagnosis was based on TEM analyses of brush biopsies taken from the respiratory epithelium of the nasal cavity, some also confirmed by genetic testing. Out of the 16 participants with PCD 10 had abnormal TEM, 5 had normal TEM but were confirmed PCD by genetic testing. One participant was categorized as PCD based on having persistent symptoms from lower and upper airways, chronic middle ear disease with hearing loss, infertility and situs anomalies in combination with low nNO. CF diagnosis was based on the algorithm defined by the European Cystic Fibrosis Society [9]. All groups were matched based on age and sex.

2.2. Measurement of nNO

nNO measurements by aspiration while doing oral exhalation against resistance during maximally 30 s were performed according to ATS and ERS guidelines [10] with three devices: the Analyzer CLD 88 (Eco Medics AG, Switzerland), Medisoft FeNO+ (MGC Diagnostics, USA) and NIOX Vero (Circassia Group PLC, UK). The EcoMedics device is a chemiluminescence analyzer, while the Medisoft and NIOX devices are electrochemical analyzers. All measurements were performed at room temperature and used according to standard user operating guidelines. For the fast responding EcoMedics device, an acceptable NO plateau was defined as a 4 s exhalation with <10% variance. If such a plateau was reached early, the sampling time could be less than 30 s. The Medisoft and NIOX devices have a sampling time of 30 s. However, the measuring settings are defined by the manufacturer and cannot be changed, thus we cannot report a real-time plateau for the electrochemical devices. The whole portion of sampled air (125–150 ml) was fed over the relatively slow-responding electrochemical sensors in these devices. The expiratory counter-pressure was at least 5 cm H2O for all devices. All three devices were located in the same room where the ambient NO levels were constantly low (<50 ppb). The flow rate was calibrated each morning for the EcoMedics device (only device where this is applicable). Two measurements, one from each nostril were performed with each device. In case of suspected leakage, the maneuver was repeated and the mean of the two highest measurements were used for analyses. All measurements were performed using a single-patient-use fixed plastic nasal olive inserted tightly into the nostril to prevent leakage.

The different devices used slightly different sampling flow rates (0.246–0.3 for EcoMedics, 0.25 Medisoft and 0.3 l min−1 for NIOX) and providing the measured nNO concentrations in parts per billion might introduce a bias, because nNO values are dependent on the sampling flow-rate (higher values with lower sampling flow-rate). To compensate, nNO output was calculated as mean concentration of right and left nares together × flow sampling rate (l min−1): e.g. 1000 ppb × 0.33 l min−1 = 330 nl min−1 [11].

Two participants in the control group had a technical error from the right nostril measurement (classified as leakage as values differed 10-fold between nostrils), so the mean concentration of two measurements from the left nostril (repeated) was used in the analyses. Safety procedures were followed according to the hospital's guidelines.

2.3. Statistical analyses

A linear relationship estimation was used with Pearson's coefficient (r) to assess the correlation between devices. The bias between the devices with the limits of agreement (95% of paired differences) were assessed using the Bland–Altman method.

Receiver operating characteristic (ROC) curve analysis with measurements of area under the curve (AUC) was used to evaluate the sensitivity and specificity of each device and an optimal cut-off value was obtained prioritizing higher sensitivity, as nNO testing is mostly used as a screening tool [6].

The differences between mean levels of nNO between the groups was assessed using nested one-way ANOVA test and corrected for multiple comparisons by controlling the false discovery rate (FDR) using a two-stage step-up method of Benjamini, Krieger and Yekutieli [12]. The 95% CI intervals for the mean differences between the devices were calculated using Tukey's multiple comparisons test.

In order to assess the variability of the measurements, which should be <10% [13], the coefficient of variation (CV) of two of the highest measurements was calculated (CV = (SD/) * 100).

All the statistical analyses and graphs were performed using GraphPad Prism software version 9.5.1 (GraphPad Software, USA) and Stata SE version 17.0 (StataCorp, USA). A p-value of <0.05 was considered statistically significant.

One control and one PCD patient, both adults, were removed from analyses due to inability of performing the test due to dyspnea. The characteristics of the study populations are shown in table 1.

Table 1. Characteristics (percentage, mean ± standard deviation and mean (range)) of study population and nasal nitric oxide (nNO) values using three different devices.

 All n = 44 Controls n = 16 PCD n = 16 CF n = 12 Women (%)30383117Age (years)32 (11–58; ±12)35 (11–58; ±13)33 (13–54; ±12.8)27 (16–42; ±8)Height (cm)173 ± 10173 ± 10171.8 ± 10.4176 ± 9Weight (kg)72 ± 2668 ± 2677.2 ± 30.569 ± 19ICS usage (%)3212.55033Nasal steroid usage (%)6.8012.58.3nNO values (ppb)    EcoMedics385 ± 301684 ± 21074 ± 61374 ± 136Medisoft478 ± 404886 ± 32189 ± 89488 ± 183NioX430 ± 350775 ± 26780 ± 68405 ± 162nNO output (nl min−1)    EcoMedics106 ± 82187 ± 5820 ± 16106 ± 39Medisoft119 ± 102221 ± 8022 ± 22121 ± 46NIOX128 ± 105232 ± 8024 ± 20121 ± 49

There was a significant difference between nNO levels in all the groups during exhalation against resistance. In the control group the mean difference was 191 nl min−1 higher than in the PCD group (95% CI: 153–230; p < 0.001) and 97 nl min−1 higher than in the CF (57–137; p < 0.001). In the PCD group the mean difference was 94 nl min−1 lower than in the CF group (−135, −54; p < 0.001) (figure 1)

Figure 1. Mean nNO values with 95% confidence intervals using the three devices in three groups.

Download figure:

Standard image High-resolution image 3.1. Comparison between devices

The Pearson's correlation coefficient for EcoMedics vs Medisoft was 0.98 (95% CI: 0.96–0.99), for EcoMedics vs NIOX 0.98 (0.97–0.99) and for Medisoft vs NIOX was 0.97 (0.95–0.98).

There was a statistically significant difference between the mean values in all individuals measured with the three devices (p < 0.001), with a mean on 103 for EcoMedics, 122 for Medisoft and 123 (nl min−1 ) for NIOX device (table 1).

The Bland–Altman plots in figure 2 show higher nNO output values obtained with the Medisoft (figure 2(A)) and NIOX (figure 2(B)) devices as compared to the EcoMedics device, with a bias of −19 nl min−1 (95% CI: −73–35) and −21 nl min−1 (−73–31) accordingly. The bias between Medisoft and NIOX device was −2 nl min−1 (−50–46) (figure 2(C)). The difference increased with higher nNO values.

Figure 2. Bland–Altman plots showing difference between the EcoMedics and Medisoft (A), EcoMedics and Medisoft (B) and Medisoft and NIOX (C) devices. The dashed line indicates the mean of differences between the two paired measurements, the dotted lines indicate the 95% limits of agreement.

Download figure:

Standard image High-resolution image

All devices reached a 100% sensitivity and specificity in distinguishing PCD subject from healthy subjects Cut-off values with highest sensitivity/specificity (with 95% CI of 79.6–100) for the three devices are given below.

Best discrimination between controls and PCD subjects:

The median CV for the EcoMedics device was 2% (interquartile range: 0.6%–4.2%), for the Medisoft 3.6% (1.7%–6.3%) and 2.5% (1.1%–4.8%) for NIOX. There was no statistically significant difference between the CV of the controls and PCD/CF patients. In the PCD patients the variation was numerically larger (around 8% for the electrochemical devices). The median inter-device CV 14.8% (10.3%–25.9%).

In this study, we aimed to assess the differences in nNO output values using three different devices among healthy controls, PCD and CF patients and to evaluate the sensitivity and specificity of these devices to distinguish between healthy subjects and subjects with PCD. Our results demonstrated significant differences in nNO levels during exhalation against resistance between all three groups as well as differences in the mean values between the devices. Furthermore, with increasing values all devices showed greater difference between themselves. One possible reason for these variations could be the varying sampling durations. If plateau is reached with the EcoMedics device within 30 s, the sampling would be shorter than the sampling with the Medisoft or NIOX devices (both 30 s). And consequently, the measured values could be to some extent lower. Moreover, the sampling flow rate can vary between the devices. Nevertheless, since the configuration parameters of the electrochemical devices remain unmodifiable, and the precise sampling duration remains unknown. To mitigate the introduction of a systematic error, the sequence of device utilization was randomized for each participant. A different breathing maneuver between devices and high ambient NO values could potentially impact the measurement results. However, he ambient NO in our sampling room was constantly low and the breathing maneuver is done in a way to ensure velum closure with all devices, and should thus have minimal effect on the results. The variability of our intra-device measurements was lower than previously reported in other studies [14, 15]. However, this is to be expected when only using the two highest measurements.

The three devices used in our study all demonstrated a 100% sensitivity and specificity in distinguishing between controls and PCD patients at different cut-off values. This suggests that any of these devices could potentially be used in PCD diagnosis, and can be of clinical importance when choosing a research or diagnostic device, as the price, running costs and ease of use of different devices vary greatly [7, 11]. However, due genetic heterogeneity all PCD patients do not have low levels of nNO. Hence, the reliance on nNO levels as a solitary diagnostic assay for PCD should be used with caution [16]. Moreover, the low number of participants and the monocentric nature of our study limits our study results in establishing diagnostic thresholds and should be replicated in other centers with larger sampling sizes.

In conclusion, our study suggests that electrochemical devices can potentially be used as diagnostic tools for PCD. However, device-specific cut-offs are needed, and we propose cut-off values that need to be further validated in other populations.

The data cannot be made publicly available upon publication due to legal restrictions preventing unrestricted public distribution. The data that support the findings of this study are available upon reasonable request from the authors.

Kjell Alving reports receiving personal payment, equipment, and material for a clinical study outside of the present manuscript from NIOX. The other authors report no COI.

This research was funded by Swedish Heart and Lung Foundation and research support from the Swedish state under the ALF agreement between the Swedish government and the county councils.

The research was conducted in accordance with the principles embodied in the Declaration of Helsinki and in accordance with local statutory requirements.

All participants (or their parent or legal guardian in the case of children under 16) gave written informed consent to participate in the study and ethical approval was granted by the Swedish Ethical Review Authority (Dnr: 2019-02603).