Introduction

Viruses such as SARS-CoV-2 and influenza A/B are highly transmissible and have overlapping symptoms.1 2 Rapid and accurate differential diagnosis is crucial in preventing further viral spread and making appropriate decisions about treatment and resource utilisation.3 Additionally, as viruses cause about 90% of upper and 30% of lower respiratory infections,4 5 their diagnosis can help prevent unnecessary use of antibiotics and/or additional investigations.6

In a hospital setting, rapid identification of infected patients and their appropriate cohorting on admission is a priority to prevent nosocomial transmission. Due to their high sensitivity and specificity, real-time reverse transcription PCR (RT-PCR) tests are recommended for the diagnosis of RNA viruses (such as SARS-CoV-2 and influenza).7 However, laboratory-based PCR tests can be costly and have a variable turnaround time ranging from 3 to 24 hours or more. In the setting of an ED, where early triage and isolation decisions are of the essence, these limitations are particularly relevant.

Progress has been made with novel PCR platforms that can be used at the point of care (POC), thus reducing the turnaround time to approximately 20 min (excluding sample transfer and preparation time). However, the current cost of POC PCR platforms prevents them from being a long-term solution in the ED setting, although the technology may become less expensive over time. Consequently, antigen tests/lateral flow devices (LFDs) are attractive for their speed (turnaround time of approximately 15 min), cost-effectiveness and ease of use.8 Unlike POC PCR platforms, LFDs do not require a dedicated testing area or additional pipetting steps by trained operators. Rather, the sample can be applied immediately on acquisition and results read in as little as 5 min. The drawback with LFDs is their lower sensitivity at low viral concentrations compared with PCR.9 Despite this limitation, LFDs can be a useful tool for rapid diagnosis, helping to reduce virus transmission as demonstrated during the SARS-CoV-2 pandemic.9

Winter represents the peak time for the presentation of patients with respiratory symptoms. As different respiratory viruses cause similar symptoms, it is important to make a specific diagnosis as quickly as possible. Since the SARS-CoV-2 pandemic, there has been a continuous drive to innovate diagnostic tools, which includes the development of ‘multiplex’ LFDs (M-LFDs) for simultaneous detection of multiple viruses from a single sample. Compared with singleplex LFDs, M-LFDs represent further improvements in speed, simplicity and cost efficiency in detecting common viruses. However, few studies have evaluated M-LFD use for SARS-CoV-2 and influenza in clinical practice, specifically in the ED setting.10–12 In this study, we conducted a preliminary laboratory evaluation of one M-LFD for detection of influenza A, B and SARS-CoV-2, followed by an evaluation of its real-world performance during one winter season at an ED.

MethodsLaboratory evaluation of SureScreen M-LFD

Methodology of the preliminary laboratory evaluation of the SureScreen SARS-CoV-2 + Influenza A&B Antigen Combo Rapid Test Cassette (SureScreen M-LFD) with laboratory-grown virus and PCR-negative clinical samples is detailed in online supplemental methods.

Real-world study designSetting

Effectiveness of the SureScreen M-LFD in clinical practice was conducted in the ED at St Thomas’ Hospital, London, UK (approximately 200 000 attendees and 50 000 emergency admissions annually). Recruitment and prospective data collection took place during the respiratory infection season from 1 December 2022 to 21 April 2023. National surveillance data from October 2022 to April 2023 identified that influenza A was predominant (94.6%, vs influenza B 5.4%). Of the influenza A viruses, 66.1% were H3N2 and 33.9% were H1N1; all influenza B viruses belonged to subclade V1A3.13

Inclusion and exclusion criteria

Participants formed a consecutive series. Those ≥18 years of age who were admitted to the ED with respiratory symptoms suspected of influenza or SARS-CoV-2 infection and received both M-LFD and PCR testing were eligible. Exclusion criteria included age <18 years, lack of clinical suspicion for influenza or SARS-CoV-2 infection and clinical unsuitability for sample collection (as stipulated by a healthcare worker). The exclusion of patients <18 years of age was to avoid confounding the results with a false co-infection with influenza A and B that could be detected in patients aged 2–17 years who were given live attenuated influenza vaccine nasal spray (Fluenz Tetra) as per the national influenza immunisation programme.

Endpoints

The primary endpoint was the sensitivity of the M-LFD to detect influenza A and B compared with PCR as the standard of truth. The secondary endpoint was the sensitivity of the M-LFD to detect SARS-CoV-2 versus PCR.

Ethics

This study was conducted in accordance with the Declaration of Helsinki14 and, where appropriate, UK Medical Devices Regulations 2002,15 Medical Devices (Coronavirus Test Device Approvals) Regulations 2021,16 MHRA In Vitro Diagnostic Devices Guidelines 202117 and the Data Protection Act 2018.18 This service evaluation was conducted under the approved ‘UK Health Security Agency (UKHSA) standardised protocol for clinical studies evaluating the safety and performance of In Vitro Diagnostic Devices (IVDD) for detecting COVID-19 and other infectious diseases’.

Patient and public involvement

Patients or the public were not involved in the design, conduct, reporting, or dissemination plans of our research.

Testing with M-LFD and PCRTesting pathway

The testing pathway is summarised in figure 1A. The pathway was initiated if a participant admitted to ED had presented with respiratory symptoms consistent with a possible infection with influenza A/B or SARS-CoV-2. During the initial medical review, healthcare professionals (registered nurses, physicians, physician associates and nurse practitioners) took anterior nares swabs and performed the test according to the manufacturer’s instructions. The M-LFD was visually read and digitally captured (ClearScreen Reader by TestCard) for review and transmission to the electronic patient record. Samples from the same participants (using a separate swab) were also tested with the POC RT-PCR testing platform (Roche cobas Liat system).

Figure 1

Schema of (A) the testing pathway and (B) the participant flow. *This could be for a variety of reasons including clinical instability, transfer, elopement, non-recording, testing done in another location and testing done in the last 24 hours with proof of result. M-LFD, multiplex lateral flow device; POC, point of care.

RT-PCR cycle threshold values

RT-PCR cycle threshold (Ct) values can be used as a proxy for the virus amount present in a person’s nasal or oral cavity rather than a direct measure. It depends on both the quality of the swabbing technique and the efficiency of the release of the virus into the transport medium.

For RT-PCR detection of the pathogens, Ct values have been reported previously for SARS-CoV-2.19 For influenza A and B, Ct values were determined by an internal laboratory validation using 100 excess clinical samples from the previous winter season. Samples were tested with both SureScreen M-LFD and PCR (Roche cobas Liat system). The M-LFD limit of sensitivity correlated to Ct 26–27 for influenza A; however, a final Ct threshold for influenza B was not set due to the low sample number.

Data analysesEvaluating test results

M-LFD and PCR test results were either concordant, which meant true positives (both positive) and true negatives (both negative), or discordant, comprising false positives (M-LFD positive, PCR negative) and false negatives (M-LFD negative, PCR positive). Sensitivity analyses included only PCR-positive samples to ensure that only true positives and false negatives were included. All other analyses involved all participants with an evaluable result for both M-LFD and PCR tests (positive or negative) to avoid confounding the analysis with void test results.

Sample size

As no prior evaluation has been conducted for influenza A/B, there was no baseline sensitivity to use for non-inferiority. A sample of 100 participants with a positive PCR test for influenza A/B was deemed sufficient to determine a baseline sensitivity of the M-LFD compared with PCR. Using normal approximation and assuming sensitivity to be 90% with a lower confidence limit of 80%, 86 positive participants would provide 80% power to observe this sensitivity.

Statistical analyses

Data analyses were performed using R V.4.1.1 with the packages of ‘binom’ to calculate 95% CIs. Descriptive statistics are presented for all variables. Continuous variables are summarised by the number of observations (N), mean, SD, median, minimum and maximum.

M-LFD sensitivities for SARS-CoV-2 and influenza A/B were statistically compared using a two-sided χ2 test with continuity correction (proportion test function in R). Sensitivity was also analysed using logistic regression models with log10 viral concentration and age as continuous independent variables and sex as a categorical independent variable. After model fitting, logistic regression curves were generated for the predicted probability of a true positive in relation to viral concentration.

Exploratory analyses of the sensitivity, specificity and positive and negative predictive values were performed between subgroups by sex and age.

ResultsLaboratory evaluation

In the preliminary performance evaluation, SureScreen M-LFD showed 100% specificity to the tested pathogens, with no identified false positives. The sensitivity to influenza A was 100% (5/5 positive results) at viral titres of 8×104–9.9×105 focus-forming units (FFU) per mL, but strain-dependent at 8–9.9×103 FFU/mL (5/5 positive for H3N2 A/Darwin/9/21 and 1/5 for H1N1 A/Victoria/2570/19; very weak band intensities for both strains) and 0% at 8×101–9.9×102 FFU/mL. For influenza B, sensitivity was 100% at 1×106–1×107 FFU/mL, but strain-dependent at 1–4.7×105 FFU/mL (5/5 positive for B/Austria/1359417/21 and 1/5 positive for B/Phuket/3073/13; very weak band intensities for both strains) and 0% at 4.7×102–4.7×104 FFU/mL. For SARS-CoV-2, SureScreen M-LFD positively identified the virus in all 25 samples at 1×102 and 1×103 plaque-forming units (PFU) per mL (online supplemental table S1).

Real-world study participants

During the study period as outlined in the supplemental information, 16 667 patients were admitted to the ED of which 1501 had respiratory symptoms that were consistent with COVID-19 or influenza. Of 1501 adults admitted to ED with respiratory symptoms, 808 met all eligibility criteria and were included in the study (figure 1B). The remaining individuals were excluded, predominantly because they had M-LFD test results without a concurrent PCR (figure 1B).

Of these 808 participants, 402 (49.8%) were female and 406 (50.2%) male, with a mean age of 47.2 and 46.5 years, respectively (range 18–75 years in both groups). The distribution of age and sex is shown in online supplemental figure S1. Proportions of participants in the age groups 18–39, 40–59 and 60–80 years were 36.1%, 35.8% and 28.1%, respectively.

Concordance of the M-LFD and PCR test results

Positive and negative results from the SureScreen M-LFD and PCR for all three pathogens are summarised in table 1. No participant tested positive for more than one infection and no M-LFD result indicated a different infection to the PCR test results.

Table 1

Summary of PCR and M-LFD test results for influenza A, B and SARS-CoV-2

Concordance rates between the M-LFD and the PCR results for individual pathogens were >90%, with an overall concordance of 85.6% (table 2). The highest proportion of false negatives occurred for SARS-CoV-2 (9.1%; table 2).

Table 2

Concordance of M-LFD and PCR test results in detecting influenza A, B and SARS-CoV-2

Sensitivity and specificity

For all evaluable samples, the M-LFD test sensitivity (95% CI) was 67.0% (56.9% to 76.1%) for influenza A, 94.1% (71.3% to 99.9%) for influenza B and 48.2% (39.7% to 56.8%) for SARS-CoV-2 (figure 2A). The sensitivity for SARS-CoV-2 was statistically significantly lower than that for influenza A (p=0.0057) and influenza B (p=0.00088). Specificity (95% CI) of the M-LFD was 99.4% (98.6% to 99.8%) for influenza A, 100% (99.5%to 100%) for influenza B and 99.3% (98.3% to 99.8%) for SARS-CoV-2 (figure 2B).

Figure 2

Forest plot of (A) sensitivity and (B) specificity values for influenza A and B and SARS-CoV-2.

A summary of Ct values for PCR-positive samples is shown in table 3. Ct values for samples with false-negative M-LFD results (mean 31.7–33.1 across all three pathogens) were higher than those for true positives (21.2–23.1). From the distribution of Ct values among samples positive for influenza A and SARS-CoV-2, those with Ct 27–29 yielded mixed M-LFD results, and those with Ct ≥30 returned false negatives (online supplemental figure S2).

Table 3

Ct values for PCR-positive samples of influenza A, B and SARS-CoV-2, shown by M-LFD results

In a logistic regression analysis, a 98% probability was reached when Ct values were <25 for influenza A and SARS-CoV-2 (figure 3). Regression analysis was not performed for influenza B due to the small sample size (n=17). Details of the univariate and multivariate regression analysis of sensitivity for influenza A and SARS-CoV-2 are shown in online supplemental table S2.

Figure 3

Logistic regression curve of the predicted probability of a positive M-LFD test result for influenza A and SARS-CoV-2 in relationship with RT-PCR Ct values. Shaded areas represent 95% CIs of the predicted probabilities. Probabilities were based on multivariable regression models fitted to the data observed in the study and were calculated for a 40-year-old woman across Ct values from 10 to 40. The regression analysis was not performed for influenza B due to the small sample size (n=17). Ct, cycle threshold; M-LFD, multiplex lateral flow device; RT-PCR, reverse transcription PCR.

Exploratory subgroup analyses did not reveal any significant differences in the M-LFD sensitivity, specificity or positive and negative predictive values for any of the pathogens between sexes and age groups (online supplemental tables S3, S4).

Discussion

The real-world performance of the SureScreen M-LFD in testing for influenza A, B and SARS-CoV-2 in an emergency care setting was consistent with our preliminary laboratory evaluation. While a small number of false positives occurred, there was no cross-reactivity. The false negatives all corresponded with high Ct values and thus low sample viral concentrations. The concordance rate for individual pathogens with PCR testing was over 90%, with an overall concordance rate of 85.6%. The specificity was high (≥99.3%) for all three pathogens.

The overall sensitivity of the M-LFD for SARS-CoV-2 in our study (48.2%) was lower than that observed in a previous evaluation of a singleplex SARS-CoV-2 LFD at the ED of St Thomas’ Hospital in December 2020 to March 2021 (sensitivity 70.7%).20 While individuals presenting in the ED were screened for influenza/COVID-19 symptoms, the time of symptom onset was not recorded. It is likely that those presenting with symptoms were in the early phase of infection, which could mean low viral concentration at the sites of sampling. It has been previously documented that LFD sensitivity was dependent on viral concentration and symptom status and can diagnose individuals with the highest viral concentrations, associated with the most infectious COVID-19 cases.21

Evaluation of the singleplex SureScreen SARS-CoV-2 Antigen Rapid Test Cassette in community testing sites from February 2021 to April 2021 showed a sensitivity of 72.5% (95% CI 67.1% to 77.4%) for assisted testing and 71.7% (65.7% to 77.1%) for self-testing.21 However, no adjustments for viral concentration, symptom presence or the timing of peak viral concentration (which has been shown to vary from at the time of symptom onset at the start of the pandemic to more recent observations showing a peak 3–4 days post-onset)22 were undertaken in this study, and so this discrepancy might not reflect a difference in actual device performance.

Sensitivity of the SureScreen M-LFD to influenza B (94.1%) was statistically significantly higher than for influenza A (67.0%). For influenza, no prior performance data are available for comparison.

Other studies set in emergency care evaluating M-LFDs for detecting SARS-CoV-2 and influenza have reported mixed results. In a Dutch study evaluating the Roche SARS-CoV-2 and Influenza A/B Rapid Antigen Test in 1740 adults, the sensitivity was 67.7% for SARS-CoV-2 and 52.7% for influenza A/B.10 Some studies also included children—while some evidence suggests a slightly lower viral concentration of SARS-CoV-2 in children versus adults, which might affect LFD sensitivity, this was not deemed sufficiently significant to warrant a different testing approach.22 In a French paediatric ED study involving 263 children, the COVID-VIRO ALL IN TRIPLEX M-LFD showed an overall sensitivity of 91.6% for influenza and 88.9% for SARS-CoV-2, which increased, respectively, to 92.3% and 100.0% in a subgroup of samples with Ct <32.11 Lastly, a study of the Fluorecare SARS-CoV-2 and Influenza A/B and RSV Antigen Combo Test with 178 adults and children in a Belgian ED reported a sensitivity of 80.8% for influenza A, 65.9% for influenza B and 77.8% for SARS-CoV-2.12

Performance was within an expected range when compared with PCR across all Ct values. Although Ct values cannot be directly compared between different assays, platforms and laboratories, we note that influenza A-positive samples had a higher Ct value (23.1) than influenza B (21.2) and SARS-CoV-2 (23.1), suggesting a lower viral concentration in influenza A samples. The sensitivity for influenza A and SARS-CoV-2 decreased rapidly for samples with Ct >25 (influenza B not evaluable due to sample size), suggesting that the sensitivity of the SureScreen M-LFD declined with decreasing viral concentration. A similar observation was reported for the COVID-VIRO ALL IN TRIPLEX M-LFD based on the ad hoc subgroup analysis of samples with Ct <3211 and for Fluorecare M-LFD that showed limited virus detection in samples with Ct >25.12 In fact, Ct values of 30 and 25 have been observed as typical efficacy thresholds for more or less analytically sensitive LFDs, respectively.22

Despite the limitations in sensitivity observed for multiple M-LFDs, their ability to identify people with a high viral concentration is valuable because such individuals are likely to be the most infectious.23 This is particularly crucial in the hospital ED setting to reduce the risk of nosocomial infections and ensure an appropriate use of resources. For example, in the Dutch study, the authors estimated that a total of 8712 hours (median ≤6 hours 59 min) per patient of unnecessary isolation measures were saved over the 5-month study period.10 On the contrary, the decreased sensitivity of LFDs may render it necessary to re-test symptomatic individuals with a negative LFD result with another, more sensitive test. Nevertheless, the ability of LFDs to correctly diagnose the majority of individuals still represents a fast and potentially cost-effective pathway. Rapid diagnosis of an individual may impact patient treatment where patients can be quickly assessed for COVID-19 and influenza and then prescribed antiviral therapy if necessary. Additionally, diagnosis using M-LFDs impacts infection control and patient flow through the hospital, as it allows for appropriate isolation room use in ED settings, thereby minimising transmission of infection. While testing with LFDs does not rule out bacterial co-infection, those patients tend to present with a more systemic illness and so may be managed differently from the outset.

No co-infections were identified in our study. In general, co-infection with SARS-CoV-2 and influenza appears to be rare (0.7% (95% CI 0.4% to 1.3%) of SARS-CoV-2 patients globally24 and 0.03% (4/15 000) in a UK community survey 2022–2 02325). Co-infection with influenza A and B is also very rare; in a study from Northwest England (2007–2011), only five samples with such co-infection were identified out of 10 501 samples positive for any respiratory virus (0.05%).26 However, influenza B does not occur commonly every year like influenza A, and so the co-infection rate depends on the studied time period. For example, a rate of 1.6% was observed in a Madrid hospital between December 2014 and March 2015.27

As this was a real-world evaluation in an ED setting, there are some limitations. While individuals were screened for symptoms indicative of influenza or COVID-19-related symptoms, the time of symptom onset was not recorded. Operational issues led to some individuals missing tests and thus their exclusion from the study. For example, fast-track admission could lead to the M-LFD test being omitted. Additionally, if the clinical staff felt the individual would not be admitted, a fast-track PCR may have been skipped. Analysis of the overall demographics of the group showed no statistical difference, and the impact of excluding individuals with only one test was likely minimal. This further highlights some of the challenges of validation of POC testing in difficult patient areas. The demographic information for the study population was restricted to age and sex. However, several factors mean that the study population is likely representative of the UK population accessing emergency care. First, the large number of attendees to St Thomas’ Hospital (approximately 200 000 per year) provides a heterogeneous population for analysis. For example, the proportions of participants across age groups in this study (36.1% aged 18–39, 35.8% aged 40–59 and 28.1% aged 60–80 years) were similar to those in the UK general population (36.1%, 33.2% and 25.0%, respectively).28 Additionally, as St Thomas’ Hospital’s location (Lambeth, London) is among the 10% most ethnically diverse local authorities in England,28 it is likely that the study participants were similarly diverse. Put together, results found in this study may be generalisable to M-LFD use in other settings and/or the general public. Previous research has suggested performance of COVID-19 LFDs is similar in adults and children,29 however, as children were not included in this evaluation further work would be required to confirm performance in this setting. Lastly, no formal acceptability of the test was undertaken as part of the evaluation. However, informal debriefing of staff suggested they found the test easy to administer and helpful in patient management.

Knowing the sensitivity and specificity of the SureScreen M-LFD in a real-world setting, if it is assumed to be broadly generalisable to other M-LFDs, contributes to assessments of where and how such technology can be used. It would be reasonable to assume that other M-LFDs that match or surpass the laboratory evaluation performance also at least match the real-world performance seen in this study. While this analysis is most relevant to an ED admission setting, with the paucity of other real-world data, it could be used as part of considerations in other settings and other cases, such as outbreak detection in care homes and prisons.

In conclusion, our findings suggest that the SureScreen M-LFD can be an effective tool to identify individuals most likely to be infectious with influenza A, B or SARS-CoV-2 and, given the representative UK population sample, can be potentially generalised for use in other settings.