In this study, we report the results from the first post-pandemic EQA for SARS-CoV‑2 virus genome detection and compare these results to the previous rounds. The aim was to determine whether the overall performance had changed since the pandemic ended, given that specific testing circumstances have changed. As a main finding we show that the response ratio of registered laboratories for the genome detection EQA schemes continuously dropped as the pandemic progressed, from 99% to 74% at a rate of −0.3% per month (Fig. 1). This decrease may be related to a loss of interest in prioritizing SARS-CoV‑2 genome detection assays, or the impression that assays have been sufficiently validated. As there are no data on the number of test facilities that were in operation in Austria at a specific time and which test systems were used, no statement can be made as to what proportion complied with the statutory obligation to participate in EQA. The only available information in this respect is the number of 1034 pharmacies registered to carry out tests in Austria in January 2023. We note that the national SARS-CoV‑2 POCT EQA scheme at this time had only 28 participants [16], and we report variable participation in the POCT EQA scheme over time (Fig. 1). The emergence of novel genetic and antigenic variants provides an impetus for laboratories to continue monitoring genome detection assays through EQA; however, ultimately, we do not know the precise individual motivation(s) that drove participation in EQAs and, more importantly, the reasons for not reporting results when a participant has registered for a given round.

The overall performance in post-pandemic EQA for SARS-CoV‑2 virus genome detection was broadly consistent with the previous rounds as most false negative results were reported for the sample with the lowest virus load. When controlling for virus concentration, the results from the two samples with the highest concentration were slightly lower than the expected true positive ratio, but the sample with the lowest virus load was within expectations based on all previous results. When stratified by subsets of results, the observations from earlier rounds that automated test systems had higher detection ratios than manual test systems and that medical laboratories had higher detection ratios than nonmedical laboratories continued in the post-pandemic period. We acknowledge that the design of the post-pandemic schemes varied slightly from those during the pandemic, in a shift towards including other respiratory viruses in the panel. As a result, some participants may have incorporated multiplex tests to detect other respiratory viruses. Although we do not have the statistical power to analyze it here, this could be a potential confounding factor in determining whether performance has decreased relative to previous rounds.

Adding to the analysis presented in a previous study, we now separately analyzed the performance of NPT/POCT assays as a subset of the group of automated test systems. Automated test systems intended for NPT/POCT do not require delicate manual work steps and deliver clear results or a clear indication of a malfunction or measurement error [10]. Therefore, medical professionals without laboratory qualifications are authorized to also use such test systems [15]; however, our results show decreasing detection ratios (true positive results) in the order: automated laboratory systems (98.7%) > automated systems intended for NPT/POCT use (95.0%) > manual methods (89.6%) > in-house assays (73.7%) for samples with relatively low virus load (Table 4). Therefore, the automated systems intended for NPT/POCT use did not meet the expectation to deliver almost perfect performance, were surpassed by automated laboratory systems, but performed better than methods requiring manual steps.

The World Health Organization (WHO) defined a limit of detection (LOD) of NAT test systems of 1000 cp/mL as required and < 100 cp/mL as desirable [11]. In Austria, however, massive testing was prioritized above this recommendation, and the recommended LODs were not declared mandatory. This lack of enforcement of LOD regulations may partly explain why we continued to observe 11% false negative results for samples ~1000 cp/mL in the post-pandemic EQA rounds (Table 3), which is not an improvement over the > 6% false negative results for samples of similar concentration in earlier rounds (Table 4). Given that 25% of symptom-free individuals who were coincidentally identified as positive at screening had low viral loads, using only sufficiently sensitive tests should be required, at least for testing asymptomatic individuals [12,13,14]. As Austrian laboratories were not incentivized to improve SARS-CoV‑2 diagnostic methods, and the existence of unprecedented shortages of reagents and consumables in the early phases of the pandemic, it is possible that participants could not switch to better performing assays, or were reluctant to do so, even if feedback from participation in the EQAs indicated that their assay of choice had low performance.

However, it must be stated that the EQA schemes we report here were not strictly designed for NPT/POCT assays as they are designed to be implemented on primary human samples. For example, some participants with POCT systems would have had to use a swab to remove some of the fluid from the provided sample, in contrast to methods where RNA could be extracted directly from the provided material and concentrated. Theoretically, this would have diluted the test sample, which may explain the loss of sensitivity for the low-concentration sample for NPT/POCT test systems compared to other automated methods.

We also report the results of nonmedical laboratories and specifically categorize pharmacies as a subset of the group of nonmedical laboratories. As mentioned above, a small fraction of all pharmacies registered to perform SARS-CoV‑2 testing participated in the reported EQA schemes. Of the 359 test results submitted by pharmacies over 6 rounds, 16 (4.5%) were reported from automated test systems, 73 (20.3%) were reported from automated POCT test systems, and the majority (270, 75.2%) were reported from manual test systems, the systems with the lowest overall performance, in general, and those that require the most technical competence; however, when interpreting these findings, it is worth reiterating the fact that we do not know the ultimate motivations of the participants, nor, for example, whether their participation is intended to test/validate new assays not in routine use.

As with all studies on EQA data, a limitation of this study is that results can only be analyzed as they were reported by participants. It must be trusted that they were generated properly. We cannot assume the trends we observed represent the testing performance in Austria, as we do not know if more laboratories than those that participated in an EQA round were in operation and what performance their test systems had. Nonetheless, the data show the dynamics of test performances across laboratory type and assay type from the start of the pandemic. We were limited in our comparisons to previous rounds by statistical sensitivity (or statistical power) due to relatively small sample sizes and small effect sizes. A post hoc power analysis (not shown) suggested that we achieved a power (1 − β) of only 0.29 with a sample size of 327 results comparing whether the observed true positive ratio of 95.4% in the post-pandemic round was significantly different from pandemic rounds (Table 3); however, the principal asset of these data is the existence of > 6000 results available from the beginning of the pandemic. We can say with some confidence that the overall performance is high, and individual laboratories can receive excellent feedback based on this large dataset for monitoring their performance and determining whether improvements are necessary. Our results are similar to those reported by other EQA providers analyzing performance over time for SARS-CoV‑2 nucleic acid testing [17, 18]. Even if we continue to see a decline in response ratios in the upcoming years, our dataset provides essential information for health authorities on the overall quality and accuracy of SARS-CoV‑2 monitoring. This provides confidence for estimating the incidence in the population to monitor trends and dynamics in the virus circulation.