A few weeks ago, the WHO re-opened its investigation of the origins of SARS-CoV-2, which will be a crucial part of worldwide efforts to prevent and mitigate pandemics. The study by Carrat et al. [1] about the apparent circulation of the virus in France before the beginning of 2020 will certainly be an important element of this investigation. This study purports to demonstrate, through ELISA-S serological tests against the spike protein of the virus as well as serum neutralization (SN) titration, that the SARS-CoV-2 virus was circulating in France prior to the first reported cases in early 2020.

However, an in-depth reading of Carrat et al.’s study shows major biases that we believe are not appropriately addressed in the Discussion section.

The specificity of the ELISA-S test is estimated at 97.5% [2]. From November to mid-December of 2019, the positive test rate is below 2.5%; hence, this could consist exclusively of false positives.

The article states that the specificity of SN “was estimated at 100% over thousands of blood donors sampled in 2017–2018” citing Gallian et al. [3]. However, the latter study assessed a 100% specificity based only on 464 samples. The mention of “thousands of blood donors” is untrue and should be corrected in the Carrat et al. article [1]. Furthermore, the point estimate of 100% specificity was not accompanied by a confidence interval. To estimate this we use Hanley’s formula, which states that if a certain event did not occur in a sample with n subjects (n greater than 30), the interval from 0 to 3/n is a 95% confidence interval for the rate of occurrences in the population. We thus infer that with 464 samples, the specificity is in the range (0.9935–1) with a confidence interval of 95%, and hence could be as low as 99.35%.

SN titers were carried out only for ELISA-S values > 0.7, which could cause a selection bias. Indeed, we know that ELISA-S false positives can be the reaction to other common cold coronaviruses. It is also possible that the false positives resulting from SN tests could be connected to the marginal effects of cross-reactive immunity [4]. This would imply that the false-positive results of the two tests could be correlated, as a result of the selection bias that we have just mentioned. To avoid this bias, the rate of false-positive SN results (i.e., 0.65%, according to our calculations of specificity shown in the preceding paragraph) would need to be calculated on the basis of the total number of samples tested with ELISA-S, rather than the number of SN tests that were subsequently carried out. Figure 1 appears to show that during the period in question (November to mid-December of 2019) around 3,800 samples in total were tested, leading to 20 positive SN tests. This gives an overall SN positive test rate of 0.53%, which seems entirely compatible with the likely false-positive rate.

It is regrettable that additional negative controls were not performed as part of Carrat et al.’s analyses. The CONSTANCES cohort started recruiting in 2012, which means that earlier samples could have been analyzed as negative controls and even been used to infer the false positive rate. SN testing could have been assigned to random samples among the negative ELISA-S samples, instead of selecting only those with values > 0.7. We believe that these additional analyses should be performed as a matter of urgency.

Another issue not discussed by Carrat et al. is the fact that ELISA-S testing showed a huge increase in positive results in the tenth week (beginning on January 6th, 2020), which was not associated with a higher number of SN positive results in the same week. In fact, their Fig. 2 shows a dramatic increase in ELISA-S and SN positive results combined at that tenth week. But this increase is actually almost entirely due to ELISA-S results. When the combined test results are split, SN positive results remain essentially constant (on average 2.2 per week before January 6th, 2020 and 2.4 per week after) while ELISA-S positive results rise dramatically (on average 7.7 per week before and 28.4 after). It is unclear why ELISA-S positive results would increase while SN positive results did not change. Of note, this observation implies that the SN positive rate dropped dramatically at the tenth week since the absolute number of SN positive results remained constant while there was an increase in tested samples due to the increase in ELISA-S positive results.

Finally, and perhaps most starkly, these results imply a seroprevalence of 5.0% among the French population in January 2020, at a time when most medical professionals had not seen a single case of the novel coronavirus. Such an implicit prevalence is difficult to reconcile with the actual seroprevalence of around 5.7% that was measured in mid-May 2020, after the first Covid-19 wave resulted in thousands of hospitalizations and deaths [5].

In summary, it is entirely possible that all of the positive tests in 2019 (Elisa-S and SN) were false positives. Hence, contrary to the claims of Carrat el al., there does not appear to be any reliable evidence that SARS-CoV-2 was circulating in France before mid-December 2019 or early 2020.