ABSTRACTBACKGROUND

Point of care serological assays are a promising tool in COVID-19 diagnostics but do have limitations. This study evaluated the sensitivity of five rapid antibody assays and explored factors influencing their sensitivity to detect SARS-CoV-2-specific IgG and IgM antibodies.

METHODS

Finger-prick blood samples from 102 participants, within two to six weeks of PCR-confirmed COVID-19 diagnosis, were tested for IgG and IgM on five rapid serological assays. The assay sensitivities were compared, and patient factors evaluated in order to investigate potential associations with assay sensitivity.

RESULTS

Sensitivity ranged from 36% to 69% for IgG and 13% to 67% for IgM. Age was the only factor significantly influencing the likelihood of a detectable IgG or IgM response. Individuals aged 40 years and older had an increased likelihood of a detectable IgG or IgM antibody response by rapid antibody assay.

CONCLUSION

Rapid serological assays demonstrate significant variability when used in a real-world clinical context. There may be limitations in their use for COVID-19 diagnosis amongst the young.

BACKGROUNDRapid serological assays, used at the point of care (POC), pose a promising clinical tool in the diagnosis of Coronavirus disease 2019 (COVID-19), particularly in low- and middle-income countries where diagnostic resources are scarce. These lateral flow chromatographic immunoassays qualitatively detect immunoglobulin G (IgG) and immunoglobulin M (IgM) antibodies to the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) on a venous or finger-prick whole blood sample without the need for specialized equipment. Such assays are useful for rapid antibody testing in surveillance programmes in outbreak settings or in high seroprevalence areas. The assays may assist in the diagnosis of suspected COVID-19 in patients who test negative for SARS-CoV-2 by polymerase chain reaction (PCR) on naso- or oropharyngeal swabs.(Theel, Theel et al., 2020Theel ES Slev P Wheeler S Couturier MR Wong SJ Kadkhoda K. The Role of Antibody Testing for SARS-CoV-2: Is There One?.) In addition, they require minimal operator training and have a turnaround time of under thirty minutes.(Riccò, Riccò et al., 2020Riccò M Ferraro P Gualerzi G et al.Point-of-Care Diagnostic Tests for Detecting SARS-CoV-2 Antibodies: A Systematic Review and Meta-Analysis of Real-World Data.)Some valid concerns about the performance quality of these rapid assays exist, and most available rapid assays have been subjected only to single centre internal validation studies, in small populations.() Furthermore, the threshold antibody titre required to generate a detectable result on these devices is poorly described. Reported overall IgG/IgM sensitivities range from 18.4 to 93.3% and vary depending on disease severity and duration since symptom onset.(Riccò, Riccò et al., 2020Riccò M Ferraro P Gualerzi G et al.Point-of-Care Diagnostic Tests for Detecting SARS-CoV-2 Antibodies: A Systematic Review and Meta-Analysis of Real-World Data.)

This study critically evaluated the sensitivity of five rapid antibody assays for detection of SARS-CoV-2-specific IgM and IgG antibodies, on finger-prick blood samples amongst patients with COVID-19 confirmed by PCR on nasopharyngeal or oropharyngeal swab. Importantly, this study also investigates patient factors that influence the sensitivity of such assays.

METHODS AND MATERIALS

This study was approved by the University of the Witwatersrand Human Research Ethics Committee (Medical) (M200697). Written informed consent was obtained from all participants and patient data were anonymised prior to analysis.

 Study participants

Adult participants (≥18 years old) were recruited at the Charlotte Maxeke Johannesburg Academic Hospital in Johannesburg, South Africa. Randomly selected inpatients and outpatients were invited to participate if they had laboratory confirmed COVID-19 by RT-PCR on a naso- or oropharyngeal swab prior to interview and testing. Participant numbers were restricted by the number of assay cassettes available.

Clinical and biographical data were collected using an electronic database (REDCap 10.6.2, Vanderbilt University) by means of a self-administered participant questionnaire. Variables collected included demographics (age, sex, self-reported ethnicity), co-morbidities and if the participant was a healthcare worker (HCW). Participants provided details of previous PCR testing including the number of previous tests done, the date and result of each test, the route of sampling (oropharyngeal or nasopharyngeal), symptoms experienced at the time of the positive test, the date of onset of symptoms and the severity of disease (asymptomatic, mild, moderate, severe, critical) classified according to published criteria.(National Institutes of Health 2020

National Institutes of Health. COVID-19 Treatment Guidelines Panel. Coronavirus Disease 2019 (COVID-19) Treatment Guidelines. Available at: https://www.covid19treatmentguidelines.nih.gov/. Accessed 5 June 2020..

) Rapid Antibody AssaysFive rapid immunochromatographic antibody assays were evaluated in this study and performed for each participant, namely:1

2019-nCoV-IgG/IgM Rapid Test (whole blood, serum or plasma), Lot 200505, Dynamiker Biotechnology Company Ltd, Tianjin, China (Dynamiker)

2

2019-nCoV IgG/IgM Rapid Test Cassette (whole blood, serum or plasma), Lot NCP20030123, AllTest Biotech Company Ltd, Hangzhou, China (AllTest)

3

2019-nCoV Ab Test (Colloidal Gold) (serum, plasma, venous whole blood), Lot 20200402, Innovita Biotechnology Company Ltd, Tangshan, China (Innovita)

4

Medical Diagnostech COVID-19 IgG/IgM Rapid test (whole blood, serum or plasma), Lot 200703, Altis Biologics (Pty) Ltd, Pretoria, South Africa (Altis)

5

Cellex qSARS-CoV-2 IgG/IgM Cassette Rapid Test (serum, plasma, whole blood), Lot WI1106C-DH-GZ-20200511, Cellex, Jiangsu, China (Cellex)

A single drop (10-20μL) of whole blood drawn by fingertip puncture was deposited in the sample well of each test cassette. Two to five drops of reagent buffer were then added to the sample well and results read 15-20 minutes later according to the specific manufacturer's instructions. Assays three and five had not previously been validated on finger-prick specimens. Two readers (N.I. and/or J.Z. and/or B.O.) read the cassettes with the naked eye and a third reader (J.V.) settled any disputes. A test was considered valid if a control line was visualized and was interpreted as positive if the control line and the line for IgG or IgM or both was seen. Each of the five kits were evaluated once on each participant with no replication of testing on any participants. All test cassettes and reagents for each of the five different kits were from the same manufacturing batch.

 Statistical analysis

Data were analysed using Prism 8.4.3 (GraphPad Software Inc, La Jolla, California) using standard non-parametric statistical tests as appropriate. Continuous data were expressed as medians with interquartile ranges (IQRs) and categorical variables presented as numbers and percentages. Fisher's exact tests were used to compare results in the age, time and severity groups and Spearman's correlation co-efficients were used to assess agreement between test assays. The multivariate logistic regression analysis was performed using IBM SPSS Statistics 26.0 (IBM Corporation, Armonk, New York).

Time since positive SARS-CoV-2 PCR testWe hypothesized that the dynamic antibody responses during and after COVID-19 may influence the detection of IgG and IgM SARS-CoV-2-specific antibodies.(Zhao, 2020) The association between time since the SARS-CoV-2 PCR test and the detection of IgG and IgM antibodies was explored. There was no difference in IgG sensitivity between those presenting within 30 days of a positive PCR test compared to those presenting after 30 days as shown in Table 2. However, a significantly lower proportion of individuals tested more than 30 days after a positive SARS-CoV-2 PCR test had a detectable IgM response by the AllTest assay (23% (≤30 days) vs 6% (>30 days), p = 0.02, OR 0.2228(0.0644 - 0.7993), Table 3). There was no significant difference in IgM sensitivity in the two time groups for the other four rapid antibody assays (p>0.05 for Dynamiker, Innovita, Altis and Cellex, Table 3).In order to better define the temporal relationship between time elapsed since COVID-19 and the sensitivity of the rapid assays to detect SARS-CoV-2-specific IgG and IgM antibodies, we further stratified individuals by time since positive SARS-CoV-2 PCR test and compared the sensitivity of the assays between these strata (Figure 3). There was great heterogeneity in the sensitivity to detect IgM between assays and no consistent pattern of sensitivity of assays according to time since positive PCR test. Contrasting patterns of sensitivity for IgM by time are illustrated by the Dynamiker and AllTest assays. There was a significantly higher sensitivity for detection of IgM in individuals testing more than 40 days after a positive PCR test when compared to those testing less than 21 days when using the Dynamiker assay (88.9% (>40 days) vs 64% (Figure 3). By contrast, when using the AllTest assay the sensitivity for IgM was lower in the group testing more than 40 days after a positive PCR test (0% (>40 days) vs 24% (Figure 3). For IgG there was a trend towards increasing sensitivity associated with increased time since positive PCR test for all of the rapid assays. This was significant for the Dynamiker and Cellex assays where sensitivities were higher in individuals testing more than 40 days after a positive PCR test when compared to those testing 21-30 days thereafter (Dynamiker: 88.9% (>40 days) vs 60.7% (21-30 days), p=0.049; Cellex: 88.9% (>40 days) vs 60.7% (21-30 days), p=0.049, Figure 3).

Figure 3Bar graphs show the sensitivity of each rapid antibody assay in detecting SARS-CoV-2-specific IgG and IgM antibodies stratified by time since a positive SARS-CoV-2 PCR test (<21 days: n=25; 21-30 days: n=28; 31-40 days: n= 31 and >40 days: n= 18). Significant differences are shown above the relevant columns and P values were derived using Fisher's exact tests.

DISCUSSION

Rapid serological assays are increasingly becoming an essential component of surveillance of outbreaks and retrospective diagnoses of COVID-19. These assays have appeal as they are rapid, inexpensive and user-friendly. However, data evaluating their true clinical performance on finger-prick analyses at the POC, in a real-world clinical context, is urgently needed.

In this present study, tested against the reference gold standard PCR, the sensitivity of five rapid antibody assays ranged from 36% to 69% for IgG and 13% to 67% for IgM. This is significantly lower than that reported in previous validation studies of these rapid tests.(Dynamiker Biotechnology (Tianjin) Co., Ltd., 2020; Mølbak, 2020Performance Evaluation of the 2019-nCOV IgG/IgM Rapid Test From Dynamiker Biotechnology (Tianjin) Co., Ltd. Study Report
Division of Infectious Diseases Preparedness.; Lassaunière, ) The South African Health Products Regulatory Authority (SAHPRA) specifies that a minimum clinical sensitivity of 85% within 95% confidence intervals be proven prior to registration.(SAHPRA, SAHPRA. MD007 2020

SAHPRA. MD007: Specification criteria for COVID-19 serological test kits. 2020; Available at: https://www.sahpra.org.za/wp-content/uploads/2020/07/MD007-Specifications-Serological-Test-kits-v2-22072020.pdf. Accessed 23 January 2021.

) All of these rapid antibody assays fall below that benchmark when used in our cohort. The wide spread of the results of our sensitivity analysis are comparable with findings of existing studies. In a pooled analysis of 1030 POC antibody assays, by Riccò et al, combined sensitivity for IgG and IgM, ranged from 18.4% to 93.3% and with an average of 64.8% (95%CI 54.5–74.0).(Riccò, Riccò et al., 2020Riccò M Ferraro P Gualerzi G et al.Point-of-Care Diagnostic Tests for Detecting SARS-CoV-2 Antibodies: A Systematic Review and Meta-Analysis of Real-World Data.) We also report poor correlation between the results of the five assays in comparison to each other, particularly when testing for IgM. This finding has also been previously described, and prompted Van Elslande et al to question the need for measuring IgM SARS-CoV-2 antibodies at all.(Cassaniti, Cassaniti et al., 2020Cassaniti I Novazzi F Giardina F et al.Performance of VivaDiag COVID-19 IgM/IgG Rapid Test is inadequate for diagnosis of COVID-19 in acute patients referring to emergency room department.; Van Elslande, Van Elslande et al., 2020Van Elslande J Houben E Depypere M et al.Diagnostic performance of seven rapid IgG/IgM antibody tests and the Euroimmun IgA/IgG ELISA in COVID-19 patients.) The heterogenous clinical sensitivity performance of IgM in the antibody assays in our study further suggest limited clinical utility. Given the poor correlation between the Innovita and all other kits, we would not recommend use of this kit on fingerprick blood samples.Our analysis of patient factors associated with an improved sensitivity of the rapid antibody tests revealed an association between age and sensitivity. All of the assays demonstrated improved sensitivities in those patients aged >40 years for IgG and all the assays except the Altis for IgM. The higher sensitivity of these tests in older participants may indicate higher antibody titres in these individuals and therefore a higher likelihood of detection of such antibody responses by the rapid assays. Older patients with COVID-19 have been shown to have higher SARS-CoV-2-specific IgG and IgM antibody titres than younger patients, although the reason for this is unclear.(Huang, Huang et al., 2020Huang M. Lu QB. Zhao H. et al.Temporal antibody responses to SARS-CoV-2 in patients of coronavirus disease 2019.; Wu F, 2020) It is likely that increased SARS-CoV-2-specific antibody titres correlate with more severe COVID-19, and that increased age has been correlated with severe disease and worse outcome.(Zhao, Zhou et al., 2020Clinical course and risk factors for mortality of adult inpatients with COVID-19 in Wuhan, China: a retrospective cohort study.; Zhou, Zhao et al., 2020Zhao J Yuan Q Wang H et al.Antibody Responses to SARS-CoV-2 in Patients With Novel Coronavirus Disease 2019.). When both age and disease severity were included in a multivariate logistic regression analysis, only age was associated with an increased likelihood of a positive rapid IgG or IgM antibody response, thereby suggesting that age may play a role independently of its association with disease severity. Our cohort primarily consisted of participants that had mild COVID-19 with 80% of the cohort in the mild to moderate category. Due to the fact that younger individuals typically experience milder COVID-19 disease and have an increased rate of asymptomatic infection, our findings may suggest a significant impediment to the use of these assays to assess seroprevalence in younger participants.The impact of time since diagnosis of COVID-19 by positive PCR test and the sensitivity of the rapid assays is intriguing. The trend towards increased sensitivity to detect IgG and increased time since positive PCR is as expected and is consistent with previous studies.(Deeks, 2020; Wu JL, 2020) For IgM, the two kits with the highest sensitivity to detect IgM (Dynamiker and Cellex) showed highest sensitivity when tested amongst individuals more than 40 days post PCR test. Although generally believed to be relatively short-lived, SARS-CoV-2-specific IgM antibodies have been shown to persist for up to two and a half months after SARS-CoV-2 infection and therefore may explain this.(Iyer, Iyer et al., 2020 Oct 8Iyer AS Jones FK Nodoushani A Kelly M Becker M Slater D et al.Persistence and decay of human antibody responses to the receptor binding domain of SARS-CoV-2 spike protein in COVID-19 patients.) It is not clear whether poor IgM sensitivity in some of the assays is possibly associated with higher specificity and further assessment, possibly with the use of ELISA assays, would be required to investigate this.There are limitations to this study. Although used as the reference gold standard for the diagnosis of COVID-19, PCR-based testing gives no indication of patient seropositivity. The negative serological results reported here may represent low antibody titres or low participant seroconversion, rather than a failure of the test to detect antibodies. Furthermore, two assays (Innovita and Cellex) that were included have not previously been validated on finger-prick blood samples. Although we consider the effect is likely to be small, as all other tests could be used across blood sample platforms, this may have contributed to their poor performance. Owing to inadequate access to negative control samples, we were not able to perform our own corresponding specificity analysis, however, a Cochrane Diagnostic Test Accuracy Review has shown such rapid serological assays to have very high specificity.(Deeks, Deeks et al., 2020

Deeks JJ, Dinnes J, Takwoingi Y, Davenport C, Spijker R, Taylor-Phillips S, et al. Antibody tests for identification of current and past infection with SARS-CoV-2. Cochrane Database Syst Rev. 2020 Jun 25;6(6):CD013652. doi: 10.1002/14651858.CD013652.

)ACKNOWLEDGEMENTS

None

Funding Statement

First Medical Company, Cape Town; SMD Technologies, Johannesburg and Altis Biologics, Pretoria donated test assays for use in this study but had no role in the study design, data collection, data analysis and/or writing of the manuscript

Meeting(s) where data has been presented

This data has not been previously presented at any meetings or conferences