Clinical diagnostic performance evaluation of five immunoassays for antibodies to SARS-CoV-2 diagnosis in a real-life routine care setting

¹Department of Microbiology, Faculty of Natural and Applied Sciences, Nasarawa State University, Keffi, Nasarawa State, Nigeria, ²Primary Health Care Board, Federal Capital Territory Administration, Abuja, Nigeria, ³Department of Medical Laboratory Science, Federal University of Lafia, Nasarawa, Nigeria, ⁴Medical Laboratory Services Department, Clinical Chemistry Unit, University of Abuja Teaching Hospital, Gwagwalada, Nigeria, ⁵Department of Histopathology, Medical Laboratory Services, University of Abuja Teaching Hospital, Gwagwalada, Nigeria

^&Corresponding author
Adamu Ishaku Akyala, Department of Microbiology, Faculty of Natural and Applied Sciences, Nasarawa State University, Keffi, Nasarawa State, Nigeria

Introduction    

There are several serological tests available for the diagnosis of Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2). Laboratory diagnosis based on reverse transcription-polymerase chain reaction (RT-PCR) remain the gold standard for the rapid diagnosis of acute SARS-CoV-2 infections which is essential for contact tracing and patient management [1]. Several commercial and laboratory developed assays are available but few manufacture-independent evaluations and few comparisons between assays have been published till date [2]. Moreover, the comparison between assays is hampered by the absence of accepted gold standard test as well as our incomplete knowledge of the natural history of SARS-CoV-2 infection [3]. Studies evaluating the concordance between assays are thus needed at this point in the pandemic [4].

Due to the unprecedented pandemic, there has been a quest for an antibody detection testing panel that can detect the virus in blood specimen requiring a sufficient viral load [5]. Over 40 novel SARS-CoV-2-specific antibody testing kits has been developed but there is paucity of information regarding their sensitivity, specificity and kappa level of agreement with the RT-PCR which is the gold standard [6]. There have been huge gaps in the capacity to perform a timely diagnosis using a RT-PCR testing panel and the number of samples in a limited resource setting thus an alternative testing panel as containment of public health strategies especially rapid diagnostic tests (RDTs) which is cost effective, easy to use and adapt to climatic weather and can serve as field base community-based testing panel or point-of-care testing (POCT) is required [7]. This study evaluates the diagnostic performance of five novel antibody-based RDTs for the detection of SARS-CoV-2 in serum and plasma specimens from 134HCW who are positive by RT-PCR to SARS-CoV-2. The sensitivity and specificity of this RDT is compared with RT-PCR as the gold standard.

Methods     

Study design: in our study, we evaluated five lateral flow immunoassays for the detection of SARS-CoV-2 antibodies. Patient serum samples used in this study were submitted to the routine Molecular Laboratory of Nasarawa State Infectious Disease and Research Center (NASIRDAC) in for diagnostic evaluation purposes.

Study period and serum samples: control serum samples (n=134) included archived anonymous serum obtained from healthy blood donors with no history of SARS-CoV-2 infection, between March 1^st and September 2020 (group 1, healthy control). These serum samples were donated to the Nasarawa State Infectious Disease and Research Center (NASIRDAC) in for diagnostic evaluation purposes. Case serum samples were obtained from patients with SARS-CoV-2 infection (n=134) between March 1^st and September 2020 (group 2, patients with RT-PCR-positive and group 3, patients with RT-PCR-negative, “clinically diagnosed”, that means patients with pneumonia, showing clinical and radiographic evidence compatible with COVID-19 according to the 5^th edition of guideline on diagnosis and treatment of the novel coronavirus pneumonia).

Real-time PCR assay: we used three types of automatic extractors to obtain viral RNA from clinical samples, i.e. MagCore HF16 (RBC bioscience, Taipei, Taiwan), Nimbus MicrolabSeegene (Hamilton Company, Bonaduz, Switzerland) and m2000 system (Abbott Molecular Inc. Des Plaines, IL). RNA amplification was made using two real-time PCR platforms, i.e. qCOVID-19 (Genomica, Madrid, Spain) and Allplex 2019-nCoV assay (Seegene, Seoul, South Korea) and we used the CFX96™ (Bio-Rad) real-time detection system. PCR did not have a human extraction control gene target. The extraction control gen target was a phage. These kits were used according to the manufacturer´s instructions for both the handling and the interpretation of the results.

Rapid diagnostic test: SARS-CoV-2 antibody test (lateral flow method) is an immunochromatographic assay used for rapid qualitative detection of IgM/IgG in human whole blood serum or plasma samples against SARS-CoV-2 infection. This is a medical diagnostic test that is easy to perform for preliminary or emergency medical screening of SARS-CoV-2 within 20 minutes. The test was performed according to leaflets-protocol provided from the manufacturer in the test kit packet.

Data analysis: statistical analysis was carried out using the statistical package STATA/IC version 13.1 (StataCorp, TX, USA). Continuous data are expressed as median and IQR, while categorical data were expressed as frequencies and percentages. Comparisons between variables were made using two-tailed Fisher´s exact test or t test. For these comparisons, a p value less than or equal to 0.05 was considered significant. Sensitivity and specificity were calculated by means of Excel (version 16.0, Microsoft, Washington, USA) using the following definitions in Excel: sensitivity = 100 x [true positive/ (true positive + false negative)] specificity = 100 x [true negative/ (true negative + false positive)]. The agreement between the different serological diagnostic techniques was expressed by the Kappa index and percentage of agreement. A Kappa value of more than 0.75 indicates good agreement between tests, while a value of less than 0.4 indicates poor agreement.

Ethical consideration: ethical approval was obtained from the Ethical Review Board of Nasarawa State Ministry of Health. Prior to enrollment in the study, all participants were informed as consent on the objectives and background of the study. Information was provided toward the risks and benefits of the current study. Similarly, a designated questionnaire and data were collected after obtaining returned informed consent. Anonymity and confidentiality of the study participants were maintained.

Results     

There were five testing kits considered for evaluation. Information about the manufacturer, assay commercial name, country of manufacturer, target, volume required, test waiting time, interpretation method and regulatory status of the kits were collected (Table 1). The 2019-nCoV IgG/IgM antibody determination kit, the Tigsun COVID-19 combo IgM/IgG rapid test (lateral flow), rapid response COVID-19 IgG/IgM test cassette (whole blood/serum/plasma), COVID-19 IgM-IgG combined antibody rapid test and iChroma COVID-19 Abtest kits were validated.

All the kits had certified regulatory status, had visual method of interpretation and all used blood sampling targeting immunoglobulins G and M. The kits had a turnaround time (TAT) of a range between 5-20 minutes though the iChroma COVID-19 Ab test kit had the shortest TAT of five minutes and required only a drop of blood (Table 2). The 2019-nCoV IgG/IgM antibody determination kit had a sensitivity of 33%, 100% specificity, accuracy of 68.5% and Kappa coefficient of 0.7 and rapid response COVID-19 IgG/IgM test cassette had a sensitivity of 33%, specificity of 100% and accuracy of 72.5% with Kappa coefficient 0.7. The Tigsun COVID-19 combo IgM/IgG rapid test (lateral flow), positive, COVID-19 IgM-IgG combined antibody rapid test and iChroma COVID-19 AbRT all had sensitivity of zero percent (Table 2).

Discussion     

Evidence shows that there are enormous challenges in diagnostic approaches that require rapid and accurate identification of cases of SARS-CoV-2 infections and asymptomatic cases, however, enormous constraints in diagnostics that can rapidly and accurately identify infected persons exist in low resource settings. Approaches that can detect disease progression in order to classify patients for appropriate care and that can thereby prevent exacerbation of the disease, have been recommended [8].

Diagnostic testing is a major component of outbreak detection and emergency response. At the beginning of the coronavirus outbreak globally, there was pressure to expand to an effective response to the testing capacity for the novel coronavirus [8]. In some countries there was a need to get clearance before test kits could be shipped in deployed and approved for use [9]. As the public health institute coordinating the coronavirus pandemic response in Nigeria, the Nigeria Centre for Disease Control (NCDC) set up and scaled up diagnostic capacity for testing across the country.

As part of the scaling up it was important to use best practices for timely detection of the virus causing the pandemic. The gold standard for SARS-CoV-2 detection is RT-PCR, however, to scale up testing capacity with a good TAT, it was necessary to incorporate use of rapid test kits which could be adopted to increase coverage. Based on the results of this study, the 2019-nCoV IgG/IgM antibody determination kit and the rapid response COVID-19 IgG/IgM test cassette had low sensitivity, optimal sensitivity and above average accuracy compared to RT-PCR. Though the COVID-19 IgM-IgG combined antibody rapid test had fair sensitivity, above average accuracy and low sensitivity alongside the other two kits with low specificity. These screening test kits´ sensitivity were much lower than recently conducted validation studies in Sweden and China [10]. Though the optimal sensitivity specificity was the same for the study in Sweden and higher than the studies in China, the test kits however, this study did not differentiate for IgG or IgM sensitivity separately, which was the case in the European and Chinese [10].

In another study in China where a combined IgG and IgM test kit was assessed and validated, the test´ sensitivity was more than two times higher than the sensitivity values in this study, accuracy was lower in this study compared to the previous study in China specificity was however higher in this study [10]. This study validated five combined IgG/IgM antibody-based detection tests and compared them to RT-PCR. With the overwhelming number of SARS-CoV-2 cases and potentially large numbers of asymptomatic cases, the use of RDT Kits with high sensitivity, specificity and accuracy is essential to detecting and tackling SARS-CoV-2 infections.

Conclusion     

The huge impact of the SARS-CoV-2 emergence in public health justifies extensive sero-epidemiological studies to survey its spread in various populations and numerous settings. There is a burst of serologic assays rolling out in different formats, including simple rapid tests. Our study shows that specificity may be highly variable among available immunoassays for antibody to SARS-CoV-2. Poor specificity of an assay in a population where prevalence and incidence of SARS-CoV-2 are low will lead to irrelevant data. Our study, as others, stresses on the absolute necessity to use only carefully validated assays to provide epidemiological data useful to public health decision makers.

Limitations: the data analyzed in this study is for a sub-population and may influence the generalizability of this results, compared in a larger population.

Competing interests     

The authors declare no competing interests.

Authors' contributions     

AIA conceptualized and designed the study; AIA and JRA analyzed the data; ACO revised the manuscript for intellectual and scientific content and developed the results and discussion section; OMAB and AOGE reviewed and revised the manuscript. All the authors have read and agreed to the final manuscript.

Acknowledgments     

We thank Dr Liman Mohammad of the Department of Epidemiology, Nasarawa State Ministry of Health, Lafia, Nigeria, for the data access and the Nigeria Center for Disease Control for laboratory diagnosis of the novel SARS-CoV-2.

Tables     

Table 1: characteristics of five commercially available SARS-CoV-2 antibodies (Ab)-based-detection tests in Nigeria (n=134)

Table 2: performance evaluation of five SARS-CoV-2 antibodies (Ab)-based-detection tests compared to RT-PCR in Nasarawa State, Nigeria

References