Article

Original Article

Ann Lab Med 2022; 42(1): 71-78

Published online January 1, 2022 https://doi.org/10.3343/alm.2022.42.1.71

Copyright © Korean Society for Laboratory Medicine.

Performance Comparison of Five SARS-CoV-2 Antibody Assays for Seroprevalence Studies

Younhee Park, M.D.1,* , Ki Ho Hong, M.D.1,*,† , Su-Kyung Lee, M.S.2 , Jungwon Hyun, M.D.2 , Eun-Jee Oh, M.D.3 , Jaehyeon Lee, M.D.4 , Hyukmin Lee, M.D.1 , Sang Hoon Song, M.D.5 , Seung-Jung Kee, M.D.6 , Gye Cheol Kwon, M.D.7 , Su Hwan Kim, B.S.8 , Hyeon-Nam Do, M.S.8 , Ah-Ra Kim, M.S.8 , June-Woo Lee, Ph.D.8 , Sung Soon Kim, Ph.D.8 , and Hyun Soo Kim, M.D., Ph.D.2

1Department of Laboratory Medicine, Severance Hospital, Yonsei University College of Medicine, Seoul, Korea; 2Department of Laboratory Medicine, Hallym University Dongtan Sacred Heart Hospital, Hallym University College of Medicine, Hwaseong, Korea; 3Department of Laboratory Medicine, Seoul St. Mary’s Hospital, College of Medicine, The Catholic University of Korea, Seoul, Korea; 4Department of Laboratory Medicine, Jeonbuk National University Medical School and Hospital, Jeonju, Korea; 5Department of Laboratory Medicine, Seoul National University Hospital and College of Medicine, Seoul, Korea; 6Department of Laboratory Medicine, Chonnam National University Medical School and Hospital, Gwangju, Korea; 7Department of Laboratory Medicine, College of Medicine, Chungnam National University, Daejeon, Korea; 8Center for Vaccine Research, National Institute of Infectious Diseases, National Institute of Health, Korea Disease Control and Prevention Agency, Osong, Korea

Correspondence to: Hyun Soo Kim, M.D., Ph.D.
Department of Laboratory Medicine,
Hallym University Dongtan Sacred Heart Hospital, Hallym University College of Medicine, 7 Keunjaebong-gil, Hwaseong 18450, Korea
Tel: +82-31-8086-2775
Fax: +82-31-8086-2789
E-mail: hskim0901@empas.com

* These two authors contributed equally to this work.

At the time of the experiment, the affiliation of Ki Ho Hong was the Department of Laboratory Medicine, Seoul Medical Center, Seoul, Korea

Received: March 5, 2021; Revised: April 12, 2021; Accepted: July 25, 2021

This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

Background: Seroprevalence studies of coronavirus disease 2019 (COVID-19) cases, including asymptomatic and past infections, are important to estimate the scale of the disease outbreak and to establish quarantine measures. We evaluated the clinical performance of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) antibody assays available in Korea for use in seroprevalence studies.
Methods: The sensitivity, specificity, cross-reactivity, and interference of five SARS-CoV-2 antibody assays were evaluated using the following: 398 serum samples from confirmed COVID-19 patients, 510 negative control samples from before 2018 (pre-pandemic), 163 serum samples from patients with SARS, Middle East respiratory syndrome (MERS), and other viral infections, and five samples for the interference study.
Results: The sensitivities of the five assays ranged from 92.2% to 98%, and their specificities, including cross-reactivity and interference, ranged from 97.5% to 100%. The agreement rates were excellent (kappa >0.9). Adjustment of the cutoff values could be considered through ROC curve analysis. The positive predictive values of the individual assays varied from 3.5% to 100% at a 0.1% prevalence but were as high as ≥95% when two assays were combined.
Conclusions: The prevalence of COVID-19 in Korea is considered to be exceptionally low at present; thus, we recommend using a combination of two or more SARS-CoV-2 antibody assays rather than a single assay. These results could help select SARS-CoV-2 antibody assays for COVID-19 seroprevalence studies in Korea.

Keywords: COVID-19, SARS-CoV-2, Antibody, Seroprevalence

Coronavirus disease 2019 (COVID-19), which originated in Wuhan, China in December 2019, is caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) [1]. More than 100 million people have been infected with SARS-CoV-2 and more than two million deaths due to COVID-19 have been reported worldwide in approximately one year [2]. The number of patients with confirmed disease includes only those who have been tested positive for SARS-CoV-2 following a hospital visit [3]. Therefore, the actual number of COVID-19 positive cases has been underestimated. To determine the size of the infected population and to establish quarantine measures, accurate serological testing is required.

Seroprevalence studies have been conducted in many countries, including the United States, the United Kingdom, Spain, and Korea [4-8]. In less than a year, several types of antibody assays have been developed worldwide. However, comparative studies on the performance of assays available in Korea to determine seroprevalence have not yet been conducted. The available antibody assays mainly use recombinant spike (S) proteins, nucleocapsid (N) proteins, receptor-binding domains, S1 antigens, and combinations of these antigens to detect IgG, IgM, and total antibody levels [9-16].

We evaluated the clinical performance of COVID-19 antibody assays available in Korea for seroprevalence studies. We further estimated the positive predictive values (PPVs) of individual and two combined assays using the sensitivities and specificities obtained from this study and the expected prevalence in Korea. We also investigated cross-reactivity using serum samples from patients with antibodies to various viruses and bacteria, autoimmune disease, or monoclonal gammopathy.

Clinical samples

Serum samples, leftover from laboratory tests and designated to be discarded, from 398 patients diagnosed as having COVID-19 at two hospitals (Seoul Medical Center, Seoul, Korea and Hallym University Dongtan Sacred Heart Hospital, Hwaseong, Korea) and the Korea Disease Control and Prevention Agency (KDCA) were collected between March and September 2020 and stored at –70°C until analysis. The dates of symptom onset and hospital admission were obtained retrospectively from the medical records at the two hospitals. Serum samples of 510 negative controls, collected before 2018 (pre-pandemic period), were obtained from the National Biobank of Korea, the KDCA, and the High-Risk Human Serum Bank of Chung-Ang University (Seoul, Korea). A total of 168 samples were tested for cross-reactivity, including 136 residual serum samples of patients with antibodies to other viruses (human (h)CoV-229E, -NL63, -OC43, and -HKU1; adenovirus; influenza A virus; influenza B virus; human metapneumovirus; parainfluenza virus type 1/2/3/4; respiratory syncytial virus; rhinovirus; Mycoplasma pneumoniae; Chlamydia pneumoniae; hepatitis A/B/C virus; Epstein-Barr virus; cytomegalovirus; herpes simplex virus; mumps virus; and varicella zoster virus) collected from two hospitals (Severance Hospital, Seoul, Korea and Hallym University Dongtan Sacred Heart Hospital). Serum samples positive for fluorescence anti-nuclear antibody, rheumatoid factor, and monoclonal gammopathy were collected from the same two hospitals.

Twenty-three serum samples positive for antibodies to SARS-CoV-1, Middle East respiratory syndrome (MERS)-CoV, hCoV-229E, -NL63, -OC43, and -HKU1, influenza B virus, respiratory syncytial virus, and adenovirus were purchased from Trina Bioreactives (Nänikon, Switzerland). Five serum samples from patients with monoclonal gammopathy were collected from the Korea Institute of Radiological and Medical Sciences Radiation Biobank (Seoul, Korea), and four MERS-CoV convalescent serum samples were provided by the International Vaccine Institute (Seoul, Korea) (Fig. 1). The Public Institutional Review Board designated by the Ministry of Health and Welfare (http://public.irb.or.kr/) approved the study protocol and waived the need for informed consent (P01-202008-31-002).

Figure 1. Samples tested in this study.
Abbreviations: CAU, Chung-Ang University; C. pneumoniae, Chlamydia pneumoniae; CMV, cytomegalovirus; EBV, Epstein-Barr virus; FANA, fluorescence anti-nuclear antibody; HAV, hepatitis A virus; HCV, hepatitis C virus; hCoV, human coronavirus; HSV, herpes simplex virus; HUDSHH, Hallym University Dongtan Sacred Heart Hospital; IVI, International Vaccine Institute; KDCA, Korea Disease Control and Prevention Agency; KIRMS, Korea Institute of Radiological and Medical Sciences; MERS-CoV, Middle East respiratory syndrome coronavirus; M. pneumoniae, Mycoplasma pneumoniae; PIV, parainfluenza virus; RSV, respiratory syncytial virus; SARS-CoV-1, severe respiratory syndrome coronavirus 1; SH, Severance Hospital; SMC, Seoul Medical Center; VZV, varicella zoster virus.

SARS-CoV-2 antibody assays

In August 2020, the KDCA received applications from 18 manufacturers to evaluate the performance of 21 SARS-CoV-2 antibody assays. It was not feasible to evaluate all 21 assays given the limited amount of serum samples; hence, a screening evaluation was required. After the Evaluation Committee did a preliminary assessment of the data submitted by the manufacturers, five assays were selected for performance evaluation: Elecsys Anti-SARS-CoV-2 assay on the Cobas e801 platform (Roche Diagnostics, Mannheim, Germany), Abbott Architect SARS-CoV-2 IgG assay on the Architect i2000SR platform (Abbott Laboratories, Abbott Park, IL, USA), Atellica IM SARS-CoV-2 Total assay on the Atellica platform (Siemens, Munich, Germany), SD Biosensor Standard E COVID-19 Total Ab assay (SD Biosensor, Suwon, Korea), and LG Chem AdvanSure SARS-CoV-2 IgG assay (LG Chem, Seoul, Korea). SD Biosensor and LG Chem ELISAs were performed using the Epoch Microplate Spectrophotometer and ELx50 Filter Microplate Washer (both from BioTek Instruments, Winooski, VT, USA). The principle, instrument, antibody detected, reagents used, cutoff value, sample volumes, and time to first results of all assays are listed in Table 1. All assays were carried out according to the manufacturers’ instructions. Most assays were performed at Hallym University Dongtan Sacred Heart Hospital by one laboratory technician and one scientific researcher, and the Atellica IM SARS-CoV-2 total assay was performed at Severance Hospital by one laboratory technician.

Table 1 . Characteristics and sensitivity and specificity of the five SARS-CoV-2 antibody assays evaluated in this study

Roche (Roche Diagnostics, Mannheim, Germany)Abbott (Abbot Laboratories, Abbott Park, IL, USA)Siemens (Siemens, Munich, Germany)SD Biosensor (SD Biosensor, Suwon, Korea)LG Chem (LG Chem, Seoul, Korea)
Characteristics of the assays
Product nameElecsys anti-SARS-CoV-2SARS-CoV-2 IgGSARS-CoV-2 total (COV2T)STANDARD E COVID-19 Total AbAdvanSure SARS-CoV-2 IgG (S1)
AnalyzerElecsys Cobas e801Architect i2000SRAtellica IMELISAELISA
PrincipleECLIACMIACLIAELISAELISA
Target antibodyTotal (IgG+IgM)IgGTotal (IgG+IgM)Total (IgG+IgM)IgG
Used reagent antigenNucleoproteinNucleoproteinRBDSpike+NucleoproteinS1
Sample typeSerum, plasmaSerum, plasmaSerum, plasmaSerum, plasmaSerum, plasma
Sample volume20 µL25 µL50 µL50 µL10 µL
Cut-off value (unit)1.0 (COI)1.4 (index)1.0 (index)NC+0.3 (OD)1.0 (S/CO)
Time to first result (min)182915150150
Sensitivity and specificity of the assays according to days after symptom onset (or admission)
1–7 (N = 13)15.4% (2/13)15.4% (2/13)15.4% (2/13)53.8% (7/13)46.2% (6/13)
8–14 (N = 65)86.2% (56/65)86.2% (56/65)95.4% (62/65)98.5% (64/65)92.3% (61/65)
15–21 (N = 84)98.8% (83/84)98.8% (83/84)98.8% (83/84)100% (84/84)100% (84/84)
22–28 (N = 43)100% (43/43)100% (43/43)100% (43/43)100% (43/43)100% (43/43)
> 28 (N = 97)96.9% (93/96)94.8% (92/97)98.9% (95/96)98.9% (95/96)98.9% (95/96)
Unknown (N = 96)97.9% (94/96)93.8% (90/96)97.9% (94/96)100% (96/96)100% (96/96)
Total sensitivity (N = 398)93.5% (371/397, 95% CI 90.6–95.7)92.2% (367/398, 95% CI 90.0–95.3)95.7% (380/397, 95% CI 93.2–97.5)98.0% (389/397, 95% CI 96.1–99.1)97.0% (385/397, 95% CI 94.5–98.2)
Total specificity
Pre-pandemic controls (N = 510)99.6% (508/510)99.2% (506/510)100% (510/510)99.6% (508/510)97.2% (496/510)
Including cross-reactivity and interference (N = 683)99.7% (681/683, 95% CI 98.9–100)99.4% (679/683, 95% CI 98.5–99.8)100% (683/683, 95% CI 99.5–100)99.3% (678/683, 95% CI 98.3–99.8)97.5% (666/683, 95% CI 95.9–98.4)

Abbreviations: CMIA, chemiluminescence microparticle immunoassay; CLIA, chemiluminescence immunoassay; CI, confidence interval; COI, cutoff index; ECLIA, electrochemiluminescence immunoassay; ELISA, enzyme-linked immunosorbent assay; NC, negative control; OD, optical density; RBD, receptor-binding domain; SARS-CoV-2, severe acute respiratory syndrome coronavirus 2.



Performance evaluation of the five SARS-CoV-2 antibody assays

Sensitivity and specificity

The diagnostic sensitivity and specificity (and the 95% confidence intervals) of the five assays were calculated according to days after symptom onset using MedCalc software version 19.8 (MedCalc Software, Ostend, Belgium). The date of symptom onset was not available for some patients.

Cross-reactivity and interference

The potential cross-reactivity with other coronaviruses, such as SARS-CoV-1, MERS, respiratory viruses, and other viruses, bacteria, autoimmune diseases, and monoclonal gammopathy was evaluated using patient serum samples that were positive for antibodies against the relevant pathogen. For interference testing, Hb (10 mg/mL), bilirubin (0.4 ng/mL), a triglyceride mix (20 mg/mL), or zanamivir (10 mg/mL; all from Sigma-Aldrich) were spiked into serum samples that were negative for these antibodies. The results of spiked samples were compared with those of non-spiked samples and considered acceptable if the difference was <15%.

Agreement and correlation

The agreement and correlations between the results of the five SARS-CoV-2 antibody assays were calculated using MedCalc software and were reported as correlation graphs. For the Siemens Atellica IM SARS-CoV-2 Total assay, antibody titers <0.05 were considered 0, and those >10 were considered 10. The upper limits of detection values were determined to be 4.0 for the SD Biosensor assay and 4.5 for the LG Chem assay. Positive, negative, and total agreement between assays were evaluated using Cohen’s kappa statistics and categorized as poor (<0.00), slight (0.00–0.20), fair (0.21–0.40), moderate (0.41–0.60), substantial (0.61–0.80), or nearly perfect (0.81–1.00).

ROC curve analysis

ROC curve analyses and areas under the ROC curve (AUC) calculations for the five assays were performed using the MedCalc software. P≤0.05 was considered statistically significant. For each ROC curve, we recalculated the cutoff values so that the Youden index (=sensitivity+specificity–1) was maximized, and the sensitivity and specificity were determined from the calculated cutoff values.

PPVs and negative predictive values (NPVs) for individual and two combined assays using varying seroprevalence settings

PPVs and NPVs of individual and two combined assays using the calculated sensitivity and specificity values and the predictive seroprevalence of 10%, 5%, 2%, 1%, and 0.1% were calculated using a US Food and Drug administration online calculator [17].

Sensitivity and specificity

The diagnostic sensitivities were 93.5% for the Roche assay, 92.2% for the Abbott assay, 95.7% for the Siemens assay, 98.0% for the SD Biosensor assay, and 97.0% for the LG Chem assay. The diagnostic specificities were 99.7% for the Roche assay, 99.4% for the Abbott assay, 100% for the Siemens assay, 99.3% for the SD Biosensor assay, and 97.5% for the LG Chem assay (Table 1). The positivity rate was low in the first week after symptom onset but was nearly 100% three to four weeks after symptom onset. The SD Biosensor assay showed the highest sensitivity and the Siemens assay showed the highest specificity.

Cross-reactivity and interference

The Abbott, Roche, and Siemens assays showed negative cross-reactivity for 33 species of viruses and bacteria in 168 positive serum samples (Supplemental Data Table S1). The SD Biosensor and LG Chem assays each showed three false-positive results in the cross-reactivity study. No samples showed interference for Hb, a triglyceride mix, bilirubin, albumin, or zanamivir.

Agreement and correlation

The agreement among all assays was nearly perfect (kappa>0.90) (Table 2); however, correlations among the assays were non-linear (Supplemental Data Fig. S1), and several samples exceeded the upper limit of detection in the Siemens, SD Biosensor, and LG Chem assays. The Siemens and SD Biosensor assays showed the highest agreement rate at 98.7% (95% CI, 97.8–99.3; kappa, 0.966). Discordant results among the assays are shown in Supplemental Data Table S2.

Table 2 . Agreement rates between the five SARS-CoV-2 antibody assays

A/BRoche/ AbbottRoche/ SiemensRoche/SD BiosensorRoche/ LG ChemAbbott/ SiemensAbbott/SD BiosensorAbbott/LGSiemens/ SD BiosensorSiemens/LG ChemSD Biosensor/ LG Chem
Positive/Positive (total number)364368371371363366366380378385
Positive/Negative (total number)95227440217
Negative/Positive (total number)612233117283614249
Negative/Negative (total number)700694683675692682674685675669
Positive agreement of A to B (%) (95% CI)98.4 (96.5–99.4)96.8 (94.5–98.4)94.2 (91.4–96.3)92. 3 (78.2–94.7)95.5 (92.9–97.4)92.9 (89.9–95.2)91.0 (87.8–93.6)96.5 (94.1–98.0)94.0 (91.2–96.1)97.8 (95.7-99.0)
Negative agreement of A to B (%) (95% CI)98.7 (97.6–99.4)99.3 (98.3–99.8)99.7 (98.9–100)99.7 (98.9–100)99.0 (97.9–99.6)99.4 (98.5–99.8)99.4 (98.5–99.8)100 (99.5–100)99.7 (98.9–100)97.5 (96.1-98.6)
Positive agreement of B to A (%) (95% CI)97.6 (95.5–98.9)98.7 (96.9–99.6)99.5 (98.1–99.9)99.5 (98.1–99.9)98.1 (96.1–99.2)98.9 (97.3–99.7)98.9 (97.3–99.7)100 (99.0-100)99.5 (98.1–99.9)95.8 (93.3-97.5)
Negative agreement of B to A (%) (95% CI)99.2 (98.2–99.7)98.3 (97.1–99.1)96.7 (95.2–97.9)95.6 (93.8–97.0)97.6 (96.2–98.6)96.1 (94.4–97.4)94.9 (93.0–96.4)98.0 (96.7–98.9)96.6 (94.9–97.8)98.7 (97.5-99.4)
Total agreement (%) (95% CI)98.6 (97.7–99.2)98.4 (97.5–99.1)97.7 (96.6–98.5)96.9 (95.7–97.9)97.8 (96.7–98.6)97.0 (95.8–98.0)96.3 (95.0–97.3)98.7 (97.8–99.3)97.6 (96.5–98.4)97.6 (96.5-98.4)
Kappa0.9610.9620.9460.9300.9430.9340.9180.9660.9420.949

Abbreviations: CI, confidence interval; SARS-CoV-2, severe acute respiratory syndrome coronavirus 2.



ROC curve and cutoff analysis

The AUCs calculated from ROC curve analysis were 0.976, 0.987, 0.984, 0.994, and 0.987 for the Roche, Abbott, Siemens, SD Biosensor, and LG Chem assays, respectively (Table 3). The optimal cutoff values, calculated using the Youden index method, were 0.188 cutoff index (COI) for the Roche assay, 0.44 index for the Abbott assay, 0.57 index for the Siemens assay, 0.404 OD for the SD Biosensor assay, and 1.163 S/CO (signal/cutoff) for the LG Chem assay, which differed from the original manufacturers’ cutoff values. Using the calculated cutoff values, the sensitivities of the Roche, Abbot, and Siemens assays increased (Roche, from 93.5% to 96.5%; Abbott, from 92.2% to 96.2%; Siemens, from 95.7% to 96.7%), whereas the specificities decreased (Roche, from 99.7% to 98.1%; Abbott, from 99.4% to 99.0%; Siemens, from 100% to 99.6%). By contrast, the sensitivities and specificities of the SD Biosensor and LG Chem assays did not significantly differ when using the manufacturers’ or calculated optimal cutoff values (Table 3).

Table 3 . ROC curve analysis and calculated cutoff values for the five SARS-CoV-2 antibody assays

RocheAbbottSiemensSD BiosensorLG Chem
AUC (P)0.976 (P < 0.001)0.987 (P < 0.001)0.984 (P < 0.001)0.994 (P < 0.001)0.987 (P < 0.001)
Manufacturer’s cutoff1.0 COI1.4 index1.0 index(NC+0.3) OD1.0 S/CO
Sensitivity % (95% CI) according to the manufacturer’s cutoff93.5 (90.6–95.7)92.2 (90.0–95.3)95.7 (93.2–97.5)98.0 (96.1–99.1)97.0 (94.5–98.2)
Specificity % (95% CI) according to the manufacturer’s cutoff99.7 (98.9–100)99.4 (98.5–99.8)100 (99.5–100)99.3 (98.3–99.8)97.5 (95.9–98.4)
Cutoff calculated based on the Youden index0.19 COI0.44 index0.57 index0.40 OD1.16 S/CO
Sensitivity % (95% CI) according to the calculated cutoff96.5 (94.2–98.1)96.2 (93.9–97.9)96.7 (94.5–98.2)97.7 (95.7–99.0)96.7 (94.5–98.2)
Specificity % (95% CI) according to the calculated cutoff98.1 (96.8–99.0)99.0 (97.9–99.6)99.6 (98.7–99.9)99.4 (98.5–99.8)98.0 (96.6–98.9)

Abbreviations: AUC, area under the curve; COI, cutoff index; NC, negative control; OD, optical density; S/CO, signal/cutoff; CI, confidence interval; SARS-CoV-2, severe acute respiratory syndrome coronavirus 2.



PPVs and NPVs for individual and two combined assays using determined sensitivity, specificity, and seroprevalence

The lower the prevalence rate (from 10% to 0.1%), the lower is the PPV. The Siemens assay showed the highest specificity of 100% (95.2%; PPVs calculated using the lowest value of the 95% CI of the calculated specificity are shown in parentheses because the specificity was calculated as 100%, even at the lowest prevalence rate) among the five assays at a 10% prevalence and the highest specificity of 100% (15.1%) at a 0.1% prevalence (Table 4). When the predicted prevalence rate of 0.1% in Korea was considered, the PPV was as low as 24.2% for the Roche assay and 13.7% for the Abbott assay. However, the PPVs increased when two assays were both positive or when the orthogonal test algorithm (i.e., employing two assays in sequence when the first assay yields a positive result) [17] was used (Table 4).

Table 4 . PPVs when one assay or two combined assays were positive for five exemplary populations with 10%, 5%, 2%, 1%, and 0.1% SARS-CoV-2 prevalence

PPV (%) when one assay or two combined assays are both positive*

SARS-CoV-2 10% prevalenceSARS-CoV-2 5% prevalenceSARS-CoV-2 2% prevalenceSARS-CoV-2 1% prevalenceSARS-CoV-2 0.1% prevalence
Roche97.3 (90.8)94.4 (82.4)86.7 (64.4)76.3 (47.2)24.2 (8.2)
Abbott94.6 (87.4)89.3 (76.6)76.0 (56.0)61.6 (38.6)13.7 (5.9)
Siemens100 (95.2)100 (90.3)100 (78.4)100 (64.2)100 (15.1)
SD Biosensor93.7 (86.5)87.6 (75.2)73.2 (54.1)57.5 (36.8)10.8 (5.5)
LG Chem80.3 (72.2)65.9 (55.2)42.8 (32.3)27.0 (19.1)3.5 (2.3)
Roche+Abbott100 (99.8)100 (99.7)99.9 (99.1)99.8 (98.2)98.1 (84.7)
Roche+Siemens100 (99.9)100 (99.9)100 (99.7)100 (99.4)100 (94.0)
Roche+SD Biosensor100 (99.8)100 (99.6)99.9 (99.1)99.8 (98.1)97.7 (83.6)
Abbott+Siemens100 (99.9)100 (99.8)100 (99.6)100 (99.1)100 (91.7)
Abbott+SD Biosensor100 (99.7)99.9 (99.5)99.8 (98.7)99.5 (97.3)95.5 (78.2)
Siemens+SD Biosensor100 (99.9)100 (99.8)100 (99.5)100 (99.0)100 (91.1)

*Calculated using online calculators on the US Food and Drug administration [13]; PPVs calculated using the lowest value of the 95% CI of the calculated specificity are shown in parentheses.

Abbreviations: CI, confidence interval; PPV, positive predictive particle; SARS-CoV-2, severe acute respiratory syndrome coronavirus 2.


The sensitivities of the five assays were all >93%, and the sensitivities were low in the first week after symptom onset but were nearly 100% three to four weeks after symptom onset. These results are similar to or better than those in previous studies [10, 12-20]. In a previous study, some SARS-CoV-2 antibody assays showed a sensitivity of 100%; however, that study used only samples collected two weeks after symptom onset [17]. The assay sensitivity may vary across studies depending on how many of the samples that are collected in the first week after symptom onset (especially on days 0–3) are included. In this study, the SD Biosensor and LG Chem assays had higher sensitivity but lower specificity than the Roche, Abbott, and Siemens assays. The Roche, Abbott, and Siemens assays use high cutoff values to increase specificity, according to the CDC guidelines [21]. The sensitivity and specificity and the cutoff may vary depending on the purpose of the assay. If the assay is used for diagnostic screening, a high sensitivity is preferred. However, the specificity should be high when the assay is used to investigate seroprevalence or the effect of vaccination. Laboratories can adjust assay cutoff values according to the intended purpose.

We extensively investigated cross-reactivity using serum samples of patients with antibodies to various viruses and bacteria, autoimmune disease, or monoclonal gammopathy. The Roche, Abbott, and Siemens assays showed negative results for 163 samples tested for cross-reactivity, whereas the SD Biosensor and LG Chem assays showed false-positive results in three out of the 163 samples. These false-positive results seemed to be due to nonspecific reactions as only one sample yielded a positive result in each species. We found no cross-reactivity in serum with anti-SARS-CoV-1 antibody, which is considered the most similar to the anti-SARS-CoV-2 antibody, or in hypergammaglobulinemia serum.

This study was the first to estimate the PPVs of individual and two combined assays based on the orthogonal test algorithm, using the sensitivities and specificities calculated in this study and the expected prevalence in Korea. As the prevalence in Korea is expected to be <0.1% at present, if only one assay was used to determine the seroprevalence, the PPV was as low as 24.2% (8.2%) for the Roche assay, 13.7% (5.9%) for the Abbott assay, 100% (15.1%) for the Siemens assay, 57.5% (10.8%) for the SD Biosensor assay, and 3.5% (2.3%) for the LG Chem assay (Table 4). When two assays were combined, the PPV increased to >95% for all combinations.

We evaluated the performances of SARS-CoV-2 antibody assays available in Korea during September 2020, and the assay reagents were still in development during this period; hence, the sensitivity and specificity of all assays could be improved, and the cutoff values could be adjusted. Presently, other SARS-CoV-2 antibody assays have been developed and are available in Korea.

One limitation of this study is that neutralizing antibody assays that may represent personal immunity were not included [16]. Because most vaccines use the S protein as an immunogen, SARS-CoV-2 antibody assays targeting the N protein, such as the Abbott and Roche assays evaluated in this study, would not be useful to evaluate the vaccine response. However, the use of combinations of assays targeting the N and S proteins may allow discriminating natural infections from vaccinations.

In summary, this study was performed to select appropriate SARS-CoV-2 antibody assays for implementation in a large-scale seroprevalence study in Korea. We estimated PPVs of individual and two combined assays based on the orthogonal test algorithm, using calculated sensitivity and specificity and expected prevalence of COVID-19 in Korea. Because the prevalence of COVID-19 in Korea is considered exceptionally low at present, we recommend using a combination of two or more assays rather than a single assay.

We appreciate Soyeon Yun for excellent technical support and data collection. The negative control samples used in this study were provided by the National Biobank of Korea, KDCA, and High-Risk Human Serum Bank (NRF-2017M3A9B8069581) at Chung-Ang University. The monoclonal gammopathy samples used in this study were provided by the Korea Institute of Radiological and Medical Sciences Radiation Biobank. MERS-CoV convalescent serum samples were kindly provided by Dr. Manki Song (International Vaccine Institute, Korea). Serum collection was funded by the Coalition for Epidemic Preparedness Innovations.


All authors have accepted responsibility for the entire content of this submitted manuscript and approved the submission. Kim HS designed the study, analyzed the data, and wrote the manuscript. Park Y and Hong KH analyzed the data and wrote the manuscript. Lee SK, Kim SH, Do HN, and Kim AR performed the assays and helped analyze the data. Hyun J, Oh EJ, Lee J, Lee H, Song SH, Kee SJ, Kwon GC, Lee JW, and Kim SS helped analyze the data and review the manuscript.

  1. Zhu N, Zhang D, Wang W, Li X, Yang B, Song J, et al. A novel coronavirus from patients with pneumonia in China, 2019. N Engl J Med 2020;382:727-33.
    Pubmed KoreaMed CrossRef
  2. World Health Organization. Coronavirus disease (COVID-19) pandemic. https://www.who.int/emergencies/diseases/novel-coronavirus-2019 (Updated on Jan 2021).
  3. Gao Z, Xu Y, Sun C, Wang X, Guo Y, Ma K. A systemic review of asymptomatic infections with COVID-19. J Microbiol Immunol Infect 2020;54:12-6.
    CrossRef
  4. Anand S, Montez-Rath M, Han J, Bozeman J, Kerschmann R, Beyer P, et al. Prevalence of SARS-CoV-2 antibodies in a large nationwide sample of patients on dialysis in the USA: a cross-sectional study. Lancet 2020;396:1335-44.
    Pubmed KoreaMed CrossRef
  5. Clarke C, Prendecki M, Dhutia A, Ali MA, Sajjad H, Shivakumar O, et al. High prevalence of asymptomatic COVID-19 infection in hemodialysis patients detected using serologic screening. J Am Soc Nephrol 2020;31:1969-75.
    Pubmed KoreaMed CrossRef
  6. Pollán M, Pérez-Gómez B, Pastor-Barriuso R, Oteo J, Hernán MA, Pérez-Olmeda M, et al. Prevalence of SARS-CoV-2 in Spain (ENE-COVID): a nationwide, population-based seroepidemiological study. Lancet 2020;396:535-44.
    Pubmed KoreaMed CrossRef
  7. Noh JY, Seo YB, Yoon JG, Seong H, Hyun H, Lee J, et al. Seroprevalence of anti-SARS-CoV-2 antibodies among outpatients in Southwestern Seoul, Korea. J Korean Med Sci 2020;35:e311.
    Pubmed KoreaMed CrossRef
  8. Song SK, Lee DH, Nam JH, Kim KT, Do JS, Kang DW, et al. IgG Seroprevalence of COVID-19 among individuals without a history of the coronavirus disease infection in Daegu, Korea. J Korean Med Sci 2020;35:e269.
    Pubmed KoreaMed CrossRef
  9. Cheng MP, Yansouni CP, Basta NE, Desjardins M, Kanjilal S, Paquette K, et al. Serodiagnostics for severe acute respiratory syndrome-related coronavirus 2: a narrative review. Ann Intern Med 2020;173:450-60.
    Pubmed KoreaMed CrossRef
  10. Deeks JJ, Dinnes J, Takwoingi Y, Davenport C, Spijker R, Taylor-Phillips S, et al. Antibody tests for identification of current and past infection with SARS-CoV-2. Cochrane Database Syst Rev 2020;6:CD013652.
    Pubmed KoreaMed CrossRef
  11. Kontou PI, Braliou GG, Dimou NL, Nikolopoulos G, Bagos PG. Antibody tests in detecting SARS-CoV-2 infection: a meta-analysis. Diagnostics (Basel) 2020;10:319.
    Pubmed KoreaMed CrossRef
  12. Mekonnen D, Mengist HM, Derbie A, Nibret E, Munshea A, He H, et al. Diagnostic accuracy of serological tests and kinetics of severe acute respiratory syndrome coronavirus 2 antibody: a systematic review and meta-analysis. Rev Med Virol 2021;31:e2181.
    Pubmed CrossRef
  13. Hong KH, Lee SW, Kim TS, Huh HJ, Lee J, Kim SY, et al. Guidelines for Laboratory Diagnosis of Coronavirus Disease 2019 (COVID-19) in Korea. Ann Lab Med. 2020;40:351-60.
    Pubmed KoreaMed CrossRef
  14. Aoki K, Takai K, Nagasawa T, Kashiwagi K, Mori N, Matsubayashi K, et al. Combination of a SARS-CoV-2 IgG Assay and RT-PCR for Improved COVID-19 Diagnosis. Ann Lab Med. 2021;41:568-76.
    Pubmed KoreaMed CrossRef
  15. Kim IH, Kang BH, Seo SH, Park YE, Kim GJ, Lee SW, et al. Early Laboratory Preparedness of the Korea Disease Control and Prevention Agency and Response to Unknown Pneumonia Outbreak from Wuhan, China, in January 2020. Ann Lab Med. 2021;41:532-9.
    Pubmed KoreaMed CrossRef
  16. Yun S, Ryu JH, Jang JH, Bae H, Yoo SH, Choi AR, et al. Comparison of SARS-CoV-2 Antibody Responses and Seroconversion in COVID-19 Patients Using Twelve Commercial Immunoassays. Ann Lab Med 2021;41:577-87.
    Pubmed KoreaMed CrossRef
  17. Irsara C, Egger AE, Prokop W, Nairz M, Loacker L, Sahanic S, et al. Evaluation of four commercial, fully automated SARS-CoV-2 antibody tests suggests a revision of the Siemens SARS-CoV-2 IgG assay. Clin Chem Lab Med 2021;59:1143-54.
    Pubmed CrossRef
  18. Schnurra C, Reiners N, Biemann R, Kaiser T, Trawinski H, Jassoy C. Comparison of the diagnostic sensitivity of SARS-CoV-2 nucleoprotein and glycoprotein-based antibody tests. J Clin Virol 2020;129:104544.
    Pubmed KoreaMed CrossRef
  19. National SARS-CoV-2 Serology Assay Evaluation Group. Performance characteristics of five immunoassays for SARS-CoV-2: a head-to-head benchmark comparison. Lancet Infect Dis 2020;20:1390-400.
    Pubmed KoreaMed CrossRef
  20. Centers for Disease Control and Prevention. Interim guidelines for COVID-19 antibody testing. https://www.cdc.gov/coronavirus/2019-ncov/lab/resources/antibody-tests-guidelines.html (Updated on Mar 2021).