Introduction

The new pandemic associated to SARS-CoV-2 virus, causes a disease known as COVID-19¹ . Besides of documented rapid spreading, the virus is associated with a case fatality risk of at least 1.4%, specially related to older or other comorbidities such as cardiac involvement, a critical situation that is worsened given the lack of a specific treatment or vaccine¹ . Spreading of this new coronavirus in humans could be measured if positive (symptomatic and asymptomatic) cases could be diagnosed, identified and isolated, in order to define individual treatments when necessary, as well as the application of various collective health measures³ .

Given the specificity of antigen-antibody reactions, serological tests based on this immunochemistry are commonly used for diagnostic or prognostic purposes for various infectious diseases. Serological tests do not usually meet strict diagnostic standards, but if used in combination with molecular tests, they can be very useful in surveillance strategies. When well-characterized antibodies are available, the detection of antigens in biological samples meets the criteria to be a diagnostic test. If what is being searched for in a serological test are the antibodies that the infected individual has generated as a response to the pathogen, this cannot be characterized as a diagnostic test, and all that can be affirmed is that the individual has been or is currently is infected, regardless of whether that individual is symptomatic or not. In the case of COVID-19, it is safe to assume that an infected asymptomatic individual is highly infectious. Because of this, it is important to complement a serological test with a follow-up molecular test, in order to confirm whether the individual is currently infected⁸ . Therefore, combined use of molecular and serological tests can help guide public policies regarding epidemiologic surveillance, particularly once the disease has spread widely within a population^7-9. However, the performance of some of these tests has been questioned because of their low sensitivity, increasing the percentage of false negatives (asymptomatic but infectious individuals).

With the understanding that SARS-CoV-2 infection may be asymptomatic or mild in about half of the cases, the immune response is expected to play a critical role in natural disease control, as has been shown for other newly-onset viral respiratory diseases impacting public health, like SARS and MERS^10,11. The number of infected individuals that have recovered is also expected to be significant, so that they manage to limit the spread, in what is known as the generation of collective or herd immunity.

However, the quality of both molecular and serological tests is a sensitive concern for governments. If these tests do not show high levels of sensitivity and specificity, false negative results (undetected infectious individuals) would be responsible for dissemination hotspots, which could not be properly isolated and monitored⁹ . The concern is understandable: due to the urgency of testing, some health agencies have relaxed the rules for health registration mechanisms, which has in turn led to the use of evidence with questionable sensitivity and specificity. With molecular testing these parameters are relatively clear; with serological tests they are not.

Commercial immunochromatographic tests to detect antiIgG and anti-IgM antibodies against SARS-CoV-2 virus have the following characteristics:

i) They show widely varying ranges of sensitivity and specificity, parameters which are closely correlated to the time of infection^6,12,13. However, few commercial tests indicate in their inserts the variables related to the kinetics of the infection, an aspect relevant to their interpretation and clinical correlation.

ii) The manufacturers show validation of their tests with variable numbers of serum samples, and generally do not report the results of these validations in the product insert. When it is reported, the ranges vary from 20 to 1,300 samples ⁶ . This leads to a lack of reliability with some of these tests, especially those with limited sizes in the validation population.

iii) Very few manufacturers report positivity correlation rates with a molecular test. In those cases where it is reported, the values range from 37.5% to 95% for IgG, and 81.25% to 97.1% for IgM. Given that the possibility of finding IgG antibodies increases after the 10th day post-infection, a high positivity rate will allow for the detection of individuals in earlier stages of the disease.

Although the COVID-19 pandemic has accelerated the production of molecular and serological tests for diagnostic and epidemiological purposes, these products must be accompanied by the proper health records of their countries of origin and that of other countries that may decide to endorse them. These are usually processes that review documentation only, and do not entail an analytical verification of the claims set forth in the documentation. After the controversies regarding the use of some immunochromatographic tests of Chinese origin, the Chinese Ministry of Commerce, the General Consumer Administration of China and the Chinese National Medical Products Administration (NMPA) announced on March 31 the elimination of the “Fast Track” route for the registration of such products¹⁴. After this date, Chinese authorities decided to require additional evidence from manufacturers, so these products do not currently have export permits. Apparently, only a few companies were able to meet the more stringent conditions now required to obtain a sanitary registration in China.

For the case of Colombia, an adequate selection of the serological tests to be used for the detection of anti-SARS-CoV-2 IgM/IgG has to take into account the provisions referred to in Decree 3770 of 2004 of the country’s Ministry of Health and Social Protection. This means reviewing in detail the sensitivity and specificity ranges of the tests, ensuring they are in accordance with the time points of infection with SARSCoV-2. Only those that report positivity correlation rates with molecular tests and report the number of samples with which the developers carried out the validation of the test, should be considered. Furthermore, as part of the support and quality requirements, it is desirable that the test should have a current sanitary registration in the country of origin, and an endorsement or certification from other regulatory agencies such as the FDA or the European Union, or that they are listed by regulatory agencies such as the World Health Organization. Considering the shortage of diagnostic products, the Colombian Ministry of Health and the consensus of experts have decided to introduce serological tests as part of the response in the context of care and follow-up¹⁵.

A serological test that is useful for seroconversion and seroprevalence studies, either for management of either individual cases or for epidemiological surveillance, should identify the generation of a humoral immune response (neutralizing antibodies) in all symptomatic or asymptomatic infected individuals.

Materials and Methods

Study Design

Demo samples of “fast” serological tests were received from companies or persons interested in marketing them, in varying number and conditions.

Verification of the packaging, as well as test presentations, were performed for each test prior to its use and summarized in Table 1. A random internal number was assigned to each test. The number of samples run for each test was in accordance to the available number of demo tests delivered by each supplier.

Table 1 Serological anti-IgG and anti-IgM lateral flow assays, with a cassette presentation. The assigned number corresponds to the internal identification code of the test. The number of tests used corresponds to the number of serum samples used, as well as to the available demo tests. The observations record the presentation of each of the tests. N/A: Demo test was sent without information. 

Tabla 2

Table 2 Characteristics of each patient whose sample was used in this performance check of “rapid” immunochromatographic and ELISA serological tests. * Both sera samples provided from same patient took with a day of difference. a Negative volunteer (N); b Hospitalized symptomatic patients (SH); c Asymptomatic volunteers (A); d Symptomatic patients in house hospitalization (SD). ND: No Data; NA: Does Not Apply 

The ELISA NovaTec® test for IgM, IgG and IgA was given by Quimiolab. The sera were tested according to the manufacturer’s instructions. For the calculation of the arbitrary units of immunoglobulins (AUI), the average optical densities (at 450 nm) for the cut-off control were first established. AUIs of each serum sample was calculated as follows: OD of the serum sample x the dilution factor/cut-off control.

Serum Samples

The serum samples correspond to individuals positive for SARSCoV-2, as diagnosed by RT-PCR as follows: A: asymptomatic; SD: symptomatic patients hospitalized at home with clinical follow-up of public health system in Bogotá; SH: inpatients (general ward or intensive care Unit) of Clínica Colsanitas in Bogotá. Negative patients (N): who were symptomatic from other disease (Non-COVID-19) and RT-PCR negative for SARS-CoV-2.

According to the incubation period reported by Lauer et al ¹⁶ the likely date of infection was calculated based on the reported onset of symptoms in patients positive for COVID-19 (5.5 days). Thus, the serums of patients positive for COVID-19 were collected between 12- and 35-days post-infection.

Results

In this preliminary serological test of the verification of characteristics of the different tests analyzed, we report a negative match rate of 100% for all the assays tested: No anti-IgG or anti-IgM was detected in patients with RT-PCR for SARSCoV-2 negative, according to immunochromatographic tests. 3 out of 8 (37.5%) negative patients for SARS-CoV-2 showed positive results by ELISA-measured IgA, which showed a low specificity of this technique.

A 100% positive match rate level for IgG was found for tests identified as 3, 5, and 9. The lowest match rate corresponds to 70% regarding the molecular test. For positive match rates for IgM, a 100% rate was found for tests identified as 3 and 6. The lowest match rate for IgM corresponds to 30% (number test 8). The result of rapid test identified with number 1 was associated with a recognition of 50% of the sera from COVID-19 patients. Although the ability to discriminate two immunoglobulins is announced in the test insert, the cassette only allows showing a single recognition band for both IgM and IgG, preventing the characterization of the evolution of the infection. Test number 4 showed IgG recognition in 9 of the 10 positive control sera analyzed, 90% for IgG and 70% for IgM, while test 10 showed a recognition by sera positive control for IgG and IgM of 77% for each immunoglobulin.

A comparative analysis of the recognition of positive control serums according to the evolution time of the infection, for assays that had the highest number of tests (marks number 4, 6, 8 and 10, with 20 tests each) is showed in Table 3.

Table 3 Immunochromatographic test reading. Results of the immunochromatographic assay with respect to the time of infection (5.5 days before appearance of the onset of COVID-19 symptoms,). 

Neg: Negative, no band appearance on the cassette. Pos: Positive, clear bands appear on the cassette. Pos D: Positive weak, corresponds to a faint band in the cassette membrane. ND: No Data: invalid test (no mark on the control band).

In line with our analysis of the different tests, figure 1 shows the sera recognition bands of patients identified as 18, 19 and 20, taken on days 12 to 17, 18 to 23 and 24 to 35 after infection, respectively.

Figure 1 Recognition of representative sera of COVID-19 patients [with positive molecular (RT-PCR) diagnosis]. The characteristics at the intensity level of the bands could be correlated with the concentration or titters of anti SARS-CoV-2 antibodies, and therefore with the sensitivity of the test. Letter C in each cassette corresponds to control of procedure (other antibodies in sera). Letters G, M, or IgG or IgM, correspond to antibodies bind to the cassette membrane (anti-immunoglobulin IgG or anti-immunoglobulin IgM). 

With the use of ELISA, positives could be more objectively discriminated from negatives (weak recognition bands generated by immunochromatography), and although the result is not rapid, the reliability and sensitivity are improved with respect to immunochromatography. The results of the preliminary immunoglobulin analysis are showed in Table 4. For IgM analyzed by ELISA, 25 % (2 out of 8 individuals) of recognition was found by groups of asymptomatic and symptomatic individuals with mild symptoms (hospitalized at home). 75 % of hospitalized symptomatic individuals had detectable levels of IgM. The two patients hospitalized for COVID-19 who were negative for IgM had between 12 and 17 days after symptoms appeared. Patients negative for SARS-CoV-2 were not recognized by the IgM ELISA test. At the IgG level, 62.5 % of COVID-19 patients followed at home were positive for this immunoglobulin, while 8/8 (100 %) of hospitalized patients were positive. When analyzing the level of antibodies represented in the AUI for each of the symptomatic COVID-19 groups (SD and SH, Table 4), at least twice as much as AUIs are observed in the hospitalized versus home-monitored group. ELISA for IgA shows a nonspecific recognition, represented by a 37.5 % positivity in the group of negatives for SARS-CoV-2 (hospitalized by Non-COVID-19 cause). 25 % and 87.5 % of home-monitored and hospitalized symptomatic individuals, respectively, showed positivity by IgA ELISA tests.

Table 4 IgM, IgG and IgA anti protein N SARS-CoV-2 production. AUI: Arbitrary Immunoglobulin Units. Results correspond to AUI calculated for each immunoglobulin. Shaded boxes represent results that are above the cutoff values. The cutoff value for IgM is >3.8, for IgG > 6.0 and for IgA > 4.0. 

Figure 1. Recognition of representative sera of COVID-19 patients [with positive molecular (RT-PCR) diagnosis]. The characteristics at the intensity level of the bands could be correlated with the concentration or titters of anti SARS-CoV-2 antibodies, and therefore with the sensitivity of the test. Letter C in each cassette corresponds to control of procedure (other antibodies in sera). Letters G, M, or IgG or IgM, correspond to antibodies bind to the cassette membrane (anti-immunoglobulin IgG or anti-immunoglobulin IgM).

Discussion

Lateral-flow immunodetection assays (immunochromatography for serology) have been employed in diagnostics since the 1980s¹⁷. The first available tests were designed for the detection of human chorionic gonadotropin, secreted in pregnancy. The quality of immunochromatographic tests and their value for detecting the corresponding analyte against which the detection antibodies have been developed, depend on several variables (ways to reach lower detection limits of Lateral Flow Immunoassays) that are increasingly better controlled (the design of the test, the material used in the recipient where sample containing the analyte is deposited, the material used for the interaction and movement of the reagents, the material in which the antigen-antibody reaction occurs, and the place where the reagents and biological material not coupled in the reaction are deposited) ^(18,19. The sensitivity and specificity of immunochromatographic tests depend on these variables. Sensitivity therefore depends not only on the title or concentration of antigens, but on their ability to bind and remain in these supports and materials that allow immunodetection. If these variables are well controlled, the intensity in the coloration of the bands suggests a greater amount of analyte. In theory, a semi-quantitative titration could be made.

The sensitivity and specificity of each of serological tests must be carefully analyzed, as well as the correlation rates with molecular tests. Moreover, the number of samples with which the manufacturing companies carried out the respective validations is another important factor that must be considered. This study corresponds to a verification (not a validation) of the performance of eight different tests. Although an analytical method has been previously standardized, it is necessary to confirm whether it works properly, before proceeding to routine use. To this procedure by which the per formance of the method is evaluated to demonstrate that it meets the requirements for the intended use, which were established as a result of the validation (to identify IgG and IgM, specific for SARS proteins- CoV-2), is called secondary validation or verification. Unlike validation (which corresponds to the process necessary to demonstrate the performance characteristics set out when a method has been designed or developed, or when extensions or modifications have been made to a standardized method as a manufacturer or user), the verification process it is not always framed in analysis with statistical design. When it comes to qualitative procedures or subjective testing, positive and negative controls should be incorporated into verification processes, whenever possible. Validation is also necessary when seeking to demonstrate equivalence of the results obtained by two methods. Since higher antibody titers and seroconversions are detected in most individuals with severe or not severe symptomatic COVID-19 patients, but it could be negative in asymptomatic or pauci-symptomatic patients^20-21. On the other hand, positive and negative control sera were confirmed by RT-PCR for SARS-CoV-2, and in this validation the RT-PCR result was considered the gold standard, although it has also been shown to be negative, especially in those with a longer period of time after the onset of symptoms⁶.

Although manufacturers do not reveal the characteristics of the anti-antibodies used, it is known that the system in which these are produced, and the variable affinity they present with the Fc fraction of immunoglobulins, in particular with IgM, induces greater complexity in their detection¹⁸. The diversity of hybridomas, of purification methods and of materials used as support, are factors that influence the sensitivity of each test, and are presumed to have been considered in the design of the technique, as well as in the reports that accompany the inserts in each kit.

Considering the pattern of immunity against SARS-CoV-2, it has been proven that, as it is against other microbial agents, it is a predominantly humoral (IgG-based) response, and targeted against the N protein, which can be detected starting at day 4 post-infection, with an average seroconversion of 14 days in infected individuals^13,22. In another study in which antibodies were detected through ELISA, it was found that the average time point of seroconversion will occur between day 12 and 14, for IgM and IgG, respectively, and after this period increase rapidly from day 15, achieving an average seroconversion of 94.3% for IgM and 79.8% for IgG¹². This study indicates that the combination of RT-PCR and antibody detection improves diagnosis, even in the early phase (first week). Although it is clarified that high antibody titers are not associated with the clinical evolution of the infected individuals. On the other hand, Jin et al. has described the positive rate and titer variance of IgG are higher than that of IgM in COVID-19²³.

According to the speed and scale of the COVID-19 pandemic and the urgent public health needs of each country and difficulties in obtaining supplies around the world, health authorities that have decided to use these serological tests have based their decisions on the review of the serological commercial and developers manufacturer’s dossiers and inserts, focusing their attention on criteria such as the reported sensitivity and specificity of these serological tests. Considering all of the above, it is important to establish clear criteria that allow the definition of the best serological tests, based on the information they provide, which will then need to verified and used for the purposes already described. This study showed that only for one of the serological tests referred to, the correlation rate with the molecular test corresponds to the information registered in the insert.

If a comparative validation test is considered, a larger number of samples will be required. Since this study was conducted with serum samples, it is recommended to perform a comparison with whole blood samples, with and without anticoagulant, although other authors (Li Z et al,) have evaluated this aspect and shown that differences in the type of sample used are not relevant¹³.

Inserts of the developers of the immunochromatographic assays indicate that the intensity of the color of the bands should not be associated with the anti-SARS-CoV-2 antibody titer. However, if a high affinity capture antibody is used, low reactogenicity plotted in weak intensity bands, if it could be correlated with the detected antibodies level. Despite the fact that in our study the number of positive control sera may be a limiting factor, as observed in Table 3, a trend towards the production of IgM and even detectable IgG can be postulated before 17 days post-infection (7 days from the report of symptoms) by some of the tests (6 and 10). Although they are not the same patients followed over time, the group of patients whose sera were collected between the 18th and 23rd post-infection, exhibited the most marked production of both immunoglobulins. A striking result suggests that after 24 days post-infection the antibody titers for both immunoglobulins (more marked for IgM) decline to patterns similar to those found before 17 days post-infection.

Considering all of the above, ELISA or chemoluminescence assays can be used to analyze processes such as seroconversion because higher sensitivity has been reported. However, not only technology is important: the proper selection of serum samples in the secondary validation phases is essential. Given that higher antibody titers as well as greater seroconversion rates are detected in individuals symptomatic for COVID-19, the positive control sera used in primary validation are collected from inpatients with severe or critical COVID-19 cases, which represents a limitation to evaluate sensitivity (including the asymptomatic individuals “innocent viral spreaders” ⁽²⁴).

This trend could be problematic considering that the use of immunochromatographic serological tests has been suggested as a strategy to identify possible asymptomatic infectious carriers. In a preliminary study with sera from individuals positive for SARS-CoV-2 diagnosed by RT-PCR but asymptomatic, it was found that for two rapid serological tests, no more than 30% of the individuals were positive for any of the immunoglobulins. This may be because the viral load of patients with severe to critical symptomatic COVID-19, may be responsible for a detectable antibody response, while asymptomatic carriers have been associated with low viral load, which leads to low or null seroconversion²⁰.

Taking into account the above, an optimal performance test should allow correlating the patient’s immune status with their clinical evolution; also identify infectious asymptomatic patients who warrant isolation and follow-up. However, with the information obtained so far, the sensitivity of the tests for this epidemiological group would be lower than for the group of COVID-19 symptomatic patients. Both techniques appear to have limited use in asymptomatic carrier screening.

As a recommendation, decision makers are suggested to discuss in detail the following information considered in inserts or dossiers, and to choose tests with these characteristics:

For field studies it is desirable to choose tests whose cassettes include in the same line, with a clear separation, anti-IgG and anti-IgM results. For field studies, we do not recommend the use of tests with separate cassettes, or with independent lines for IgG and IgM, because it represents a double sample, and greater manipulation and exposure at the time of sampling. As in other in vitro analytical tests for diagnostic purposes, it would be desirable to have properly characterized control serums that allow for the verification of the performance of each test to be checked periodically in each laboratory.

It is possible that as the SARS-CoV-2 pandemic progresses, serological tests based on the detection of IgM / IgG antibodies will play a fundamental role in epidemiological surveillance. Besides, these tests can also play a role in the definition of seroprevalence, screening, identifying the recovered asymptomatic population that is able to return safely to the but the tests must be deployed appropriately.

With the data from this study we can suggest that the N protein used as an antigen in commercial serological tests (immunochromatography and ELISA), is a good immunogen for the follow-up of humoral response (IgA, IgM and IgG) in symptomatic COVID-19 patients (after two weeks of onset of symptoms). However, this protein is not identified by sera of asymptomatic patients. If the projected use of immunochemical tests involves the detection of infectious asymptomatic individuals in the framework of epidemiological surveillance (for particular isolations and follow-ups), such tests have a very limited usefulness. And although the quality of the tests and their performance in the field are associated with the sensitivity of these immunochemical techniques, in the case of COVID-19, the nature of the infection and the immunogenic characteristics of SARS-CoV-2 appear also to determine their value as a screening test. Recently, it has been described that the use of a recombinant Spike (S) protein from SARS-CoV-2 expressed in CHO cells in an ELISA test, allowed the early detection of antibodies in asymptomatic RT-PCR positive individuals (medical staff and close contacts of patients with COVID-19) ⁽²⁴. This justify the need to incorporate protein S as an antigen preferred in serological tests, in order to improve its scope and usefulness in epidemiological surveillance and cluster monitoring strategies (mainly asymptomatic individuals).

Lastly, determining seroconversion in healthcare workers, while at the same time monitoring them with a molecular test, is going to be of vital importance in the follow-up and care of this population. Understanding and validating serological and molecular tests is an essential component for the design of public health measures to response to the pandemic. Probably the integration in the strategies of molecular diagnosis and serological tests (with viral antigens as S protein alone or complementary to N protein) will contribute of important form not only in the containment of the epidemic but in its mitigation.