The epidermal growth factor receptor (EGFR) contains oncogenic mutations in non-small cell lung cancer (NSCLC). At diagnosis, approximately 90% of EGFR mutation-positive NSCLC patients harbor an in-frame deletion in exon 19 (E19del) or a p.Leu858Arg mutation (L858R) [1]. NSCLC patients with these mutations demonstrate progression-free survival advantages when treated with first- or second-generation EGFR-tyrosine kinase inhibitors (TKIs) [2]. Despite this transient response, most patients eventually become resistant [3^,4]. The secondary p.Thr790Met mutation (T790M) is the most common cause (approximately 60%) of acquired resistance to EGFR-TKIs [3^,4^,5]. Recently, third-generation EGFR-TKIs that can overcome T790M-associated resistance have become available [5^,6]. EGFR mutation testing is essential to screen candidates before EGFR-TKI treatment and select patients for T790M-targeted therapy [7].

Although a tissue biopsy is recommended for EGFR mutation testing, the risk of complications during the biopsy, lack of available tumor tissue, and low-quality tissue samples can limit mutation detection. Plasma cell-free DNA (cfDNA) testing offers a minimally invasive alternative when tissue-based assays are infeasible, and EGFR-mutant levels in the plasma may predict treatment response [8^,9^,10].

In Korea, two commercial plasma EGFR mutation assay kits are used in clinical laboratories with approval from the relevant governmental agency: cobas EGFR Mutation test v2 (cobasv2; Roche Molecular Systems, Pleasanton, CA, USA) and PANAMutyper-R-EGFR (Mutyper; Panagene, Daejeon, Korea). Both employ an analog quantitative PCR (qPCR)-based method [11^,12]. However, the comparative performance of these kits in terms of EGFR mutation detection and semi-quantification remains unclear. Additionally, both manufacturers provide product claims for their assays with plasma, but not other types of body fluid samples, including pleural effusion supernatant (PE-SUP). We analyzed plasma and PE-SUP samples using cobasv2 and Mutyper and compared the results to determine which kit is more efficient. We also evaluated whether PE-SUP samples are assayable, similar to plasma samples, using the two kits, or which option is more efficient when plasma and PE-SUP are available.

Study population and samples

This two-step combined study was performed at Wonkwang University Hospital, Iksan, Korea, from January 2012 through August 2018. The initial step, which included candidate enrollment, was performed prospectively, and the case-controlled study of selected samples was conducted retrospectively. During the clinical sample enrollment process, dipotassium-EDTA (K2-EDTA)-treated plasma or PE-SUP was prepared by centrifugation for 15 minutes at 2,500×g; each prepared sample was stored at −75℃ until DNA extraction. The EGFR-mutant group comprised samples originating from patients with lung adenocarcinoma (LADC) and EGFR mutations (as per tissue-based and/or cells of cytology-based assay results [TC] genotype). The EGFR-wild type group comprised (1) samples originating from patients with LADC and EGFR-wild type (as per TC genotype), (2) samples originating from patients with benign pulmonary disease without any evidence of a malignancy, and (3) samples originating from healthy individuals. The sample size of the EGFR-wild type group was limited to approximately 20% of the total sample size, as the undetected types of the EGFR-mutant group could yield reasonable specificity (i.e., E19del was not detected in plasma from patients with L858R, and vice versa). We excluded samples with (1) insufficient volume (2) delayed centrifugation following collection (>four hours), and (3) in vitro hemolysis according to a visual inspection.

After reviewing the medical records of the candidates, we retrospectively selected 156 clinical samples (Table 1; 98 plasma and 58 PE-SUP) from 116 individuals through the biobank of Wonkwang University Hospital. Among these 116 individuals, 82 provided a single sample, and 34 provided two to four samples (no participant provided the same type of sample more than three times).

Additionally, 47 serially diluted contrived (SDC) samples (plasma/ PE-SUP mixture samples) were prepared using separate aliquot parts of samples from 12 of the 116 individuals (six LADC patients and six healthy individuals). In detail, four different types of EGFR-mutant stock solutions were prepared using K2-EDTA-treated plasma/PE-SUP samples from six LADC patients harboring EGFR-mutants and each stock solution was serially diluted with a pooled normal plasma sample harboring EGFR-wild type mutants. The four stock solutions contained at least one of the three major types of EGFR-mutants: E19del, L858R, and T790M [1^,6^,13]. They were composed as follows: (1) stock #1 (one plasma and one PE-SUP mixture containing three types of EGFR mutants); (2) stock #2 (two PE-SUP mixtures containing three types of EGFR mutants); (3) stock #3 (one PE-SUP sample containing E19del-type mutant); and (4) stock #4 (one PE-SUP sample containing L858R- and T790M-type mutants).

This study was approved by the Institutional Review Board (IRB) of Wonkwang University Hospital (IRB No. 2017-02-029). Samples and medical records of the individuals were obtained after they provided a written informed consent.

DNA extraction

Plasma or PE-SUP cfDNA was isolated using the cobas cfDNA sample preparation kit (Roche Molecular Systems), according to the manufacturer's instructions. For all clinical plasma samples, plasma cfDNA was extracted from a starting volume of 2 mL and eluted in 100 µL of elution buffer. Two equal volumes of eluted cfDNA samples were homogenized (200 µL of cfDNA was extracted from 4 mL of each plasma sample) to ensure evenly matched comparative conditions between the two kits. For PE-SUP cfDNA extraction, 0.5 mL (25% of the plasma sample volume) of PE-SUP was used as the starting volume (the volume was determined by preliminary assessment). Except for the starting volume of PE-SUP, all other extraction processes were conducted in the same manner for all samples.

Each cfDNA of 47 SDC samples was prepared as per the clinical sample extraction (100/110 µL of cfDNA extracted from a 2.0/2.2 mL starting volume), duplication (final volume of 200/220 µL from a 4.0/4.4 mL starting volume), and homogenizing processes.

Detection and semi-quantification of EGFR mutations

To ensure an even-handed comparison of the plasma assay kits, immediately prior to target DNA amplification, ~220 µL of cfDNA elute was thoroughly mixed to maximize the homogeneity of the DNA content. Subsequently, two aliquots (75 µL for cobasv2 and 30 µL for Mutyper) of the cfDNA elute were amplified according to the manufacturer's instructions. PCR amplification of cobasv2 was performed in three separate wells per sample, and each well composition was as follows: 25 µL of cfDNA, 20 µL of master mix reagent, and 5 µL of magnesium acetate. PCR amplification of Mutyper was performed in six separate wells, and each well composition was as follows: 5 µL of cfDNA, 19 µL of PCR reagent, and 1 µL of Taq DNA polymerase.

Definition of overall/sample-specific data and modified approach for reference genotypes

In the analysis of 156 clinical samples, overall data was defined as the summated results obtained from plasma and PE-SUP together, and sample-specific data was defined as the result obtained from either plasma or PE-SUP individually. In the analysis of 47 SDC samples, overall sensitivity was defined as the summated sensitivity obtained from both more-dilute and less-dilute samples (dilution ratio: ≥ or <1:1,000, respectively).

The reference genotypes in each case were defined according to the corresponding TC genotypes (N=137) analyzed using direct sequencing and/or a peptide nucleic acid-clamping assay during routine diagnostic and follow-up processes. The following three exceptional conditions (N=19; see Supplemental Data Table S1) that were relevant to the actual false-negative results of tissue testing [8^,10^,11^,12^,14^,15^,16^,17], detection of multiple minor-type mutants (any EGFR-mutants except for the three major types) from one sample, or the limited status for tissue acquisition from individuals without any malignancy were defined using a modified approach: (1) when T790M was concordantly identified by both plasma assay kits in the plasma and/or PE-SUP of an NSCLC patient harboring any sensitizing mutations (as per TC genotype) and an additional aliquot of plasma/PE-SUP from the patient showed the same result using one of the two kits, it was defined as a correctly identified additional T790M (because of the known high specificities of the two plasma assay kits and tumor heterogeneity); (2) when multiple minor types of EGFR-mutants were identified by either a plasma assay kit or TC genotype and when among these, one or more types were consistent with the TC genotype, such a case was defined as a correctly identified minor-type EGFR-mutant; (3) an individual without any malignancy was defined as the wild type.

Statistical analysis

Fisher's exact test was used for comparing accuracy, sensitivity, specificity, and limits of detection (LODs). Weighted κ-values, with 95% confidence intervals, were calculated to evaluate concordance. The values were classified as follows: 0.81–1.0, very good; 0.61–0.80, good; 0.41–0.60, moderate; 0.21–0.40, fair; and <0.20, poor. Spearman's rho (ρ) with P was calculated between semi-quantitatively measured EGFR-mutant levels (semi-quantitative indexes, SQI) by each kit. Steiger's Z-test was used to evaluate which kit could better reflect the effect of dilution on SQI. To determine which kit is more sensitive for detecting relatively less-abundant EGFR-mutants, LODs for the three major EGFR-mutant genotypes and sensitivities depending on dilution ratio of the genotypes were compared using SDC samples. MedCalc version 17.9 (MedCalc Software, Ostend, Belgium) and Cocor [18] were used for statistical analysis. P<0.05 was considered statistically significant.

Comparison of kits using clinical samples

Accuracy, sensitivity, specificity, and concordance

Cobasv2 tended to show higher accuracy in terms of detecting the presence/absence of EGFR mutations than Mutyper overall (73% [114/156] vs 69% [108/156]) and in plasma samples (62% [61/98] vs 55% [54/98]), but the difference was not significant (Table 2). Similarly, overall and sample-specific sensitivities of cobasv2 and Mutyper were not significantly different. Both kits showed high overall and sample-specific specificities (≥92%) without significant differences. Concordance between kits for overall EGFR mutation presence/absence was very good (92%, κ=0.85). Plasma (89%, κ=0.77) and PE-SUP (98%, κ=0.96) samples showed good and very good concordance, respectively, between the kits.

Detection rates for paired liquid samples

Comparisons of the 31 paired samples revealed remarkable differences in accuracy (P<0.05) and sensitivity (P<0.05) between the two sample types for EGFR mutation detection (Table 3). Regardless of kit, PE-SUP had higher accuracy and sensitivity (97% and 96%, respectively) than plasma (55–71% and 46–65%, respectively).

Comparison of kits according to EGFR-mutant genotype

A detailed comparison of the two kits in terms of EGFR-mutant genotype (Table 4) demonstrated that the greatest difference in accuracy was for the E19del type (3% [5/156]). However, accuracy and sensitivity did not differ significantly between the kits for each of the three major types and other minor types of EGFR-mutants. Both kits showed high specificities (≥99%) for the three major types as well as the minor types of EGFR-mutants.

The EGFR mutation detection concordances according to the three major genotypes between the two kits were all >94% (very good, κ≥0.87). The minor-type concordance rate was 99%; however, the degree of concordance (good, κ=0.74) was lower than that of the major types. Overall EGFR-mutant genotype concordance between the two kits was 98% (very good, κ=0.92).

Correlation between SQIs

SQIs measured using cobasv2 and Mutyper showed strong positive correlations for the three major EGFR-mutants (E19del, ρ=0.80; L588R, ρ=0.95; and T790M, ρ=0.96, P<0.0001).

Comparison of kits using SDC samples

Sensitivity based on EGFR-mutant abundance

The predicted LOD ranges of the two kits (Fig. 1) overlapped with each other in six of the nine matched LOD comparisons using SDC samples; however, in some cases, they were not clearly definable or distinguishable from incidental detection failures. Two LODs of the cobasv2 series (dilution #1-T790M-series and dilution #2-L858R-series) and one LOD of the Mutyper series (dilution #4-T790M-series) were higher than those of their counterparts (2/9 vs 1/9, P>0.05). Further, in the more-dilute samples, cobasv2 (43% [15/35]) tended to have higher sensitivity than Mutyper (20% [7/35]; P=0.07). In the less-dilute samples, sensitivity did not differ between the kits (Table 5).

Comparison of kits according to EGFR-mutant genotype

Accuracy and sensitivity did not differ significantly between the kits for each of the three major types of EGFR-mutants (Table 5). Both cobasv2 and Mutyper showed 100% specificities, not only for each of the three major types of EGFR-mutants but also for minor-type EGFR-mutants.

False-negative results from cobasv2

Although cobasv2 had higher sensitivity than Mutyper in the relatively small number of mutant-containing samples, cobasv2 produced unexpected simultaneous false-negative results for all three types of EGFR mutations (“no mutation detected” in the dilution #2 series with the 1:38 dilution ratio in the cobasv2 column; Fig. 1). This mutant-genotype-independent abrupt detection failure occurred in a certain range of dilution ratios (1:20 to 1:50); i.e., E19del or L858R were detected in far more dilute (dilution #2 series with 1:75 to 1:6,000 dilution ratio) samples. However, Mutyper did not show this unexpected EGFR mutation detection failure for all types. All three types of false-negative results (“wild” in the Mutyper column in Fig. 1) were observed in samples with a >1:75 dilution ratio in all dilution series. Furthermore, the unexpected detection failure using cobasv2 was observed during replicate analysis using the remaining identical cfDNA aliquots (i.e., in the replicated assay, the 1:38 dilution ratio sample of dilution #2 series presented a “no mutation” using cobasv2, whereas it presented “EGFR mutations [E19del and L858R]” using Mutyper). A vaguely similar false-negative pattern using cobasv2 was also observed in the 1:24 dilution ratio sample of dilution #3 series.

Semi-quantification ability of kits

Of the nine relationship comparisons (the correlation coefficients between the dilution effect and SQIs), six comparisons did not show significant differences (Fig. 1). Only three comparisons showed significant differences (Fig. 1); in two comparisons (dilution #1-E19del-series and dilution #4-L858R-series), cobasv2-SQIs showed a closer relationship with dilution effect than Mutyper-SQIs (cobasv2-ρ>Mutyper-ρ) and in one comparison (dilution #2-L858R-series), Mutyper-SQIs showed a closer relationship with dilution effect than cobasv2-SQIs (Mutyper-ρ>cobasv2-ρ). Thus, no difference (2/9 vs 1/9, P>0.05) was observed between cobasv2-SQI and Mutyper-SQI in terms of the dilution effect of SDC samples.

Liquid biopsy is emerging as a promising approach for noninvasive assessment of cancer gene profiles; its actual usefulness varies remarkably depending on the methods adopted, including DNA extraction, detection, and quantification [10^,11]. In this regard, performance characterizations of current methods are important. We compared the performance of cobasv2 and Mutyper in terms of EGFR mutation detection and semi-quantification, using large numbers of EGFR-mutant-harboring samples and their firmly homogenized cfDNA elutes and verified whether PE-SUP samples could be assayed using the two commercial plasma assay kits.

Comparison of diagnostic parameters using 156 clinical samples showed similar performance between the kits. Although, cobasv2 showed slightly higher accuracy and sensitivity than Mutyper in plasma samples, these differences were not significant. This pattern held regardless of the specific genotype among the three major EGFR mutants. The current mutation analysis paradigm recommends the use cfDNA EGFR as a “rule in” assay when tissue samples are limited or difficult to obtain, as tissue and plasma EGFR mutation assays are not mutually exclusive and because of the relatively high specificity and rapid turnaround time of plasma EGFR mutation assays [7^,10]. The specificity levels in our study were nearly 100% for both kits; thus, they are suitable for the “rule in” approach of the cfDNA EGFR mutation assay. Moreover, the major EGFR genotypes identified by cobasv2 and Mutyper showed a high degree of concordance, and EGFR-mutant SQIs measured using the two kits had strong positive correlations. Taken together, these results indicate that both kits have statistically indistinguishable performance in terms of detection, as well as substantially parallel values for respective clinical samples from patients with major/wild-type EGFR-mutants.

Lung carcinomas are most often detected at stage IV [19]. Therefore, most PE-SUP samples, including the malignant PE-SUP samples, were obtained from stage IV patients (possibly greater chance of releasing tumor cells into PE). Hence, the plasma/PE-SUP paired-comparison findings could not reflect those of lower-stage (I–III) patients. Nonetheless, these paired comparisons provided evidence that PE-SUP can be suitably detected by both kits, at least in stage IV patients with PE. Moreover, even with a quarter of the plasma sample volume, PE-SUP is more efficient than plasma for detecting EGFR mutations in NSCLC patients with PE. Our findings somewhat contrast those of Liu et al. [20], who did not find a significant difference between plasma and PE-SUP samples, although they observed a slightly higher detection rate in PE-SUP than in plasma. However, our results are in line with the findings of Yeo et al. [21], who demonstrated remarkable EGFR mutation detection superiority in PE-SUP than in serum samples. A possible explanation for the difference between our findings and those of Liu et al. [20] is that we used paired samples for each patient, as did Yeo et al. [21], or that we used many more samples than Liu et al. did.

For SDC samples, a marginal superior tendency (2/9 vs 1/9) was noted for cobasv2 in the LOD comparison of the three genotypes. This superior tendency of cobasv2 was more prominent in the sensitivity comparison based on dilution ratio (23% sensitivity difference in more-dilute samples vs 0% sensitivity difference in less-dilute samples). This implies that cobasv2 may have higher sensitivity than Mutyper for samples with low EGFR-mutant abundance, but not for samples with high EGFR-mutant abundance.

However, we noticed unexpected false-negative results for all types of EGFR mutations using cobasv2 in much-less-dilute samples. We postulate that the net effect of these false results using cobasv2 in less-dilute samples and the more prominent superior tendency of cobasv2 in more highly dilute samples contributes to the very similar overall detection performance of the two kits, as indicated by the comparisons using 156 clinical samples.

With regard to the unexpected false-negative results obtained from cobasv2, we hypothesize that amplification/detection using cobasv2 is vulnerable to inhibition by certain materials contained in clinical liquid samples or/and agents related to cfDNA extraction [22^,23^,24] based on the following points: (1) the identical replicate findings (false-negatives using cobasv2 and true positives using Mutyper) using the remaining cfDNA aliquots; (2) a previous manufacturer's report regarding sample-specific potential inhibition (i.e., false-negatives generated only for plasma samples, but not for formalin-fixed paraffin-embedded tumor samples) using cobasv2 [25]; and (3) cobasv2 required a relatively large-volume ratio (25 µL/50 µL) of the extracted cfDNA elute (undiluted cfDNA extract volume/total PCR reaction volume) per well than Mutyper did (5 µL/25 µL). The third point indicates a higher concentration of cfDNA elute (this may be linked to the higher concentration of a certain PCR inhibitor that contained in cfDNA elute) in the cobasv2 PCR mixture than in the Mutyper PCR mixture.

Recent studies related to clinical outcomes have highlighted the importance of EGFR-mutant quantification [10^,11]. We have shown that the general semi-quantification abilities of cobasv2 and Mutyper were not remarkably different.

Our results should be considered in the light of some limitations. We could provide only limited information for minor types, as opposed to major types, because of the low number of positive samples. Moreover, due to limited resources, we could not identify the exact cause of the false-negative results obtained using cobasv2. Further studies with a more specialized experimental design and larger sample sizes are needed.

In conclusion, cobasv2 and Mutyper have an overall similar performance in terms of EGFR mutation detection and semi-quantification in clinical (plasma/PE-SUP) samples. cobasv2 tends to have higher sensitivity than Mutyper, especially for relatively less-abundant EGFR-mutants in SDC samples. PE-SUP samples are assayable using cobasv2 and Mutyper for the detection of EGFR mutations; for this purpose, PE-SUP samples are more efficient than plasma samples when these two kits are used.

Mutation detection and semi-quantification using two plasma EGFR mutation assay kits with 47 serially diluted contrived samples. The numbers in the green-, yellow-, and blue-tinged boxes indicate the SQIs of the mutation-detected samples and the color tone indicates the SQI rank (light tones indicate lower SQIs and dark tones indicate higher SQIs). The en dash (–) in each white-colored box (except for the en dash indicating EGFR-mutant genotype) indicates that at least one of the EGFR mutations was detected in the corresponding sample using the corresponding plasma assay kits. NM (no mutation detected by cobasv2 analysis) or W (wild type by Mutyper analysis) written in blue in each white box indicates that no EGFR mutations were detected in the corresponding samples. The red-bordered boxes indicate the predicted limit of detection values of the two kits for the three major EGFR-mutant genotypes. The same upper-case or lower-case superscript letter in each dilution series box indicates the same batch analysis. The asterisk in each box indicates that the same mutant was observed following replicate analysis using the remaining aliquots of the extracted DNA samples. The correlation coefficients (ρ) between the dilution effect and SQI rank and P written in black were obtained using Spearman's rank correlation analysis. P or not significant (NS; P>0.05) written in white was obtained using Steiger's Z-test (St-Z), which served to calculate the difference between the two correlation coefficients (cobasv2-ρ vs Mutyper-ρ).

Abbreviations: EGFR, epidermal growth factor receptor; E19del, an in-frame deletion in exon 19; L858R, p.Leu858Arg mutation; T790M, p.Thr790Met mutation; cobasv2, cobas EGFR Mutation test v2; Mutyper, PANAMutyper-R-EGFR; SQI, semi-quantitative index; St-Z, Steiger's Z-test; NS, not significant.