Standardization Status of Total Cholesterol Concentration Measurement: Analysis of Korean External Quality Assessment Data
2021; 41(4): 366-371
Ann Lab Med 2021; 41(4): 419-423
Published online July 1, 2021 https://doi.org/10.3343/alm.2021.41.4.419
Copyright © Korean Society for Laboratory Medicine.
1Department of Laboratory Medicine, Severance Hospital, Yonsei University College of Medicine, Seoul, Korea; 2Department of Laboratory Medicine Research Institute of Bacterial Resistance, Yonsei University College of Medicine, Seoul, Korea
Correspondence to: Sang-Guk Lee, M.D., Ph.D.
Department of Laboratory Medicine, Severance Hospital, Yonsei University College of Medicine, 50-1 Yonsei-ro, Seodaemun-gu, Seoul 03722, Korea
* Current affiliation: Department of Laboratory Medicine Hallym University Sacred Heart Hospital, Anyang, Korea
This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.
Procalcitonin (PCT) is a useful bacterial infection biomarker with the potential for guiding antibiotic therapy. We evaluated the concordance of three automated PCT immunoassays: Kryptor (BRAHMS GmbH, Hennigsdorf, Germany), Atellica IM 1600 (Siemens Healthcare Diagnostics, Munich, Germany), and Cobas e801 (Roche Diagnostics, Mannheim, Germany). In 119 serum samples with a PCT concentration <5.00 μg/L, Kryptor (reference assay) was compared with the other two immunoassays by Spearman’s rank correlation, regression analysis, and concordance at two antibiotic stewardship medical decision points: 0.25 and 0.50 μg/L. The Atellica IM 1600 and Cobas e801 results showed high correlations with those of Kryptor, with correlation coefficient (ρ) values of 0.97 and 0.99, respectively. However, negative biases were observed in both immunoassays (slope/y-intercept: 0.75/–0.00 for Atellica IM 1600; 0.88/–0.01 for Cobas e801). Atellica IM 1600 and Cobas e801 demonstrated excellent concordance with Kryptor at both medical decision points, with linearly weighted κ values of 0.90 and 0.92, respectively, despite discrepancies, which were more prominent at the 0.25 μg/L medical decision point. Based on these biases and discrepancies, the alternate use of different PCT immunoassays in repeat examinations is inadvisable. Standardization is required before comparing the results of different PCT immunoassays.
Keywords: Procalcitonin, Standardization, Concordance, Immunoassay, Kryptor, Atellica IM 1600, Cobas e801
Procalcitonin (PCT) is a useful biomarker of bacterial infection and its resolution [1-3]. Repeated PCT concentration assessment can be used to monitor the therapeutic response to guide medical decisions on early discontinuation of antibiotics and support personalized antibiotic management [2, 3]. Multiple studies have shown that PCT-guided antibiotic initiation and discontinuation can shorten the duration of antibiotic treatment, which can in turn reduce the associated side effects and improve clinical outcomes [4-7]. Recently, two medical decision points have been suggested for PCT-guided antibiotic stewardship: 0.25 and 0.50 μg/L for non-intensive care unit (ICU) and ICU patients, respectively .
Sensitive and accurate determination of PCT concentration is required for its use as an antibiotic stewardship guide. Numerous fully automated PCT immunoassays are available . However, because PCT reference material and methods are lacking, the comparability of these immunoassays is not guaranteed, necessitating comparative studies. Kryptor (BRAHMS GmbH, Hennigsdorf, Germany) was the first fully automated PCT immunoassay  and it was used in several clinically important studies to measure PCT concentrations [4, 6, 10]. Thus, most published comparative studies have used Kryptor as the reference method [11-20].
We examined whether PCT concentrations measured using Atellica IM 1600 (Siemens Healthcare Diagnostics, Munich, Germany) and Cobas e801 (Roche Diagnostics, Mannheim, Germany) are equivalent to those measured using Kryptor in terms of PCT-guided antibiotic stewardship employing the medical decision points of 0.25 and 0.50 μg/L. To the best of our knowledge, this is the first study to evaluate Atellica IM 1600 in this regard. Our results would improve the understanding of how PCT concentrations can be influenced by different immunoassay principles employing the same BRAHMS PCT antibody.
For this comparison study, clinical serum samples were collected from patients at Severance Hospital, Seoul, Korea, from March to April 2019. A routine PCT immunoassay was performed using Cobas e601 upon collection. Immediately thereafter, 119 serum samples with a PCT concentration <5.00 μg/L were selected to increase the number of samples with PCT concentrations near the medical decision points, a strategy not applied previously. Selected samples were divided into three identical aliquots and stored at –70°C. All aliquots were thawed at 20°C for an hour and immediately analyzed simultaneously for PCT concentration using three different immunoassays: Kryptor (BRAHMS PCT-sensitive KRYPTOR), Atellica IM 1600 (Atellica IM BRAHMS PCT), and Cobas e801 (Elecsys BRAHMS PCT). Detailed descriptions and technical data of these three immunoassays are summarized in Supplemental Data Table S1. This study was reviewed and the need for informed consent for reviewing medical records of the study population has been waived by the Institutional Review Board of Yonsei University of Medicine (4-2020-0013).
The PCT concentrations measured by Atellica IM 1600 and Cobas e801 were compared with those measured by Kryptor (used as the reference assay). Spearman’s rank correlation coefficient (ρ) values, Passing-Bablok regression, Bland–Altman difference plots, and Cohen’s kappa coefficient (κ) were used for statistical analyses. The linearly weighted kappa coefficient across two PCT medical decision points, 0.25 and 0.50 μg/L, was used to assess overall concordance. All data were analyzed using Analyse-it (Analyse-it Software Ltd., Leeds, UK). The statistical significance threshold was set at
All PCT concentrations measured by Kryptor were within the analytical measurement range of 0.03–4.63 μg/L (median: 0.30 μg/L; interquartile range: 1.26). Atellica IM 1600 and Cobas e801 failed to detect PCT concentrations below the limit of detection (Supplemental Data Table S1) in 12 and six samples, respectively. These samples were excluded from the correlation and comparison analyses. Correlation analyses of the Atellica IM 1600 and Cobas e801 PCT concentrations with the Kryptor PCT concentrations yielded strong ρ values of 0.97 and 0.99, respectively (
Next, we investigated the influence of these systemic biases on the clinical interpretation of the PCT concentrations. To this end, the PCT concentrations of all 119 samples measured by all three immunoassays were classified into three categories based on the two medical decision points (0.25 and 0.50 μg/L). The overall concordance across the two PCT medical decision points (0.25 and 0.50 μg/L) of Atellica IM 1600 vs. Kryptor and Cobas e801 vs. Kryptor was 89.9 and 92.4%, respectively, with excellent linearly weighted κ values (κ=0.90 for Atellica IM 1600; κ=0.92 for Cobas e801; Table 1). However, compared with Kryptor, Atellica IM 1600 underestimated the PCT concentration in 11 samples and overestimated it in one sample, whereas Cobas e801 underestimated the PCT concentration in nine samples (no overestimation; Table 2). Notably, the discordance did not surpass either medical decision point in the Atellica IM 1,600 vs Kryptor or Cobas e801 vs Kryptor comparison, i.e., no sample was <0.25 µg/L by one method and >0.50 by the other (Table 2). Thirteen of the 119 samples (10.9%) measured by all three immunoassays were classified discordantly by at least one immunoassay; 10 at the 0.25 µg/L and three at the 0.50 μg/L medical decision point.
These results indicate that despite excellent concordance, both Atellica IM 1600 and Cobas e801 still risk leading to premature cessation of antibiotics and treatment failure due to underestimation of the PCT concentration compared with Kryptor. These discrepancies are more pronounced at the lower 0.25 μg/L medical decision point. Thus, cautious interpretation of PCT concentrations measured by these immunoassays is needed, especially when they are near the lower 0.25 μg/L medical decision point. Atellica IM 1600 could not be compared with other instruments because of a lack of published data. The Cobas e801 vs Kryptor comparison results are largely consistent with previous data . Although Lippi,
Our study has some limitations. First, we did not include immunoassays employing antibodies other than BRAHMS PCT. However, the evaluated immunoassays showed biases and discrepancies, supporting the necessity of reference materials and standardization methods [15, 17, 19, 21]. Second, duplicated sample measurements were not available because of volume limitations. Considering the number of samples used in this study, the possibility of random errors owing to the use of a single replicate would have minimally affected the overall results. The strengths of this study are that it was the first to evaluate Atellica IM 1600 for PCT concentration measurement and that a large number of samples with concentrations close to the medical decision points were analyzed.
In conclusion, this study demonstrated that the Atellica IM 1600 and Cobas e801 immunoassays provide strongly correlated and largely concordant PCT concentrations compared with Kryptor, despite the different assay principles. However, both immunoassays manifested modest negative bias compared with Kryptor, which could, in some cases, result in discordant medical decisions at the 0.25 and 0.50 μg/L medical decision points. Therefore, to successfully implement PCT-guided antibiotic stewardship, rigorous evaluation of immunoassay comparability is essential on the implementation of alternative PCT immunoassays. In addition, alternate use of different PCT immunoassays in repeat examinations should be avoided, because it can hamper the interpretation of PCT concentrations during longitudinal patient assessment. Furthermore, efforts should be made to develop and establish reference materials and methods for PCT concentration measurement, which would improve comparability and ultimately achieve standardization of the various PCT immunoassays.
We are grateful to the medical laboratory technologists at the Clinical Chemistry Division of the Department of Laboratory Medicine for assistance.
Cho HW and Lee SG contributed to study conception and design; Kim SH was involved in data acquisition and maintenance; Cho HW and Cho Y analyzed and validated the data using statistical analysis tools; Cho HW wrote the original draft; Jeong SH and Lee SG critically revised the manuscript; and Lee SG provided resources and administered the project. All authors have reviewed and agreed to the published version of the manuscript.
No potential conflicts of interest relevant to this article are reported.