Article

Original Article

Ann Lab Med 2021; 41(4): 366-371

Published online July 1, 2021 https://doi.org/10.3343/alm.2021.41.4.366

Copyright © Korean Society for Laboratory Medicine.

Standardization Status of Total Cholesterol Concentration Measurement: Analysis of Korean External Quality Assessment Data

Young Ahn Yoon, M.D.1 , Yong-Wha Lee, M.D., Ph.D.2 , Sollip Kim, M.D., Ph.D.3 , Kyunghoon Lee, M.D.4 , Ph.D., Hyung-Doo Park, M.D., Ph.D.5 , Sail Chun, M.D., Ph.D.6 , and Won-Ki Min, M.D., Ph.D.6

1Department of Laboratory Medicine, Soonchunhyang University Cheonan Hospital, Soonchunhyang University College of Medicine, Cheonan, Korea; 2Department of Laboratory Medicine and Genetics, Soonchunhyang University Bucheon Hospital, Soonchunhyang University College of Medicine, Bucheon, Korea; 3Department of Laboratory Medicine Inje University, Ilsan Paik Hospital, Goyang, Korea; 4Department of Laboratory Medicine, Seoul National University Bundang Hospital and College of Medicine, Seongnam, Korea; 5Department of Laboratory Medicine and Genetics, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, Korea; 6Department of Laboratory Medicine, University of Ulsan College of Medicine and Asan Medical Center, Seoul, Korea

Correspondence to: Yong-Wha Lee, M.D., Ph.D.
Department of Laboratory Medicine and Genetics, Soonchunhyang University Bucheon Hospital, Soonchunhyang University College of Medicine, 170 Jomaru-ro, Wonmi-gu, Bucheon 14584, Korea
Tel: +82-32-621-5943
Fax: +82-32-621-5944
E-mail: lywmd@schmc.ac.kr

Sail Chun, M.D., Ph.D.
Department of Laboratory Medicine, Asan Medical Center, University of Ulsan College of Medicine, 88 Olympic 43-gil, Songpa-gu, Seoul 05505, Korea
Tel: +82-2-3010-4513
Fax: +82-2-478-0884
E-mail: sailchun@amc.seoul.kr

Received: May 26, 2020; Revised: September 7, 2020; Accepted: January 20, 2021

This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

Background: Total cholesterol concentration measurement is important in the diagnosis of dyslipidemia and evaluation of cardiovascular disease risk factors. Measurement reliability for obtaining an accurate total cholesterol concentration requires procedure standardization. We evaluated the standardization status for total cholesterol concentration measurement through Korean external quality assessment (EQA) data analysis.
Methods: This study involved 1,670 laboratories that participated in the EQA of total cholesterol concentration measurements in 2019 for 32 products from different manufacturers. The target concentrations of three quality control (QC) materials (samples A, B, and C) were measured using the reference method and compared with EQA data. The performance criteria for total cholesterol concentration measurement were based on the National Cholesterol Education Program guidelines, with ±3% inaccuracy.
Results: The target values and inaccuracies of the QC material based on the reference method measurements were 254.65±7.64, 108.30±3.25, and 256.29±7.69 mg/dL (6.59±0.20, 2.80±0.08, and 6.63±0.20 mmol/L) for samples A, B, and C, respectively. The performance criteria were not met in 42.7% laboratories for sample A, 68.4% of laboratories for sample B, and 38.0% laboratories for sample C.
Conclusions: Despite significant efforts to accurately measure total cholesterol concentrations, further actions are needed for measurement standardization. Manufacturers reporting values that differ from target values should check calibrator traceability; additional efforts to accurately measure total cholesterol concentrations are required for laboratories that use products from these manufacturers.

Keywords: Inaccuracy, Total cholesterol, External quality assessment, Standardization

Total cholesterol concentration is used to diagnose dyslipidemia and determine the risk factors for cardiovascular diseases. The recent National Cholesterol Education Program (NCEP) guidelines [1] set specific medical decision points for serum total cholesterol (e.g., desirable concentrations: <200 mg/dL [5.2 mmol/L]; high and undesirable concentrations: >240 mg/dL [6.2 mmol/L]). The medical decision points for total cholesterol concentration were based on national population studies conducted by the United States Centers for Disease Control and Prevention (CDC) and were standardized to the decision points determined using the Abell-Kendall reference method [2]. To use these decision points, clinical total cholesterol concentration measurement procedures must be standardized and harmonized within clinically meaningful limits. Using results that are neither standardized nor harmonized may lead to significantly erroneous clinical decisions. Manufacturers have made various efforts to prepare products for measuring total cholesterol concentration accurately for routine clinical use. However, accurate total cholesterol concentration measurement remains challenging with respect to traceability to the reference material, altered matrix characteristics of calibrators and controls, and other factors [2].

The degree of standardization and harmonization of total cholesterol concentration measurement procedures can be evaluated by the external quality assessment (EQA). EQA is a widely accepted tool for monitoring and improving method performance in clinical laboratories and plays a central role in achieving trueness verification standardization and harmonization [3]. The Korean Association of External Quality Assurance Service (KEQAS) has been the nation’s top authorized EQA institute for the standardization and quality management of laboratory tests in Korea since its foundation in 1965 [4]. The KEQAS was certified as an EQA provider according to ISO/IEC 17043 in 2015 [5].

Only a few studies have evaluated the accuracy of total cholesterol concentration measurement worldwide using EQA, and no such studies have been conducted in Korea [6, 7]. We report the total cholesterol concentration measurement standardization status of products from various manufacturers using KEQAS data analysis for the first time in Korea; our findings can indirectly indicate reagent performance.

This retrospective study involved 1,670 laboratories that participated in the KEQAS assessment of 32 products from different manufacturers for measuring total cholesterol concentration in 2019. Total cholesterol concentrations in three quality control (QC) materials (samples A, B, and C) were measured in the first round of KEQAS in February 2019. The samples included a human serum powder (Bio-Rad Laboratories, Hercules, CA, USA) and were shipped to the participating laboratories in a box with ice packs under refrigeration. The total cholesterol concentration in each sample was measured using routine methods without modification of routine practice. The laboratories reported their results, methods, and instruments used.

The target values of the three QC materials (A, B, and C) were measured at the National Medical Reference Laboratory (NMRL) of the Korea Disease Control and Prevention Agency (KDCA), a member laboratory of the Cholesterol Reference Method Laboratory Network that participates in an EQA scheme for reference laboratories of the Joint Committee on Traceability in Laboratory Medicine (JCTML). Total cholesterol concentration was measured by isotope-dilution gas chromatography-mass spectrometry (ID/GC/MS), which was certified by the JCTML, and then converted to the predicted Abell-Kendall method value using a previously reported equation [8-11]. The target values of the three QC materials were determined as the means of three replicates.

Statistical analysis

Results from participating laboratories were evaluated using the standard deviation index (SDI) among peer groups in the KEQAS in April 2019. SDI was calculated as follows: (laboratory result – peer group mean)/peer group SD. An SDI >3 was considered unacceptable by the KEQAS. The peer group mean and SD were not presented when the number of laboratories per manufacturer was <10; the corresponding manufacturer data were excluded from the comparative analysis with a target value, although these data were used to calculate the total mean value. Therefore, data from 1,598 laboratories and 24 manufacturers were used in the comparative analysis. Manufacturer standardization degree was evaluated based on the performance criteria suggested by the NCEP [1, 12]; the performance criteria have a bias of ≤3% of the target value. The NCEP-based performance criteria and the KEQAS acceptable range based on the peer group mean were compared. We examined whether the KEQAS acceptable range for each laboratory group using the same manufacturer’s product met the performance criteria determined using the standard NMRL measurement procedure. Statistical analyses were performed using Microsoft Excel (Microsoft Corporation, Redmond, WA, USA).

The mean and SD of the total cholesterol concentration measurements reported by different manufacturers are summarized in Table 1. The KEQAS acceptable range (≤3 SD), target value measured at the NMRL of the KDCA, and the performance criteria are shown in Fig. 1. The NCEP-based performance criteria of samples A, B, and C by the NMRL were 254.65±7.64, 108.30±3.25, and 256.29±7.69 mg/dL (6.59±0.20, 2.80±0.08, and 6.63±0.20 mmol/L), respectively. The total mean values of samples A, B, and C in the KEQAS data from 1,670 laboratories were 254.5, 103.7, and 255.3 mg/dL (6.58, 2.68, and 6.60 mmol/L), respectively. Values in 42.7%, 68.4%, and 38.0% of the laboratories using samples A, B, and C, respectively, were outside the performance criteria range. In contrast to those for samples A and C, the total mean value of sample B was outside the performance criteria range. The acceptable ranges for all manufacturers were wider than the range for the target value performance criteria. Additionally, in the case of a few manufacturers, the peer group means were outside the target value performance criteria range. Three manufacturers using sample A, nine using sample B, and four using sample C had a peer group mean outside the performance criteria range. At lower concentrations (sample B), participating laboratories showed poor accuracy compared that at high concentrations (sample A and C). In the case of peer groups with high SD, even if the peer group mean falls within the performance criteria, many laboratories were found to be out of the performance criteria because the acceptable range was wide as 3 SD.

Table 1 . Statistics of laboratories participating in total cholesterol concentration measurement

ManufacturerLaboratories (N)Sample A*Sample B*Sample C*

Mean ± SD (mmol/L)
Total1,6706.58 ± 0.212.68 ± 0.136.60 ± 0.20
Sekisui Medical, Tokyo, Japan2856.61 ± 0.162.74 ± 0.076.62 ± 0.13
Roche Diagnostics, Mannheim, Germany2736.44 ± 0.122.58 ± 0.066.46 ± 0.14
Beckman Coulter Inc., Brea, CA, USA2246.66 ± 0.162.57 ± 0.096.68 ± 0.17
Wako Diagnostics, Osaka, Japan986.60 ± 0.182.76 ± 0.066.58 ± 0.16
Shenzhen Mindray Bio-Medical Electronics, Shenzhen, China916.65 ± 0.122.72 ± 0.066.65 ± 0.15
Stanbio Laboratory, Boerne, TX, USA826.36 ± 0.182.62 ± 0.146.45 ± 0.24
JW Medical, Seoul, Korea766.65 ± 0.262.81 ± 0.116.66 ± 0.20
Biosystems, Barcelona, Spain646.79 ± 0.232.69 ± 0.096.90 ± 0.18
Kyowa Medex, Tokyo, Japan586.57 ± 0.092.75 ± 0.056.57 ± 0.10
Siemens Healthineers, Erlangen, Germany496.39 ± 0.142.73 ± 0.066.38 ± 0.13
GDLKorea, Anyang, Korea476.62 ± 0.192.74 ± 0.116.57 ± 0.13
ELITechGroup, Puteaux, France336.54 ± 0.302.72 ± 0.146.67 ± 0.35
DiaSys Diagnostic Systems, Holzheim, Germany316.78 ± 0.282.68 ± 0.156.80 ± 0.25
Hanlab, Chungju, Korea276.61 ± 0.102.75 ± 0.056.61 ± 0.09
IVDLab, Uiwang, Korea276.66 ± 0.222.79 ± 0.136.75 ± 0.26
LC Diagnostics, Quezon, The Philippines226.78 ± 0.162.78 ± 0.116.82 ± 0.24
Asan Pharm, Seoul, Korea216.67 ± 0.212.82 ± 0.096.75 ± 0.27
Spinreact, Girona, Spain156.76 ± 0.212.75 ± 0.066.86 ± 0.16
Shinyang Diagnostics, Seoul, Korea146.45 ± 0.222.55 ± 0.036.48 ± 0.21
Erba diagnostics, Miami Lakes, FL, USA146.49 ± 0.212.70 ± 0.086.51 ± 0.16
Shino-Test Corporation, Tokyo, Japan136.33 ± 0.132.65 ± 0.066.34 ± 0.12
Embiel, Gunpo, Korea126.49 ± 0.262.61 ± 0.126.58 ± 0.24
GeneMatrix, Yongin, Korea126.74 ± 0.292.88 ± 0.106.78 ± 0.19
Shinsung Pharm, Suwon, Korea106.71 ± 0.362.94 ± 0.196.79 ± 0.21

*The target value and performance criteria of sample A, B, and C: 6.59±0.20, 2.80±0.08, and 6.63±0.20 mmol/L. Manufacturers with <10 laboratories are not listed in this table but were counted in the total number.

Abbreviation: SD, standard deviation.


Figure 1. Distribution of total cholesterol concentrations according to manufacturers, and target values and performance criteria of quality control materials (samples A, B, and C). The peer group mean is the average value of all laboratories in the same manufacturer group. Numbers from 1 to 24 were given to each manufacturer group in ascending order of peer group mean. (A) Data obtained using sample A. (B) Data obtained using sample B. (C) Data obtained using sample C. Abbreviation: SD, standard deviation.

Although all manufacturers have tried to maintain traceability to standard material, the actual inaccuracy of products from various manufacturers was revealed by comparing the true values of QC materials used in the KEQAS. This approach of using the peer group mean as the reference has the potential to create errors when evaluating EQA results. Although the KEQAS data, which are based on peer group mean and 3 SD, suggest that <0.27% of laboratories were outside the allowed range, this study, based on true values and performance criteria, showed that a significant number of laboratories did not meet the performance criteria (42.7% using sample A, 68.4% using sample B, and 38.0% using sample C). The acceptable ranges for all manufacturers in the KEQAS were outside the range of performance criteria. A high SD group has a high probability of deviating from the performance criteria, indicating that a high SD manufacturer is more vulnerable to QC than other manufacturers.

The KEQAS results show that, despite efforts to standardize total cholesterol concentration measurement worldwide, differences in test results still exist across manufacturers and can lead to serious errors in the practical application of various clinical guidelines, including those of the NCEP. There are a variety of factors underlying errors in test results performed in clinical laboratories, and analytical errors depend largely on the manufacturers. The reproducibility of the equipment and the traceability of the calibrator affect the test results. In approaches using peer group mean, analytical performance depends on the precision of the peer group using the same manufacturer’s product, since the peer group mean is regarded as the target value. Therefore, as long as manufacturers only deal with reproducibility issues, the EQA results of user laboratories are not of concern. Statistically, only a small number of laboratories outside the 3 SD (<0.27%) will obtain unacceptable results; thus, biases originating from a specific manufacturer cannot be detected by interpreting the EQA results using the peer group mean. For globally standardized tests such as total cholesterol, accuracy-based EQA can detect bias and imprecision simultaneously because of the target value of the QC materials.

There are many options for analytical performance criteria based on clinical outcomes, including the clinician’s expert opinion, biological variability, and accrediting agency. We selected the performance criteria based on published professional recommendations [1, 12].

Although QC through the KEQAS has improved in many clinical laboratories in Korea, additional efforts are required to keep pace with the recent standardization of lipid testing worldwide. Monitoring EQA result trends and providing feedback to individual manufacturers are also necessary. One of the factors underlying the differences between manufacturers may be calibrator traceability. Some calibrators have been identified as untraceable; even those marked as traceable they show different values when measured using a standard procedure.

Overall industry performance can be elucidated using data from these accreditation and regulatory agencies (such as the KEQAS). However, the manufacturers in this study were not ranked, because the test results were not determined solely by the manufacturer’s products. Pre-analytical elements, such as technicians, transportation, vial-filling imprecision for the QC materials, varying storage stability, shipping, and in-use stability of QC materials owing to handling at user laboratories could also affect the test results. This study has a limitation that a commutability test to exclude matrix effects was not performed [13].

In conclusion, we demonstrated the biases between each peer group means and target values in total cholesterol concentration measurements. EQA data represent a useful source for comparable analysis of in vitro diagnostics. Although significant efforts have been made to accurately measure total cholesterol concentrations to date, further actions are needed by manufacturers to standardize total cholesterol concentration measurement and determine calibrator traceability accurately.

Yoon YA, Kim S, Chun S and Lee YW designed the study, and Yoon YA drafted the manuscript. Lee YW, Kim S, and Lee K provided study materials. Lee YW, Kim S, Lee K, Park HD, Chun S and Min WK have reviewed and revised the manuscript.

No potential conflicts of interest relevant to this article are reported.

This work was supported by the research fund of the Korean Association of External Quality Assessment Service (KEQAS 2019-01) and the Soonchunhyang University Research Fund.

  1. National Cholesterol Education Program (NCEP) Expert Panel on Detection, Evaluation, and Treatment of High Blood Cholesterol in Adults (Adult Treatment Panel III). Third report of the National Cholesterol Education Program (NCEP) Expert Panel on Detection, Evaluation, and Treatment of High Blood Cholesterol in Adults (Adult Treatment Panel III) final report. Circulation 2002;106:3143-421.
    Pubmed CrossRef
  2. Cholesterol Reference Method Laboratory Network. Total cholesterol certification protocol for manufacturers. http://www.cdc.gov/labstandards/pdf/crmln/RevisedTCprotocolOct04.pdf (Updated on Apr 8, 2020)
  3. Miller WG. The role of proficiency testing in achieving standardization and harmonization between laboratories. Clin Biochem 2009;42:232-5.
    Pubmed CrossRef
  4. Kim S, Lee K, Park HD, Lee YW, Chun S, Min WK. Schemes and performance evaluation criteria of Korean Association of External Quality Assessment (KEQAS) for improving laboratory testing. Ann Lab Med 2021;41:230-9.
    Pubmed CrossRef
  5. Kim H, Kim S, Yun YM, Um TH, Chang J, Lee KS, et al. Status of quality control for laboratory tests of medical institutions in Korea: analysis of 10 years of data on external quality assessment participation. Healthcare (Basel) 2020;8:75.
    Pubmed KoreaMed CrossRef
  6. Sundvall J, Leiviskä J, Alfthan G, Vartiainen E. Serum cholesterol during 27 years: assessment of systematic error and affecting factors and their role in interpreting population trends. Clin Chim Acta 2007;378:93-8.
    Pubmed CrossRef
  7. Ross JW, Miller WG, Myers GL, Praestgaard J. The accuracy of laboratory measurements in clinical chemistry: a study of 11 routine chemistry analytes in the College of American Pathologist Chemistry Survey with fresh frozen serum, definitive methods, and reference methods. Arch Pathol Lab Med 1998;122:587-608.
  8. Ellerbe P, Myers GL, Cooper GR, Hertz HS, Sniegoski LT, Welch MJ, et al. A comparison of results for cholesterol in human serum obtained by the Reference Method and by the Definitive Method of the National Reference System for cholesterol. Clin Chem 1990;36:370-5.
    Pubmed CrossRef
  9. Abel LL, Levy BB, Brodie BB, Kendall FE. A simplified method for the estimation of total cholesterol in serum and demonstration of its specificity. J Biol Chem 1952;195:357-66.
    CrossRef
  10. Bowers GN, Jr, Fassett JD, White E 5th. Isotope dilution mass spectrometry and the National Reference System. Anal Chem 1993;65:475R-9R.
    Pubmed CrossRef
  11. Edwards SH, Kimberly MM, Pyatt SD, Stribling SL, Dobbin KD, Myers GL. Proposed serum cholesterol reference measurement procedure by gas chromatography-isotope dilution mass spectrometry. Clin Chem 2011;57:614-22.
    Pubmed CrossRef
  12. Warnick GR, Kimberly MM, Waymack PP, Leary ET, Myers GL. Standardization of measurements for cholesterol, triglycerides, and major lipoproteins. Lab Med 2008;39:481-90.
    CrossRef
  13. Long Q, Qi T, Zhang T, Wang J, Zeng J, Yan Y, et al. Commutability assessment of candidate external quality assessment materials for aminotransferase approaches in China. Ann Lab Med 2021;41:68-76.
    Pubmed KoreaMed CrossRef