Accuracy Assessment of Five Equations Used for Estimating the Glomerular Filtration Rate in Korean Adults
2017; 37(5): 371-380
Ann Lab Med 2021; 41(4): 372-379
Published online July 1, 2021 https://doi.org/10.3343/alm.2021.41.4.372
Copyright © Korean Society for Laboratory Medicine.
Tae-Dong Jeong, M.D., Ph.D.1 , Eun-Jung Cho, M.D., Ph.D.2 , Kyunghoon Lee, M.D., Ph.D.3 , Woochang Lee, M.D., Ph.D.4 , Yeo-Min Yun, M.D., Ph.D.5 , Sail Chun, M.D., Ph.D.4 , Junghan Song, M.D., Ph.D.3 , and Won-Ki Min, M.D., Ph.D.4
1Department of Laboratory Medicine, Ewha Womans University College of Medicine, Seoul, Korea; 2Department of Laboratory Medicine, Hallym University Dongtan Sacred Heart Hospital, Hallym University College of Medicine, Hwaseong, Korea; 3Department of Laboratory Medicine, Seoul National University Bundang Hospital and College of Medicine, Seongnam, Korea; 4Department of Laboratory Medicine, University of Ulsan College of Medicine and Asan Medical Center, Seoul, Korea; 5Department of Laboratory Medicine, Konkuk University School of Medicine, Konkuk University Medical Center, Seoul, Korea
Correspondence to: Won-Ki Min, M.D., Ph.D.
Department of Laboratory Medicine, University of Ulsan College of Medicine and Asan Medical Center, 88 Olympic-ro 43-gil, Songpa-gu, Seoul 05505, Korea
This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.
Background: Accurate serum creatinine (Cr) concentration measurement is essential for evaluating kidney function. In 2011, the Korean Association of External Quality Assessment Service (KEQAS) launched an accuracy-based Cr proficiency testing (ABCr PT) survey. We analyzed long-term data of the KEQAS ABCr PT survey collected between 2011 and 2019 to assess recent trends in Cr assays in Korea.
Methods: The ABCr PT survey including three commutable fresh-frozen serum samples was performed twice a year. The target Cr concentration was assigned using isotope-dilution mass spectrometry. We analyzed data obtained from the participating laboratories, calculated the yearly bias, and evaluated bias trends for the major reagents and instruments. Outliers were excluded from all analysis.
Results: The mean percentage bias based on the total data of all participating laboratories was 10.8% in the 2011-A survey and 0.2% in 2019-B survey. Bias for the major reagents and instruments differed depending on the manufacturer. Enzymatic assays generally showed desirable bias ranging from –3.9% to 3.2% at all Cr concentrations and lower interlaboratory variability than non-enzymatic assays (enzymatic vs. non-enzymatic, 3.3%–7.2% vs. 6.3%–9.1%).
Conclusions: Although the mean percentage bias of Cr assays tends to decrease over time, it is necessary to continuously strive to improve Cr assay accuracy, especially at low concentrations.
Keywords: Accuracy, Bias, Creatinine, Proficiency testing, Survey
The estimated glomerular filtration rate (eGFR) is the single best indicator for evaluating kidney function . eGFR calculations recommended by organizations such as the Kidney Disease Improving Global Outcomes (KDIGO) and National Kidney Disease Education Program (NKDEP) are mainly based on serum creatinine (Cr) and cystatin C, and Cr is widely used in current clinical practice . Cr-based eGFR values form the basis for medical decision making, including diagnosis, classification, prognosis, and determination of treatment policies, for chronic kidney disease and/or acute kidney injury [2-5]. Accurate Cr concentration measurement is essential to maximize the value of the laboratory test data. Clinical laboratories should use a calibrator that is traceable by an isotope dilution mass spectrometry (IDMS) measurement procedure, and the precision and accuracy of Cr concentration measurement should be continuously monitored through internal quality control and proficiency testing (PT) [6-8].
The Korean Association of External Quality Assessment Service (KEQAS) has been conducting the accuracy-based Cr PT (ABCr PT) survey since 2011; we previously reported the results of this survey conducted between 2011 and 2017 . The mean percentage bias for all participating laboratories significantly decreased from 11.1% in 2011 to 2.4% in 2017 . However, the previous study included outliers, and detailed analyses according to the assay principle (i.e., enzymatic vs. Jaffe method) or target Cr concentration (i.e., low vs. high) were not performed and their effects on bias were not evaluated. Until 2019, participation in the KEQAS ABCr PT survey was voluntary; however, 2020 onward, all laboratories participating in KEQAS PT have to participate in the ABCr PT survey. More small- to medium-sized laboratories may participate from 2020; however, to date, mainly medium- to large-sized laboratories were included. As laboratory size can affect analytical performance , we thought it would be meaningful to analyze bias trends based on KEQAS ABCr PT survey data collected until 2019. We analyzed KEQAS ABCr PT survey data from 2011 to 2019 to identify trends in Cr assay standardization across participating laboratories and to explore ways to improve the quality of Cr assays based on long-term ABCr PT survey data.
We retrospectively reviewed the KEQAS ABCr PT survey data. This study did not involve human subjects and was exempted from the approval of Institutional Review Board of Ewha Womans University Seoul Hospital. The KEQAS ABCr PT survey was performed twice a year (A and B trials in sequential order) from 2011 to 2019. Three samples of fresh frozen serum were used in each survey and were processed according to the Clinical & Laboratory Standards Institute document C37-A, with modification . Venous blood was drawn from donors into plastic blood bags without anticoagulant. Within 5 minutes of blood collection, plasma was separated by centrifugation at 1,500×g, 4°C for 8 minutes. The plasma was transferred to a clean and sterile borosilicate glass bottle and allowed to clot at room temperature for 4 hours. The serum was separated by centrifugation at 2,700 ×g for 20 minutes. Cr was spiked to prepare high Cr concentration serum pools. Serum pools were incubated at 4°C for 14 hours under constant, low-speed, magnetic stirrer mixing to ensure homogeneity. The serum pools were then filtered through a 0.22-µm hydrophilic membrane and aliquoted into glass vials at 4°C. The samples were stored at –70°C until shipment to ABCr PT survey participants. The target Cr concentration was measured by IDMS in the Reference Material Institute for Clinical Chemistry Standards (Kawasaki, Japan).
The distribution of Cr data from each participating laboratory was visually observed, and any difference greater than ±50% from the target Cr concentration was considered an outlier and excluded from further analyses. Detailed participant responses and the number of outliers are described in Table 1.
To investigate bias (measured Cr concentration–target Cr concentration) trends in Cr assays for all participants from 2011 to 2019, the mean percentage bias and 95% confidence interval (CI) for the mean were calculated after outlier elimination. For the bias analysis, samples were classified into three groups based on the target Cr concentrations (<88.40 µmol/L [1.00 mg/dL], 88.40–175.92 µmol/L [1.00–1.99 mg/dL], and 176.80–264.32 µmol/L [2.00–2.99 mg/dL]), and into two groups based on the assay principle (enzymatic vs. non-enzymatic methods) of Cr concentration measurement. The non-enzymatic assays were subdivided into kinetic Jaffe with compensation, kinetic Jaffe without compensation, and rate-blanked and compensated kinetic Jaffe. The assay principles reported may differ from those used in actual laboratory practice, as they were simply divided into enzymatic and non-enzymatic methods. The interlaboratory coefficient of variation (CV) depending on the Cr assay principle was calculated. To evaluate the cumulative ABCr PT survey participation effect, bias was analyzed by grouping the responses in order of survey participation. In total, 328 laboratories participated in the ABCr PT survey over nine years. Nine of these laboratories participated only once, and 32 laboratories participated in all 18 surveys. Based on the first bias results, post-hoc analysis was conducted using Dunnett’s test. Forest plots were used to identify bias trends for major reagents and instruments that were used by more than 10 participants per trial of the ABCr PT survey.
For statistical analysis, MedCalc Statistical Software version 18.10.2 (MedCalc Software Bvba, Ostend, Belgium) and Analyse-it for Microsoft Excel 5.51.1 (Analyse-it Software, Leeds, UK) were used.
The mean percentage bias (95% CIs) for all participants was 10.8% (9.5%–12.2%) for 2011-A, and 0.2% (–0.5%–0.9%) for 2019-B. Between 2011 and 2019, the mean percentage bias significantly decreased (
The bias differed depending on the Cr concentration. For the group with Cr concentrations <1.00 mg/dL, the mean percentage bias was 12.8% in 2011 and 3.5% in 2019. The group with Cr concentrations of 1.00–1.99 mg/dL showed 2.5%–6.3% positive bias between 2011 and 2016, and a ±1.5% decrease after 2017. The group with Cr concentrations of 2.00–2.99 mg/dL showed a bias of ±3% regardless of the year (Fig. 2A).
The mean percentage bias for enzymatic assays was within ±4% from 2011 to 2019, and the interlaboratory %CV was 3.3%–7.2%. The mean percentage bias for non-enzymatic methods was 9.8% in 2011 and within ±3.5% since 2014. The interlaboratory %CV of non-enzymatic assays was 6.3%–9.1% (Fig. 2B and Table 2).
Compared with that in the first participation, the bias significantly decreased after more than four participations (1st vs. 4th, difference in mean percentage bias –2.06%, 95% CI –3.35%– –0.77%,
Beckman Coulter reagents showed 6.8%–7.2% positive bias in 2011–2013, <1% positive bias in 2014–2018, and –2.5% negative bias in 2019. Roche reagents had an overall –0.1%–4.1% positive bias. Sekisui reagents showed 6.0%–12.7% positive bias between 2011 and 2017, which decreased to 2.7% in 2019. Siemens reagents had 2.1%–6.1% positive bias between 2011 and 2013, and –5.5% negative bias as of 2014 (Fig. 3A). The bias for Beckman Coulter, Roche, and Siemens instruments was similar to that for the assay reagents. The Hitachi instrument had 4.1%–9.7% positive bias between 2011 and 2017 and 2.6%–5.1% in 2018–2019. The Toshiba instrument had a 13.9% positive bias in 2011, which gradually decreased to –1.2% in 2019 (Fig. 3B).
The strength of this study was that the bias trend analysis for Cr assays was based on KEQAS ABCr PT survey data collected over nine years. The mean percentage bias for all participants, excluding outliers, significantly decreased from 10.8% for 2011-A to 0.2% for 2019-B. The decrease in bias was the most prominent for non-enzymatic assays, with Cr concentrations <1.0 mg/dL. This trend was similar to that in previous studies [10, 12, 13].
For the Beckman Coulter and Siemens platforms, the annual bias had decreased significantly since 2014. This is mainly because an IDMS-traceable calibrator was used in the participating laboratories. Standardized Cr assays with an IDMS-traceable calibrator yielded Cr concentrations 0.1–0.2 mg/dL lower than non-IDMS-traceable assays, indicating that only changes in the calibrator can be expected to improve positive bias in Cr assays [14, 15].
Constant positive bias, probably due to a systematic error, was observed for the Sekisui reagents. The bias especially increased in 2016–2017 and resulted in more than a half of the participants using these reagents reporting “unacceptable” results for low-concentration samples. The main reason for the increased bias in this period was the relatively large number of low-concentration (Cr<1.0 mg/dL) samples. Some laboratories discussed this issue with the manufacturer to decrease the bias and used compensated kinetic Jaffe assays to minimize the constant positive bias. As a result, the bias has decreased as of 2018 (i.e., mean %bias: 11.7% in 2016; 12.7% in 2017; 5.9% in 2018; and 2.7% in 2019). This indicates that assay standardization can be greatly improved when clinical laboratories, manufacturers, and external quality assessment (EQA) agencies combine efforts.
The optimum, desirable, and minimum acceptable bias considering biological variation for Cr assays are 1.7%, 3.4%, and 5.1%, respectively, by the NKDEP and 1.9%, 3.7%, and 5.6%, respectively, by the latest European Federation of Clinical Chemistry and Laboratory Medicine data [16, 17]. ABCr PT survey participants using an enzymatic method had a mean percentage bias of –3.9%–3.2% from 2011 to 2019, satisfying the desirable bias criterion regardless of year and Cr concentration. Enzymatic methods avoid interference by Jaffe-like chromogens, such as glucose, protein, and acetone, allowing more accurate Cr concentration measurements than the Jaffe method [18, 19].
Participants who used non-enzymatic methods exceeded the minimum acceptable bias criterion in 2011, with a 9.8% mean percentage bias, but generally met the desirable bias criterion after 2014. However, unlike the enzymatic method, bias significantly differed depending on the Cr concentration (i.e.,<1.0 mg/dL vs. ≥1.0 mg/dL) for the non-enzymatic method. As of 2014, samples with a Cr concentration ≥1.0 mg/dL showed a –1.3%–2.0% bias and usually met the optimum bias criterion, whereas samples with a Cr concentration <1.0 mg/dL showed 3.6%–7.2% mean percentage bias, resulting in many cases not meeting the minimum acceptable bias criterion. Although the Cr bias for ABCr PT survey participants gradually decreases over time, the non-enzymatic method still overestimates Cr concentration and consequently underestimates eGFR, especially in low-concentration samples. Relatively larger bias in the Jaffe method than in the enzymatic method at low Cr concentrations has been reported [10, 13, 20].
Conversely, a negative bias tendency was observed in high-concentration samples. There was –2% to –3% negative bias in samples with a concentration >2.0 mg/dL. In the Italian study, the bias distribution in 227.2 μmol/L (2.57 mg/dL) samples was –8.4% to –1.0% in 2011 . Similarly, in the 2011 College of American Pathologists survey, the bias for 2.77 mg/dL samples was –4.9%–0.9%, and that of 4.09 mg/dL samples was –4.7%–0.4%, revealing an overall negative bias . The different bias tendencies between low- and high-concentration samples are assumed to be a calibration effect. Currently, the surest way to decrease bias in low-concentration samples and inter-laboratory variation is to use enzymatic Cr assays. However, this method has practical limitations, including high cost.
This study had some limitations. First, the Cr assay status in small laboratories was not represented. Second, although outliers were removed, the definition of an outlier lacks scientific rationale; the mean percentage bias in Cr concentration measurement for the participating laboratories in the KEQAS ABCr PT survey may vary depending on how the outlier is defined. Third, the bias for the detailed Jaffe method was not analyzed.
In conclusion, the long-term KEQAS ABCr PT survey demonstrated the Cr assay standardization status across participants and provided information for improving assay quality. Overall, the bias in Cr assay tends to decrease with participation in the KEQAS ABCr PT survey; however, continuous efforts are needed to decrease the bias for low-concentration samples. Clinical laboratories should consistently monitor Cr assay accuracy by using standardized calibrators and reagents that maintain metrological traceability, and by participating in an ABCr PT program. Coordinated efforts of clinical laboratories, manufacturers, and EQA agencies may further standardize Cr assay.
The KEQAS provided the ABCr PT survey data. We acknowledge all participating laboratories for their significant contribution to improving the quality of Cr assays in Korea.
TDJ designed the study, analyzed the data, and wrote the draft; WKM conceived the study, analyzed the data, and finalized the draft; EJC and KL discussed the data; WL, YMY, and SC discussed the data and reviewed the manuscript. All authors read and approved the final manuscript.