There is a persistent need for early identification of patients at risk of cardiac surgery-associated acute kidney injury (AKI) because of the increased need for kidney replacement therapy (KRT) and increased risk of in-hospital mortality [1].

Despite advances in the conceptual understanding of the pathophysiology and epidemiology of cardiac surgery-associated AKI [2], consensus definitions and staging of AKI remain based on relative changes in serum creatinine (sCr) concentration, a decline in urine output (UO) [3], and the provision of KRT [4].

Several prospective observational studies have assessed whether the use of kidney tubular damage biomarkers such as urine neutrophil gelatinase-associated lipocalin (NGAL) improves early kidney risk assessment when measured in addition to sCr concentration [5^-8]. The performance of such biomarkers as predictors of subsequent changes in sCr concentrations is poor, which may be influenced by various confounders such as the choice of the consensus definition of AKI, clinical setting, or type of sample used to measure NGAL (urine or plasma) [9].

Patients testing positive for kidney tubular damage biomarkers may not fulfill the traditional AKI criteria and may still have worse outcomes compared to those with negative biomarker findings. This recognition led to the proposal that new AKI diagnostic criteria should equally include both glomerular function markers (sCr) and kidney damage biomarkers (e.g., NGAL or other injury biomarkers) based on thresholds; however, there is still no consensus of the appropriate thresholds for such assessments [10]. Multiple statistical methodologies have been suggested for determining the “optimal” cutoff values for such tasks [11]. Therefore, using multiple methodologies, studies have determined and reported study-specific optimal NGAL cutoff concentrations to define NGAL positivity or negativity [5^-7, 12^-15].

We hypothesized that differences in the methodology of NGAL cutoff value determination and the system chosen for AKI classification influence patients’ clinical AKI-phenotype group allocation and the associated risk of adverse events according to the proposed AKI matrix system [10, 16]. Using data from two independent prospective cardiac surgery study cohorts [17, 18], we aimed to explore the extent to which such changes in methodologies and classification systems affect the calculated risk of adverse outcome [15, 16]. We hypothesized that patient outcomes and the magnitude of the attributed risk of adverse events between clinical AKI-phenotype groups vary according to the methodology used for NGAL cutoff selection and the choice of classification system for sCr/UO-based AKI.

Study design

We used the existing data of 200 patients at an increased risk of AKI from the Berlin study cohort of a randomized, multicenter study on cardiac surgery (NCT00672334) performed at the German Heart Center Berlin between May 2008 and January 2012 [17]. Urine samples were obtained upon patients’ admission to the intensive care unit (ICU) and were immediately centrifuged at 5,000 rpm (~ 3,075 g) and stored at −80°C. Following completion of patient enrollment, NGAL concentrations in urine samples were measured using badge analysis as described below.

The second study cohort included 103 consecutively enrolled adult patients undergoing open-heart surgery at the University Clinic Magdeburg, Germany, which aimed to explore postoperative clinical kidney risk assessment using urine sampled at ICU admission for each patient and NGAL measured individually within 60 minutes according to routine laboratory diagnostics [18].

In both cohorts, NGAL measurements were performed in the central laboratory at the Institute of Clinical Chemistry and Pathobiochemistry of the University Clinic Magdeburg, using a standardized clinical platform assay for urinary NGAL (ng/mL; ARCHITECT, Abbott Diagnostics, Abbott Park, IL, USA), which is a Conformité Européenne (CE)-certified biomarker for the diagnosis of AKI in Germany. The CE certification mark indicates conformity with the health, safety, and environmental protection standards for products sold within the European Economic Area. The measurement interval of the ARCHITECT urine NGAL assay is 10.0–1,500.0 ng/mL, with an imprecision of ≤10% total CV. When using an automated dilution procedure, the assay can report values of up to 6,000.0 ng/mL. Further data on assay performance and handling requirements used in this study are described elsewhere [19]. The sCr concentrations were measured via routine laboratory diagnostics using an enzymatic method standardized by isotope dilution-mass spectroscopy on a Cobas 8000 modular analyzer (Roche Diagnostics, Mannheim, Germany). Complete sCr and urinary NGAL data were available for 199/200 and 100/103 patients of the Berlin and Magdeburg cohorts, respectively. Patients with missing data were excluded from analysis.

Laboratory investigators were blinded to the sample sources and clinical outcomes. In the Berlin study cohort, Medicine Ethics Committee, Charité University, Berlin, Germany approved the study (ZS EK 11 654/07), and written informed consent was obtained from each patient [17]. In the study performed at the University Clinic Magdeburg, Germany, the local institutional Review Board categorized the study as an audit of current clinical practice and approved prospective data collection and use for the purpose of this study. Patients received written study information, but the need for written consent was waived by the ethics committee (Ethics Committee, University of Magdeburg, Case 49/13) [18]. Full study details have been described previously [17, 18].

Definition of sCr/UO-based AKI

sCr/UO-based AKI was defined according to the Renal risk, Injury, Failure, Loss of kidney function, End-stage (RIFLE) [3] or Kidney Disease: Improving Global Outcomes (KDIGO) classification system [4], considering increases in postoperative sCr concentrations from baseline (preoperative stage) and the decline in UO criteria in both cohorts. Both classification systems define AKI as an increase in sCr concentration by a factor of 1.5-times from baseline or a decline in UO to less than 0.5 mL/kg/hr for up to 6–12 hours. In contrast to RIFLE, the KDIGO classification additionally considers an increase of ≥0.3 mg/dL (≥26.5 µmol/L) within 48 hours as AKI stage 1.

Methodology of NGAL cutoff selection

We used five methods to derive candidate NGAL cutoff values for clinical AKI phenotypes. In both datasets, the area under the ROC curve (AUC) value was calculated separately for the dichotomous outcome measure sCr/UO-based AKI according to RIFLE and KDIGO classification criteria fulfilling at least RIFLE-risk or KDIGO stage 1, respectively. We chose candidate cutoff values based on study-patient ROC curve analysis [11, 20], where each point on the ROC curve corresponds to a potential cutoff value and is associated with a sensitivity and specificity value pair. Threshold selection represents a compromise between sensitivity and specificity (Fig. 1) [21]. Previously, to derive the NGAL and sCr/UO-based AKI matrix, there has been no preference of favoring either high sensitivity or specificity over the compromise for an optimal balance of both indices [22]. Similarly, for the present derivation of cutoff values, no weighting of sensitivity and specificity was applied.

Figure 1. Derivation of a 2×2 contingency table according to a chosen NGAL cutoff value in an example distribution of patients with and without sCr/UO-based AKI. (A) Example distribution of sCr/UO-based AKI and AKI-free patients according to NGAL concentration on the X-axis. When these distributions overlap in their NGAL concentrations, type 1 (FP) and type 2 (FN) errors are introduced. The ROC space was then defined by the FPR (1–specificity) and the TPR. An ROC curve for a dichotomous outcome measure is generated by a cohort’s finite set of 2×2 cell matrices or contingency tables, where each represents a trade-off between specificity and sensitivity pairs. Decreasing the cutoff value would result in fewer FNs but consecutively increase the number of FPs (B). Increasing the cutoff value would result in fewer FPs but consecutively more FNs (C). The proportions of attributed clinical AKI phenotypes corresponding to the figure can be derived depending on the chosen NGAL cutoff values in the 2×2 table [¹⁶, ³³].
Abbreviations: AKI, acute kidney injury; FN, false negative; FP, false positive; FPR, false positive rate; NGAL, neutrophil gelatinase-associated lipocalin; RIFLE, Risk, Injury, Failure, Loss, and End-stage; sCr, serum creatinine; TPR, true positive rate; UO, urine output.

To determine an optimal cutoff value to differentiate between patients with the condition (sCr/UO-AKI) and those free of the condition (AKI-free), various methods have been proposed aiming to narrow the gap to the point in the ROC space that mathematically maximizes the AUC to “perfect test prediction”; that is, the point at coordinates (0, 1) in the upper-left corner of the ROC space.

The first method is based on previous studies [5, 13] that selected the maximum Youden index [23] as the cutoff value, which maximizes the sum of sensitivity and specificity by maximizing the distance from the diagonal chance line (y=x) connecting (0, 0) to (1, 1) in the ROC space. The chance line itself approximates an AUC value of 0.5. The Youden index for a given point on the ROC curve is calculated as follows:

The maximum Youden index is the point on the ROC curve farthest from the chance line [11, 24].

The second method is one that balances sensitivity and specificity in assessing when the test sensitivity is equivalent to the test specificity. This point on the curve is located on the line connecting the upper-left corner (0, 1) to the lower-right corner (1, 0) of the ROC space. Hypothetically, at the point where sensitivity equals specificity, the product of these two indices (sensitivity×specificity) is at its maximum [11]. In this study, we chose the point closest to the equivalence of sensitivity and specificity, referred to as sensitivity≈specificity.

The third method minimizes the distance of a candidate cutoff value to the point on the curve closest to perfection (0, 1). This cutoff value should correspond to the optimal cutoff value chosen from all potential points available on the ROC curve [24].

Mathematically, searching for the shortest radius originating from (0, 1), the square of the distance from each data point on the ROC curve to (0, 1) is first calculated as:

where TPF is the true-positive fraction and FPF is the false-positive fraction [24]. The distance D to (0, 1) of any given point on the ROC curve is then calculated as follows:

Finally, two cutoff values were adopted from two meta-analyses investigating the ability of urinary NGAL to predict sCr/UO-based AKI [9, 25].

Derivation of the AKI phenotype matrix

Patients were allocated to the clinical AKI phenotype matrix groups derived from each candidate cutoff value of 2×2 contingency table cells from the ROC curve with regard to meeting or not meeting the criteria for any stage of sCr/UO-based AKI, separately for RIFLE and KDIGO (functional impairment, dichotomization).

NGAL positivity (+), indicating the presence of structural or tubular damage, or NGAL negativity (–) was determined according to whether the urinary NGAL concentration measured at ICU admission after cardiac surgery was above or equal (≥) or lower (<) than each candidate cutoff value, respectively.

In summary, the following four different combinations defining the patient groups could be distinguished considering the RIFLE/KDIGO and NGAL statuses individually for each candidate cutoff value (Fig. 2): NGAL(–)/sCr/UO(–), NGAL(+)/sCr/UO(–), NGAL(–)/sCr/UO(+), and NGAL(+)/sCr/UO(+).

Figure 2. Matrix of attributed clinical AKI phenotypes derived from the NGAL test-based 2×2 contingency table. (A) Attributed clinical AKI phenotypes plotted in a 2×2 contingency table or “matrix.” Functional impairment is attributed to increased sCr concentrations or reduced UO and defined by positive RIFLE or KDIGO criteria. Structural damage is attributed to increased NGAL concentrations above a candidate threshold concentration. Three scenarios of potential functional and structural kidney impairment can be distinguished. (B) Corresponding test classification matrix: NGAL(–)/sCr)/UO(–), TN, no kidney impairment; NGAL(+)/sCr/UO(–), FP, subclinical AKI (AKI stage 1 S); NGAL(–)/sCr/UO(+), FN, hemodynamic AKI, volume depletion, diminished kidney functional reserve (AKI stage 1–3A); NGAL(+)/sCr/UO(+), TP, AKI with functional and structural impairment (AKI stage 1–3B).
Abbreviations: AKI, acute kidney injury; FN, false negative; FP, false positive; KDIGO, Kidney Disease Improving Global Outcomes; NGAL, neutrophil gelatinase-associated lipocalin; sCr, serum creatinine; TN, true negative; TP, true positive; UO, urine output.

Outcome measures

The predefined outcome measures were acute KRT initiation, in-hospital mortality, and the combination of KRT or in-hospital mortality.

Statistical analysis

The odds ratio (OR) with 95% confidence interval (CI) was calculated to assess the associated difference in the risk of developing an outcome measure between the patient groups derived from the corresponding AKI phenotype matrix. When there were groups with zeros (N=0), we introduced a corrective factor of 0.5 for all cells. The two groups of zeros were not corrected or compared with respect to systematic differences. We then calculated the magnitude of risk disparity between the methodologies used for cutoff value selection with the following formula:

Since the OR is a relative measure of risk, a value of 1.0 would correspond to the assumption of no difference in the magnitude of risk of developing an outcome of interest between the cutoff selection methodologies “a” and “b.” SPSS version 28.0 (IBM Corp., Armonk, NY, USA), R Environment for Statistical Computing [26], and Microsoft Excel 365 version 16.43 (Redmond, WA, USA) were used for statistical analyses.

In the Magdeburg cohort, 14 (14.0%) patients had AKI according to the RIFLE classification and 23 (23.0%) had AKI according to the KDIGO classification, whereas in the Berlin cohort, 24 (12.1%) patients had AKI according to the RIFLE classification and 59 (29.6%) patients had AKI according to the KDIGO classification. The detailed perioperative patient characteristics of both cohorts are presented in Supplemental Data Table S1 and in Supplemental Data Fig. S1. The ROC curves and associated AUC values of NGAL in predicting RIFLE- and KDIGO-defined AKI in both cohorts are presented in Supplemental Data Fig. S2. All derived cutoff values and their associated statistical metrics are listed in Table 1.

Table 1 . Risk assessment metrics according to various methods used to calculate the urine NGAL cutoff concentration for AKI (all stages) defined by the RIFLE and KDIGO classification systems in the Magdeburg and Berlin cohorts

Methodology	NGAL cutoff*, ng/mL	Sensitivity	Specificity	D†	Youden J‡	DOR (95% CI)	LR+ (95% CI)	LR– (95% CI)	PPV (95% CI)	NPV (95% CI)	AKI prevalence, %
Magdeburg cohort
RIFLE AKI classification used as dependent outcome measure
Sensitivity ≈ specificity	10.6	0.714	0.709	0.408	0.423	6.10 (1.75–21.28)	2.46 (1.54–3.92)	0.40 (0.17–0.93)	0.29 (0.14–0.44)	0.94 (0.88–1.00)	14.0
Lowest distance D to (0,1)	11.75	0.714	0.733	0.391	0.447	6.85 (1.95–24.00)	2.67 (1.65–4.32)	0.39 (0.17–0.90)	0.30 (0.15–0.46)	0.94 (0.88–1.00)	14.0
Maximum Youden index	45.55	0.500	0.953	0.502	0.453	20.50 (4.80–87.46)	10.75 (3.61–32.00)	0.52 (0.31–0.89)	0.64 (0.35–0.92)	0.92 (0.87–0.98)	14.0
Albert, et al. [9]§	66.3	0.357	0.965	0.644	0.322	15.37 (3.14–75.23)	10.24 (2.75–38.14)	0.67 (0.45–0.99)	0.63 (0.29–0.96)	0.90 (0.84–0.96)	14.0
Haase, et al. [25]||	159.1	0.214	0.965	0.787	0.179	7.55 (1.35–42.12)	6.14 (1.37–27.46)	0.81 (0.62–1.07)	0.50 (0.10–0.90)	0.88 (0.82–0.95)	14.0
KDIGO AKI classification used as dependent outcome measure
Sensitivity ≈ specificity	8.0	0.565	0.571	0.611	0.136	1.73 (0.68–4.44)	1.32 (0.85–2.05)	0.76 (0.46–1.26)	0.28 (0.15–0.41)	0.81 (0.71–0.92)	23.0
Lowest distance D to (0,1)	9.5	0.565	0.688	0.535	0.253	2.87 (1.10–7.46)	1.81 (1.11–2.96)	0.63 (0.39–1.03)	0.35 (0.20–0.51)	0.84 (0.75–0.93)	23.0
Maximum Youden index	45.55	0.348	0.961	0.653	0.309	13.16 (3.12–55.43)	8.93 (2.58–30.93)	0.68 (0.50–0.92)	0.73 (0.46–0.99)	0.83 (0.75–0.91)	23.0
Albert, et al. [9]§	66.3	0.261	0.974	0.739	0.235	13.24 (2.46–71.35)	10.04 (2.17–46.43)	0.76 (0.59–0.97)	0.75 (0.45–1.05)	0.82 (0.74–0.89)	23.0
Haase, et al. [25]||	159.1	0.174	0.974	0.826	0.148	7.89 (1.34–46.37)	6.70 (1.31–34.25)	0.85 (0.70–1.03)	0.67 (0.29–1.04)	0.80 (0.72–0.88)	23.0
Berlin cohort
RIFLE AKI classification used as dependent outcome measure
Sensitivity ≈ specificity	16.85	0.625	0.623	0.532	0.248	2.75 (1.14–6.64)	1.66 (1.15–2.38)	0.60 (0.35–1.02)	0.19 (0.10–0.27)	0.92 (0.88–0.97)	12.1
Lowest distance to (1,0)	50.0	0.625	0.771	0.439	0.396	5.28 (2.15–12.93)	2.60 (1.73–3.91)	0.49 (0.29–0.83)	0.26 (0.15–0.38)	0.94 (0.90–0.98)	12.1
Maximum Youden index	50.0	0.625	0.771	0.439	0.396	5.28 (2.15–12.93)	2.60 (1.73–3.91)	0.49 (0.29–0.83)	0.26 (0.15–0.38)	0.94 (0.90–0.98)	12.1
Albert, et al. [9]§	79.3	0.542	0.811	0.495	0.353	5.09 (2.09–12.36)	2.87 (1.78–4.64)	0.56 (0.36–0.88)	0.28 (0.15–0.41)	0.93 (0.89–0.97)	12.1
Haase, et al. [25]||	149.25	0.458	0.846	0.563	0.304	4.64 (1.88–11.43)	2.97 (1.70–5.18)	0.64 (0.44–0.93)	0.29 (0.15–0.43)	0.92 (0.88–0.96)	12.1
KDIGO AKI classification used as dependent outcome measure
Sensitivity ≈ specificity	14.25	0.627	0.629	0.526	0.256	2.85 (1.52–5.34)	1.69 (1.26–2.26)	0.59 (0.42–0.85)	0.42 (0.31–0.52)	0.80 (0.73–0.87)	29.7
Lowest distance to (1,0)	14.75	0.627	0.636	0.521	0.263	2.93 (1.56–5.51)	1.72 (1.28–2.31)	0.59 (0.41–0.84)	0.42 (0.32–0.52)	0.80 (0.73–0.88)	29.7
Maximum Youden index	36.5	0.508	0.779	0.539	0.287	3.64 (1.90–6.95)	2.30 (1.54–3.42)	0.63 (0.48–0.83)	0.49 (0.37–0.62)	0.79 (0.72–0.86)	29.7
Albert, et al. [9]§	79.3	0.356	0.821	0.668	0.177	2.54 (1.28–5.05)	1.99 (1.22–3.27)	0.78 (0.64–0.96)	0.46 (0.31–0.60)	0.75 (0.68–0.82)	29.7
Haase, et al. [25]||	149.25	0.305	0.857	0.710	0.162	2.63 (1.27–5.46)	2.14 (1.22–3.74)	0.81 (0.68–0.97)	0.47 (0.31–0.63)	0.75 (0.68–0.81)	29.7

*Considered a positive result if greater than or equal to the cutoff; ^†D²=(1–TPF)²+FPF²=(1–sensitivity)²+(1–specificity)²; ^‡Youden index (J) defined as TPF–FPF=sensitivity+specificity−1; ^§We chose the value closest to that suggested by Albert, et al. [9] (81 ng/mL); ^||We chose the value closest to that suggested by Haase, et al. [25] (150 ng/mL).

Abbreviations: AKI, acute kidney injury; KDIGO, Kidney Disease Improving Global Outcomes; NGAL, neutrophil gelatinase-associated lipocalin; NPV, negative predictive value; PPV, positive predictive value; RIFLE, Risk, Injury, Failure, Loss, End-stage; TPF, true positive fraction; FPF, false positive fraction.

The cutoff concentrations calculated from the corresponding ROC curves differed according to both the methodology used and the choice of RIFLE or KDIGO classification, ranging from 10.6 to 159.1 ng/mL in the Magdeburg cohort and from 16.85 to 149.25 ng/mL in the Berlin cohort. In both cohorts, the findings obtained when using the sensitivity≈specificity and lowest distance methods were similar. In the Berlin cohort, using the RIFLE classification, the lowest distance method yielded the same cutoff value as the maximum Youden index. Lower cutoff values subsided with distinctly higher sensitivity for sCr/UO-based AKI (RIFLE and KDIGO), as opposed to specificity improvements for higher cutoff values.

The fraction of potential subclinical AKI (AKI stage 1 S) shifted from 2% to 33% and from 10.1% to 26.1% for the KDIGO classification in the Magdeburg and Berlin cohorts, respectively (Table 2). With increasing cutoff values, more patients were eventually allocated to the NGAL-negative group.

Table 2 . Patients’ allocation and outcome according to NGAL/RIFLE and NGAL/KDIGO groups, N (%)

Cutoff methodology	sCr/UO-AKI	NGAL cutoff* (ng/mL)	NGAL(–)/ sCr/UO(–)†	NGAL(+)/ sCr/UO(–)‡	NGAL(–)/ sCr/UO(+)§	NGAL(+)/ sCr/UO(+)ll
Allocation
Magdeburg cohort
Sensitivity ≈ specificity	RIFLE	10.6	61 (61.0)	25 (25.0)	4 (4.0)	10 (10.0)
Lowest distance D to (0,1)	RIFLE	11.75	63 (63.0)	23 (23.0)	4 (4.0)	10 (10.0)
Maximum Youden index	RIFLE	45.55	82 (82.0)	4 (4.0)	7 (7.0)	7 (7.0)
Albert, et al. [9]¶	RIFLE	66.3	83 (83.0)	3 (3.0)	9 (9.0)	5 (5.0)
Haase, et al. [25]**	RIFLE	159.1	83 (83.0)	3 (3.0)	11 (11.0)	3 (3.0)
Sensitivity ≈ specificity	KDIGO	8.0	44 (44.0)	33 (33.0)	10 (10.0)	13 (13.0)
Lowest distance D to (0,1)	KDIGO	9.5	53 (53.0)	24 (24.0)	10 (10.0)	13 (13.0)
Maximum Youden index	KDIGO	45.55	74 (74.0)	3 (3.0)	15 (15.0)	8 (8.0)
Albert, et al. [9]¶	KDIGO	66.3	75 (75.0)	2 (2.0)	17 (17.0)	6 (6.0)
Haase, et al. [25]**	KDIGO	159.1	75 (75.0)	2 (2.0)	19 (19.0)	4 (4.0)
Berlin cohort
Sensitivity ≈ specificity	RIFLE	16.85	109 (54.8)	66 (33.2)	9 (4.5)	15 (7.5)
Lowest distance D to (0,1)	RIFLE	50.0	133 (66.8)	42 (21.1)	9 (4.5)	15 (7.5)
Maximum Youden index	RIFLE	50.0	133 (66.8)	42 (21.1)	9 (4.5)	15 (7.5)
Albert, et al. [9]¶	RIFLE	79.3	142 (71.4)	33 (16.6)	11 (5.5)	13 (6.5)
Haase, et al. [25]**	RIFLE	149.25	148 (74.4)	27 (13.6)	13 (6.5)	11 (5.5)
Sensitivity ≈ specificity	KDIGO	14.25	88 (44.2)	52 (26.1)	22 (11.1)	37 (18.6)
Lowest distance D to (0,1)	KDIGO	14.75	89 (44.7)	51 (25.6)	22 (11.1)	37 (18.6)
Maximum Youden index	KDIGO	36.5	109 (54.8)	31 (15.6)	29 (14.6)	30 (15.1)
Albert, et al. [9]¶	KDIGO	79.3	115 (57.8)	25 (12.6)	38 (19.1)	21 (10.6)
Haase, et al. [25]**	KDIGO	149.25	120 (60.3)	20 (10.1)	41 (20.6)	18 (9.1)
KRT initiation
Magdeburg cohort
Sensitivity ≈ specificity	RIFLE	10.6	0 (0.0)	0 (0.0)	0 (0.0)	6 (60.0)
Lowest distance D to (0,1)	RIFLE	11.75	0 (0.0)	0 (0.0)	0 (0.0)	6 (60.0)
Maximum Youden index	RIFLE	45.55	0 (0.0)	0 (0.0)	2 (28.6)	4 (57.1)
Albert, et al. [9]¶	RIFLE	66.3	0 (0.0)	0 (0.0)	3 (33.3)	3 (60.0)
Haase, et al. [25] **	RIFLE	159.1	0 (0.0)	0 (0.0)	5 (45.5)	1 (33.3)
Sensitivity ≈ specificity	KDIGO	8.0	0 (0.0)	0 (0.0)	0 (0.0)	6 (46.2)
Lowest distance D to (0,1)	KDIGO	9.5	0 (0.0)	0 (0.0)	0 (0.0)	6 (46.2)
Maximum Youden index	KDIGO	45.55	0 (0.0)	0 (0.0)	2 (13.3)	4 (50.0)
Albert, et al. [9]¶	KDIGO	66.3	0 (0.0)	0 (0.0)	3 (17.6)	3 (50.0)
Haase, et al. [25]**	KDIGO	159.1	0 (0.0)	0 (0.0)	5 (26.3)	1 (25.0)
Berlin cohort
Sensitivity ≈ specificity	RIFLE	16.85	2 (1.8)	3 (4.5)	1 (11.1)	7 (46.7)
Lowest distance D to (0,1)	RIFLE	50.0	2 (1.5)	3 (7.1)	1 (11.1)	7 (46.7)
Maximum Youden index	RIFLE	50.0	2 (1.5)	3 (7.1)	1 (11.1)	7 (46.7)
Albert, et al. [9]¶	RIFLE	79.3	4 (2.8)	1 (3.0)	1 (9.1)	7 (53.8)
Haase, et al. [25]**	RIFLE	149.25	4 (2.7)	1 (3.7)	2 (15.4)	6 (54.5)
Sensitivity ≈ specificity	KDIGO	14.25	0 (0.0)	0 (0.0)	3 (13.6)	10 (27.0)
Lowest distance D to (0,1)	KDIGO	14.75	0 (0.0)	0 (0.0)	3 (13.6)	10 (27.0)
Maximum Youden index	KDIGO	36.5	0 (0.0)	0 (0.0)	3 (10.3)	10 (33.3)
Albert, et al. [9]¶	KDIGO	79.3	0 (0.0)	0 (0.0)	5 (13.2)	8 (38.1)
Haase, et al. [25]**	KDIGO	149.25	0 (0.0)	0 (0.0)	6 (14.6)	7 (38.9)
In-hospital mortality
Magdeburg cohort
Sensitivity ≈ specificity	RIFLE	10.6	0 (0.0)	0 (0.0)	0 (0.0)	3 (30.0)
Lowest distance D to (0,1)	RIFLE	11.75	0 (0.0)	0 (0.0)	0 (0.0)	3 (30.0)
Maximum Youden index	RIFLE	45.55	0 (0.0)	0 (0.0)	1 (14.3)	2 (28.6)
Albert, et al. [9]¶	RIFLE	66.3	0 (0.0)	0 (0.0)	2 (22.2)	1 (20.0)
Haase, et al. [25]**	RIFLE	159.1	0 (0.0)	0 (0.0)	2 (18.2)	1 (33.3)
Sensitivity ≈ specificity	KDIGO	8.0	0 (0.0)	0 (0.0)	0 (0.0)	3 (23.1)
Lowest distance D to (0,1)	KDIGO	9.5	0 (0.0)	0 (0.0)	0 (0.0)	3 (23.1)
Maximum Youden index	KDIGO	45.55	0 (0.0)	0 (0.0)	1 (6.7)	2 (25.0)
Albert, et al. [9]¶	KDIGO	66.3	0 (0.0)	0 (0.0)	2 (11.8)	1 (16.7)
Haase, et al. [25]**	KDIGO	159.1	0 (0.0)	0 (0.0)	2 (10.5)	1 (25.0)
Berlin cohort
Sensitivity ≈ specificity	RIFLE	16.85	1 (0.9)	4 (6.1)	2 (22.2)	6 (40.0)
Lowest distance D to (0,1)	RIFLE	50.0	1 (0.8)	4 (9.5)	2 (22.2)	6 (40.0)
Maximum Youden index	RIFLE	50.0	1 (0.8)	4 (9.5)	2 (22.2)	6 (40.0)
Albert, et al. [9]¶	RIFLE	79.3	3 (2.1)	2 (6.1)	3 (27.3)	5 (38.5)
Haase, et al. [25]**	RIFLE	149.25	3 (2.0)	2 (7.4)	3 (23.1)	5 (45.5)
Sensitivity ≈ specificity	KDIGO	14.25	1 (1.1)	2 (3.8)	2 (9.1)	8 (21.6)
Lowest distance D to (0,1)	KDIGO	14.75	1 (1.1)	2 (3.9)	2 (9.1)	8 (21.6)
Maximum Youden index	KDIGO	36.5	1 (0.9)	2 (6.5)	2 (6.9)	8 (26.7)
Albert, et al. [9]¶	KDIGO	79.3	2 (1.7)	1 (4.0)	4 (10.5)	6 (28.6)
Haase, et al. [25]**	KDIGO	149.25	2 (1.7)	1 (5.0)	4 (9.8)	6 (33.3)
KRT initiation or in-hospital mortality
Magdeburg cohort
Sensitivity ≈ specificity	RIFLE	10.6	0 (0.0)	0 (0.0)	0 (0.0)	6 (60.0)
Lowest distance D to (0,1)	RIFLE	11.75	0 (0.0)	0 (0.0)	0 (0.0)	6 (60.0)
Maximum Youden index	RIFLE	45.55	0 (0.0)	0 (0.0)	2 (28.6)	4 (57.1)
Albert, et al. [9]¶	RIFLE	66.3	0 (0.0)	0 (0.0)	3 (33.3)	3 (60.0)
Haase, et al. [25]**	RIFLE	159.1	0 (0.0)	0 (0.0)	5 (45.5)	1 (33.3)
Sensitivity ≈ specificity	KDIGO	8.0	0 (0.0)	0 (0.0)	0 (0.0)	6 (46.2)
Lowest distance D to (0,1)	KDIGO	9.5	0 (0.0)	0 (0.0)	0 (0.0)	6 (46.2)
Maximum Youden index	KDIGO	45.55	0 (0.0)	0 (0.0)	2 (13.3)	4 (50.0)
Albert, et al. [9]¶	KDIGO	66.3	0 (0.0)	0 (0.0)	3 (17.6)	3 (50.0)
Haase, et al. [25]**	KDIGO	159.1	0 (0.0)	0 (0.0)	5 (26.3)	1 (25.0)
Berlin cohort
Sensitivity ≈ specificity	RIFLE	16.85	3 (2.8)	5 (7.6)	2 (22.2)	11 (73.3)
Lowest distance D to (0,1)	RIFLE	50.0	3 (2.3)	5 (11.9)	2 (22.2)	11 (73.3)
Maximum Youden index	RIFLE	50.0	3 (2.3)	5 (11.9)	2 (22.2)	11 (73.3)
Albert, et al. [9]¶	RIFLE	79.3	6 (4.2)	2 (6.1)	3 (27.3)	10 (76.9)
Haase, et al. [25] **	RIFLE	149.25	6 (4.1)	2 (7.4)	4 (30.8)	9 (81.8)
Sensitivity ≈ specificity	KDIGO	14.25	1 (1.1)	2 (3.8)	4 (18.2)	14 (37.8)
Lowest distance D to (0,1)	KDIGO	14.75	1 (1.1)	2 (3.9)	4 (18.2)	14 (37.8)
Maximum Youden index	KDIGO	36.5	1 (0.9)	2 (6.5)	4 (13.8)	14 (46.7)
Albert, et al. [9]¶	KDIGO	79.3	2 (1.7)	1 (4.0)	7 (18.4)	11 (52.4)
Haase, et al. [25]**	KDIGO	149.25	2 (1.7)	1 (5.0)	8 (19.5)	10 (55.6)

*A positive test result is determined if greater than or equal to the cutoff value; ^†No damage or functional impairment [10, 16]; ^‡Subclinical AKI (stage 1 S); kidney tubular injury indicated by positive NGAL, no functional impairment detected [10, 16]; ^§Hemodynamic AKI (stage 1–3A); no tubular injury detected, functional impairment present [10, 16]; ^llAKI stage 1–3B; tubular injury and functional impairment present [10, 16]; ^¶We chose the value closest to that suggested by Albert, et al. [9] (81 ng/mL); **We chose the value closest to that suggested by Haase, et al. [25] (150 ng/mL).

Abbreviations: KDIGO, Kidney Disease Improving Global Outcomes; KRT, kidney replacement therapy; NGAL, neutrophil gelatinase-associated lipocalin; RIFLE, Risk, Injury, Failure, Loss, and End-Stage Kidney Disease; sCr, serum creatinine; UO, urine output.

When the highest cutoff values were chosen, more patients with adverse events (i.e., those who were more likely to have higher NGAL concentrations) were reassigned to the NGAL-negative group, which resulted in groups with NGAL-negative status having more events than those with NGAL-positive status (OR <1.0; Table 2 and Table 3, Fig. 1C).

Table 3 . Odds ratios for the risk of the outcome measures between the NGAL/RIFLE and NGAL/KDIGO groups

Cutoff methodology	sCr/UO-AKI	NGAL(+)/sCr/UO(–)* vs. reference†	NGAL(–)/sCr(+)‡ vs. reference	NGAL(+)/sCr/UO(+)§ vs. reference	NGAL(+)/sCr(+) vs. NGAL(–)/sCr/UO(+)
KRT initiation
Magdeburg cohort
Sensitivity ≈ specificity	RIFLE	NC	NC	177.67 (8.57–3,682.96)	13.00 (0.55–306.22)
Lowest distance D to (0,1)	RIFLE	NC	NC	183.44 (8.85–3,801.49)	13.00 (0.55–306.22)
Maximum Youden index	RIFLE	NC	75.00 (3.19–1,762.00)	212.14 (9.45–4,763.31)	2.83 (0.37–21.89)
Albert, et al. [9]¶	RIFLE	NC	89.92 (4.18–1,934.88)	233.80 (9.35–5,847.08)	2.60 (0.33–20.79)
Haase, et al. [25]**	RIFLE	NC	141.31 (7.01–2,847.09)	100.20 (3.22–3,121.90)	0.71 (0.07–7.22)
Sensitivity ≈ specificity	KDIGO	NC	NC	77.13 (3.92–1,516.64)	18.20 (0.88–374.91)
Lowest distance D to (0,1)	KDIGO	NC	NC	92.73 (4.73–1,818.94)	18.20 (0.88–374.91)
Maximum Youden index	KDIGO	NC	27.59 (1.25–607.18)	149.00 (6.90–3,218.77)	5.40 (0.83–35.33)
Albert, et al. [9]¶	KDIGO	NC	36.45 (1.79–743.94)	151.00 (6.46–3,527.54)	4.14 (0.63–27.32)
Haase, et al. [25]**	KDIGO	NC	57.28 (3.00–1,093.38)	64.71 (2.21–1,891.82)	1.13 (0.13–9.70)
Berlin cohort
Sensitivity ≈ specificity	RIFLE	2.55 (0.41–15.66)	6.69 (0.55–81.94)	46.81 (8.32–263.50)	7.00 (0.69–70.75)
Lowest distance D to (0,1)	RIFLE	5.04 (0.81–31.24)	8.19 (0.67–100.18)	57.31 (10.20–321.99)	7.00 (0.69–70.75)
Maximum Youden index	RIFLE	5.04 (0.81–31.24)	8.19 (0.67–100.18)	57.31 (10.20–321.99)	7.00 (0.69–70.75)
Albert, et al. [9]¶	RIFLE	1.08 (0.12–9.97)	3.45 (0.35–33.85)	40.25 (9.20–176.03)	11.67 (1.14–119.55)
Haase, et al. [25]**	RIFLE	1.38 (0.15–12.89)	6.55 (1.08–39.79)	43.20 (9.19–203.09)	6.60 (0.97–44.93)
Sensitivity ≈ specificity	KDIGO	NC	31.77 (1.58–640.36)	67.58 (3.83–1,190.96)	2.13 (0.56–8.14)
Lowest distance D to (0,1)	KDIGO	NC	32.13 (1.59–647.54)	68.35 (3.88–1,204.32)	2.13 (0.56–8.14)
Maximum Youden index	KDIGO	NC	28.92 (1.45–577.18)	112.17 (6.32–1,990.32)	3.88 (1.02–14.81)
Albert, et al. [9]¶	KDIGO	NC	37.93 (2.04–703.57)	145.44 (7.94–2,662.96)	3.84 (1.10–13.32)
Haase, et al. [25]**	KDIGO	NC	44.13 (2.43–802.56)	157.17 (8.43–2,931.68)	3.56 (1.03–12.35)
In-hospital mortality
Magdeburg cohort
Sensitivity ≈ specificity	RIFLE	NC	NC	57.40 (2.69–1,222.58)	4.20 (0.17–101.54)
Lowest distance D to (0,1)	RIFLE	NC	NC	59.27 (2.78–1,261.93)	4.20 (0.17–101.54)
Maximum Youden index	RIFLE	NC	38.08 (1.41–1,031.09)	60.00 (2.40–1,497.18)	1.58 (0.14–17.41)
Albert, et al. [9]¶	RIFLE	NC	55.67 (2.44–1,269.49)	37.11 (1.09–1,259.86)	0.67 (0.05–8.95)
Haase, et al. [25]**	RIFLE	NC	43.95 (1.96–985.05)	66.80 (1.79–2,495.20)	1.52 (0.10–22.74)
Sensitivity ≈ specificity	KDIGO	NC	NC	29.67 (1.42–619.31)	7.00 (0.32–152.96)
Lowest distance D to (0,1)	KDIGO	NC	NC	35.67 (1.71–742.79)	7.00 (0.32–152.96)
Maximum Youden index	KDIGO	NC	15.41 (0.60–397.42)	57.31 (2.48–1,324.62)	3.72 (0.40–34.43)
Albert, et al. [9]¶	KDIGO	NC	24.35 (1.11–532.69)	41.18 (1.50–1,134.17)	1.69 (0.18–15.98)
Haase, et al. [25]**	KDIGO	NC	21.57 (0.99–469.65)	64.71 (2.21–1,891.82)	3.00 (0.29–30.74)
Berlin cohort
Sensitivity ≈ specificity	RIFLE	6.97 (0.76–63.74)	30.86 (2.48–383.23)	72.00 (7.79–665.30)	2.33 (0.36–15.30)
Lowest distance D to (0,1)	RIFLE	13.89 (1.51–128.04)	37.71 (3.04–467.80)	88.00 (9.54–811.96)	2.33 (0.36–15.30)
Maximum Youden index	RIFLE	13.89 (1.51–128.04)	37.71 (3.04–467.80)	88.00 (9.54–811.96)	2.33 (0.36–15.30)
Albert, et al. [9]¶	RIFLE	2.99 (0.48–18.65)	17.38 (3.01–100.17)	28.96 (5.85–143.28)	1.67 (0.29–9.45)
Haase, et al. [25]**	RIFLE	3.87 (0.61–24.32)	14.50 (2.59–81.29)	40.28 (7.75–209.29)	2.78 (0.48–16.03)
Sensitivity ≈ specificity	KDIGO	3.48 (0.31–39.35)	8.70 (0.75–100.74)	24.00 (2.88–200.14)	2.76 (0.53–14.38)
Lowest distance D to (0,1)	KDIGO	3.59 (0.32–40.63)	8.80 (0.76–101.89)	24.28 (2.91–202.41)	2.76 (0.53–14.38)
Maximum Youden index	KDIGO	7.45 (0.65–85.05)	8.00 (0.70–91.53)	39.27 (4.67–330.09)	4.91 (0.94–25.53)
Albert, et al. [9]¶	KDIGO	2.35 (0.21–27.03)	6.65 (1.17–37.88)	22.60 (4.18–122.30)	3.40 (0.84–13.84)
Haase, et al. [25]**	KDIGO	3.11 (0.27–35.95)	6.38 (1.12–36.24)	29.50 (5.35–162.61)	4.63 (1.11–19.19)
KRT initiation or in-hospital mortality
Magdeburg cohort
Sensitivity ≈ specificity	RIFLE	NC	NC	177.67 (8.57–3,682.96)	13.00 (0.55–306.22)
Lowest distance D to (0,1)	RIFLE	NC	NC	183.44 (8.85–3,801.49)	13.00 (0.55–306.22)
Maximum Youden index	RIFLE	NC	75.00 (3.19–1,762.00)	212.14 (9.45–4,763.31)	2.83 (0.37–21.89)
Albert, et al. [9]¶	RIFLE	NC	89.92 (4.18–1,934.88)	233.80 (9.35–5,847.08)	2.60 (0.33–20.79)
Haase, et al. [25]**	RIFLE	NC	141.31 (7.01–2,847.09)	100.20 (3.22–3,121.90)	0.71 (0.07–7.22)
Sensitivity ≈ specificity	KDIGO	NC	NC	77.13 (3.92–1,516.64)	18.20 (0.88–374.91)
Lowest distance D to (0,1)	KDIGO	NC	NC	92.73 (4.73–1,818.94)	18.20 (0.88–374.91)
Maximum Youden index	KDIGO	NC	27.59 (1.25–607.18)	149.00 (6.90–3,218.77)	5.40 (0.83–35.33)
Albert, et al. [9]¶	KDIGO	NC	36.45 (1.79–743.94)	151.00 (6.46–3,527.54)	4.14 (0.63–27.32)
Haase, et al. [25]**	KDIGO	NC	57.28 (3.00–1,093.38)	64.71 (2.21–1,891.82)	1.13 (0.13–9.70)
Berlin cohort
Sensitivity ≈ specificity	RIFLE	2.90 (0.67–12.54)	10.10 (1.44–70.66)	97.17 (19.22–491.30)	9.63 (1.38–67.25)
Lowest distance D to (0,1)	RIFLE	5.86 (1.34–25.65)	12.38 (1.77–86.51)	119.17 (23.62–601.29)	9.63 (1.38–67.25)
Maximum Youden index	RIFLE	5.86 (1.34–25.65)	12.38 (1.77–86.51)	119.17 (23.62–601.29)	9.63 (1.38–67.25)
Albert, et al. [9]¶	RIFLE	1.46 (0.28–7.59)	8.50 (1.79–40.39)	75.56 (16.40–348.05)	8.89 (1.40–56.58)
Haase, et al. [25]**	RIFLE	1.89 (0.36–9.92)	10.52 (2.51–44.10)	106.50 (18.76–604.57)	10.13 (1.47–69.94)
Sensitivity ≈ specificity	KDIGO	3.48 (0.31–39.35)	19.33 (2.04–183.31)	52.96 (6.61–423.97)	2.74 (0.77–9.76)
Lowest distance D to (0,1)	KDIGO	3.59 (0.32–40.63)	19.56 (2.06–185.40)	53.57 (6.69–428.79)	2.74 (0.77–9.76)
Maximum Youden index	KDIGO	7.45 (0.65–85.05)	17.28 (1.85–161.36)	94.50 (11.62–768.34)	5.47 (1.53–19.59)
Albert, et al. [9]¶	KDIGO	2.35 (0.21–27.03)	12.76 (2.52–64.53)	62.15 (12.06–320.26)	4.87 (1.49–15.95)
Haase, et al. [25]**	KDIGO	3.11 (0.27–35.95)	14.30 (2.90–70.62)	73.75 (13.76–395.16)	5.16 (1.54–17.27)

*NGAL(+)/sCr/UO(–), Subclinical AKI (stage 1 S); kidney tubular injury indicated by positive NGAL; ^†The reference group was NGAL(–)/sCr/UO(–), No damage or functional impairment [10, 16]. ^‡NGAL(–)/sCr(+), Hemodynamic AKI (stage 1–3A); ^§NGAL(+)/sCr(+), AKI stage 1–3B; tubular injury and functional impairment present [10, 16]; ^¶We chose the value closest to that suggested by Albert, et al. [9] (81 ng/mL); **We chose the value closest to that suggested by Haase, et al. [25] (150 ng/mL).

NC: When there were groups with zeros, N=0, we introduced a corrective factor of 0.5 to all cells. The two groups of zeros were not corrected for and were not compared with respect to systematic differences.

Abbreviations: KDIGO, Kidney Disease Improving Global Outcomes; KRT, kidney replacement therapy; NC, not calculable; NGAL, neutrophil gelatinase-associated lipocalin; RIFLE, Risk, Injury, Failure, Loss, and End-stage; sCR, serum creatinine; UO, urine output; OR, odds ratio.

The proportion of patients with kidney impairment additionally identified by urinary NGAL only in relation to the proportion of patients diagnosed as having AKI by conventional sCr/UO-based criteria differed by up to 8.33–16.49 times and 2.46–2.6 times between the lowest and highest candidate cutoffs for the RIFLE and KDIGO classifications in the Berlin and Magdeburg cohorts, respectively (Supplemental Data Fig. S3 and S4). The distribution of patients according to the derived cutoff values allocated to the NGAL/sCr/UO groups and their outcomes are shown in Table 2. The ORs illustrating the differences in the risk of developing adverse events between these groups are provided in Table 3.

Finally, the magnitude of the difference in the calculated risk (fraction of ORs) for adverse events between the group distributions varied considerably when the cutoff concentration was changed within the RIFLE or KDIGO classification (up to 18.33- and 16.11-times risk difference, respectively). This variation in the calculated risk was even more prominent when comparing the various cutoff methodologies between the RIFLE and KDIGO classification systems, reaching up to a 12.79-times difference in the Berlin cohort and up to a 25.67-times difference in the Magdeburg cohort for KRT initiation. As an example, such distinct changes in calculated risk resulted from an approximate doubling of the NGAL cutoff value from 79.3 to 149.3 ng/mL and the change of the NGAL cutoff value from approximately 10 to 160 ng/mL, respectively (Fig. 3, Supplemental Data Fig. S5A–D).

Figure 3. Heat maps illustrating the magnitude of difference of calculated risk. Heat map examples show the magnitude of difference of calculated risk derived by the division of ORs from Table 3 for the development of the outcome measures of KRT initiation or in-hospital mortality (A, B) or in-hospital mortality (C). A factor of 1.0 indicates no difference between the methodologies. (A) For attributed subclinical phenotype (AKI stage 1 S) NGAL(+)/sCr/UO(–) vs. NGAL(–)/sCr/UO(–) reference groups in the Berlin cohort. As an example, the calculated risk (OR) between these groups varied by a factor of 4 when choosing 50 vs. 79.3 ng/mL as a cutoff concentration for NGAL using the RIFLE classification. (B) NGAL(+)/sCr/UO(+) vs. NGAL(–)/sCr(+) groups in the Magdeburg cohort. For example, the calculated risk (OR) between these groups varied by a factor of 16.1 when choosing 9.5 vs. 159.1 ng/mL as a cutoff concentration for NGAL using the KDIGO classification. The highest variation observed was a 25.67-times difference of the calculated risk between using KDIGO and 8.0 ng/mL as the NGAL cutoff vs. RIFLE and 159.1 ng/mL as the NGAL cutoff concentration. (C) Such variation was also present comparing attributed hemodynamic/pre-kidney AKI phenotypes NGAL(–)/sCr(+) vs. NGAL(–)/sCr(–) as reference for the outcome in-hospital mortality in the Berlin cohort. The calculated risk (OR) between these groups varied by a factor of 2.17 when choosing RIFLE and 50.0 vs. 79.3 ng/mL or a 5.91-times difference when choosing KDIGO with 149.25 ng/mL over RIFLE and 50.0 ng/mL as the cutoff concentration for NGAL.
Abbreviations: AKI, acute kidney injury; KRT, kidney replacement therapy; KDIGO, Kidney Disease: Improving Global Outcome classification; NGAL, neutrophil gelatinase-associated lipocalin; OR, odds ratio; RIFLE, Risk, Injury, Failure, Loss of kidney function, End-stage; sCr, serum creatinine; UO, urine output.

Using two independent cardiac surgery cohorts, we assessed the extent to which the methodology for NGAL cutoff value selection and AKI classification system affected patients’ clinical AKI-phenotype allocation and the associated risk of KRT initiation or in-hospital mortality.

We found an increased risk of KRT initiation and in-hospital mortality for patients testing NGAL-positive. This increased risk for adverse events found between clinical phenotype groups varied considerably in magnitude (up to a 25.7-times difference) according to the underlying sCr/UO classification system of AKI and the NGAL cutoff selection methodology applied.

At ICU admission, NGAL positivity carried more prognostic information over NGAL-negative findings, regardless of whether the kidney function acutely declined (positive RIFLE/KDIGO criteria), confirming the findings of a previous study [5]. However, multiple methods of determining attributed “optimal” cutoff values to define biomarker positivity have been suggested [11]. These thresholds were derived from ROC curves to predict sCr and UO-based AKI [5, 12, 13].

Previous studies on NGAL predicting adverse outcomes in various AKI settings selected certain cutoffs based on published values, manufacturer recommendations, or calculated cutoff values from the study population [5^-7, 12^-15]. Considering a variety of confounders such as the AKI classification system and the collection timing of samples for NGAL testing, a recent meta-analysis derived summary ROC curves and NGAL cutoff values for the prediction of sCr-based AKI [9], showing high variability in the data among the included studies, which may be closely related to the present findings on the risk disparity for adverse events.

Considering that higher NGAL concentrations likely reflect more severe kidney injury than lower concentrations, the associated risk of adverse events and specificity will steadily increase as the measured concentration increases [27]. Although cardiac surgery-associated AKI is common, severe adverse complications such as KRT initiation or mortality are less frequent [28, 29], which seems to be consistent with the lower frequency of high NGAL concentrations in the present cohorts.

Given the small samples investigated in these cohorts with relatively few adverse events, our data indicate that there may be study-specific “sweet-spot thresholds” where the risk prediction metrics (Table 1) will be the highest, resulting in favorable allocation of patients according to such specific NGAL cutoff values that more distinctively differentiate groups of relative risk of adverse events [30]. It is therefore reasonable that the applied cutoff value selection methods may identify the same cutoff values as “optimal” in a small cohort but identify different cutoff values as “optimal” in larger cohorts [24]. This may also explain the high level of heterogeneity of previously reported cutoff values [9]. Even a small shift in the cutoff concentration (ng/mL) may ultimately lead to a potentially meaningful clinical difference in outcome distribution and consecutive risk disparity, as the respective cutoff 2×2 table is subject to change (Figs. 1 and 2) [31].

Extending conventional AKI classification criteria to a clinically applicable AKI phenotype matrix-based system representing different pathophysiological phenotypes of AKI (Fig. 2) is desirable [32]. Characterization of clinical AKI phenotypes in such a matrix-based system is directly dependent on the cutoff value chosen for dichotomization [33]. In such a matrix-based framework, a biomarker may be of additional prognostic value only if it clearly differentiates the incremental risk of adverse events within the derived matrix-based groups rather than specifically showing good diagnostic performance for sCr/UO-based AKI according to a high AUC value [34].

We found that the statistical methodology for NGAL cutoff value selection may be a relevant factor potentially leading to inconsistent findings in patients’ phenotype allocation and in the calculation of the attributed risk of adverse events among different clinical phenotypes of AKI. We consider that the magnitude of such confounding factors will likely depend on the variability across patient cohorts and that calculations of risk differences may be affected by cohort sample size variations. Complementarily, in the aforementioned recent meta-analysis on NGAL cutoff values [9], we demonstrated that the underlying meta-analysis approach may also lead to considerable differences in findings for discriminatory estimators such as the AUC or derived cutoff values [35].

Many caveats remain to be resolved to more accurately define AKI phenotypes [32]. NGAL was previously associated with mortality in cardiogenic shock [36], but concentrations and discriminative ability may also be influenced by systemic conditions such as sepsis [37] or variations in urine flow [38], potentially confounding the findings. We did not correct the outcome data for the presence of septic conditions as influencing factors since supplemental data were not recorded [17, 18]; however, including C-reactive protein, a surrogate parameter for sepsis, did not improve the risk assessment of AKI phenotypes in a previous analysis of this cohort [5]. In the present study, the risk parameters and cutoff values were derived from and were dependent on the investigated cohorts; therefore, they may not be generalizable. Nevertheless, the internal validity of our findings is strengthened by the similarities in risk variability within two independent, well-defined cardiac surgery cohorts with a consequent timing of biomarker sampling. We do not preclude the conclusion that other biomarkers for AKI risk prediction may also be affected by changing biomarker threshold selections [39].

Changing cutoff concentrations or threshold methodology may also change the previous pathophysiological narrative and associated understanding of AKI phenotypes such as subclinical AKI [16, 33]. We suggest considering potential confounders [9] and emphasize that great care should be taken when establishing a cutoff concentration for clinical use. The variability in risk in our findings highlights the complementary prognostic relevance of kidney functional parameters and kidney injury biomarkers and the necessity to interpret both in conjunction with the clinical context. Further studies are needed to assess the potential clinical relevance of the cutoff value selection methodology [40] and to outline the associated risk disparity for the clinical phenotypes of AKI.

In summary, using two independent cardiac surgery cohorts, we confirmed our hypothesis that the magnitude of the attributed risk of adverse events varies according to the methodology used for NGAL cutoff selection and the classification system of sCr/UO-based AKI. Further studies are needed to address alternative methods and evaluate the advantages and limitations of different approaches, in addition to those based on the ROC curve and dichotomization.

None.

Albert C, Albert A, Bellomo R, and Braun-Dullaeus RC contributed to the conception and design of the study. Haase M and Albert C acquired patient data. Albert C and Albert A performed formal analysis, interpretation and illustration of the data, and wrote the first draft of the manuscript. Albert C supervised the study. All authors participated in drafting and/or revising the paper and provided intellectual content of critical importance to the work, read and approved the final manuscript for publication, and agreed to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Albert A has received honoraria speaking for Abbott Diagnostics on unrelated work. Haase M has received honoraria speaking for Abbott Diagnostics, Alere, Biosite Inc., and Siemens Healthineers. Albert C has received honoraria speaking for Siemens Healthineers. All other authors declare no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Albert C was supported by grants from the Dr. Werner Jackstädt Foundation, Wuppertal, Germany. Haase M was supported by grants from the German Heart Foundation (Deutsche Stiftung für Herzforschung, Frankfurt am Main, Germany) and from the Else Kröner-Fresenius-Stiftung, Bad Homburg, Germany.