Kidney Week

Abstract: FR-PO0009

External Validation of Temporal XGBoost Model for Predicting Seven-Day CRRT-Free Survival in Critically Ill Patients on CRRT

Session Information

• Artificial Intelligence: Clinical Prediction, Decision Support, and Digital Kidney Care
  October 23, 2026 | Location: Exhibit Hall A, Convention Center
  Abstract Time: 10:00 AM - 12:00 PM

Category: Artificial Intelligence, Digital Health, and Data Science

• 300 Artificial Intelligence, Digital Health, and Data Science

Authors

• Yang, Joanna, Icahn School of Medicine at Mount Sinai, New York, New York, United States

Joanna Yang, MS

• Kauffman, Justin, Icahn School of Medicine at Mount Sinai, New York, New York, United States

Justin Kauffman, MS

• Pranto, Shehan Irteza, University of Alabama at Birmingham Health System, Birmingham, Alabama, United States

Shehan Irteza Pranto, MS

• Takeuchi, Tomonori, University of Alabama at Birmingham Health System, Birmingham, Alabama, United States

Tomonori Takeuchi, MD

• Goldstein, Stuart, Cincinnati Children's Hospital Medical Center, Cincinnati, Ohio, United States

Stuart Goldstein, MD, FASN

• Chen, Jin, University of Alabama at Birmingham Health System, Birmingham, Alabama, United States

Jin Chen, PhD

• Lambert, Joshua, University of Cincinnati, Cincinnati, Ohio, United States

Joshua Lambert, PhD

• Nadkarni, Girish N., Icahn School of Medicine at Mount Sinai, New York, New York, United States

Girish N. Nadkarni, MD, MPH

• Chan, Lili, Icahn School of Medicine at Mount Sinai, New York, New York, United States

Lili Chan, MD, MS

• Neyra, Javier A., University of Alabama at Birmingham Health System, Birmingham, Alabama, United States

Javier A. Neyra, MD, MS, FASN

Background

Decisions to continue continuous renal replacement therapy (CRRT), transition to intermittent dialysis, or withdraw therapy in critically ill patients with acute kidney injury lack a validated bedside prognostic tool. We previously developed an XGBoost model using EHR data to predict 7-day CRRT-free survival with internal AUROC 0.84. Generalizability across sites is unknown; therefore, we performed external validation in two independent cohorts and tested whether site-specific fine-tuning improved performance.

Methods

The pre-trained temporal XGBoost model for predicting 7-day CRRT-free survival was applied zero-shot to two external cohorts (Mount Sinai, University of Alabama at Birmingham/UAB) defined using CRRTnet inclusion and exclusion criteria. Site-specific fine-tuning was then performed on each cohort using an 80:20 train-test split, with hyperparameters optimized using stratified 5-fold cross-validated randomized search. Performance was evaluated using AUROC and AUPRC.

Results

Mount Sinai (n=748) and UAB (n=3,890) differed from CRRTnet in 7-day CRRT-free survival prevalence (43.4% and 22.8% vs 28.2%; Figure 1A). Zero-shot performance declined from internal validation (AUROC 0.84, AUPRC 0.76) to AUROC 0.70, AUPRC 0.53 at Mount Sinai and AUROC 0.76, AUPRC 0.39 at UAB. Following site-specific fine-tuning, discrimination recovered to AUROC 0.78, AUPRC 0.68 at Mount Sinai and AUROC 0.84, AUPRC 0.54 at UAB (Figure 1B).

Conclusion

Zero-shot performance declined at both external sites, consistent with differences in patient populations and CRRT practices. Site-specific fine-tuning improved performance, with AUROC recovering to within 0.06 of internal performance, suggesting that adapting the existing model to local data is sufficient for use across sites. Limitations include cross-site differences in data collection and CRRT processes. Next steps include prospective evaluation, adjusting predicted risks to match observed outcomes, and federated training.

Funding

• NIDDK Support