Abstract: SA-PO046
Race and Ethnicity Do Not Influence the Performance of Fistula Failure Machine Learning Model
Session Information
- Diversity and Equity: Other Research
November 04, 2023 | Location: Exhibit Hall, Pennsylvania Convention Center
Abstract Time: 10:00 AM - 12:00 PM
Category: Diversity and Equity in Kidney Health
- 900 Diversity and Equity in Kidney Health
Authors
- Lama, Suman Kumar, Fresenius Medical Care Holdings Inc, Waltham, Massachusetts, United States
- Willetts, Joanna, Fresenius Medical Care Holdings Inc, Waltham, Massachusetts, United States
- Monaghan, Caitlin, Fresenius Medical Care Holdings Inc, Waltham, Massachusetts, United States
- Chaudhuri, Sheetal, Fresenius Medical Care Holdings Inc, Waltham, Massachusetts, United States
- Larkin, John W., Fresenius Medical Care Holdings Inc, Waltham, Massachusetts, United States
- Maddux, Franklin W., Fresenius Medical Care Holdings Inc, Waltham, Massachusetts, United States
- Eneanya, Nwamaka D., Fresenius Medical Care Holdings Inc, Waltham, Massachusetts, United States
- Kraus, Michael A., Fresenius Medical Care Holdings Inc, Waltham, Massachusetts, United States
- Sor, Murat, Azura Vascular Care, Malvern, Pennsylvania, United States
- Usvyat, Len A., Fresenius Medical Care Holdings Inc, Waltham, Massachusetts, United States
Background
Risk stratification models are important decision support aids in medicine yet can inadvertently introduce bias if social factors, such as race and ethnicity, are included in models that predict biological outcomes. (Obermeyer et al., Science 2019). The inclusion of social factors in model development may not have a significant impact on model performance. In the development of a model to predict an arteriovenous fistula (AVF) failure event in a hemodialysis (HD) patient, we investigated the effect of inclusion/exclusion of race and ethnicity as predictor variables on model performance.
Methods
AVF failure was defined by the change from active to unusable status within 30 days. We built two machine learning (ML) algorithms (XGBoost) using HD patient data from Jan to Dec 2018 at large integrated kidney disease healthcare provider. Models were trained using demographics, treatment, laboratory, comorbidity, clinical notes, hospitalization data. Model A included race and ethnicity and Model B did not. Both models considered approximately 2,400 predictor variables. Dataset was randomly split into 60% training, 20% validation 20% testing data. Unseen testing data was used to evaluate the model’s performance.
Results
Models were developed using data on approximately 67000 patients (approximately 14000 patients had events). Model A & B showed an area under the curve (AUC) of 0.76 vs 0.75, sensitivity of 0.53 vs 0.53, and specificity of 0.84 vs 0.84 respectively (Figure 1).
Conclusion
ML model performance was not affected by race or ethnicity data, suggesting they should be excluded from models of biological outcomes to minimize biases that could disproportionally classify risk and lead to disparities in care. Patient factors that may affect biology are critical to consider and validate during model development, and should span beyond race and ethnicity.
Figure 1: AUROC of AVF failure model with and without race and ethnicity data
Funding
- Commercial Support – Fresenius Medical Care