Abstract: TH-PO047
A Risk-Based Reinforcement Learning Algorithm to Predict Intradialytic Hypotension During Kidney Replacement Therapy
Session Information
- AI, Digital Health, Data Science - I
November 02, 2023 | Location: Exhibit Hall, Pennsylvania Convention Center
Abstract Time: 10:00 AM - 12:00 PM
Category: Augmented Intelligence, Digital Health, and Data Science
- 300 Augmented Intelligence, Digital Health, and Data Science
Authors
- Lint, Annabelle H., University of Pittsburgh, Pittsburgh, Pennsylvania, United States
- McLaverty, Brian P., University of Pittsburgh, Pittsburgh, Pennsylvania, United States
- Parker, Robert S., University of Pittsburgh, Pittsburgh, Pennsylvania, United States
- Murugan, Raghavan, University of Pittsburgh, Pittsburgh, Pennsylvania, United States
- Kashani, Kianoush, Mayo Foundation for Medical Education and Research, Rochester, Minnesota, United States
- Clermont, Gilles, University of Pittsburgh, Pittsburgh, Pennsylvania, United States
Background
Kidney replacement therapy (KRT) after acute kidney injury is associated with intradialytic hypotension (IDH) in up to 30% of sessions. IDH is associated with premature termination of therapy and is an independent predictor of mortality. Thus, there is a critical need to develop a clinical decision support tool that predicts IDH and preemptively suggests optimal therapy.
Methods
We constructed a reinforcement learning (RL) algorithm that generates personalized, dynamic treatment plans for IDH during intermittent KRT. The RL model was trained and validated using clinical data from a cohort of 277 patients undergoing 1,595 hemodialysis treatments in a tertiary care center. A random forest classifier was used to generate individualized risk trajectories for IDH, which were incorporated with patients’ physiological features to define a discrete state space. The RL was trained for an optimal action every 15 minutes. To ensure that selected actions aligned with clinical intuition, a bias term prevented the RL from over-recommending clinically rare actions. Finally, a Q-learning algorithm used risk rewards as intermittent feedback to learn a personalized sequence of optimal actions.
Results
The RL agent tended to recommend interventions more frequently than clinicians, with an RL intervention frequency of 41.7% compared to a clinician intervention frequency of 18.4%. The most frequently recommended clinical intervention by both clinicians and the RL agent was a change in ultrafiltration rate (clinician, 17.8%, RL, 40.8%), followed by administering mannitol or vasopressors (clinician, 0.6%, RL, 0.8%). Implementing the learned policy decreased the occurrence of IDH and increased fluid goal achievment in silico , as seen in Table 1 below.
Conclusion
Dynamic risk forecasting and RL generated risk prediction of IDH and personalized treatment plans for IDH during intermittent KRT. These dynamic policies recommend more than twice as many interventions as clinicians. We showed RL agents could decrease the frequency of IDH while meeting therapeutic goals and improving clinical outcomes.
Table 1.
| IDH [count (%)] | Fluid Goals Achieved [count (%)] | |
| RL Treatment Policy | 1817 (7.3) | 23265 (93.1) |
| Clinician Treatment Policy | 253 (15.0) | 1082 (64.2) |
Funding
- NIDDK Support