Abstract: TH-PO002
Assessing Drug Interactions with Tacrolimus: Evaluating ChatGPT as a Promising Resource for Drug Interactions with Tacrolimus
Session Information
- AI, Digital Health, Data Science - I
November 02, 2023 | Location: Exhibit Hall, Pennsylvania Convention Center
Abstract Time: 10:00 AM - 12:00 PM
Category: Augmented Intelligence, Digital Health, and Data Science
- 300 Augmented Intelligence, Digital Health, and Data Science
Authors
- Garcia Valencia, Oscar Alejandro, Mayo Clinic Division of Nephrology and Hypertension, Rochester, Minnesota, United States
- Bentall, Andrew J., Mayo Clinic Division of Nephrology and Hypertension, Rochester, Minnesota, United States
- Issa, Naim S., Mayo Clinic Division of Nephrology and Hypertension, Rochester, Minnesota, United States
- El Ters, Mireille, Mayo Clinic Division of Nephrology and Hypertension, Rochester, Minnesota, United States
- Craici, Iasmina, Mayo Clinic Division of Nephrology and Hypertension, Rochester, Minnesota, United States
- Thongprayoon, Charat, Mayo Clinic Division of Nephrology and Hypertension, Rochester, Minnesota, United States
- Schinstock, Carrie A., Mayo Clinic Division of Nephrology and Hypertension, Rochester, Minnesota, United States
- Cheungpasitporn, Wisit, Mayo Clinic Division of Nephrology and Hypertension, Rochester, Minnesota, United States
Background
Tacrolimus is the most common immunosuppressant in kidney transplant recipients. However, its blood concentration can be affected by other medications and thus, assessing drug interactions is essential to optimize its efficacy and minimize toxicity. ChatGPT is a high-capacity language model proficient in natural language processing tasks. This study aims to evaluate the accuracy of ChatGPT in assessing drug interactions involving tacrolimus.
Methods
We used the Micromedex Drug Interactions and Lexicomp databases to evaluate ChatGPT's ability to correctly assess if a drug interaction with Tacrolimus was present and in what direction should the tacrolimus dose be adjusted. A total of 79 drugs were chosen based on the risk rating of D (consider therapy modification) or X (avoid combination) . Each question was executed twice using ChatGPT (03/2023, OpenAI), and we determined the concordance between the 2 runs, conducted 2 weeks apart. An additional assessment was conducted using ChatGPT to draw a conclusion from our study.
Results
The overall accuracy of ChatGPT in assessing the presence of drug interactions with tacrolimus was found to be 96% on the 1st run and 98% on the 2nd run. The direction of adjustment was correct in 87% on both runs. The concordance in correct answers was 96% for the need of adjustment and 86% on the direction. ChatGPT itself acknowledged these results, emphasizing that while it exhibits promising accuracy, it should not be the sole source of information for clinical decision-making. Comprehensive and reliable assessment of drug interactions and dosing adjustments should involve consultation with additional resources and healthcare professionals.
Conclusion
ChatGPT demonstrated an accuracy exceeding 96% regarding the presence of drug interactions involving tacrolimus. However, when asked about the adjustment direction, the accuracy dropped to 87%. The integration of ChatGPT in clinical practice should be accompanied by caution and should not replace consultation with healthcare professionals or the utilization of comprehensive resources. Future steps should focus on refining ChatGPT's capabilities to ensure its safe and effective use in assessing drug interactions involving tacrolimus.