Abstract: PUB227
Indications for Methenamine Use After Kidney Transplantation Derived from a Large Language Model
Session Information
Category: Transplantation
- 2002 Transplantation: Clinical
Authors
- Ngouonga, Alvine Amelia, University of California San Francisco School of Medicine, San Francisco, California, United States
- Bicki, Alexandra, University of California San Francisco, San Francisco, California, United States
Background
Recurrent urinary tract infections (UTIs) affect up to one-third of kidney transplant recipients. Oral methenamine may hold promise as a non-antibiotic strategy for UTI prophylaxis; its real-world use remains poorly characterized due to historic reliance on structured clinical data. The objective of this study was to define reasons for methenamine use after kidney transplantation by abstracting rationale from free-text clinical documentation.
Methods
We performed a retrospective cohort study of kidney transplant recipients at our institution (2013-2024) using clinical data deidentified via natural language processing. Authors manually coded reasons for methenamine use/non-use (n=44) to create a single adjudicated validation set before scaling to a large language model (LLM)-assisted abstraction (n=173).
Results
Among 173 kidney transplant recipients with over 3,000 clinical notes mentioning methenamine, 125 (72%) had ever-use. The leading explicit reason for use was a prior UTI caused by extended-spectrum beta-lactamase-producing bacteria (36% of users). The leading reason for non-use was contingency planning that was never required (e.g., “if patient has another UTI, will consider methenamine”; 61% of non-users). Human interrater reliability was lower for methenamine use vs. non-use (κ=0.74 vs. 0.82, respectively). Agreement between LLM and human adjudication was higher among those with methenamine use vs. non-use (κ=0.85 vs. 0.62, respectively; Table 1). Human agreement was strongest when reasons for non-use required inference, whereas the model’s performance declined in those scenarios.
Conclusion
To our knowledge, this is the largest single retrospective cohort exploring methenamine use for UTI prophylaxis among kidney transplant recipients, with scaling enabled by an LLM. The most common indication for use was a prior antibiotic-resistant UTI, suggesting a role for non-antibiotic prophylaxis. Next steps include further structuring free-text data elements to assess real-world effectiveness.
Acknowledgment
AI Disclosure: UCSF's implementation of ChatGPT Enterprise (detailed in Methods) is certified to process deidentified health information; data are private, secure, and not used to train public models.
Interrater reliability for human vs. large language model assessment of reasons for methenamine use and non-use after kidney transplantation
| Comparison | Scenario | Raw agreement (%) | Kappa (κ) |
| Human 1 vs. Human 2 | Methenamine use | 91% | 0.74 |
| Methenamine non-use | 85% | 0.82 | |
| Human vs. Large language model | Methenamine use | 90% | 0.85 |
| Methenamine non-use | 76% | 0.62 |
Funding
- NIDDK Support