Kidney Week

Abstract: SA-PO0083

Validation of Large Language Model-Based Phenotyping for Kidney Stone Disease in the Mayo Clinic Biobank

Session Information

• Artificial Intelligence: Kidney Imaging, Pathology, and Precision Phenotyping
  October 24, 2026 | Location: Exhibit Hall A, Convention Center
  Abstract Time: 10:00 AM - 12:00 PM

Category: Artificial Intelligence, Digital Health, and Data Science

• 300 Artificial Intelligence, Digital Health, and Data Science

Authors

• Zaher, Ahmed Abdelnaby Aly Mahmoud, Mayo Clinic Minnesota, Rochester, Minnesota, United States

Ahmed Abdelnaby Aly Mahmoud Zaher, MBChB

• Alshareef, Abdulmueti, Mayo Clinic Minnesota, Rochester, Minnesota, United States

Abdulmueti Alshareef, MD

• Jayachandran, Muthuvel, Mayo Clinic Minnesota, Rochester, Minnesota, United States

Muthuvel Jayachandran, PhD

• McDonnell, Shannon K., Mayo Clinic Minnesota, Rochester, Minnesota, United States

Shannon K. McDonnell, MS

• Larson, Nicholas B., Mayo Clinic Minnesota, Rochester, Minnesota, United States

Nicholas B. Larson, PhD

• Harris, Peter C., Mayo Clinic Minnesota, Rochester, Minnesota, United States

Peter C. Harris, PhD

• Lieske, John C., Mayo Clinic Minnesota, Rochester, Minnesota, United States

John C. Lieske, MD, FASN

Background

Kidney stone disease (KSD) phenotyping in large cohorts often relies on structured electronic health record (EHR) data and surveys, which may miss cases documented only in clinical notes or radiology reports. Large language model (LLM)-assisted review of unstructured text may improve KSD case recognition and aid studies of KSD risk factors.

Methods

We performed a validation study within the Mayo Clinic Biobank (MCBB). Of 52,884 MCBB participants, 416 were randomly sampled for clinical notes and radiology report abstraction. Notes and reports were filtered for KSD-related text using Mayo Data Explorer (MDE) and classified using the MDE LLM tool. ICD/survey phenotyping used ICD codes and survey data. Chart review of screen-positive and sampled screen-negative cases served as the reference standard.

Results

LLM phenotyping identified 83 positives, of whom 73 were confirmed by chart review, yielding a PPV of 88.0% (95% CI, 79.2%–93.3%) (Table 1). ICD/survey phenotyping identified 67 positives, of whom 54 were confirmed, yielding a PPV of 80.6% (95% CI, 69.6%–88.3%). Using equal-sized random negative samples of 91 subjects, both methods had an NPV of 94.5% (95% CI, 87.8%–97.6%). Overlap analysis showed 40 chart-confirmed cases identified by both methods, 33 by LLM only, and 14 by ICD/survey only. The combined approach identified 110 positives, of whom 87 were chart-confirmed, yielding a PPV of 79.1% (95% CI, 70.6%–85.6%). Among 85 participants negative by both methods, 84 were confirmed as non-KSD and 1 had KSD, yielding a combined-method NPV of 98.8% (95% CI, 93.6%–99.8%).

Conclusion

In this MCBB validation study, LLM phenotyping using clinical notes and radiology reports had a higher yield of validated cases than ICD/survey phenotyping. The combined approach identified the largest number of validated cases, although with lower PPV. Participants negative by both methods had a high NPV, supporting combined LLM and ICD/survey phenotyping for KSD identification in biobank studies. Future analysis of false-positive and false-negative cases may further refine LLM phenotyping.

Table 1

Funding

• Private Foundation Support