Abstract: FR-PO0028
Integrating Clinical Context to Improve Large Language Model Interpretation of Image-Based Nephropathology Questions
Session Information
- Artificial Intelligence: Clinical Prediction, Decision Support, and Digital Kidney Care
October 23, 2026 | Location: Exhibit Hall A, Convention Center
Abstract Time: 10:00 AM - 12:00 PM
Category: Artificial Intelligence, Digital Health, and Data Science
- 300 Artificial Intelligence, Digital Health, and Data Science
Authors
- Mohan, Arjunmohan, Mayo Clinic Minnesota, Rochester, Minnesota, United States
- Thongprayoon, Charat, Mayo Clinic Minnesota, Rochester, Minnesota, United States
- Craici, Iasmina, Mayo Clinic Minnesota, Rochester, Minnesota, United States
- Miao, Jing, Mayo Clinic Minnesota, Rochester, Minnesota, United States
- Cheungpasitporn, Wisit, Mayo Clinic Minnesota, Rochester, Minnesota, United States
Background
Large language models (LLMs) are increasingly used in medical education and clinical decision support, but their performance in image-intensive subspecialties such as nephropathology remains poorly characterized. We evaluated how three LLMs (GPT-5.2, Claude Sonnet 4.5, and Gemini 3) perform on kidney pathology questions, focusing on how accuracy changes with increasing levels of clinical context. The goal was to quantify the relative contribution of image data, structured question prompts, and full clinical vignettes to model performance.
Methods
We used 21 questions from KSAP nephropathology modules (Volumes 7–11) and tested each model across three conditions:
Round 1: Image only, with a prompt to provide the most likely diagnosis
Round 2: Image plus question stem and answer options
Round 3: Full clinical vignette, question, and image
Responses were scored against KSAP answer keys. Performance was further stratified by domain (diagnosis, management/next best step, immunofluorescence interpretation). Paired improvement from Round 2 to Round 3 was assessed using McNemar’s test for within-model comparisons.
Results
Accuracy improved consistently with additional context. In Round 1, GPT-5.2 and Claude each achieved 42.9% accuracy, while Gemini reached 61.9%. Across models, diabetic glomerulosclerosis was the most frequently generated incorrect response. In Round 2, accuracy increased to 61.9% (GPT-5.2) and 71.4% (Claude), but decreased for Gemini (42.9%). With full clinical context in Round 3, performance improved substantially: GPT-5.2 reached 95.2%, while Claude and Gemini each reached 85.7%. From Round 2 to Round 3, correct responses increased most in ‘diagnosis’ and ‘next best step/treatment’ domains, while IF staining pattern questions showed minimal change. McNemar testing showed significant improvement from Round 2 to 3 for GPT-5.2 (p=0.016) and Gemini (p=0.004), but not for Claude.
Conclusion
LLM performance in nephropathology is highly dependent on clinical context. Image-only interpretation frequently leads to misclassification, whereas adding clinical information results in marked accuracy gains. GPT-5.2 achieved the highest overall performance with full clinical vignette, highlighting the importance of multimodal input for developing reliable AI tools in nephropathology.