Abstract: FR-PO0023
Context-Aware Classification of Fatigue Status in Electronic Health Record Notes Using Large Language Models in CKD and CVD Cohorts
Session Information
- Artificial Intelligence: Clinical Prediction, Decision Support, and Digital Kidney Care
October 23, 2026 | Location: Exhibit Hall A, Convention Center
Abstract Time: 10:00 AM - 12:00 PM
Category: Artificial Intelligence, Digital Health, and Data Science
- 300 Artificial Intelligence, Digital Health, and Data Science
Authors
- Chavooshi, Malihehsadat, Baylor College of Medicine, Houston, Texas, United States
- Gregg, Lucile Parker, Baylor College of Medicine, Houston, Texas, United States
- Aminnia, Shiva, Baylor College of Medicine, Houston, Texas, United States
- Elkhidir, Sabri Elamin abbas, Baylor College of Medicine, Houston, Texas, United States
- Razjouyan, Javad, Baylor College of Medicine, Houston, Texas, United States
- Maghsoudi, Arash, Baylor College of Medicine, Houston, Texas, United States
Background
Fatigue is a common and clinically complex symptom in people with chronic kidney disease (CKD) and cardiovascular disease (CVD) that is not accurately captured by structured data items such as diagnosis codes. We applied a large language model (LLM) to identify fatigue in electronic health record (EHR) notes.
Methods
Using data from the Veterans Health Administration from 1999-2025, we randomly selected 10,000 clinical notes mentioning fatigue from patients with CKD (N=5000) or CVD (N=5000). Analysis was performed at the snippet level with a ±50-word window extracted around each mention of fatigue, yielding 6033 snippets among people with CKD and 5973 snippets among people with CVD. A random sample of 1800 notes (900 CKD & 900 CVD) was selected for manual annotation. Each snippet was manually annotated for fatigue as affirmed, negated, or not applicable (NA) for snippets that confirmed, denied, or were not related to the patient’s experience of fatigue, respectively. Annotation was performed by two physicians, with disagreements resolved by a third senior physician. We reported inter-annotator agreement using Cohen’s kappa. Using few-shot prompting with a subset of the annotated snippets, we trained an LLM (Claude Sonnet 3.5) with expert-guided prompt engineering to classify cases as affirmed, negated, or NA for fatigue. We evaluated the model performance using accuracy, precision, recall, F1-score, and negative predictive value (NPV), using manually annotated cases as the gold standard.
Results
After excluding entries without valid fatigue mentions, 1646 snippets were included for analysis. Inter-annotator agreement was high (Cohen’s κ=0.85). The LLM achieved an overall accuracy of 93.3%. For the affirmed (yes) vs. rest (no or NA), negated (no) vs. rest, and NA vs. rest classifications, accuracy, precision, recall, specificity, F1-score, and NPV are presented in the Table.
Conclusion
The LLM accurately identified fatigue in EHR notes. Automated detection of fatigue in EHR notes could enable novel, large-scale epidemiologic research about fatigue.
| Metric | Fatigue affirmed | Fatigue negated | NA |
| Accuracy | 94.7% | 95.9% | 96.0% |
| Precision | 93.2% | 97.7% | 67.2% |
| Recall | 95.1% | 93.9% | 77.1% |
| F-1 score | 94.2% | 95.7% | 71.8% |
| NPV | 96.0% | 94.4% | 98.4% |
Funding
- Veterans Affairs Support