Kidney Week

Abstract: FR-PO0040

Artificial Intelligence (AI)-Assisted Identification of Nephrology Clinical Trials: Automated vs. Manual Screening of ClinicalTrials.gov Repository

Session Information

• Artificial Intelligence: Clinical Prediction, Decision Support, and Digital Kidney Care
  October 23, 2026 | Location: Exhibit Hall A, Convention Center
  Abstract Time: 10:00 AM - 12:00 PM

Category: Artificial Intelligence, Digital Health, and Data Science

• 300 Artificial Intelligence, Digital Health, and Data Science

Authors

• Tanjy, Ahmad, The Evidence Based Practice and Impact Center, Department of Internal Medicine, University of Kansas Medical Center, Kansas City, Kansas, United States

Ahmad Tanjy, MD

• Azzam, Muayad, The Evidence Based Practice and Impact Center, Department of Internal Medicine, University of Kansas Medical Center, Kansas City, Kansas, United States

Muayad Azzam, MD

• Moreira Lopes, Lucca, The Evidence Based Practice and Impact Center, Department of Internal Medicine, University of Kansas Medical Center, Kansas City, Kansas, United States

Lucca Moreira Lopes, MD, MPH

• Mohammad, Nashwa, The Evidence Based Practice and Impact Center, Department of Internal Medicine, University of Kansas Medical Center, Kansas City, Kansas, United States

Nashwa Mohammad

• Mustafa, Reem Adel, The Evidence Based Practice and Impact Center, Department of Internal Medicine, University of Kansas Medical Center, Kansas City, Kansas, United States

Reem Adel Mustafa, MD, PhD, MPH, FASN

Background

Reviewing of clinical trial registries provides an understanding of the nephrology research landscape. Manual screening is labor-intensive. AI-assisted tools offer the potential to accelerate study identification, but their accuracy for nephrology-specific trials has not been validated.

Methods

We conducted a study comparing AI-assisted screening to manual reviewer screening for nephrology trials on ClinicalTrials.gov. Using an updated set of 113 MeSH and 39 non-MeSH nephrology terms, we searched ClinicalTrials.gov and uploaded 12,605 results to Otto, an AI-assisted systematic review platform, for title/abstract and full-text screening. Manual screening data was obtained from a prior study in which two independent reviewers screened 5,412 studies from inception of ClinicalTrials.gov to November 2018. The primary analysis was restricted to 5,143 studies identified by both searches. Using manual review as the reference standard, we calculated accuracy, precision (positive predictive value), recall (sensitivity), F1 score, specificity, and negative predictive value (NPV).

Results

Among 5,143 overlapping studies, 4,509 were concordantly included and 259 were concordantly excluded by both methods. Screening through Otto achieved an accuracy of 92.7%, recall of 95.3%, precision of 96.8%, and F1 score of 0.96. Specificity was 63.2% and NPV was 53.6%, reflecting a tendency toward over-inclusion. Otto screening accepted 151 studies that manual reviewers had excluded, while missing 224 that manual reviewers had included. An additional 269 studies (5%) in the manual dataset were not captured by the updated search, likely attributable to differences between database versions and MeSH term revisions over time.

Conclusion

AI-assisted screening through Otto demonstrated high recall, precision, and F1 score for identifying nephrology trials, with a tendency toward over-inclusion leading to a lower specificity. These findings suggest that AI-assisted tools may serve as effective first-pass screeners, reducing reviewer burden while maintaining high capture rates.