Abstract: SA-PO1001
Learning with NEPHRON-AI: Exploring Student Interactions with an Artificial Intelligence Tutoring Tool in a Preclerkship Nephrology Course
Session Information
- Top Trainee Posters - 3
October 24, 2026 | Location: Exhibit Hall A, Convention Center
Abstract Time: 01:06 PM - 01:12 PM
Category: Educational Research
- 1000 Educational Research
Authors
- Fai, Caitlin P., Tufts University School of Medicine, Boston, Massachusetts, United States
- Blanco, Maria A., Tufts University School of Medicine, Boston, Massachusetts, United States
- Gilbert, Scott J., Tufts University School of Medicine, Boston, Massachusetts, United States
- Drew, David A., Tufts University School of Medicine, Boston, Massachusetts, United States
Background
Generative artificial intelligence (AI) tools are increasingly used in health professions education. Limited evidence describes learner engagement. This study characterized medical students' interaction patterns with an AI tutoring tool.
Methods
We developed NEPHRON-AI, a custom tutor chatbot built on Playlab (Tufts University licensed), using retrieval-augmented generation restricted to course materials. The chatbot scaffolded prompting and resisted requests for new practice questions or passive information delivery. Use was optional. We analyzed de-identified transcripts using content analysis and randomly sampled 300 prompts from 23 users during peak usage, 36 hours before the final exam. Two coders applied inductive and deductive coding by Bloom's Taxonomy, prompt type, and nephrology topic. Discrepancies were resolved by consensus.
Results
A total of 118/203 (58%) students engaged with the chatbot, submitting 6,989 prompts during the 3-week course. Individual usage was classified as high (>90 prompts; n=28), middle (30-89; n=33), low (2-29; n=49), or dropout (1; n=8). Analysis included 300 prompts from 11 high, 7 middle, and 5 low users, 36 hours before the final exam. Common topics included osmolality and ADH, metabolic alkalosis, glomerulonephritis pathology, CKD staging, and RAAS regulation. High users' prompts reflected remembering (22%) and understanding levels (54%), with frequent clarification and validation of knowledge. They also reached applying (11%) and analyzing levels (6%) while using chatbot scaffolding. Low users' prompts reflected remembering (40%) and understanding (51%). They rarely sought clarification or validation and wrote targeted prompts without follow-up. Middle users showed similar patterns to low users, though some prompts reached higher cognitive levels, with brief scaffolding engagement. Middle users most frequently requested high-yield summaries.
Conclusion
NEPHRON-AI was highly used in a first-year nephrology course. Students' interactions reflected how they aligned the tool with their learning needs. High users treated the chatbot as an interactive tutor and engaged with cognitive scaffolding to deepen understanding. Low users used it for retrieval of information. Intentional AI configuration can support higher cognitive engagement while reinforcing course learning objectives. Commonly requested topics may inform curricular review.