Abstract: FR-PO055
Creation and Validation of a Formative Assessment Tool for Nephrology Fellows' Clinical Reasoning in a National Cohort
Session Information
- Educational Research
November 03, 2023 | Location: Exhibit Hall, Pennsylvania Convention Center
Abstract Time: 10:00 AM - 12:00 PM
Category: Educational Research
- 1000 Educational Research
Authors
- Boyle, Suzanne, Lewis Katz School of Medicine at Temple University, Philadelphia, Pennsylvania, United States
- Martindale, James R., University of Virginia School of Medicine, Charlottesville, Virginia, United States
- Parsons, Andrew S., University of Virginia School of Medicine, Charlottesville, Virginia, United States
- Sozio, Stephen M., Johns Hopkins Medicine, Baltimore, Maryland, United States
- Hilburg, Rachel, University of Pennsylvania Perelman School of Medicine, Philadelphia, Pennsylvania, United States
- Bahrainwala, Jehan Z., University of Pennsylvania Perelman School of Medicine, Philadelphia, Pennsylvania, United States
- Chan, Lili, Icahn School of Medicine at Mount Sinai, New York, New York, United States
- Stern, Lauren D., Boston University School of Medicine, Boston, Massachusetts, United States
- Warburton, Karen M., University of Virginia School of Medicine, Charlottesville, Virginia, United States
Background
Deficits in clinical reasoning are common among graduate medical learners.We created and validated an instrument to assess clinical reasoning in a national cohort of first-year nephrology fellows and established performance thresholds for remedial coaching.
Methods
Experts in nephrology education and clinical reasoning remediation designed an instrument to measure clinical reasoning through a written patient-encounter note from a web-based, simulated AKI consult (https://sites.temple.edu/rene/case-2/).The instrument measured clinical reasoning in three domains (Problem Representation, Differential Diagnosis with Justification, Diagnostic Plan with Justification). Inter-rater reliability was established in a pilot cohort of first-year fellows using a two-way random effects agreement intraclass correlation coefficient (ICC) model.The instrument was then administered to a larger cohort of first-year fellows to establish performance standards for coaching using the Hofstee method.
Results
Both the pilot cohort (n=15 fellows; 4 training programs) and study cohort (n=61 fellows; 20 training programs) were representative of first-year nephrology fellows nationwide. The ICCs for Problem Representation, Differential Diagnosis, and Diagnostic Plan were 0.90, 0.70, and 0.50. Passing thresholds (% total points) in Problem Representation, Differential Diagnosis, and Diagnostic Plan were 59%, 57%, and 62%. Fifty-nine percent (n=36) met the threshold for coaching in at least one domain.
Conclusion
We provide validity evidence for a simulated AKI consult for formative assessment of clinical reasoning in nephrology fellows.The majority met criteria for coaching in at least one of three reasoning domains, demonstrating a need for assessment and instruction in clinical reasoning.
The simulated consult measures three domains, which map to a part of the linear clinical reasoning pathway.
Funding
- Private Foundation Support