Abstract: SA-PO1052
ChatGPT and Gemini Responses to the Organ Procurement and Transplantation Network's Kidney Transplantation FAQ Using Natural Language Processing
Session Information
- Transplantation: Clinical - Postkidney Transplant Outcomes and Potpourri
November 08, 2025 | Location: Exhibit Hall, Convention Center
Abstract Time: 10:00 AM - 12:00 PM
Category: Transplantation
- 2102 Transplantation: Clinical
Authors
- Han, Hwarang Stephen, Dell Seton Medical Center at The University of Texas, Austin, Texas, United States
- Lee, Jihye, The University of Texas at Austin, Austin, Texas, United States
Background
Artificial intelligence powered chatbots hold great potential. We aimed to compare the responses to kidney transplant evaluation and listing frequently asked questions (FAQs) from the Organ Procurement & Transplantation Network (OPTN) with those generated by ChatGPT and Gemini.
Methods
Analysis compared responses to the OPTN FAQs about transplant evaluation with answers generated by ChatGPT 4 and Gemini 1.5 Flash.
We conducted a Natural Language Processing (NLP) analysis using Linguistic Inquiry and Word Count (LIWC) software.
We examined three LIWC text scores: analytic thinking, social reference, and future focus. Analysis of Variance (ANOVA) was used to independently assess group differences.
Results
Significant differences across all text scores: analytic thinking (F(2, 51) = 6.02, p = .004), social references (F(2, 51) = 21.43, p < .001), focus on the future (F(2, 51) = 7.12, p = .002). We performed post hoc comparisons using Tukey’s Honestly Significant Difference (HSD) test to identify pairwise differences and present the results in Figure 1.
Post hoc comparisons showed that OPTN had lower analytic thinking scores than both ChatGPT (M = -18.11, 95% CI [-32.11, -4.11], p = .008) and Gemini (M = -16.65, 95% CI [-30.65, -2.65], p = .016), with no significant difference between Gemini and ChatGPT (M = -1.46, 95% CI [-15.46, 12.54], p = .966).
For social references, OPTN scored higher than both ChatGPT (M = 5.92, 95% CI [3.44, 8.39], p < .001) and Gemini (M = 5.68, 95% CI [3.21, 8.15], p < .001), with no significant difference between Gemini and ChatGPT (M = 0.23, 95% CI [-2.24, 2.70], p = .972).
Regarding focus on the future, OPTN showed higher scores than ChatGPT (M = 1.72, 95% CI [0.48, 2.97], p = .004) and Gemini (M = 1.65, 95% CI [0.40, 2.90], p = .007), with no significant difference between Gemini and ChatGPT (M = 0.07, 95% CI [-1.17, 1.32], p = .989).
Conclusion
ChatGPT and Gemini used language associated with analytic thinking and long-term perspectives, while OPTN’s responses included more social references, indicating a greater emphasis on community-driven health management.
Box plots of text scores in responses by ChatGPT, Gemini, and OPTN: Analytic thinking (left), social references (center), and future focus (right)