Abstract: PUB046
Artificial Intelligence (AI)-Augmented Workflows for Clinical Algorithm Analysis and Optimization
Session Information
Category: Artificial Intelligence, Digital Health, and Data Science
- 300 Artificial Intelligence, Digital Health, and Data Science
Authors
- Bergling, Karin, Renal Research Institute, New York, New York, United States
- Hamilton, H. Emel, Fresenius Medical Care, Waltham, Massachusetts, United States
- Rivenbark, Amanda Short, Fresenius Medical Care, Waltham, Massachusetts, United States
- Lombardi, Cathy, Fresenius Medical Care, Waltham, Massachusetts, United States
- Scoggins, Tory, Fresenius Medical Care, Waltham, Massachusetts, United States
- Usvyat, Len A., Renal Research Institute, New York, New York, United States
- Zhang, Hanjie, Renal Research Institute, New York, New York, United States
Background
Clinical algorithms are often presented as text-heavy documents, embedding key information within complex tables and nested workflows. These may encode logic, thresholds, and actions in ways that are difficult for optical character recognition (OCR) and rule-based tools to interpret. We evaluate an AI-based document extraction tool combined with a large language model (LLM) to streamline translation into structured, LLM-readable formats, identify logic-impacting issues, and facilitate flowchart visualization.
Methods
A six-page algorithm addressing hyporesponse to anemia management was processed using an AI-based document extraction tool (Landing AI: www.landing.ai), which converted complex tables into structured JSON and markdown formats (Figure 1). The extracted content was then input into the o1 LLM (OpenAI), prompted to generate a flowchart in Mermaid syntax and render the logic visually using Mermaid.js. The LLM was also prompted to assess for logical inconsistencies, ambiguities, or missing criteria. Performance was assessed manually by a physician throughout the process.
Results
The AI extraction tool demonstrated high accuracy, with only minor errors that did not affect interpretation or decision logic. The LLM successfully identified several, though not all, issues, including infinite loops, conflicting or ambiguous decision thresholds, and typographical errors. A sample issue is shown in Figure 1. The algorithm was effectively translated into a flowchart, however some human input was needed to guide the flow correctly, likely due to identified logic issues remaining unresolved.
Conclusion
AI-augmented information extraction demonstrated strong performance and proved to be a key enabler for unlockingthe potential of the capabilities of LLMs. This approach supports downstream tasks such as identifying logical inconsistencies, refining decision pathways, and automated visualization, which may streamline optimization and quality efforts.
AI-augmented workflow for translating a clinical algorithm into structured logic, Mermaid-based flowchart and review for logic flaws and ambiguities. One flagged issue (Step 11 to 6 loop) is shown as an example.
Funding
- Commercial Support – Renal Research Institute LLC