98%
Accuracy on the curated 50-scenario subset
On par with the 94.4% correct-disposition rate from the five-nurse expert panel on that same subset.
Approach
Agentic AI in hospital triage procedures
Triage guidelines are decision trees, not prose. A fluent model that invents a disposition is a patient-safety event. This article is how Vstorm treats that problem — proved out on Schmitt-Thompson Clinical Content, not on a stock hospital photo.
The production proof
The numbers on this page are STCC's. They stay STCC's.
We built this architecture for Schmitt-Thompson Clinical Content: HIPAA-aware guideline-executing triage, guidelines as the exclusive source of truth, disposition derived from the tree — not invented by the model. Evaluation: 591 expert-validated scenarios spanning 14 dispositions, plus a curated 50-scenario open benchmark.
98% on the open 50-scenario set (on par with a 94.4% nurse-panel correct-disposition rate). 93% accuracy / 94% weighted F1 on the full 591-scenario suite, versus 44% for a raw LLM. Full engineering write-up: the STCC case study. This is not Medicare Advantage pre-appointment time and not Synera node-wiring.
44% → 98%
Raw LLM baseline to guideline-executing performance
Same open benchmark. The gain is executable guidelines, not a larger model.
94%
Weighted F1 on 591 expert-validated scenarios
Fourteen dispositions across the full acuity spectrum. Disposition accuracy is 93%.
About this article
This is a Vstorm approach piece, not a second STCC case study. It sits on the case-study route as an unlisted editorial. The production system it describes is Schmitt-Thompson Clinical Content — one of the most widely deployed clinical decision-support publishers in North America. Read the full build: reaching expert-level medical triage with guideline-executing LLMs.
Our approach
What we build into a clinical AI system
- Guideline content stays the single source of truth — the model does not reason beyond it
- Every recommendation traces to a specific guideline and disposition, not a general inference
- The system reports when it cannot find a confident match, rather than a plausible-sounding answer
- HIPAA-aware data handling is in the architecture from the first Proof of Value
- Narrow agents and structured handoffs — not one model doing the whole tree at once
The problem
Why triage is a hard problem for AI
Clinical triage guidelines are rarely prose. They are decision trees: yes/no questions, ranked by urgency, each pointing to a disposition such as go to the emergency department now or safe to manage at home. A nurse or intake clinician works that logic under time pressure, often with incomplete information from the patient in front of them.
General-purpose language models are trained to interpret natural language, not branching conditionals and urgency tiers. Pointed at this content they can misread the logic and still answer with confidence. In most domains that is an annoyance. In triage, a confidently wrong disposition is a patient-safety incident, not a UX bug. The bar is not mostly right. It is traceable, and honest about its limits. A system that occasionally guesses is worse than one that says it does not know.
How we deliver it
Extract, retrieve the tree, refuse to invent
Proof of Value is a scored scenario set — not a fluent demo on a made-up patient.
Treat the guideline as the only source of truth
Map the tree: questions, urgency, dispositions. The model does not get to add medical knowledge from pretraining. Incomplete patient context is extracted first, not guessed later.
- Decision-tree inventory
- Urgency and disposition map
- HIPAA-aware data path
Proof of Value on nurse-validated scenarios
Retrieve against a structured, pre-indexed guideline, not raw PDF prose. Evaluate relevant branches in parallel. Report no-match instead of generating a disposition. Score against scenarios that already cleared the expert CRD gate.
- Structured retrieval
- Parallel branch eval
- Honest no-match
Harden the misses that are not hallucination
On STCC, remaining gaps were deterministic retrieval and matching — in active development — not invented content. Production follows TriStorm: a working Proof of Value, then hardening.
- Expert-level accuracy bar
- Matching-gap backlog
- Production hardening
How it works
Narrow agents, structured handoffs, a tree that stays a tree
Patient context first. Retrieval against the indexed guideline. Branch evaluation. A disposition only when the tree supports it. The same isolation idea shows up in Vstorm's hybrid agent-graph work — specialists and typed handoffs — but those manufacturing SQL numbers stay on that page. Clinical proof stays on STCC.
Extract patient context What is known. What is not guessed.
↓
Retrieve the guideline tree Structured index — not raw prose
↓
Evaluate relevant branches In parallel, against the tree
↓
Disposition or no-match Never a fluent invention
Clinical triage agent — guidelines as exclusive source of truth
Results
Read the STCC breakdown for the engineering
98% on the open 50-scenario benchmark — on par with a 94.4% nurse-panel correct-disposition rate. 93% accuracy / 94% weighted F1 on 591 expert-validated scenarios spanning 14 dispositions, versus 44% for a raw LLM. Remaining gaps are retrieval and matching, not invented content. Full STCC engineering case study.
Work with us
Ready to see how agentic AI supports triage and clinical workflows?
Meet directly with our founders and PhD AI engineers. We will walk through real implementations from 30+ agentic projects and the practical steps to integrate them into your workflows.