The Triage Algorithm Problem: When Software Decides Who Waits
The Triage Algorithm Problem: When Software Decides Who Waits
The Triage Algorithm Problem: When Software Decides Who Waits
Every emergency department in Canada uses a structured triage system. In most provinces, that system is CTAS, the Canadian Triage and Acuity Scale. In the US, it is usually ESI, the Emergency Severity Index. Both were designed as decision-support frameworks for experienced triage nurses. Both are now being augmented, and in some cases partially replaced, by machine learning models that claim to do the job faster and more consistently.
I have concerns about where this is headed.
What triage algorithms actually do
CTAS assigns patients to one of five levels based on chief complaint, vital signs, and clinical modifiers. A CTAS 1 is resuscitation. A CTAS 5 is non-urgent. The system relies heavily on the triage nurse's clinical judgment and the ability to read a patient's overall appearance. ML-augmented triage tools try to replicate this by ingesting structured data from the EHR at arrival: vitals, age, chief complaint text, and sometimes historical visit data.
Epic's deterioration index is one example. It runs in the background and flags patients whose physiological trajectory suggests they may worsen. The APPROVE study, published in 2022, evaluated an ML triage tool against ESI and found that the algorithm matched or outperformed human triage in identifying high-acuity patients when the presenting complaint was chest pain or shortness of breath. For those categories, the data is relatively clean and the feature set is well-defined.
The problem is everything else.
Where bias enters
Elderly patients present atypically. An 82-year-old with an acute MI may have no chest pain at all, presenting instead with fatigue or confusion. Triage algorithms trained on chief complaint text will systematically undertriage these patients because the input does not match the high-acuity pattern the model learned. A 2021 retrospective analysis by Obermeyer and colleagues found that pain-assessment algorithms in emergency settings consistently underestimated pain severity in Black patients, partly because the training data reflected existing documentation biases.
This is a specific, measurable failure mode. The algorithm is faithfully reproducing patterns from biased training data, at scale, in a context where the consequences are measured in minutes of delayed care.
The workflow question
There is a second problem that gets less attention. When an ML triage tool disagrees with the nurse's assessment, who wins? In most current implementations, the algorithm's output is advisory. The nurse can override it. But there is documented evidence that advisory systems create anchoring effects. If the screen says CTAS 3, a nurse who might have assigned CTAS 2 based on gut feeling has to actively argue against the machine. Some will. Many, especially under the time pressure of a busy ED, will not.
I think the right design pattern for ML-augmented triage is disagreement detection, not score replacement. The algorithm should flag cases where its assessment diverges significantly from the nurse's initial read, triggering a second look rather than overriding clinical judgment. This is a different product than what most vendors are building. Most vendors want to automate the triage decision because automation is what sells. Disagreement detection is harder to market but closer to what the clinical environment actually needs.
What I would want to see
Before any ML triage tool goes live in a Canadian ED, I want to see three things: prospective validation on a demographically representative patient population, a published analysis of undertriage rates stratified by age and ethnicity, and a clear governance framework for who is accountable when the algorithm gets it wrong. We have none of these consistently right now, and the systems are already deployed.