Articles · Health
One added line sometimes moved the higher risk patient back
UT Southwestern researchers gave three open source AI models matched pairs of emergency patients. One added stigmatizing line sometimes moved the higher risk patient back.
Read the story
UT Southwestern Medical Center published the release on October 20th, 2026. It says the study appeared in npj Digital Medicine. The team asked three open source AI language models to predict which patients were more likely to deteriorate and who should be seen first.
The records came from Stanford Medicine in California and Beth Israel Deaconess Medical Center in Massachusetts. Each patient was paired with another of similar starting severity. Philip Jarrett is the first author and Andrew Jamieson the senior author.
Before and after the added line
With no demographic, social or stigmatizing detail in the record, all three models picked out higher risk patients at rates well above chance. In some cases a detail unrelated to a patient's immediate condition led the models to lower the priority of patients who later deteriorated seriously. In some instances nearly 10% of patients were wrongly downgraded once stigmatizing detail was added.
One pair set the neutral "established patient with multiple chronic conditions" against "17 visits in the past 12 months". Some of the strongest effects came when records described patients as frequent emergency users or noted past psychiatric emergencies. Age shifted some decisions too. The researchers think that reflected the greater medical risk of older age.
The earlier preprint
The authors posted an earlier version on Research Square dated May 4th, 2026. It had not been peer reviewed. That version tested two models on 999 pairs from MIMIC-IV-ED and 997 from MC-MED.
It tested 10 added conditions, each from a distinct demographic or stigma domain. Its most consistent harmful shifts came when the added line indicated psychiatric history, frequent emergency use, homelessness, substance use or altered age. Race, language and insurance status did not produce harmful shifts.
What the study can't say
Jarrett said bias varies by model, so each AI tool should be tested against real patient outcomes before use. The release says hospitals increasingly explore AI to help prioritize care. The preprint notes that both of its datasets came from academic medical centers in the United States. How the results carry to community hospitals is unknown.
Institutional funds from the UT Southwestern Office of the President supported the study. Follow-up research has grant funding from the National Foundation of Emergency Medicine.
Sources for this story
- UT Southwestern Medical Center news release
- Research Square preprint rs-9249025 v1, abstract page
- Research Square preprint rs-9249025 v1, full text
Claim-by-claim verification · 42 claims
UT Southwestern Medical Center published its news release on the emergency triage study on October 20th, 2026.
Published on: October 20, 2026
The release carries a Dallas dateline.
DALLAS – Oct. 20, 2026 – Artificial intelligence (AI)-driven tools used in emergency triage
According to the release, the study appeared in the journal npj Digital Medicine.
a study published in npj Digital Medicine
First author Philip Jarrett said stigmatizing language can steer clinicians away from the patients who most need care.
stigmatizing language can steer clinicians away from the patients who need care the most,” said first author Philip Jarrett
Philip Jarrett is an assistant professor of emergency medicine at UT Southwestern.
Philip Jarrett, M.D., M.B.A., is Assistant Professor of Emergency Medicine at UT Southwestern.
Andrew Jamieson is the senior author of the study.
said the study’s senior author, Andrew Jamieson
Jamieson is an assistant professor in the Lyda Hill Department of Bioinformatics at UT Southwestern.
Andrew Jamieson, Ph.D., is Assistant Professor in the Lyda Hill Department of Bioinformatics at UT Southwestern.
Jamieson said a single added detail sometimes moved a model's priority away from the patient who later deteriorated.
a single added detail sometimes shifted a model’s priority away from the patient who later deteriorated,” said the study’s senior author
The published study tested three open-source large language models, according to the release.
asked three open-source AI large language models (LLMs) to predict
The models were asked to predict which patients were more likely to deteriorate and which should get priority for care.
to predict which patients were more likely to deteriorate and who should be prioritized for care
The patient records came from Stanford Medicine in California and Beth Israel Deaconess Medical Center in Massachusetts, not from a Texas hospital.
Stanford Medicine in California and Beth Israel Deaconess Medical Center in Massachusetts
Each patient was paired with another whose initial condition severity was similar.
They paired patients with similar initial condition severity levels
With no demographic, social or stigmatizing detail added, all three models picked out higher-risk patients at rates well above chance.
Before demographic, social, or potentially stigmatizing information was added, all three models identified higher-risk patients at rates well above chance.
In some cases, information unrelated to a patient's immediate medical condition led the models to lower the priority of patients who later deteriorated seriously.
In some cases, information unrelated to a patient’s immediate medical condition caused the models to lower the priority of patients who later experienced serious deterioration.
The release says nearly 10% of patients were incorrectly downgraded in some instances once stigmatizing information was added.
In some instances, nearly 10% of patients were incorrectly downgraded after stigmatizing information was added.
Some of the strongest effects appeared when records described patients as frequent ED users or noted a history of psychiatric emergencies.
Some of the strongest effects were seen when records described patients as frequent ED users or included a history of psychiatric emergencies.
The models responded to some substance use scenarios differently than the researchers expected.
In some substance use scenarios, the models responded differently than researchers expected, highlighting the complexity of how these systems interpret clinical language.
Age also moved some decisions, though the researchers believe that likely reflected the greater medical risk of older age rather than age bias.
Age also influenced some decisions, although the researchers believe those effects likely reflected the greater medical risks often associated with older age rather than age-related bias.
The release pairs a neutral wording with a stigmatizing one. The neutral line is "established patient with multiple chronic conditions". The stigmatizing line is "17 visits in the past 12 months".
they compared neutral language and stigmatizing language: “established patient with multiple chronic conditions” vs. “17 visits in the past 12 months”
Another wording pair set the neutral "opioid use disorder, on maintenance therapy" against the stigmatizing "active IV drug use, heroin, last use today."
“opioid use disorder, on maintenance therapy” vs. “active IV drug use, heroin, last use today.”
Jarrett named a main takeaway. Bias varies by model, and every AI tool needs testing against real patient outcomes before use.
One of the main takeaways from this study is that bias varies by model, so each AI tool should be tested against real patient outcomes before use,” Dr. Jarrett said.
The release says how patient information is documented could affect whether these systems make fair and accurate decisions.
the way patient information is documented could influence whether these systems make fair and accurate decisions
The release describes hospitals as increasingly exploring AI to help prioritize care.
As hospitals increasingly explore the use of AI to help prioritize care
Institutional funds from the UT Southwestern Medical Center Office of the President supported the study.
This study was supported by institutional funds provided by the UT Southwestern Medical Center Office of the President.
The research has also received grant funding from the National Foundation of Emergency Medicine for follow-up work.
It has also received grant funding from the National Foundation of Emergency Medicine for follow-up research.
The May preprint's Research Square page says it has not been peer reviewed by a journal.
This is a preprint; it has not been peer reviewed by a journal.
The preprint was posted to Research Square on May 4th, 2026.
Posted Date: May 4th, 2026
The May preprint is released under the Creative Commons Attribution 4.0 International License.
This work is licensed under a Creative Commons Attribution 4.0 International License.
The preprint's cover page says a version was published in npj Digital Medicine on September 19th, 2026.
A version of this preprint was published at npj Digital Medicine on September 19th, 2026.
In the May preprint, the two models tested were GPT-4.1 and DeepSeek v3.2.
Two models were evaluated: GPT-4.1 (OpenAI, accessed via Microsoft Azure) and DeepSeek v3.2
In the May preprint, each patient pair was matched on ESI triage level and held one patient who deteriorated and one who did not.
ESI-matched patient pairs, one deteriorator and one non-deteriorator, were evaluated by two LLMs
In the May preprint, deterioration counted ICU admission, intubation, vasopressor initiation or mechanical ventilation within 6 hours of arrival, as well as in-hospital death.
within 6 hours of arrival: intensive care unit (ICU) admission, intubation, vasopressor initiation, or mechanical ventilation, as well as in-hospital death
In the May preprint, the researchers sampled 999 pairs from MIMIC-IV-ED and 997 from MC-MED.
We sampled 999 pairs from MIMIC-IV-ED and 997 from MC-MED
The May preprint tested 10 conditions, each drawn from a distinct demographic or stigma domain.
We tested 10 conditions, each a representative exemplar from a distinct demographic or stigma domain
In the May preprint, the frequent ED visitor condition added a line reporting 17 visits in the past 12 months.
ED visit history: 17 visits in the past 12 months
In the May preprint, the psychiatric history condition added bipolar disorder and 3 prior psychiatric emergency holds.
Psychiatric history: bipolar disorder, 3 prior psychiatric emergency holds
In the May preprint, the active substance use condition described active IV drug use of heroin, last used that day.
Substance use: active IV drug use [heroin], last use today
The May preprint found no harmful shifts from race, language or insurance status.
Race, language, and insurance status did not produce harmful shifts.
In the May preprint, harmful reprioritization rates ran from 2.9% to 10.7% for psychiatric history and from 2.9% to 9.9% for frequent ED visitor status.
psychiatric history (2.9% to 10.7%), frequent ED visitor status (2.9% to 9.9%)
In the May preprint, DeepSeek v3.2 showed larger effect sizes than GPT-4.1 for the stigma conditions.
DeepSeek v3.2 consistently exhibited larger effect sizes than GPT-4.1 for stigma conditions
The May preprint says its two datasets come from academic medical centers in the United States and that generalizability to community hospitals is unknown.
the two datasets are from academic medical centers in the United States, and generalizability to community hospitals or international settings is unknown
In the May preprint, the most consistent harmful shifts came when the added line indicated psychiatric history, frequent ED use, homelessness, substance use or altered age.
the most consistent harmful shifts occurred when the added information indicated psychiatric history, frequent ED use, homelessness, substance use, or altered age