Validation of a Machine Learning Model for Early Shock Detection

Yuliya Pinevich, Adam Amos-Binks, Christie S. Burris, Gregory Rule, Marija Bogojevic, Isaac Flint, Brian W. Pickering, Christopher P. Nemeth, Vitaly Herasevich

Research output: Contribution to journalArticlepeer-review


Objectives: The objectives of this study were to test in real time a Trauma Triage, Treatment, and Training Decision Support (4TDS) machine learning (ML) model of shock detection in a prospective silent trial, and to evaluate specificity, sensitivity, and other estimates of diagnostic performance compared to the gold standard of electronic medical records (EMRs) review. Design: We performed a single-center diagnostic performance study. Patients and setting: A prospective cohort consisted of consecutive patients aged 18 years and older who were admitted from May 1 through September 30, 2020 to six Mayo Clinic intensive care units (ICUs) and five progressive care units. Measurements and main results: During the study time, 5,384 out of 6,630 hospital admissions were eligible. During the same period, the 4TDS shock model sent 825 alerts and 632 were eligible. Among 632 hospital admissions with alerts, 287 were screened positive and 345 were negative. Among 4,752 hospital admissions without alerts, 78 were screened positive and 4,674 were negative. The area under the receiver operating characteristics curve for the 4TDS shock model was 0.86 (95% CI 0.85-0.87%). The 4TDS shock model demonstrated a sensitivity of 78.6% (95% CI 74.1-82.7%) and a specificity of 93.1% (95% CI 92.4-93.8%). The model showed a positive predictive value of 45.4% (95% CI 42.6-48.3%) and a negative predictive value of 98.4% (95% CI 98-98.6%). Conclusions: We successfully validated an ML model to detect circulatory shock in a prospective observational study. The model used only vital signs and showed moderate performance compared to the gold standard of clinician EMR review when applied to an ICU patient cohort.

Original languageEnglish (US)
Pages (from-to)82-88
Number of pages7
JournalMilitary medicine
Issue number1-2
StatePublished - Jan 1 2022

ASJC Scopus subject areas

  • Public Health, Environmental and Occupational Health


Dive into the research topics of 'Validation of a Machine Learning Model for Early Shock Detection'. Together they form a unique fingerprint.

Cite this