Skip to main navigation Skip to search Skip to main content

Outlier detection to identify presence of clinical confounders to AI performance: Application to coronary angiography and LVEF prediction

  • Kuan Zhang
  • , Behrouz Rostami
  • , Shahriar Faghani
  • , Eric Chang
  • , Douglas Svestka
  • , Kenneth Fetterly
  • , Bradley J. Erickson
  • , Mohamad Alkhouli

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Increasing integration of AI in medical imaging relies on robust models trained on large patient populations. However, models with high performance metrics may struggle in clinical practice due to confounding factors such as class imbalance, variability in image acquisition and quality, and natural variability of numeric clinical data used for labels. A practical example is the use of routine coronary angiography images to monitor non-coronary findings in real time, such as reduced left ventricular ejection fraction (LVEF). We recently developed an ensemble deep learning model, integrating 3D CNN and Transformer architectures, to distinguish patients with significant reduced LVEF (≤ 40%) from those with normal or mildly reduced LVEF (> 40%) as determined by echocardiography. Despite achieving an area under the curve (AUC) of 0.87, the model’s accuracy was 77%, notably lower than the class imbalance ratio (82%). In this work, we hypothesize that prior AI model performance was limited by confounders including dataset imbalance, technical image quality, variable presence of medical devices in the images, and temporally variable clinical data used as labels. We employed K-nearest-neighbor (KNN) outlier detection using cosine similarity on our model’s final-layer features, in priorly identifying test samples prone to reduced accuracy. A human reader study confirmed that outlier samples exhibited higher incidence of confounding factors, with 54 out of 88 outliers meeting exclusion criteria, compared to just 6 out of 88 inliers. By applying conformal prediction, we provided a confidence score for each test sample, resulting in a “certain” group with improved accuracy (91%) compared to the overall performance (77%) and an “uncertain” group with lower accuracy (44%). These findings underscore the value of identifying confounding cases to improve AI performance and provide clinicians insight into the confidence that the model has in its predictions.

Original languageEnglish (US)
Title of host publicationMedical Imaging 2025
Subtitle of host publicationComputer-Aided Diagnosis
EditorsSusan M. Astley, Axel Wismuller
PublisherSPIE
ISBN (Electronic)9781510685925
DOIs
StatePublished - 2025
EventMedical Imaging 2025: Computer-Aided Diagnosis - San Diego, United States
Duration: Feb 17 2025 → Feb 20 2025

Publication series

NameProgress in Biomedical Optics and Imaging - Proceedings of SPIE
Volume13407
ISSN (Print)1605-7422

Conference

ConferenceMedical Imaging 2025: Computer-Aided Diagnosis
Country/TerritoryUnited States
CitySan Diego
Period2/17/25 → 2/20/25

Keywords

  • clinical confounders
  • coronary angiography
  • deep learning
  • left ventricular ejection fraction
  • Outlier detection
  • uncertainty quantification

ASJC Scopus subject areas

  • Electronic, Optical and Magnetic Materials
  • Atomic and Molecular Physics, and Optics
  • Biomaterials
  • Radiology Nuclear Medicine and imaging

Fingerprint

Dive into the research topics of 'Outlier detection to identify presence of clinical confounders to AI performance: Application to coronary angiography and LVEF prediction'. Together they form a unique fingerprint.

Cite this