A 25-reader performance study for hepatic metastasis detection: lessons from unsupervised learning

Research output: Chapter in Book/Report/Conference proceedingConference contribution


There is substantial variability in the performance of radiologist readers. We hypothesized that certain readers may have idiosyncratic weaknesses towards certain types of lesions, and unsupervised learning techniques might identify these patterns. After IRB approval, 25 radiologist readers (9 abdominal subspecialists and 16 non-specialists or trainees) read 40 portal phase liver CT exams, marking all metastases and providing a confidence rating on a scale of 1 to 100. We formed a matrix of reader confidence ratings, with rows corresponding to readers, and columns corresponding to metastases, and each matrix entry providing the confidence rating that a reader gave to the metastasis, with zero confidence used for lesions that were not marked. A clustergram was used to permute the rows and columns of this matrix to group similar readers and metastases together. This clustergram was manually interpreted. We found a cluster of lesions with atypical presentation that were missed by several readers, including subspecialists, and a separate cluster of small, subtle lesions where subspecialists were more confident of their diagnosis than trainees. These and other observations from unsupervised learning could inform targeted training and education of future radiologists.

Original languageEnglish (US)
Title of host publicationMedical Imaging 2022
Subtitle of host publicationPhysics of Medical Imaging
EditorsWei Zhao, Lifeng Yu
ISBN (Electronic)9781510649378
StatePublished - 2022
EventMedical Imaging 2022: Physics of Medical Imaging - Virtual, Online
Duration: Mar 21 2022Mar 27 2022

Publication series

NameProgress in Biomedical Optics and Imaging - Proceedings of SPIE
ISSN (Print)1605-7422


ConferenceMedical Imaging 2022: Physics of Medical Imaging
CityVirtual, Online


  • low contrast detection
  • reader variability
  • unsupervised learning

ASJC Scopus subject areas

  • Electronic, Optical and Magnetic Materials
  • Atomic and Molecular Physics, and Optics
  • Biomaterials
  • Radiology Nuclear Medicine and imaging


Dive into the research topics of 'A 25-reader performance study for hepatic metastasis detection: lessons from unsupervised learning'. Together they form a unique fingerprint.

Cite this