OSCARS: An Outlier-Sensitive Content-Based Radiography Retrieval System

Xiaoyuan Guo, Jiali Duan, Saptarshi Purkayastha, Hari Trivedi, Judy Wawira Gichoya, Imon Banerjee

Research output: Chapter in Book/Report/Conference proceedingConference contribution


Improving the retrieval relevance on noisy datasets is an emerging need for the curation of a large-scale clean dataset in the medical domain. While existing methods can be applied for class-wise retrieval (aka. inter-class), they cannot distinguish the granularity of likeness within the same class (aka. intra-class). The problem is exacerbated on medical external datasets, where noisy samples of the same class are treated equally during training. Our goal is to identify both intra/inter-class similarities for fine-grained retrieval. To achieve this, we propose an Outlier-Sensitive Content-based rAdiologhy Retrieval System (OSCARS), consisting of two steps. First, we train an outlier detector on a clean internal dataset in an unsupervised manner. Then we use the trained detector to generate the anomaly scores on the external dataset, whose distribution will be used to bin intra-class variations. Second, we propose a quadruplet (a, p, nintra, ninter) sampling strategy, where intra-class negatives nintra are sampled from bins of the same class other than the bin anchor a belongs to, while n_inter are randomly sampled from inter-classes. We suggest a weighted metric learning objective to balance the intra and inter-class feature learning. We experimented on two representative public radiography datasets. Experiments show the effectiveness of our approach. The training and evaluation code can be found in https://github.com/XiaoyuanGuo/oscars.

Original languageEnglish (US)
Title of host publicationICMR 2022 - Proceedings of the 2022 International Conference on Multimedia Retrieval
PublisherAssociation for Computing Machinery, Inc
Number of pages8
ISBN (Electronic)9781450392389
StatePublished - Jun 27 2022
Event2022 International Conference on Multimedia Retrieval, ICMR 2022 - Newark, United States
Duration: Jun 27 2022Jun 30 2022

Publication series

NameICMR 2022 - Proceedings of the 2022 International Conference on Multimedia Retrieval


Conference2022 International Conference on Multimedia Retrieval, ICMR 2022
Country/TerritoryUnited States


  • deep metric learning
  • medical image retrieval
  • outlier detection

ASJC Scopus subject areas

  • Computer Graphics and Computer-Aided Design
  • Human-Computer Interaction
  • Software


Dive into the research topics of 'OSCARS: An Outlier-Sensitive Content-Based Radiography Retrieval System'. Together they form a unique fingerprint.

Cite this