TY - JOUR
T1 - Learning binary and sparse permutation-invariant representations for fast and memory efficient whole slide image search
AU - Hemati, Sobhan
AU - Kalra, Shivam
AU - Babaie, Morteza
AU - Tizhoosh, H. R.
N1 - Publisher Copyright:
© 2023 The Author(s)
PY - 2023/8
Y1 - 2023/8
N2 - Considering their gigapixel sizes, the representation of whole slide images (WSIs) for classification and retrieval systems is a non-trivial task. Patch processing and multi-Instance Learning (MIL) are common approaches to analyze WSIs. However, in end-to-end training, these methods require high GPU memory consumption due to the simultaneous processing of multiple sets of patches. Furthermore, compact WSI representations through binary and/or sparse representations are urgently needed for real-time image retrieval within large medical archives. To address these challenges, we propose a novel framework for learning compact WSI representations utilizing deep conditional generative modeling and the Fisher Vector Theory. The training of our method is instance-based, achieving better memory and computational efficiency during the training. To achieve efficient large-scale WSI search, we introduce new loss functions, namely gradient sparsity and gradient quantization losses, for learning sparse and binary permutation-invariant WSI representations called Conditioned Sparse Fisher Vector (C-Deep-SFV), and Conditioned Binary Fisher Vector (C-Deep-BFV). The learned WSI representations are validated on the largest public WSI archive, The Cancer Genomic Atlas (TCGA) and also Liver-Kidney-Stomach (LKS) dataset. For WSI search, the proposed method outperforms Yottixel and Gaussian Mixture Model (GMM)-based Fisher Vector both in terms of retrieval accuracy and speed. For WSI classification, we achieve competitive performance against state-of-art on lung cancer data from TCGA and the public benchmark LKS dataset.
AB - Considering their gigapixel sizes, the representation of whole slide images (WSIs) for classification and retrieval systems is a non-trivial task. Patch processing and multi-Instance Learning (MIL) are common approaches to analyze WSIs. However, in end-to-end training, these methods require high GPU memory consumption due to the simultaneous processing of multiple sets of patches. Furthermore, compact WSI representations through binary and/or sparse representations are urgently needed for real-time image retrieval within large medical archives. To address these challenges, we propose a novel framework for learning compact WSI representations utilizing deep conditional generative modeling and the Fisher Vector Theory. The training of our method is instance-based, achieving better memory and computational efficiency during the training. To achieve efficient large-scale WSI search, we introduce new loss functions, namely gradient sparsity and gradient quantization losses, for learning sparse and binary permutation-invariant WSI representations called Conditioned Sparse Fisher Vector (C-Deep-SFV), and Conditioned Binary Fisher Vector (C-Deep-BFV). The learned WSI representations are validated on the largest public WSI archive, The Cancer Genomic Atlas (TCGA) and also Liver-Kidney-Stomach (LKS) dataset. For WSI search, the proposed method outperforms Yottixel and Gaussian Mixture Model (GMM)-based Fisher Vector both in terms of retrieval accuracy and speed. For WSI classification, we achieve competitive performance against state-of-art on lung cancer data from TCGA and the public benchmark LKS dataset.
KW - Fisher Vector Theory
KW - Image representation
KW - Multiple-instance learning
KW - Whole slide imaging
UR - http://www.scopus.com/inward/record.url?scp=85160641182&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85160641182&partnerID=8YFLogxK
U2 - 10.1016/j.compbiomed.2023.107026
DO - 10.1016/j.compbiomed.2023.107026
M3 - Article
C2 - 37267827
AN - SCOPUS:85160641182
SN - 0010-4825
VL - 162
JO - Computers in Biology and Medicine
JF - Computers in Biology and Medicine
M1 - 107026
ER -