TY - GEN
T1 - Towards Foundation Models Learned from Anatomy in Medical Imaging via Self-supervision
AU - Hosseinzadeh Taher, Mohammad Reza
AU - Gotway, Michael B.
AU - Liang, Jianming
N1 - Publisher Copyright:
© The Author(s), under exclusive license to Springer Nature Switzerland AG 2024.
PY - 2024
Y1 - 2024
N2 - Human anatomy is the foundation of medical imaging and boasts one striking characteristic: its hierarchy in nature, exhibiting two intrinsic properties: (1) locality: each anatomical structure is morphologically distinct from the others; and (2) compositionality: each anatomical structure is an integrated part of a larger whole. We envision a foundation model for medical imaging that is consciously and purposefully developed upon this foundation to gain the capability of “understanding” human anatomy and to possess the fundamental properties of medical imaging. As our first step in realizing this vision towards foundation models in medical imaging, we devise a novel self-supervised learning (SSL) strategy that exploits the hierarchical nature of human anatomy. Our extensive experiments demonstrate that the SSL pretrained model, derived from our training strategy, not only outperforms state-of-the-art (SOTA) fully/self-supervised baselines but also enhances annotation efficiency, offering potential few-shot segmentation capabilities with performance improvements ranging from 9% to 30% for segmentation tasks compared to SSL baselines. This performance is attributed to the significance of anatomy comprehension via our learning strategy, which encapsulates the intrinsic attributes of anatomical structures—locality and compositionality—within the embedding space, yet overlooked in existing SSL methods. All code and pretrained models are available at GitHub.com/JLiangLab/Eden.
AB - Human anatomy is the foundation of medical imaging and boasts one striking characteristic: its hierarchy in nature, exhibiting two intrinsic properties: (1) locality: each anatomical structure is morphologically distinct from the others; and (2) compositionality: each anatomical structure is an integrated part of a larger whole. We envision a foundation model for medical imaging that is consciously and purposefully developed upon this foundation to gain the capability of “understanding” human anatomy and to possess the fundamental properties of medical imaging. As our first step in realizing this vision towards foundation models in medical imaging, we devise a novel self-supervised learning (SSL) strategy that exploits the hierarchical nature of human anatomy. Our extensive experiments demonstrate that the SSL pretrained model, derived from our training strategy, not only outperforms state-of-the-art (SOTA) fully/self-supervised baselines but also enhances annotation efficiency, offering potential few-shot segmentation capabilities with performance improvements ranging from 9% to 30% for segmentation tasks compared to SSL baselines. This performance is attributed to the significance of anatomy comprehension via our learning strategy, which encapsulates the intrinsic attributes of anatomical structures—locality and compositionality—within the embedding space, yet overlooked in existing SSL methods. All code and pretrained models are available at GitHub.com/JLiangLab/Eden.
KW - Learning from Anatomy
KW - Self-supervised Learning
UR - http://www.scopus.com/inward/record.url?scp=85175789486&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85175789486&partnerID=8YFLogxK
U2 - 10.1007/978-3-031-45857-6_10
DO - 10.1007/978-3-031-45857-6_10
M3 - Conference contribution
AN - SCOPUS:85175789486
SN - 9783031458569
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 94
EP - 104
BT - Domain Adaptation and Representation Transfer - 5th MICCAI Workshop, DART 2023, Held in Conjunction with MICCAI 2023, Proceedings
A2 - Koch, Lisa
A2 - Cardoso, M. Jorge
A2 - Ferrante, Enzo
A2 - Kamnitsas, Konstantinos
A2 - Islam, Mobarakol
A2 - Jiang, Meirui
A2 - Rieke, Nicola
A2 - Tsaftaris, Sotirios A.
A2 - Yang, Dong
PB - Springer Science and Business Media Deutschland GmbH
T2 - 5th MICCAI Workshop on Domain Adaptation and Representation Transfer, DART 2023
Y2 - 12 October 2023 through 12 October 2023
ER -