Expert identification of visual primitives used by CNNs during mammogram classification

Jimmy Wu; Diondra Peck; Scott Hsieh; Vandana Dialani; Constance D. Lehman; Bolei Zhou; Vasilis Syrgkanis; Lester MacKey; Genevieve Patterson

doi:10.1117/12.2293890

Expert identification of visual primitives used by CNNs during mammogram classification

Jimmy Wu, Diondra Peck, Scott Hsieh, Vandana Dialani, Constance D. Lehman, Bolei Zhou, Vasilis Syrgkanis, Lester MacKey, Genevieve Patterson

Radiology

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

7 Scopus citations

Abstract

This work interprets the internal representations of deep neural networks trained for classification of diseased tissue in 2D mammograms. We propose an expert-in-the-loop inter-pretation method to label the behavior of internal units in convolutional neural networks (CNNs). Expert radiologists identify that the visual patterns detected by the units are correlated with meaningful medical phenomena such as mass tissue and calcificated vessels. We demonstrate that several trained CNN models are able to produce explanatory descriptions to support the final classification decisions. We view this as an important first step toward interpreting the internal representations of medical classification CNNs and explaining their predictions.

Original language	English (US)
Title of host publication	Medical Imaging 2018
Subtitle of host publication	Computer-Aided Diagnosis
Editors	Kensaku Mori, Nicholas Petrick
Publisher	SPIE
ISBN (Electronic)	9781510616394
DOIs	https://doi.org/10.1117/12.2293890
State	Published - 2018
Event	Medical Imaging 2018: Computer-Aided Diagnosis - Houston, United States Duration: Feb 12 2018 → Feb 15 2018

Publication series

Name	Progress in Biomedical Optics and Imaging - Proceedings of SPIE
Volume	10575
ISSN (Print)	1605-7422

Other

Other	Medical Imaging 2018: Computer-Aided Diagnosis
Country/Territory	United States
City	Houston
Period	2/12/18 → 2/15/18

Keywords

Medical image understanding
deep learning for diagnosis
expert-in-the-loop methods
interpretable machine learning

ASJC Scopus subject areas

Electronic, Optical and Magnetic Materials
Atomic and Molecular Physics, and Optics
Biomaterials
Radiology Nuclear Medicine and imaging

Access to Document

10.1117/12.2293890

Cite this

Wu, J., Peck, D., Hsieh, S., Dialani, V., Lehman, C. D., Zhou, B., Syrgkanis, V., MacKey, L., & Patterson, G. (2018). Expert identification of visual primitives used by CNNs during mammogram classification. In K. Mori, & N. Petrick (Eds.), Medical Imaging 2018: Computer-Aided Diagnosis Article 105752T (Progress in Biomedical Optics and Imaging - Proceedings of SPIE; Vol. 10575). SPIE. https://doi.org/10.1117/12.2293890

Expert identification of visual primitives used by CNNs during mammogram classification. / Wu, Jimmy; Peck, Diondra; Hsieh, Scott et al.
Medical Imaging 2018: Computer-Aided Diagnosis. ed. / Kensaku Mori; Nicholas Petrick. SPIE, 2018. 105752T (Progress in Biomedical Optics and Imaging - Proceedings of SPIE; Vol. 10575).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Wu, J, Peck, D, Hsieh, S, Dialani, V, Lehman, CD, Zhou, B, Syrgkanis, V, MacKey, L & Patterson, G 2018, Expert identification of visual primitives used by CNNs during mammogram classification. in K Mori & N Petrick (eds), Medical Imaging 2018: Computer-Aided Diagnosis., 105752T, Progress in Biomedical Optics and Imaging - Proceedings of SPIE, vol. 10575, SPIE, Medical Imaging 2018: Computer-Aided Diagnosis, Houston, United States, 2/12/18. https://doi.org/10.1117/12.2293890

@inproceedings{2c5d460b10544fc9b99c86edfdb305fd,

title = "Expert identification of visual primitives used by CNNs during mammogram classification",

abstract = "This work interprets the internal representations of deep neural networks trained for classification of diseased tissue in 2D mammograms. We propose an expert-in-the-loop inter-pretation method to label the behavior of internal units in convolutional neural networks (CNNs). Expert radiologists identify that the visual patterns detected by the units are correlated with meaningful medical phenomena such as mass tissue and calcificated vessels. We demonstrate that several trained CNN models are able to produce explanatory descriptions to support the final classification decisions. We view this as an important first step toward interpreting the internal representations of medical classification CNNs and explaining their predictions.",

keywords = "Medical image understanding, deep learning for diagnosis, expert-in-the-loop methods, interpretable machine learning",

author = "Jimmy Wu and Diondra Peck and Scott Hsieh and Vandana Dialani and Lehman, {Constance D.} and Bolei Zhou and Vasilis Syrgkanis and Lester MacKey and Genevieve Patterson",

note = "Publisher Copyright: {\textcopyright} 2018 SPIE.; Medical Imaging 2018: Computer-Aided Diagnosis ; Conference date: 12-02-2018 Through 15-02-2018",

year = "2018",

doi = "10.1117/12.2293890",

language = "English (US)",

series = "Progress in Biomedical Optics and Imaging - Proceedings of SPIE",

publisher = "SPIE",

editor = "Kensaku Mori and Nicholas Petrick",

booktitle = "Medical Imaging 2018",

}

TY - GEN

T1 - Expert identification of visual primitives used by CNNs during mammogram classification

AU - Wu, Jimmy

AU - Peck, Diondra

AU - Hsieh, Scott

AU - Dialani, Vandana

AU - Lehman, Constance D.

AU - Zhou, Bolei

AU - Syrgkanis, Vasilis

AU - MacKey, Lester

AU - Patterson, Genevieve

PY - 2018

Y1 - 2018

N2 - This work interprets the internal representations of deep neural networks trained for classification of diseased tissue in 2D mammograms. We propose an expert-in-the-loop inter-pretation method to label the behavior of internal units in convolutional neural networks (CNNs). Expert radiologists identify that the visual patterns detected by the units are correlated with meaningful medical phenomena such as mass tissue and calcificated vessels. We demonstrate that several trained CNN models are able to produce explanatory descriptions to support the final classification decisions. We view this as an important first step toward interpreting the internal representations of medical classification CNNs and explaining their predictions.

AB - This work interprets the internal representations of deep neural networks trained for classification of diseased tissue in 2D mammograms. We propose an expert-in-the-loop inter-pretation method to label the behavior of internal units in convolutional neural networks (CNNs). Expert radiologists identify that the visual patterns detected by the units are correlated with meaningful medical phenomena such as mass tissue and calcificated vessels. We demonstrate that several trained CNN models are able to produce explanatory descriptions to support the final classification decisions. We view this as an important first step toward interpreting the internal representations of medical classification CNNs and explaining their predictions.

KW - Medical image understanding

KW - deep learning for diagnosis

KW - expert-in-the-loop methods

KW - interpretable machine learning

UR - http://www.scopus.com/inward/record.url?scp=85046283040&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85046283040&partnerID=8YFLogxK

U2 - 10.1117/12.2293890

DO - 10.1117/12.2293890

M3 - Conference contribution

AN - SCOPUS:85046283040

T3 - Progress in Biomedical Optics and Imaging - Proceedings of SPIE

BT - Medical Imaging 2018

A2 - Mori, Kensaku

A2 - Petrick, Nicholas

PB - SPIE

T2 - Medical Imaging 2018: Computer-Aided Diagnosis

Y2 - 12 February 2018 through 15 February 2018

ER -

Expert identification of visual primitives used by CNNs during mammogram classification

Abstract

Publication series

Other

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this