Correlation between a deep-learning-based model observer and human observer for a realistic lung nodule localization task in chest CT

Hao Gong; Andrew Walther; Qiyuan Hu; Chi Wan Koo; Edwin A. Takahashi; David L Levin; Tucker F. Johnson; Megan J. Hora; Shuai Leng; J. G. Fletcher; Cynthia H. McCollough; Lifeng Yu

doi:10.1117/12.2513451

Correlation between a deep-learning-based model observer and human observer for a realistic lung nodule localization task in chest CT

Hao Gong, Andrew Walther, Qiyuan Hu, Chi Wan Koo, Edwin A. Takahashi, David L Levin, Tucker F. Johnson, Megan J. Hora, Shuai Leng, J. G. Fletcher, Cynthia H. McCollough, Lifeng Yu

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

2 Scopus citations

Abstract

Mathematical model observers (MOs) have become popular in task-based CT image quality assessment, since, once proven to be correlated with human observers (HOs), these MOs can be used to estimate HO performance. However, typical MO studies are limited to phantom data which only involve uniform background. In practice, anatomical background variability and tissue non-uniformity affect HO lesion detection performance. Recently, we have proposed a deep-learning-based MO (DL-MO). In this study, we aim to investigate the correlation between this DL-MO and HOs for a lung-nodule localization task in chest CT. Using a patient database that contains 50 lung cancer screening CT patient cases, 12 different experimental conditions were generated, including 4 radiation dose levels, 3 nodule sizes, 2 nodule types and 3 reconstruction types. These conditions were created by using a validated noise and lesion insertion tool. Four subspecialized radiologists performed the HO study for all 12 conditions individually in a randomized fashion. The DL-MO was trained and tested for the same dataset. The performance of DL-MO and HO was compared across all the experimental conditions. DL-MO performance was strongly correlated with HO performance (Pearson's correlation coefficient: 0.988 with a 95% confidence interval of [0.894, 0.999]). These results demonstrate the potential to use the proposed DL-MO to predict HO performance for the task of lung nodule localization in chest CT.

Original language	English (US)
Title of host publication	Medical Imaging 2019
Subtitle of host publication	Image Perception, Observer Performance, and Technology Assessment
Editors	Robert M. Nishikawa, Frank W. Samuelson
Publisher	SPIE
ISBN (Electronic)	9781510625518
DOIs	https://doi.org/10.1117/12.2513451
State	Published - 2019
Event	Medical Imaging 2019: Image Perception, Observer Performance, and Technology Assessment - San Diego, United States Duration: Feb 20 2019 → Feb 21 2019

Publication series

Name	Progress in Biomedical Optics and Imaging - Proceedings of SPIE
Volume	10952
ISSN (Print)	1605-7422

Conference

Conference	Medical Imaging 2019: Image Perception, Observer Performance, and Technology Assessment
Country/Territory	United States
City	San Diego
Period	2/20/19 → 2/21/19

Keywords

Deep learning
Lung nodule detection
Model observer
Partial least square regression
Taskbased image quality assessment
X-ray CT

ASJC Scopus subject areas

Electronic, Optical and Magnetic Materials
Biomaterials
Atomic and Molecular Physics, and Optics
Radiology Nuclear Medicine and imaging

Access to Document

10.1117/12.2513451

Cite this

Gong, H., Walther, A., Hu, Q., Koo, C. W., Takahashi, E. A., Levin, D. L., Johnson, T. F., Hora, M. J., Leng, S., Fletcher, J. G., McCollough, C. H., & Yu, L. (2019). Correlation between a deep-learning-based model observer and human observer for a realistic lung nodule localization task in chest CT. In R. M. Nishikawa, & F. W. Samuelson (Eds.), Medical Imaging 2019: Image Perception, Observer Performance, and Technology Assessment Article 109520K (Progress in Biomedical Optics and Imaging - Proceedings of SPIE; Vol. 10952). SPIE. https://doi.org/10.1117/12.2513451

Correlation between a deep-learning-based model observer and human observer for a realistic lung nodule localization task in chest CT. / Gong, Hao; Walther, Andrew; Hu, Qiyuan et al.
Medical Imaging 2019: Image Perception, Observer Performance, and Technology Assessment. ed. / Robert M. Nishikawa; Frank W. Samuelson. SPIE, 2019. 109520K (Progress in Biomedical Optics and Imaging - Proceedings of SPIE; Vol. 10952).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Gong, H, Walther, A, Hu, Q, Koo, CW, Takahashi, EA, Levin, DL, Johnson, TF, Hora, MJ, Leng, S , Fletcher, JG , McCollough, CH & Yu, L 2019, Correlation between a deep-learning-based model observer and human observer for a realistic lung nodule localization task in chest CT. in RM Nishikawa & FW Samuelson (eds), Medical Imaging 2019: Image Perception, Observer Performance, and Technology Assessment., 109520K, Progress in Biomedical Optics and Imaging - Proceedings of SPIE, vol. 10952, SPIE, Medical Imaging 2019: Image Perception, Observer Performance, and Technology Assessment, San Diego, United States, 2/20/19. https://doi.org/10.1117/12.2513451

Gong H, Walther A, Hu Q, Koo CW, Takahashi EA, Levin DL et al. Correlation between a deep-learning-based model observer and human observer for a realistic lung nodule localization task in chest CT. In Nishikawa RM, Samuelson FW, editors, Medical Imaging 2019: Image Perception, Observer Performance, and Technology Assessment. SPIE. 2019. 109520K. (Progress in Biomedical Optics and Imaging - Proceedings of SPIE). doi: 10.1117/12.2513451

Gong, Hao ; Walther, Andrew ; Hu, Qiyuan et al. / Correlation between a deep-learning-based model observer and human observer for a realistic lung nodule localization task in chest CT. Medical Imaging 2019: Image Perception, Observer Performance, and Technology Assessment. editor / Robert M. Nishikawa ; Frank W. Samuelson. SPIE, 2019. (Progress in Biomedical Optics and Imaging - Proceedings of SPIE).

@inproceedings{ce87c2acd6824482af5283707bdb3f13,

title = "Correlation between a deep-learning-based model observer and human observer for a realistic lung nodule localization task in chest CT",

abstract = "Mathematical model observers (MOs) have become popular in task-based CT image quality assessment, since, once proven to be correlated with human observers (HOs), these MOs can be used to estimate HO performance. However, typical MO studies are limited to phantom data which only involve uniform background. In practice, anatomical background variability and tissue non-uniformity affect HO lesion detection performance. Recently, we have proposed a deep-learning-based MO (DL-MO). In this study, we aim to investigate the correlation between this DL-MO and HOs for a lung-nodule localization task in chest CT. Using a patient database that contains 50 lung cancer screening CT patient cases, 12 different experimental conditions were generated, including 4 radiation dose levels, 3 nodule sizes, 2 nodule types and 3 reconstruction types. These conditions were created by using a validated noise and lesion insertion tool. Four subspecialized radiologists performed the HO study for all 12 conditions individually in a randomized fashion. The DL-MO was trained and tested for the same dataset. The performance of DL-MO and HO was compared across all the experimental conditions. DL-MO performance was strongly correlated with HO performance (Pearson's correlation coefficient: 0.988 with a 95% confidence interval of [0.894, 0.999]). These results demonstrate the potential to use the proposed DL-MO to predict HO performance for the task of lung nodule localization in chest CT.",

keywords = "Deep learning, Lung nodule detection, Model observer, Partial least square regression, Taskbased image quality assessment, X-ray CT",

author = "Hao Gong and Andrew Walther and Qiyuan Hu and Koo, {Chi Wan} and Takahashi, {Edwin A.} and Levin, {David L} and Johnson, {Tucker F.} and Hora, {Megan J.} and Shuai Leng and Fletcher, {J. G.} and McCollough, {Cynthia H.} and Lifeng Yu",

note = "Funding Information: This study was sponsored by National Institute of Biomedical Imaging and Bioengineering of the National Institutes of Health under Award Numbers R01 EB017095 and U01 EB017185. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. Publisher Copyright: {\textcopyright} 2019 SPIE.; Medical Imaging 2019: Image Perception, Observer Performance, and Technology Assessment ; Conference date: 20-02-2019 Through 21-02-2019",

year = "2019",

doi = "10.1117/12.2513451",

language = "English (US)",

series = "Progress in Biomedical Optics and Imaging - Proceedings of SPIE",

publisher = "SPIE",

editor = "Nishikawa, {Robert M.} and Samuelson, {Frank W.}",

booktitle = "Medical Imaging 2019",

}

TY - GEN

T1 - Correlation between a deep-learning-based model observer and human observer for a realistic lung nodule localization task in chest CT

AU - Gong, Hao

AU - Walther, Andrew

AU - Hu, Qiyuan

AU - Koo, Chi Wan

AU - Takahashi, Edwin A.

AU - Levin, David L

AU - Johnson, Tucker F.

AU - Hora, Megan J.

AU - Leng, Shuai

AU - Fletcher, J. G.

AU - McCollough, Cynthia H.

AU - Yu, Lifeng

N1 - Funding Information: This study was sponsored by National Institute of Biomedical Imaging and Bioengineering of the National Institutes of Health under Award Numbers R01 EB017095 and U01 EB017185. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. Publisher Copyright: © 2019 SPIE.

PY - 2019

Y1 - 2019

N2 - Mathematical model observers (MOs) have become popular in task-based CT image quality assessment, since, once proven to be correlated with human observers (HOs), these MOs can be used to estimate HO performance. However, typical MO studies are limited to phantom data which only involve uniform background. In practice, anatomical background variability and tissue non-uniformity affect HO lesion detection performance. Recently, we have proposed a deep-learning-based MO (DL-MO). In this study, we aim to investigate the correlation between this DL-MO and HOs for a lung-nodule localization task in chest CT. Using a patient database that contains 50 lung cancer screening CT patient cases, 12 different experimental conditions were generated, including 4 radiation dose levels, 3 nodule sizes, 2 nodule types and 3 reconstruction types. These conditions were created by using a validated noise and lesion insertion tool. Four subspecialized radiologists performed the HO study for all 12 conditions individually in a randomized fashion. The DL-MO was trained and tested for the same dataset. The performance of DL-MO and HO was compared across all the experimental conditions. DL-MO performance was strongly correlated with HO performance (Pearson's correlation coefficient: 0.988 with a 95% confidence interval of [0.894, 0.999]). These results demonstrate the potential to use the proposed DL-MO to predict HO performance for the task of lung nodule localization in chest CT.

AB - Mathematical model observers (MOs) have become popular in task-based CT image quality assessment, since, once proven to be correlated with human observers (HOs), these MOs can be used to estimate HO performance. However, typical MO studies are limited to phantom data which only involve uniform background. In practice, anatomical background variability and tissue non-uniformity affect HO lesion detection performance. Recently, we have proposed a deep-learning-based MO (DL-MO). In this study, we aim to investigate the correlation between this DL-MO and HOs for a lung-nodule localization task in chest CT. Using a patient database that contains 50 lung cancer screening CT patient cases, 12 different experimental conditions were generated, including 4 radiation dose levels, 3 nodule sizes, 2 nodule types and 3 reconstruction types. These conditions were created by using a validated noise and lesion insertion tool. Four subspecialized radiologists performed the HO study for all 12 conditions individually in a randomized fashion. The DL-MO was trained and tested for the same dataset. The performance of DL-MO and HO was compared across all the experimental conditions. DL-MO performance was strongly correlated with HO performance (Pearson's correlation coefficient: 0.988 with a 95% confidence interval of [0.894, 0.999]). These results demonstrate the potential to use the proposed DL-MO to predict HO performance for the task of lung nodule localization in chest CT.

KW - Deep learning

KW - Lung nodule detection

KW - Model observer

KW - Partial least square regression

KW - Taskbased image quality assessment

KW - X-ray CT

UR - http://www.scopus.com/inward/record.url?scp=85068693821&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85068693821&partnerID=8YFLogxK

U2 - 10.1117/12.2513451

DO - 10.1117/12.2513451

M3 - Conference contribution

AN - SCOPUS:85068693821

T3 - Progress in Biomedical Optics and Imaging - Proceedings of SPIE

BT - Medical Imaging 2019

A2 - Nishikawa, Robert M.

A2 - Samuelson, Frank W.

PB - SPIE

T2 - Medical Imaging 2019: Image Perception, Observer Performance, and Technology Assessment

Y2 - 20 February 2019 through 21 February 2019

ER -

Correlation between a deep-learning-based model observer and human observer for a realistic lung nodule localization task in chest CT

Abstract

Publication series

Conference

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this