TY - JOUR
T1 - Assessing the robustness of a machine-learning model for early detection of pancreatic adenocarcinoma (PDA)
T2 - evaluating resilience to variations in image acquisition and radiomics workflow using image perturbation methods
AU - Mukherjee, Sovanlal
AU - Korfiatis, Panagiotis
AU - Patnam, Nandakumar G.
AU - Trivedi, Kamaxi H.
AU - Karbhari, Aashna
AU - Suman, Garima
AU - Fletcher, Joel G.
AU - Goenka, Ajit H.
N1 - Publisher Copyright:
© The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature 2024.
PY - 2024/3
Y1 - 2024/3
N2 - Purpose: To evaluate robustness of a radiomics-based support vector machine (SVM) model for detection of visually occult PDA on pre-diagnostic CTs by simulating common variations in image acquisition and radiomics workflow using image perturbation methods. Methods: Eighteen algorithmically generated-perturbations, which simulated variations in image noise levels (σ, 2σ, 3σ, 5σ), image rotation [both CT image and the corresponding pancreas segmentation mask by 45° and 90° in axial plane], voxel resampling (isotropic and anisotropic), gray-level discretization [bin width (BW) 32 and 64)], and pancreas segmentation (sequential erosions by 3, 4, 6, and 8 pixels and dilations by 3, 4, and 6 pixels from the boundary), were introduced to the original (unperturbed) test subset (n = 128; 45 pre-diagnostic CTs, 83 control CTs with normal pancreas). Radiomic features were extracted from pancreas masks of these additional test subsets, and the model's performance was compared vis-a-vis the unperturbed test subset. Results: The model correctly classified 43 out of 45 pre-diagnostic CTs and 75 out of 83 control CTs in the unperturbed test subset, achieving 92.2% accuracy and 0.98 AUC. Model's performance was unaffected by a three-fold increase in noise level except for sensitivity declining to 80% at 3σ (p = 0.02). Performance remained comparable vis-a-vis the unperturbed test subset despite variations in image rotation (p = 0.99), voxel resampling (p = 0.25–0.31), change in gray-level BW to 32 (p = 0.31–0.99), and erosions/dilations up to 4 pixels from the pancreas boundary (p = 0.12–0.34). Conclusion: The model’s high performance for detection of visually occult PDA was robust within a broad range of clinically relevant variations in image acquisition and radiomics workflow.
AB - Purpose: To evaluate robustness of a radiomics-based support vector machine (SVM) model for detection of visually occult PDA on pre-diagnostic CTs by simulating common variations in image acquisition and radiomics workflow using image perturbation methods. Methods: Eighteen algorithmically generated-perturbations, which simulated variations in image noise levels (σ, 2σ, 3σ, 5σ), image rotation [both CT image and the corresponding pancreas segmentation mask by 45° and 90° in axial plane], voxel resampling (isotropic and anisotropic), gray-level discretization [bin width (BW) 32 and 64)], and pancreas segmentation (sequential erosions by 3, 4, 6, and 8 pixels and dilations by 3, 4, and 6 pixels from the boundary), were introduced to the original (unperturbed) test subset (n = 128; 45 pre-diagnostic CTs, 83 control CTs with normal pancreas). Radiomic features were extracted from pancreas masks of these additional test subsets, and the model's performance was compared vis-a-vis the unperturbed test subset. Results: The model correctly classified 43 out of 45 pre-diagnostic CTs and 75 out of 83 control CTs in the unperturbed test subset, achieving 92.2% accuracy and 0.98 AUC. Model's performance was unaffected by a three-fold increase in noise level except for sensitivity declining to 80% at 3σ (p = 0.02). Performance remained comparable vis-a-vis the unperturbed test subset despite variations in image rotation (p = 0.99), voxel resampling (p = 0.25–0.31), change in gray-level BW to 32 (p = 0.31–0.99), and erosions/dilations up to 4 pixels from the pancreas boundary (p = 0.12–0.34). Conclusion: The model’s high performance for detection of visually occult PDA was robust within a broad range of clinically relevant variations in image acquisition and radiomics workflow.
KW - Machine-learning
KW - Pancreas
KW - Pancreatic ductal carcinoma
KW - Radiomics
KW - X-ray computed tomography
UR - http://www.scopus.com/inward/record.url?scp=85181452037&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85181452037&partnerID=8YFLogxK
U2 - 10.1007/s00261-023-04127-1
DO - 10.1007/s00261-023-04127-1
M3 - Article
C2 - 38175255
AN - SCOPUS:85181452037
SN - 2366-004X
VL - 49
SP - 964
EP - 974
JO - Abdominal Radiology
JF - Abdominal Radiology
IS - 3
ER -