On the necessity of fine-tuned convolutional neural networks for medical imaging

Nima Tajbakhsh, Jae Y. Shin, Suryakanth R. Gurudu, R. Todd Hurst, Christopher B. Kendall, Michael B. Gotway, Jianming Liang

Research output: Chapter in Book/Report/Conference proceedingChapter

7 Scopus citations


This study aims to address two central questions. First, are fine-tuned convolutional neural networks (CNNs) necessary for medical imaging applications? In response, we considered four medical vision tasks from three different medical imaging modalities, and studied the necessity of fine-tuned CNNs under varying amounts of training data. Second, to what extent the knowledge is to be transferred? In response, we proposed a layer-wise fine-tuning scheme to examine how the extent or depth of fine-tuning contributes to the success of knowledge transfer. Our experiments consistently showed that the use of a pre-trained CNN with adequate fine-tuning outperformed or, in the worst case, performed as well as a CNN trained from scratch. The performance gap widened when reduced training sets were used for training and fine-tuning. Our results further revealed that the required level of fine-tuning differed from one application to another, suggesting that neither shallow tuning nor deep tuning may be the optimal choice for a particular application. Layer-wise finetuning may offer a practical way to reach the best performance for the application at hand based on the amount of available data. We conclude that knowledge transfer from natural images is necessary and that the level of tuning should be chosen experimentally.

Original languageEnglish (US)
Title of host publicationAdvances in Computer Vision and Pattern Recognition
PublisherSpringer London
Number of pages13
StatePublished - 2017

Publication series

NameAdvances in Computer Vision and Pattern Recognition
ISSN (Print)2191-6586
ISSN (Electronic)2191-6594

ASJC Scopus subject areas

  • Software
  • Signal Processing
  • Computer Vision and Pattern Recognition
  • Artificial Intelligence


Dive into the research topics of 'On the necessity of fine-tuned convolutional neural networks for medical imaging'. Together they form a unique fingerprint.

Cite this