Two frameworks for improving gradient-based learning algorithms

Mario Ventresca, H. R. Tizhoosh

Research output: Chapter in Book/Report/Conference proceedingChapter


Backpropagation is the most popular algorithm for training neural networks. However, this gradient-based training method is known to have a tendency towards very long training times and convergence to local optima. Various methods have been proposed to alleviate these issues including, but not limited to, different training algorithms, automatic architecture design and different transfer functions. In this chapter we continue the exploration into improving gradient-based learning algorithms through dynamic transfer function modification. We propose opposite transfer functions as a means to improve the numerical conditioning of neural networks and extrapolate two backpropagation-based learning algorithms. Our experimental results show an improvement in accuracy and generalization ability on common benchmark functions. The experiments involve examining the sensitivity of the approach to learning parameters, type of transfer function and number of neurons in the network.

Original languageEnglish (US)
Title of host publicationOppositional Concepts in Computational Intelligence
EditorsHamid Tizhoosh, Mario Ventresca
Number of pages30
StatePublished - 2008

Publication series

NameStudies in Computational Intelligence
ISSN (Print)1860-949X

ASJC Scopus subject areas

  • Artificial Intelligence


Dive into the research topics of 'Two frameworks for improving gradient-based learning algorithms'. Together they form a unique fingerprint.

Cite this