TY - GEN
T1 - Numerical condition of feedforward networks with opposite transfer functions
AU - Ventresca, Mario
AU - Tizhoosh, Hamid Reza
PY - 2008
Y1 - 2008
N2 - Numerical condition affects the learning speed and accuracy of most artificial neural network learning algorithms. In this paper, we examine the influence of opposite transfer functions on the conditioning of feedforward neural network architectures. The goal is not to discuss a new training algorithm nor error surface geometry, but rather to present characteristics of opposite transfer functions which can be useful for improving existing or to develop new algorithms. Our investigation examines two situations: (1) network initialization, and (2) early stages of the learning process. We provide theoretical motivation for the consideration of opposite transfer functions as a means to improve conditioning during these situations. These theoretical results are validated by experiments on a subset of common benchmark problems. Our results also reveal the potential for opposite transfer functions in other areas of, and related to neural networks.
AB - Numerical condition affects the learning speed and accuracy of most artificial neural network learning algorithms. In this paper, we examine the influence of opposite transfer functions on the conditioning of feedforward neural network architectures. The goal is not to discuss a new training algorithm nor error surface geometry, but rather to present characteristics of opposite transfer functions which can be useful for improving existing or to develop new algorithms. Our investigation examines two situations: (1) network initialization, and (2) early stages of the learning process. We provide theoretical motivation for the consideration of opposite transfer functions as a means to improve conditioning during these situations. These theoretical results are validated by experiments on a subset of common benchmark problems. Our results also reveal the potential for opposite transfer functions in other areas of, and related to neural networks.
KW - Feedforward
KW - Ill-conditioning
KW - Numerical condition
KW - Opposite transfer functions
UR - http://www.scopus.com/inward/record.url?scp=56349095308&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=56349095308&partnerID=8YFLogxK
U2 - 10.1109/IJCNN.2008.4634257
DO - 10.1109/IJCNN.2008.4634257
M3 - Conference contribution
AN - SCOPUS:56349095308
SN - 9781424418213
T3 - Proceedings of the International Joint Conference on Neural Networks
SP - 3233
EP - 3240
BT - 2008 International Joint Conference on Neural Networks, IJCNN 2008
T2 - 2008 International Joint Conference on Neural Networks, IJCNN 2008
Y2 - 1 June 2008 through 8 June 2008
ER -