Improve Discontinuous Output Change in SpikeProp

Haruhiko Takase; Hiroharu Kawanaka; Shinji Tsuruoka

doi:10.20965/jaciii.2014.p0361

single-jc.php

« previous

JACIII Vol.18 No.3 pp. 361-365

doi: 10.20965/jaciii.2014.p0361

(2014)

Paper:

Views over last 60 days: 1,028

Improve Discontinuous Output Change in SpikeProp

Haruhiko Takase^, Hiroharu Kawanaka^, and Shinji Tsuruoka^**

^*Graduate School of Engineering, Mie University, 1577, Kurima-Machiya, Tsu, Mie 514-8507, Japan

^**Graduate School of Regional Innovation Studies, Mie University, 1577, Kurima-Machiya, Tsu, Mie 514-8507, Japan

Received:

October 15, 2013

Accepted:

January 31, 2014

Published:

May 20, 2014

Keywords:

spiking neural network, SpikeProp, weight decay

Abstract

We try to improve discontinuous output change in SpikeProp. The problem is that small variants in input cause significant change in output. We first show that peaks in activity cause the problem. We then propose reducing the height of peaks by weight decay. Through experiments, we conclude that the square type of weight decay is suitable for solving the problem, because it reduces to more than half the patterns that cause discontinuous output change around them.

Cite this article as:

H. Takase, H. Kawanaka, and S. Tsuruoka, “Improve Discontinuous Output Change in SpikeProp,” J. Adv. Comput. Intell. Intell. Inform., Vol.18 No.3, pp. 361-365, 2014.

Data files:

References

[1] W. Maass and C. M. Bishop, “Pulsed Neurla Networks,” Cambridge, MA, MIT press, 1998.
[2] W. Maass, “Noisy Spiking neurons with Temporal Coding have more Computational Power than Sigmoidal Neurons,” Neural Information Processing Systems, pp. 211-217, 1996.
[3] A. Kasiński and F. Ponulak, “Comparison of supervised learning methods for spike time coding in spiking neural networks,” Int. J. of Applied Mathematics and Computer Science, Vol.16, No.1, pp. 101-113, 2006.
[4] S. GomesWysoski, L. Benuskova, and N. Kasabov, “Evolving spiking neural networks for audiovisual information processing,” Neural Networks, Vol.23, pp. 819-835, 2010.
[5] S. M. Bohte, J. N. Kok, and H. La Poutŕe, “Error-backpropagation in temporally encoded networks of spiking neurons,” Neurocomputing, Vol.48, pp. 17-37, 2002.
[6] D. C. Plaut, S. J. Nowlan, and G. E. Hinton, “Experiments on Learning by Back Propagation,” Technical Report CMU-CS-86-126, Pittsburgh, PA, 1986.
[7] M. Ishikawa, “Structural learning with forgetting,” Neural Networks, Vol.9, No.3, pp. 509-521, 1996.
[8] D. E. Rumelhart, J. L. McClelland, “Parrallel distributed processing: Explorations in the microstructure of cognition,” Vol.1, Cambridge, MA, MIT Press, 1986.

This article is published under a Creative Commons Attribution-NoDerivatives 4.0 Internationa License.

[1] [1] W. Maass and C. M. Bishop, “Pulsed Neurla Networks,” Cambridge, MA, MIT press, 1998.

[2] [2] W. Maass, “Noisy Spiking neurons with Temporal Coding have more Computational Power than Sigmoidal Neurons,” Neural Information Processing Systems, pp. 211-217, 1996.

[3] [3] A. Kasiński and F. Ponulak, “Comparison of supervised learning methods for spike time coding in spiking neural networks,” Int. J. of Applied Mathematics and Computer Science, Vol.16, No.1, pp. 101-113, 2006.

[4] [4] S. GomesWysoski, L. Benuskova, and N. Kasabov, “Evolving spiking neural networks for audiovisual information processing,” Neural Networks, Vol.23, pp. 819-835, 2010.

[5] [5] S. M. Bohte, J. N. Kok, and H. La Poutŕe, “Error-backpropagation in temporally encoded networks of spiking neurons,” Neurocomputing, Vol.48, pp. 17-37, 2002.

[6] [6] D. C. Plaut, S. J. Nowlan, and G. E. Hinton, “Experiments on Learning by Back Propagation,” Technical Report CMU-CS-86-126, Pittsburgh, PA, 1986.

[7] [7] M. Ishikawa, “Structural learning with forgetting,” Neural Networks, Vol.9, No.3, pp. 509-521, 1996.

[8] [8] D. E. Rumelhart, J. L. McClelland, “Parrallel distributed processing: Explorations in the microstructure of cognition,” Vol.1, Cambridge, MA, MIT Press, 1986.

Improve Discontinuous Output Change in SpikeProp

Haruhiko Takase*, Hiroharu Kawanaka*, and Shinji Tsuruoka**

Haruhiko Takase^, Hiroharu Kawanaka^, and Shinji Tsuruoka^**