Paper:
Improve Discontinuous Output Change in SpikeProp
Haruhiko Takase*, Hiroharu Kawanaka*, and Shinji Tsuruoka**
*Graduate School of Engineering, Mie University, 1577, Kurima-Machiya, Tsu, Mie 514-8507, Japan
**Graduate School of Regional Innovation Studies, Mie University, 1577, Kurima-Machiya, Tsu, Mie 514-8507, Japan
We try to improve discontinuous output change in SpikeProp. The problem is that small variants in input cause significant change in output. We first show that peaks in activity cause the problem. We then propose reducing the height of peaks by weight decay. Through experiments, we conclude that the square type of weight decay is suitable for solving the problem, because it reduces to more than half the patterns that cause discontinuous output change around them.
- [1] W. Maass and C. M. Bishop, “Pulsed Neurla Networks,” Cambridge, MA, MIT press, 1998.
- [2] W. Maass, “Noisy Spiking neurons with Temporal Coding have more Computational Power than Sigmoidal Neurons,” Neural Information Processing Systems, pp. 211-217, 1996.
- [3] A. Kasiński and F. Ponulak, “Comparison of supervised learning methods for spike time coding in spiking neural networks,” Int. J. of Applied Mathematics and Computer Science, Vol.16, No.1, pp. 101-113, 2006.
- [4] S. GomesWysoski, L. Benuskova, and N. Kasabov, “Evolving spiking neural networks for audiovisual information processing,” Neural Networks, Vol.23, pp. 819-835, 2010.
- [5] S. M. Bohte, J. N. Kok, and H. La Poutŕe, “Error-backpropagation in temporally encoded networks of spiking neurons,” Neurocomputing, Vol.48, pp. 17-37, 2002.
- [6] D. C. Plaut, S. J. Nowlan, and G. E. Hinton, “Experiments on Learning by Back Propagation,” Technical Report CMU-CS-86-126, Pittsburgh, PA, 1986.
- [7] M. Ishikawa, “Structural learning with forgetting,” Neural Networks, Vol.9, No.3, pp. 509-521, 1996.
- [8] D. E. Rumelhart, J. L. McClelland, “Parrallel distributed processing: Explorations in the microstructure of cognition,” Vol.1, Cambridge, MA, MIT Press, 1986.