Computing Higher Order Derivatives in Universal Learning Networks
Kotaro Hirasawa, Jinglu Hu, Masanao Ohbayashi, and Junichi Murata
Department of Electrical and Electronic Systems Engineering, Kyushu University. Hakozaki, Fukuoka 812-81, Japan
This paper discusses higher order derivative computing for universal learning networks that form a super set of all kinds of neural networks. Two computing algorithms, backward and forward propagation, are proposed. Using a technique called “local description” expresses the proposed algorithms very simply. Numerical simulations demonstrate the usefulness of higher order derivatives in neural network training.