RasID – Random Search for Neural Network Training
Jinglu Hu, Kotaro Hirasawa and Junichi Murata
Department of Electrical and Electronic Systems Engineering, Kyushu University. 6-10-1 Hakozaki, Higashi-ku, Fukuoka 812-8581, Japan
This paper presents a novel random search, RasID, for neural network training, that introduces a sophisticated probability density function (PDF) in a random search for generating search vectors. The PDF provides two parameters to control local search ranges and directions efficiently. This realizes an intensified search where it is easy to find good solutions locally or a diversified search to escape local minima based on success-failure of past searches. Local gradients, if available, and trend information on the criterion function surface are used to improve search performance. The proposed scheme is applied to layered neural network training and is benchmarked against deterministic and nondeterministic methods.