Paper:
Views over last 60 days: 913
Neural Network Structure Analysis Based on Hierarchical Force-Directed Graph Drawing for Multi-Task Learning
Atsushi Shibata*, Fangyan Dong**, and Kaoru Hirota*
*Department of Computational Intelligence & Systems Science, Tokyo Institute of Technology
G3-49, 4259 Nagatsuta, Midori-ku, Yokohama 226-8502, Japan
**Education Academy of Computational Life Sciences, Tokyo Institute of Technology
J3-141, 4259 Nagatsuta, Midori-ku, Yokohama 226-8501, Japan
Received:June 21, 2014Accepted:December 4, 2014Published:March 20, 2015
Keywords:neural network, network structure, multi-task learning, clustering, visualization
Abstract
A hierarchical force-directed graph drawing is proposed for the analysis of a neural network structure that expresses the relationship between multitask and processes in neural networks represented as neuron clusters. The process revealed by our proposal indicates the neurons that are related to each task and the number of neurons or learning epochs that are sufficient. Our proposal is evaluated by visualizing neural networks learned on the Mixed National Institute of Standards and Technology (MNIST) database of handwritten digits, and the results show that inactive neurons, namely those that do not have a close relationship with any tasks, are located on the periphery part of the visualized network, and that cutting half of the training data on one specific task (out of ten) causes a 15% increase in the variance of neurons in clusters that react to the specific task compared to the reaction to all tasks. The proposal aims to be developed in order to support the design process of neural networks that consider multitasking of different categories, for example, one neural network for both the vision and motion system of a robot.
Cite this article as:A. Shibata, F. Dong, and K. Hirota, “Neural Network Structure Analysis Based on Hierarchical Force-Directed Graph Drawing for Multi-Task Learning,” J. Adv. Comput. Intell. Intell. Inform., Vol.19 No.2, pp. 225-231, 2015.Data files:
References- [1] Y. Bengio, P. Lamblin, D. Popovici, and H. Larochell, “Greedy Layer-Wise Training of Deep Networks,” Advances in Neural Information Processing Systems 19, pp. 153-160, 2007.
- [2] Y. Huang, W. Wang, L. Wang, and T. Tan, “Multi-task deep neural network for multi-label learning,” 20th the Int. Conf. on Image Processing, 2013.
- [3] T. Itoh, K. Inoue, J. Doi, K. Yasumasa, and I. Yuko, “An Improvement of Force-directed Graph Layout Method”, Report of Information Processing Society of Japan, Vol.2001, No.35, pp. 7-12, 2001 (in Japanese).
- [4] Y. LeCun, C. Cortes, and C. J. C. Burges, “THE MNITS DATABASE of handwritten digits,” http://yann.lecun.com/exdb/mnist/ [Accessed August 2014]
- [5] R. B. Plam, “Deep Learning Toolbox,” https://github.com/rasmusbergpalm/DeepLearnToolBox [Accessed August 2014]
