Particle Swarm Optimization-Based Convolutional Neural Network for Handwritten Chinese Character Recognition
Yongping Dan and Zhuo Li
School of Electronic and Information, Zhongyuan University of Technology
No.41 Zhongyuan Road, Zhengzhou 450007, China
Recently, handwritten Chinese character recognition has become an important research field in computer vision. With the development of deep learning, convolutional neural networks (CNNs) have demonstrated excellent performance in computer vision. However, CNNs are typically designed manually, which requires extensive experience and may lead to redundant computations. To solve these problems, in this study, the particle swarm optimization approach is incorporated into the design of a CNN for handwritten Chinese character recognition, reducing redundant computations in the network. In this approach, each network architecture is represented by a particle, and the optimal network architecture is determined by continuously updating the particles until a global particle is identified. The experimental validation resulted in a network accuracy of 97.24% with only 1.43 million network parameters. Therefore, it is demonstrated that the proposed particle swarm optimization method can quickly and accurately find the optimal network architecture.
-  A. Darwish et al., “A survey of swarm and evolutionary computing approaches for deep learning,” Artificial Intelligence Review, Vol.53, No.3, pp. 1767-1812, 2020. https://doi.org/10.1007/s10462-019-09719-2
-  Q. Hao et al., “Research on Offline Handwritten Chinese Character Recognition Based on Deep Learning,” 2019 9th Int. Conf. on Information Science and Technology (ICIST), pp. 470-474, 2019. https://doi.org/10.1109/ICIST.2019.8836833
-  L. Xu et al., “Recognition of handwritten Chinese characters based on concept learning,” IEEE Access, Vol.7, pp. 102039-102053, 2019. https://doi.org/10.1109/ACCESS.2019.2930799
-  M. H. Sadi and A. Mahani, “Accelerating Deep Convolutional Neural Network Base on Stochastic Computing,” Integration, Vol.76, pp. 113-121, 2021. https://doi.org/10.1016/j.vlsi.2020.09.008
-  L. Meng et al., “Oracle Bone Inscription Detector Based on SSD,” New Trends in Image Analysis and Processing (ICIAP 2019), pp. 126-136, 2019. https://doi.org/10.1007/978-3-030-30754-7_13
-  Y. Zhang, “Deep convolutional network for handwritten Chinese character recognition,” Computer Science Department, Stanford University, 2015.
-  F. E. Fernandes Jr. and G. G. Yen, “Particle swarm optimization of deep neural networks architectures for image classification,” Swarm and Evolutionary Computation, Vol.49, pp. 62-74, 2019. https://doi.org/10.1016/j.swevo.2019.05.010
-  N. Passalis and A. Tefas, “Training lightweight deep convolutional neural networks using bag-of-features pooling,” IEEE Trans. on Neural Networks and Learning Systems, Vol.30, No.6, pp. 1705-1715, 2018. https://doi.org/10.1109/TNNLS.2018.2872995
-  C. Cao et al., “Skeleton-based action recognition with gated convolutional neural networks,” IEEE Trans. on Circuits and Systems for Video Technology, Vol.29, No.11, pp. 3247-3257, 2018. https://doi.org/10.1109/TCSVT.2018.2879913
-  P. Melnyk et al., “A high-performance CNNs method for offline handwritten Chinese character recognition and visualization,” Soft Computing, Vol.24, No.8, pp. 7977-7987, 2019. https://doi.org/10.1007/s00500-019-04083-3
-  X. Xiao et al., “Building fast and compact convolutional neural networks for offline handwritten Chinese character recognition,” Pattern Recognition, Vol.72, pp. 72-81, 2017. https://doi.org/10.1016/j.patcog.2017.06.032
-  W. Yang et al., “DropSample: A new training method to enhance deep convolutional neural networks for large-scale unconstrained handwritten Chinese character recognition,” Pattern Recognition, Vol.58, No.4, pp. 190-203, 2016. https://doi.org/10.1016/j.patcog.2016.04.007
-  Z. Li et al., “Deep Matching Network for Handwritten Chinese Character Recognition,” Pattern Recognition, Vol.107, Article No.107471, 2020. https://doi.org/10.1016/j.patcog.2020.107471
-  Y. Dan et al., “S-Swin transformer: simplified Swin Transformer model for offline handwritten Chinese character recognition,” PeerJ Computer Science, Vol.8, Article No.e1093, 2022. https://doi.org/10.7717/peerj-cs.1093
-  S. Lai et al., “Toward high-performance online HCCR: A CNN approach with DropDistortion, path signature and spatial stochastic max-pooling,” Pattern Recognition Letters, Vol.89, pp. 60-66, 2017. https://doi.org/10.1016/j.patrec.2017.02.011
-  C. L. Liu et al., “Online and offline handwritten Chinese character recognition: Benchmarking on new databases,” Pattern Recognition, Vol.46, No.1, pp. 155-155, 2013. https://doi.org/10.1016/j.patcog.2012.06.021
-  J. Zou et al., “Handwritten Chinese Character Recognition by Convolutional Neural Network and Similarity Ranking,” arXiv:1908.11550, 2019.
-  M. S. Amin et al., “Recognition of Pashto Handwritten Characters Based on Deep Learning,” Sensors, Vol.20, No.20, Article No.5884, 2020. https://doi.org/10.3390/s20205884
-  H. Li et al., “A Comprehensive Analysis of Low-Impact Computations in Deep Learning Workloads,” Proc. of the 2021 on Great Lakes Symp. on VLSI (GLSVLSI’21), pp. 385-390, 2021. https://doi.org/10.1145/3453688.3461747
-  R. Sharma and B. Kaushik, “Offline recognition of handwritten Indic scripts: A state-of-the-art survey and future perspectives,” Computer Science Review, Vol.38, No.6, Article No.100302, 2020. https://doi.org/10.1016/j.cosrev.2020.100302
-  W. Wang et al., “DenseRAN for offline handwritten Chinese character recognition,” 2018 16th Int. Conf. on Frontiers in Handwriting Recognition (ICFHR), pp. 104-109, 2018. https://doi.org/10.1109/ICFHR-2018.2018.00027
-  W. Luo and S. Kamata, “Radical region based CNNs for offline handwritten Chinese character recognition,” 2017 4th IAPR Asian Conf. on Pattern Recognition (ACPR), pp. 542-547, 2017. https://doi.org/10.1109/ACPR.2017.76
-  X. Yang et al., “Improving offline handwritten Chinese character recognition by iterative refinement,” 2017 14th IAPR Int. Conf. on Document Analysis and Recognition (ICDAR), pp. 5-10, 2017. https://doi.org/10.1109/ICDAR.2017.11
-  R. Guha et al., “DevNet: An efficient CNN architecture for handwritten Devanagari character recognition,” Int. J. of Pattern Recognition and Artificial Intelligence, Vol.34, No.12, Article No.2052009, 2020. https://doi.org/10.1142/S0218001420520096
-  A. Krizhevsky et al., “Imagenet classification with deep convolutional neural networks,” Advances in Neural Information Processing Systems, Vol.25, pp. 1097-1105, 2012.
-  K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” 3rd Int. Conf. on Learning Representations (ICLR2015), 2015.
-  C. Szegedy et al., “Going deeper with convolutions,” 2015 IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 1-9, 2015. https://doi.org/10.1109/CVPR.2015.7298594
-  K. He et al., “Deep Residual Learning for Image Recognition,” 2016 IEEE Conf. on CVPR, pp. 770-778, 2016.
-  D. Cireşan and U. Meier, “Multi-Column Deep Neural Networks for Offline Handwritten Chinese Character Classification,” 2015 Int. Joint Conf. on Neural Networks (IJCNN), 2015. https://doi.org/10.1109/IJCNN.2015.7280516
-  Z. Zhong et al., “High performance offline handwritten Chinese character recognition using GoogleNet and directional feature maps,” 2015 13th ICDAR, pp. 846-850, 2015. https://doi.org/10.1109/ICDAR.2015.7333881
-  N. Bi et al., “The handwritten Chinese character recognition uses convolutional neural networks with the GoogleNet,” Int. J. of Pattern Recognition and Artificial Intelligence, Vol.33, No.11, Article No.1940016, 2019. https://doi.org/10.1142/S0218001419400160
-  X.-Y. Zhang et al., “Online and offline handwritten Chinese character recognition: A comprehensive study and new benchmark,” Pattern Recognition, Vol.61, pp. 348-360, 2017. https://doi.org/10.1016/j.patcog.2016.08.005
-  Z. Li et al., “Building efficient CNNs architecture for offline handwritten Chinese character recognition,” Int. J. on Document Analysis and Recognition (IJDAR), Vol.21, No.4, pp. 233-240, 2018. https://doi.org/10.1007/s10032-018-0311-4
-  Y. Dan et al., “PF-ViT: Parallel and Fast Vision Transformer for Offline Handwritten Chinese Character Recognition,” Computational Intelligence and Neuroscience, Vol.2022, Article No.8255763 2022. https://doi.org/10.1155/2022/8255763
-  V. G. Gudise and G. K. Venayagamoorthy, “Comparison of particle swarm optimization and backpropagation as training algorithms for neural networks,” Proc. of the 2003 IEEE Swarm Intelligence Symp., pp. 110-117, 2003. https://doi.org/10.1109/SIS.2003.1202255
-  Y. Sun et al., “A particle swarm optimization-based flexible convolutional autoencoder for image classification,” IEEE Trans. on Neural Networks and Learning Systems, Vol.30, No.8, pp. 2295-2309, 2019. https://doi.org/10.1109/TNNLS.2018.2881143
-  Q. Zhu, “Handwritten Chinese Characters Recognition Based on QPSO-BP,” 2019 5th Int. Conf. on Control, Automation and Robotics (ICCAR), pp. 281-285, 2019. https://doi.org/10.1109/ICCAR.2019.8813326
-  B. Wang et al., “A hybrid GA-PSO method for evolving architecture and short connections of deep convolutional neural networks,” 16th Pacific Rim Int. Conf. on Artificial Intelligence (PRICAI2019), pp. 650-663, 2019. https://doi.org/10.1007/978-3-030-29894-4_52
-  B. Wang et al., “Evolving deep convolutional neural networks by variable-length particle swarm optimization for image classification,” 2018 IEEE Congress on Evolutionary Computation (CEC), 2018. https://doi.org/10.1109/CEC.2018.8477735
-  A. G. Howard et al., “Mobilenets: Efficient convolutional neural networks for mobile vision applications,” arXiv:1704.04861, 2017.
-  F. N. Iandola et al., “SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size,” arXiv:1602.07360, 2016.
This article is published under a Creative Commons Attribution-NoDerivatives 4.0 Internationa License.