single-jc.php

JACIII Vol.30 No.6 pp. 1846-1856
(2026)

Research Paper:

A Continual Self-Supervised Learning Framework Using a Decoder and a Clustering-Based Data Selection Method

Ryosuke Fujii* ORCID Icon, Naoki Masuyama*,† ORCID Icon, Yusuke Nojima* ORCID Icon, Chu Kiong Loo** ORCID Icon, and Liew Wei Shiung** ORCID Icon

*Osaka Metropolitan University
1-1 Gakuen-cho, Sakai, Osaka 599-8531, Japan

†Corresponding author

**University of Malaya
Kuala Lumpur 50603, Malaysia

Received:
October 17, 2025
Accepted:
June 23, 2026
Published:
November 20, 2026
Keywords:
self-supervised learning, continual learning, clustering, replay
Abstract

Continual learning aims to learn a sequence of tasks without forgetting the previously learned tasks. Promising methods for achieving continual learning include applying a replay buffer that stores previously learned samples in existing learning frameworks. Among these methods, lifelong unsupervised mixup (LUMP) interpolates the current task samples and samples from the replay buffer to prevent forgetting. LUMP randomly selects data for the replay buffer. This random selection is likely to have a negative effect on learning performance. In this study, we propose a clustering-based data selection method for the replay buffer. In addition, we introduce a denoising autoencoder to the self-supervised methods to improve the performance of the learning representations. We compare the clustering algorithms used in our framework and demonstrate the effectiveness of the proposed framework through computational experiments. Experimental results on CIFAR-100 and Tiny-ImageNet demonstrate that the proposed framework improves the average accuracy compared with LUMP while reducing forgetting. In particular, the CA+-based selection approach enhances the learning performance. The source code is available at https://github.com/Masuyama-lab/LUMP_SidAE_Clustering

Proposed continual SSL framework

Proposed continual SSL framework

Cite this article as:
R. Fujii, N. Masuyama, Y. Nojima, C. Loo, and L. Shiung, “A Continual Self-Supervised Learning Framework Using a Decoder and a Clustering-Based Data Selection Method,” J. Adv. Comput. Intell. Intell. Inform., Vol.30 No.6, pp. 1846-1856, 2026.
Data files:
References
  1. [1] Q. Sang, Z. Shu, L. Liu, C. Hu, and Q. Wu, “Image quality assessment based on self-supervised learning and knowledge distillation,” J. of Visual Communication and Image Representation, Vol.90, Article No.103708, 2023. https://doi.org/10.1016/j.jvcir.2022.103708
  2. [2] V. Rani, S. T. Nabi, M. Kumar, A. Mittal, and K. Kumar, “Self-supervised learning: A succinct review,” Archives of Computational Methods in Engineering, Vol.30, No.4, pp. 2761-2775, 2023. https://doi.org/10.1007/s11831-023-09884-2
  3. [3] D. Madaan, J. Yoon, Y. Li, Y. Liu, and S. J. Hwang, “Representational continuity for unsupervised continual learning,” Proc. of Int. Conf. on Learning Representations, 2022.
  4. [4] F. Baier, S. Mair, and S. G. Fadel, “Self-supervised siamese autoencoders,” I. Miliou, N. Piatkowski, and P. Papapetrou (Eds.), “Lecture Notes in Computer Science,” pp. 117-128, 2024. https://doi.org/10.1007/978-3-031-58547-0_10
  5. [5] J. Gui, T. Chen, J. Zhang, Q. Cao, Z. Sun, H. Luo, and D. Tao, “A survey on self-supervised learning: Algorithms, applications, and future trends,” IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol.46, No.12, pp. 9052-9071, 2024. https://doi.org/10.1109/tpami.2024.3415112
  6. [6] T. Chen, S. Kornblith, M. Norouzi, and G. Hinton, “A simple framework for contrastive learning of visual representations,” Proc. of the 37th Int. Conf. on Machine Learning, Vol.119, pp. 1597-1607, 2020.
  7. [7] K. He, H. Fan, Y. Wu, S. Xie, and R. Girshick, “Momentum contrast for unsupervised visual representation learning,” 2020 IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 9726-9735, 2020. https://doi.org/10.1109/cvpr42600.2020.00975
  8. [8] M. Caron, I. Misra, J. Mairal, P. Goyal, P. Bojanowski, and A. Joulin, “Unsupervised learning of visual features by contrasting cluster assignments,” Proc. of Advances in Neural Information Processing Systems, pp. 9912-9924, 2020.
  9. [9] J.-B. Grill, F. Strub, F. Altché, C. Tallec, P. Richemond, E. Buchatskaya, C. Doersch, B. Avila Pires, Z. Guo, M. Gheshlaghi Azar, B. Piot, K. Kavukcuoglu, R. Munos, and M. Valko, “Bootstrap your own latent: A new approach to self-supervised learning,” Proc. of Advances in Neural Information Processing Systems, pp. 21271-21284, 2020.
  10. [10] X. Chen and K. He, “Exploring simple siamese representation learning,” 2021 IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 15745-15753, 2021. https://doi.org/10.1109/cvpr46437.2021.01549
  11. [11] J. Zbontar, L. Jing, I. Misra, Y. LeCun, and S. Deny, “Barlow twins: Self-supervised learning via redundancy reduction,” Proc. of Int. Conf. on Machine Learning, pp. 12310-12320, 2021.
  12. [12] K. He, X. Chen, S. Xie, Y. Li, P. Dollar, and R. Girshick, “Masked autoencoders are scalable vision learners,” 2022 IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 15979-15988, 2022. https://doi.org/10.1109/cvpr52688.2022.01553
  13. [13] Z. Xie, Z. Zhang, Y. Cao, Y. Lin, J. Bao, Z. Yao, Q. Dai, and H. Hu, “SimMIM: a simple framework for masked image modeling,” 2022 IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 9643-9653, 2022. https://doi.org/10.1109/cvpr52688.2022.00943
  14. [14] J. Kirkpatrick, R. Pascanu, N. Rabinowitz, J. Veness, G. Desjardins, A. A. Rusu, K. Milan, J. Quan, T. Ramalho, A. Grabska-Barwinska, D. Hassabis, C. Clopath, D. Kumaran, and R. Hadsell, “Overcoming catastrophic forgetting in neural networks,” Proc. of the National Academy of Sciences, Vol.114, No.13, pp. 3521-3526, 2017. https://doi.org/10.1073/pnas.1611835114
  15. [15] F. Zenke, B. Poole, and S. Ganguli, “Continual learning through synaptic intelligence,” Proc. of the 34th Int. Conf. on Machine Learning, pp. 3987-3995, 2017.
  16. [16] A. A. Rusu, N. C. Rabinowitz, G. Desjardins, H. Soyer, J. Kirkpatrick, K. Kavukcuoglu, R. Pascanu, and R. Hadsell, “Progressive neural networks,” arXiv:1606.04671, 2016. https://doi.org/10.48550/arXiv.1606.04671
  17. [17] A. Mallya and S. Lazebnik, “PackNET: Adding multiple tasks to a single network by iterative pruning,” 2018 IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 7765-7773, 2018. https://doi.org/10.1109/cvpr.2018.00810
  18. [18] P. Buzzega, M. Boschini, A. Porrello, D. Abati, and S. Calderara, “Dark experience for general continual learning: A strong, simple baseline,” Proc. of Advances in Neural Information Processing Systems, pp. 15920-15930, 2020.
  19. [19] D. Lopez-Paz and M. A. Ranzato, “Gradient episodic memory for continual learning,” Proc. of Advances in Neural Information Processing Systems, 2017.
  20. [20] H. Zhang, M. Cisse, Y. N. Dauphin, and D. Lopez-Paz, “mixup: Beyond empirical risk minimization,” Proc. of Int. Conf. on Learning Representations, 2018.
  21. [21] Z. Shen, Z. Liu, Z. Liu, M. Savvides, T. Darrell, and E. Xing, “Un-mix: Rethinking image mixtures for unsupervised visual representation learning,” Proc. of the AAAI Conf. on Artificial Intelligence, Vol.36, No.2, pp. 2216-2224, 2022. https://doi.org/10.1609/aaai.v36i2.20119
  22. [22] D. Reynolds, “Gaussian mixture models,” S. Z. Li and A. K. Jain (Eds.), “Encyclopedia of Biometrics,” Springer, pp. 827-832, 2015. https://doi.org/10.1007/978-1-4899-7488-4_196
  23. [23] S. Lloyd, “Least squares quantization in PCM,” IEEE Trans. on Information Theory, Vol.28, No.2, pp. 129-137, 1982. https://doi.org/10.1109/tit.1982.1056489
  24. [24] B. Fritzke, “A growing neural gas network learns topologies,” Proc. of Advances in Neural Information Processing Systems, 1994.
  25. [25] G. Carpenter and S. Grossberg, “The art of adaptive pattern recognition by a self-organizing neural network,” Computer, Vol.21, No.3, pp. 77-88, 1988. https://doi.org/10.1109/2.33
  26. [26] F. Shen and O. Hasegawa, “A fast nearest neighbor classifier based on self-organizing incremental neural network,” Neural Networks, Vol.21, No.10, pp. 1537-1547, 2008. https://doi.org/10.1016/j.neunet.2008.07.001
  27. [27] C. Wiwatcharakoses and D. Berrar, “SOINN+, a self-organizing incremental neural network for unsupervised learning from noisy data streams,” Expert Systems with Applications, Vol.143, Article No.113069, 2020. https://doi.org/10.1016/j.eswa.2019.113069
  28. [28] S. Marsland, J. Shapiro, and U. Nehmzow, “A self-organising network that grows when required,” Neural Networks, Vol.15, No. 8-9, pp. 1041-1058, 2002. https://doi.org/10.1016/s0893-6080(02)00078-3
  29. [29] G. A. Carpenter, S. Grossberg, and D. B. Rosen, “Fuzzy ART: Fast stable learning and categorization of analog patterns by an adaptive resonance system,” Neural Networks, Vol.4, No.6, pp. 759-771, 1991. https://doi.org/10.1016/0893-6080(91)90056-b
  30. [30] B. Vigdor and B. Lerner, “The Bayesian ARTMAP,” IEEE Trans. on Neural Networks, Vol.18, No.6, pp. 1628-1644, 2007. https://doi.org/10.1109/tnn.2007.900234
  31. [31] L. Wang, H. Zhu, J. Meng, and W. He, “Incremental local distribution-based clustering using Bayesian adaptive resonance theory,” IEEE Trans. on Neural Networks and Learning Systems, Vol.30, No.11, pp. 3496-3504, 2019. https://doi.org/10.1109/tnnls.2019.2919723
  32. [32] L. E. B. da Silva, I. Elnabarawy, and D. C. Wunsch, “Distributed dual vigilance fuzzy adaptive resonance theory learns online, retrieves arbitrarily-shaped clusters, and mitigates order dependence,” Neural Networks, Vol.121, pp. 208-228, 2020. https://doi.org/10.1016/j.neunet.2019.08.033
  33. [33] W. Liu, P. P. Pokharel, and J. C. Principe, “Correntropy: Properties and applications in non-gaussian signal processing,” IEEE Trans. on Signal Processing, Vol.55, No.11, pp. 5286-5298, 2007. https://doi.org/10.1109/tsp.2007.896065
  34. [34] N. Masuyama, C. K. Loo, and F. Dawood, “Kernel Bayesian ART and ARTMAP,” Neural Networks, Vol.98, pp. 76-86, 2018. https://doi.org/10.1016/j.neunet.2017.11.003
  35. [35] N. Masuyama, C. K. Loo, and S. Wermter, “A Kernel Bayesian adaptive resonance theory with a topological structure,” Int. J. of Neural Systems, Vol.29, No.5, Article No.1850052, 2019, pMID: 30764724. https://doi.org/10.1142/s0129065718500521
  36. [36] N. Masuyama, C. K. Loo, H. Ishibuchi, N. Kubota, Y. Nojima, and Y. Liu, “Topological clustering via adaptive resonance theory with information theoretic learning,” IEEE Access, Vol.7, pp. 76920-76936, 2019. https://doi.org/10.1109/access.2019.2921832
  37. [37] N. Masuyama, N. Amako, Y. Nojima, Y. Liu, C. K. Loo, and H. Ishibuchi, “Fast topological adaptive resonance theory based on correntropy induced metric,” 2019 IEEE Symp. Series on Computational Intelligence (SSCI), pp. 2215-2221, 2019. https://doi.org/10.1109/ssci44817.2019.9003098
  38. [38] L. E. B. da Silva, I. Elnabarawy, and D. C. Wunsch, “Dual vigilance fuzzy adaptive resonance theory,” Neural Networks, Vol.109, pp. 1-5, 2019. https://doi.org/10.1016/j.neunet.2018.09.015
  39. [39] L. Meng, A.-H. Tan, and D. C. Wunsch, “Adaptive scaling of cluster boundaries for large-scale social media data clustering,” IEEE Trans. on Neural Networks and Learning Systems, Vol.27, No.12, pp. 2656-2669, 2016. https://doi.org/10.1109/tnnls.2015.2498625
  40. [40] N. Masuyama, Y. Nojima, Y. Toda, C. K. Loo, H. Ishibuchi, and N. Kubota, “Privacy-preserving continual federated clustering via adaptive resonance theory,” IEEE Access, Vol.12, pp. 139692-139710, 2024. https://doi.org/10.1109/access.2024.3467114
  41. [41] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” 2016 IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 770-778, 2016. https://doi.org/10.1109/cvpr.2016.90
  42. [42] A. Krizhevsky, “Learning Multiple Layers of Features From Tiny Images,” University of Toronto, 2012.
  43. [43] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “ImageNet: A large-scale hierarchical image database,” 2009 IEEE Computer Society Conf. on Computer Vision and Pattern Recognition Workshops (CVPR Workshops), pp. 248-255, 2009. https://doi.org/10.1109/cvpr.2009.5206848
  44. [44] L. Wang, X. Zhang, H. Su, and J. Zhu, “A comprehensive survey of continual learning: Theory, method and application,” IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol.46, No.8, pp. 5362-5383, 2024. https://doi.org/10.1109/tpami.2024.3367329
  45. [45] L. Kaufman and P. J. Rousseeuw, “Partitioning around medoids (program PAM),” L. Kaufman and P. J. Rousseeuw, “Finding Groups in Data: An Introduction to Cluster Analysis,” John Wiley & Sons, Inc., pp. 68-125, 1990.
  46. [46] A. Y. Ng, M. I. Jordan, and Y. Weiss, “On spectral clustering: Analysis and an algorithm,” Proc. of the 15th Int. Conf. on Neural Information Processing Systems: Natural and Synthetic, pp. 849-856, 2001.
  47. [47] T. Kohonen, “The self-organizing map,” Proc. of the IEEE, Vol.78, No.9, pp. 1464-1480, 1990. https://doi.org/10.1109/5.58325
  48. [48] Y. Zhang, L. Charlin, R. Zemel, and M. Ren, “Integrating present and past in unsupervised continual learning,” arXiv:2404.19132, 2024. https://doi.org/10.48550/arXiv.2404.19132

*This site is desgined based on HTML5 and CSS3 for modern browsers, e.g. Chrome, Firefox, Safari, Edge, Opera.

Last updated on Nov. 20, 2026