JACIII Vol.19 No.1 pp. 51-57
doi: 10.20965/jaciii.2015.p0051


Semi-Supervised Sequential Kernel Regression Models with Penalty Functions

Hengjin Tang, Sadaaki Miyamoto, and Yasunori Endo

Department of Risk Engineering, School of Systems and Information Engineering, University of Tsukuba, 1-1-1 Tennodai, Tsukuba, Ibaraki 305-8573, Japan

April 20, 2014
August 25, 2014
Online released:
January 20, 2015
January 20, 2015
kernel regression, switching regression models, semi-supervised clustering, sequential clustering, penalty functions

Switching regression models can output multiple clusters and regression models. However, there is one problem: the results have a strong dependency on the predefined number of clusters. To avoid these drawbacks, we have researched sequential extractions. In sequential extractions process, one cluster is extracted at a time using a method of noise-detection, and the number of clusters are determined automatically. We propose semi-supervised sequential kernel regression models with penalty functions. Additionally, we also find that the sensitivity against the regularization parameter λ can be alleviated by semi-supervisions using penalty functions. We show the effectiveness of the proposed method by using numerical examples.

  1. [1] R. E. Quandt, “A New Approach to Estimating Switching Regressions,” J. of the American Statistical Association, Vol.67, pp. 306-310, 1972.
  2. [2] S. M. Goldfeld and R. E. Quandt, “Techniques for Estimating Switching Regressions,” in Studies in Nonlinear Estimation (eds. by S. M. Goldfeld and R. E. Quandt), pp. 3-35, Ballinger, Cambridge, Massachusetts, 1976.
  3. [3] B. Mirkin, “The Iterative Extraction Approach to Clustering,” Lecture Notes in Computational Science and Engineering, 58, pp. 153-179, Springer, New York, 2007.
  4. [4] S. Theodoridis and K. Koutroumbas, “Pattern Recognition (4th Ed.),” Academic Press, Massachusetts, 2008.
  5. [5] S. Miyamoto, Y. Kuroda, and K. Arai, “Algorithms for Sequential Extraction of Clusters by Possibilistic Method and Comparison with Mountain Clustering,” J. of Advanced Computational Intelligence and Intelligent Informatics, Vol.12, No.5, pp. 448-453, 2008.
  6. [6] R. N. Davé and R. Krishnapuram, “Robust clustering methods: a unified view,” IEEE Trans. on Fuzzy Systems, Vol.5, No.2, pp. 270-293, 1997.
  7. [7] R. N. Davé and S. Sen, “On Generalizing the Noise Clustering Algorithms,” Proc. of the 7th IFSA World Congress, Vol.3, pp. 205-210, 1997.
  8. [8] H. Tang and S. Miyamoto, “Semi-supervised Sequential Kernel Regression Models with Pairwise Constraints,” Lecture Notes in Artificial Intelligence, 8234, pp. 166-178, 2013.
  9. [9] M. Girolami, “Mercer Kernel-Based Clustering in Feature Space,” IEEE Trans. on Neutral Networks, Vol.13, No.3, pp. 780-784, 2002.
  10. [10] J. Shawe-Taylor and N. Cristianini, Kernel Methods for Pattern Analysis, Cambridge University Press, 2004.
  11. [11] Kernel-Machines.Org, address: [Accessed March 15, 2014]
  12. [12] K. Wagstaff, C. Cardie, S. Rogers, and S. Schröedl, “Constrained K-means Clustering with Background Knowledge,” Proc. of the 8th Int. Conf. on Machine Learning, pp. 577-584, 2001.
  13. [13] S. Basu,M. Bilenko, and R. J. Mooney, “A Probabilistic Framework for Semi-Supervised Clustering,” Proc. of the 10th ACM SIGKDD Int. Conf. on Knowledge Discovery and Data Mining, pp. 59-68, 2004.
  14. [14] H. Tang and S. Miyamoto, “Sequential Extraction of Fuzzy Regression Models: Least Squares and Least Absolute Deviations,” Int. J. of Uncertainty, Fuzziness and Knowledge-Based Systems, Vol.19, Suppl.1, pp. 53-63, 2011.
  15. [15] H. Tang and S. Miyamoto, “Sequential Regression Models with Pairwise Constraints Using Noise Clusters,” J. of Advanced Computational Intelligence and Intelligent Informatics (JACIII), Vol.16, No.7, pp. 814-818, 2012.
  16. [16] O. Chapelle, B. Schölkopf, and A. Zien, “Semi-Supervised Learning,” The MIT Press, Cambridge, Massachusttes, 2006.

*This site is desgined based on HTML5 and CSS3 for modern browsers, e.g. Chrome, Firefox, Safari, Edge, IE9,10,11, Opera.

Last updated on Mar. 28, 2017