Semi-Supervised Sequential Kernel Regression Models with Penalty Functions
Hengjin Tang, Sadaaki Miyamoto, and Yasunori Endo
Department of Risk Engineering, School of Systems and Information Engineering, University of Tsukuba, 1-1-1 Tennodai, Tsukuba, Ibaraki 305-8573, Japan
Switching regression models can output multiple clusters and regression models. However, there is one problem: the results have a strong dependency on the predefined number of clusters. To avoid these drawbacks, we have researched sequential extractions. In sequential extractions process, one cluster is extracted at a time using a method of noise-detection, and the number of clusters are determined automatically. We propose semi-supervised sequential kernel regression models with penalty functions. Additionally, we also find that the sensitivity against the regularization parameter λ can be alleviated by semi-supervisions using penalty functions. We show the effectiveness of the proposed method by using numerical examples.
-  R. E. Quandt, “A New Approach to Estimating Switching Regressions,” J. of the American Statistical Association, Vol.67, pp. 306-310, 1972.
-  S. M. Goldfeld and R. E. Quandt, “Techniques for Estimating Switching Regressions,” in Studies in Nonlinear Estimation (eds. by S. M. Goldfeld and R. E. Quandt), pp. 3-35, Ballinger, Cambridge, Massachusetts, 1976.
-  B. Mirkin, “The Iterative Extraction Approach to Clustering,” Lecture Notes in Computational Science and Engineering, 58, pp. 153-179, Springer, New York, 2007.
-  S. Theodoridis and K. Koutroumbas, “Pattern Recognition (4th Ed.),” Academic Press, Massachusetts, 2008.
-  S. Miyamoto, Y. Kuroda, and K. Arai, “Algorithms for Sequential Extraction of Clusters by Possibilistic Method and Comparison with Mountain Clustering,” J. of Advanced Computational Intelligence and Intelligent Informatics, Vol.12, No.5, pp. 448-453, 2008.
-  R. N. Davé and R. Krishnapuram, “Robust clustering methods: a unified view,” IEEE Trans. on Fuzzy Systems, Vol.5, No.2, pp. 270-293, 1997.
-  R. N. Davé and S. Sen, “On Generalizing the Noise Clustering Algorithms,” Proc. of the 7th IFSA World Congress, Vol.3, pp. 205-210, 1997.
-  H. Tang and S. Miyamoto, “Semi-supervised Sequential Kernel Regression Models with Pairwise Constraints,” Lecture Notes in Artificial Intelligence, 8234, pp. 166-178, 2013.
-  M. Girolami, “Mercer Kernel-Based Clustering in Feature Space,” IEEE Trans. on Neutral Networks, Vol.13, No.3, pp. 780-784, 2002.
-  J. Shawe-Taylor and N. Cristianini, Kernel Methods for Pattern Analysis, Cambridge University Press, 2004.
-  Kernel-Machines.Org, address:
http://kernel-machines.org [Accessed March 15, 2014]
-  K. Wagstaff, C. Cardie, S. Rogers, and S. Schröedl, “Constrained K-means Clustering with Background Knowledge,” Proc. of the 8th Int. Conf. on Machine Learning, pp. 577-584, 2001.
-  S. Basu,M. Bilenko, and R. J. Mooney, “A Probabilistic Framework for Semi-Supervised Clustering,” Proc. of the 10th ACM SIGKDD Int. Conf. on Knowledge Discovery and Data Mining, pp. 59-68, 2004.
-  H. Tang and S. Miyamoto, “Sequential Extraction of Fuzzy Regression Models: Least Squares and Least Absolute Deviations,” Int. J. of Uncertainty, Fuzziness and Knowledge-Based Systems, Vol.19, Suppl.1, pp. 53-63, 2011.
-  H. Tang and S. Miyamoto, “Sequential Regression Models with Pairwise Constraints Using Noise Clusters,” J. of Advanced Computational Intelligence and Intelligent Informatics (JACIII), Vol.16, No.7, pp. 814-818, 2012.
-  O. Chapelle, B. Schölkopf, and A. Zien, “Semi-Supervised Learning,” The MIT Press, Cambridge, Massachusttes, 2006.