Semi-Supervised Pattern Classification Utilizing Fuzzy Clustering and Nonlinear Mapping of Data
Weiwei Du and Kiichi Urahama
Department of Visual Communication Design, Kyushu University, 4-9-1 Shiobaru, Fukuoka 815-8540, Japan
We present a semi-supervised algorithm for classification of arbitrarily distributed patterns. We project data into a classification space through two stages, first is a nonlinear mapping with radial basis functions and second is a linear projection with a semi-supervised locality preserving projection. Radial basis functions are arranged by fuzzy clustering of training data. This fuzzy clustering is also exploited for selection of data to be labeled for semi-supervised learning. We devise a simple semi-supervised algorithm in which data similarity is multiplicatively modulated on the basis of label information. We examine performance of the proposed classifier with experiments for synthetic and some real data and show that our method outperforms similar graph spectral algorithms and kernel semi-supervised methods.
-  P. N. Belhumeuer, J. P. Hespanha, and D. J. Kriegman, “Eigenfaces vs. fisherfaces: recognition using class specific linear projection,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.19, No.7, pp. 711-720, 1997.
-  M. Turk and A. Pentland, “Eigenfaces for recognition,” Journal of Cognitive Neuroscience, Vol.3, No.1, pp. 71-86, 1991.
-  X. Zhu, “Semi-supervised learning literature survey,” Computer Sciences TR 1530, Univ. Wisconsin, 2005.
-  M. Belkin and P. Niyogi, “Laplacian eigenmaps for dimensionality reduction and data representation,” Neural Computation, Vol.15, Issue 6, pp. 1373-1396, 2003.
-  X. He and P. Niyogi, “Locality preserving projections,” NIPS, 2003.
-  J. Shawe-Taylor and N. Cristianini, “Kernel methods for pattern analysis,” Cambridge Univ. Press, 2004.
-  S. Hoi and M. R. Lyu, “A semi-supervised active learning framework for image retrieval,” Proc. of the 2005 IEEE Computer Society Conf. on Computer Vision and Pattern Recognition (CVPR’05), pp. 302-309, 2005.
-  S. D. Kamvar, D. Klein, and C. D. Manning, “Spectral learning,” Int. Joint Conf. of Artificial Intelligence, pp. 561-566, 2003.
-  A. Balaldi and P. Blonda, “A survey of fuzzy clustering algorithms for pattern recognition,” IEEE Transactions on Systems, Man and Cybernetics, Part B, Vol.29, pp. 778-801, 1999.
-  R. N. Dave, “Characterization and detection of noise in clustering,” Pattern Recognition Letters, Vol.12, Issue 11, pp. 657-664, 1991.
-  X. Zhu, Z. Ghahrmani, and J. D. Lafferty, “Semi-supervised learning using Gaussian fields and harmonic functions,” Int. Conf. on Machine Learning, pp. 912-919, 2003.
-  D. Graham and N. Allinson, “The UMIST face database,”
-  D. J. Newman, S. Hettich, C. L. Blake, and C. J. Merz, “UCI Repository of machine learning databases,”
-  W. Du and K. Urahama, “Semi-Supervised Classification with Spectral Subspace Projection of Data,” IEICE Transactions on Information and Systems, Vol.E90-D, Issue 1, pp. 374-377, 2007.
-  J. Lu, K. N. Plataniotis, and A. N. Venetsanopoulos, “Face recognition using kernel direct discriminant analysis algorithms,” IEEE Transactions on Neural Networks, Vol.14, Issue 1, pp. 117-126, 2003.
-  O. Chapelle, J. Weston, and B. Scholkopf, “Cluster kernels for semi-supervised learning,” Advances in Neural Information Processing Systems, Vol.15, pp. 585-592, 2003.