2.1. Shannon Entropy and Tsallis Entropy

Let \(P\) be the probability distribution. The Shannon entropy of \(P\), denoted \(H_{\text{Shannon}}(P)\), is defined as follows.

2.2. Conventional Fuzzy Clustering Algorithms

4.1. Behaviors of FCFs for SFCV, EFCV, and TFCV

In the first experiment, we analyzed the behaviors of the FCFs for SFCV, EFCV, and TFCV. Furthermore, we observed a close relationship between TFCV and existing conventional algorithms. In this experiment, we applied TFCV, SFCV, and EFCV to an artificial dataset, as illustrated in Fig. 1, which formed two linear clusters in a 2D space. Notably, across all variations in the fuzzification parameter values, the proposed algorithm consistently yielded satisfactory clustering results, as shown in Fig. 2 (objects belonging to Cluster #1 are shown in green and those in Cluster #2 are shown in red), with \(v_1 \simeq v_2 \simeq (0, 0)^\mathsf{T}\), \(w_{1,1} \simeq (0, 1)^\mathsf{T}\), and \(w_{2,1} \simeq (1, 0)^\mathsf{T}\). Note that the cluster center \(v_1\), indicated by a green cross, is not visible in the figure because it coincides with \(v_2\), and the red cross marker for \(v_2\) completely overlays it.

To avoid misunderstanding, we emphasize that the purpose of this experiment was not to evaluate the clustering performance of the methods. Artificially simple dataset was deliberately selected so that the differences in algorithmic behavior with respect to the tuning parameters could be clearly observed, without the confounding effects of complex data structures. A thorough performance comparison between the proposed method and conventional methods on multiple real datasets is presented in the next subsection.

Figures 3–5 illustrate the FCFs of TFCV with various fuzzification parameter values. For comparison, Fig. 6 shows the FCF of SFCV with \(m=2\) and Fig. 7 shows the FCF of EFCV with \(\lambda=0.2\). In these figures, objects belonging to the first cluster are shown as green points, those in the second cluster as red points, and the FCFs for the first and second clusters are represented by purple and sky-blue surfaces, respectively. The base plane represents the data space and the vertical axis indicates the FCF value. Figs. 3 and 4 show the effect of the fuzzification parameter \(m\) on the FCF and the clustering results, indicating that larger values of \(m\) produce a fuzzier clustering in regions away from all objects. Figs. 3 and 5 indicate the effect of parameter \(\lambda\); smaller values of \(\lambda\) produce fuzzier clustering in regions where \(u_1(x)=u_2(x)\). Note that Figs. 4 and 6 are similar, which substantiates the theoretical result that TFCV with \(\lambda\to\infty\) is reduced to SFCV shown in the previous section. In addition, Figs. 5 and 7 are similar, which substantiates the theoretical result that TFCV with \(m\searrow1\) was reduced to EFCV, as shown in the previous section.

In Fig. 6, the FCF values \(u_1(x)\) on the subspace \(v_1+\alpha w_{1,1}\) (\(\alpha \in \mathbb{R}\)) seems to be similar to that of \(u_2(x)\) on the subspace \(v_2+\alpha w_{2,1}\) (\(\alpha \in \mathbb{R}\)). This raises to the question of the FCF value at \(x= (0, 0)^\mathsf{T}\), the intersection point of the two subspaces because the FCF constraint \(u_1(x)+u_2(x)=1\) implies that \(u_1(0)=1\) and \(u_2(0)=1\) cannot hold simultaneously. We address this question theoretically, and show that the FCF value of SFCV at \(x= (0, 0)^\mathsf{T}\) is not well-defined as it depends on the direction from which \(x\) approaches this point. First, we consider limit \(a\to0\) with \(x=(a, 0)\), where \(a\in\mathbb{R}\setminus {0}\). In this setting, the values of \(d_1(x)\) and \(d_2(x)\) are calculated using Eq. \(\eqref{eq:d-FCV-FCF}\), as follows:

TFCV generalizes both SFCV and EFCV from the perspectives of both the optimization problem and the algorithm. This fact, along with the above discussion, yields another question: whether the FCF value of TFCV is well-defined at \(x= (0, 0)^\mathsf{T}\). We show that the FCF value of TFCV at \(x= (0, 0)^\mathsf{T}\) is well-defined, inheriting the property of EFCV because we have

4.2. Comparison of Clustering Accuracy Using Real Datasets

In the second set of experiments, we assessed the clustering performance of the proposed algorithm in comparison with conventional approaches across 14 real datasets. The Breast Cancer Coimbra (BCC), Breast Cancer Wisconsin (Original) (BCW), Glass Identification (Glass), Optical Recognition of Handwritten Digits (Optdigits), Seeds, and Yeast datasets were obtained from the UCI Machine Learning repository ⁸. The Coffee and Olive datasets were retrieved from the pgmm package ⁹ in R, whereas the Crabs dataset was retrieved from the MASS package ¹⁰ in R. The Wine dataset was accessed using the rattle package ¹¹, and the Australian Institute of Sport (AIS) dataset was obtained from the Australasian Data and Story Library ¹². Bupa and Heart data were obtained from the Keel website at University of Granada ¹³, and Pima Indians Diabetes (PID) data were obtained from the Kaggle website ¹⁴.

All datasets were clustered using SFCV, EFCV, and the proposed TFCV algorithms. For each dataset, the number of clusters was fixed to match the number of ground truth classes. Table 1 summarizes the characteristics of the datasets, including the number of objects, clusters, and dimensions.

The fuzzification parameters were varied according to the following rationale. As observed in the artificial data experiments, a larger value of \(m\) yielded fuzzier clustering results, whereas a smaller value produced crisper clustering results. If \(m\) becomes excessively large, the membership degrees approach \(1/C\) for all objects, causing all cluster centers to collapse to a single point and resulting in a meaningless clustering structure. Conversely, if \(m\) is set too small, the memberships become overly crisp, forcing objects near cluster boundaries, where fuzzy assignments are inherently appropriate, to be assigned values close to \(1\) or \(0\), thereby degrading the clustering accuracy. Moreover, excessively small values of \(m\) can lead to numerical instability, eventually producing NaN values that prevent the algorithm from continuing. Similarly, a smaller value of \(\lambda\) yields fuzzier results, whereas a larger value produces crisper results. When \(\lambda\) is too small, the memberships again approach \(1/C\), causing all cluster centers to collapse and producing an uninformative clustering. On the other hand, if \(\lambda\) is set too large, the memberships become overly crisp, leading to the same degradation in accuracy near cluster boundaries. Furthermore, extremely large values of \(\lambda\) may cause numerical overflow, preventing the algorithm from proceeding. A clustering result that is excessively fuzzy and thus meaningless can be detected by the partition entropy (PE) ¹; if the PE value associated with a result is equal to the logarithm of the number of clusters, the result is discarded because such a solution does not correspond to a meaningful clustering structure. Considering this, preliminary experiments were performed to identify the parameter ranges that avoid overflow or NaN values, yielding a PE value lower than \(\ln(C)\). Balancing these considerations, we varied the parameter values as follows: \(m \in \{1+10^{-15}\), \(1+10^{-10}\), \(1+10^{-5}\), \(1+10^{-3}\), \(1+10^{-2}\), \(1.1\), \(1.5\), \(2.0\), \(3.0\), \(4.0\), \(5.0\), \(10.0\}\) and \(\lambda \in \{0.001\), \(0.01\), \(0.1\), \(0.5\), \(1.0\), \(1.5\), \(2.0\), \(5.0\), \(10\), \(100\), \(10^{3}\), \(10^{10}\}\). The dimensions of the subspace also influence clustering accuracy. In accordance with the general principle of dimensionality reduction that the data are expected to lie in relatively low-dimensional subspaces, we varied the dimension of subspace as follows: \(D'\in\{1, 2,\dots,\lceil D/2 \rceil\}\). For the Optdigits dataset, however, the dimension of subspace was varied as \(D'\in\{1, 2,\dots,10\}\) because of its higher intrinsic dimensionality. Each algorithm was executed ten times with randomly initialized memberships for every parameter setting, and the trial with the lowest objective function value was retained. Trials in which at least one object attained the highest membership value in two or more clusters were excluded from the analysis.

Clustering quality was evaluated using the adjusted Rand index (ARI) ¹⁵, which is defined for unique crisp partitions. Consequently, trials where ties occurred at maximum membership values were excluded from the evaluation. Each algorithm also involves its own fuzzification parameter along with a certain dimension of the subspace as a hyperparameter. For each hyperparameter value, the highest ARI value was selected as the representative ARI for that algorithm on the given dataset.

Table 2 presents, for each dataset, the ARI values obtained by the three methods, SFCV, EFCV, and TFCV, along with the corresponding PE values and the parameter settings \((D', m, \lambda)\) under which these values were achieved. For each dataset, the five rows correspond to ARI, PE, \(D'\), \(m\), and \(\lambda\), respectively. Because SFCV does not use \(\lambda\) and EFCV does not use \(m\), the entries that do not apply are marked with “—.” The three columns represent SFCV, EFCV, and TFCV, respectively, and the highest ARI value for each dataset is highlighted in bold. Some readers may raise concerns that the clustering results for certain datasets are excessively fuzzy, based on the PE values reported in Table 2. In particular, for two-cluster datasets, PE values close to \(\ln(2)=0.6931\) may give the impression that the memberships are nearly uniform. However, a closer inspection of the detailed results shows that such values do not necessarily correspond to degenerating clustering solutions. For the BCW dataset, although the PE value of TFCV is relatively high (\(0.6888\)), the squared Euclidean distance between the two cluster centers is \(1.1063\), indicating that the centers do not form a single point. In addition, \(35\) out of the \(683\) objects have a membership difference of at least \(0.2\) (i.e., one membership is \(\ge 0.6\) and the other is \(\le 0.4\)), confirming that a meaningful clustering structure is preserved. The PE value of SFCV (\(0.6861\)) is slightly lower and likewise does not indicate a degenerate solution. A similar interpretation applies to the PID dataset. Although the PE value of SFCV is \(0.6383\), the squared distance between the two cluster centers is \(1421.6830\), and \(526\) out of the \(768\) objects exhibit a membership difference of at least \(0.2\). These observations indicate that the obtained clustering structure is nontrivial. The PE value of TFCV (\(0.6264\)) is lower yet and does not suggest excessive fuzziness. These observations confirm that the resulting clustering is not excessively fuzzy and retains interpretable structure. Readers may also raise the following concern: EFCV, which has only the fuzzification parameter \(\lambda\), does not seem to perform as well as SFCV, which has only the parameter \(m\). If this is the case, a question may arise whether the proposed method—TFCV, which incorporates both \(m\) and \(\lambda\)—can truly achieve better performance. This concern can be addressed as follows. The experiments using artificial datasets demonstrated that \(m\) and \(\lambda\) exhibit distinct and complementary effects on fuzzification behavior. Consequently, TFCV, which simultaneously utilizes both parameters, is capable of providing greater flexibility in controlling the degree of fuzziness and therefore has the potential to achieve higher clustering accuracy than methods that rely on only one of them. For the Yeast dataset, SFCV achieves an ARI of \(0.0849\) at \(m = 1.01\), and EFCV achieves the ARI of \(0.0779\) at \(\lambda = 1000.0\). In contrast, TFCV attains a higher ARI of \(0.0952\) when evaluated with the identical parameter values \(m = 1.01\) and \(\lambda = 1000.0\) that yielded the best results for SFCV and EFCV, respectively. For Optdigits dataset, SFCV achieves an ARI of \(0.6007\) at \(m = 1.1\). However, at the same parameter value \(m = 1.1\), TFCV achieves a higher ARI of \(0.6425\) under \(\lambda = 2.0\). These values illustrate the advantage of combining the two fuzzification parameters within a unified framework. Fig. 8 provides a boxplot visualization of the ARI values shown in Table 2. This figure is included to present the relative performance of the algorithms in an intuitive manner and to illustrate the overall tendency of their clustering accuracy across the 14 datasets. The median ARI achieved by TFCV was \(0.2432\), outperforming those of both SFCV (\(0.2217\)) and EFCV (\(0.2139\)). The Wilcoxon signed-rank test with Holm correction confirmed the statistical significance of these improvements (\(p=0.0359\) and \(p=0.0152\), respectively). These results demonstrate that TFCV provides a better performance advantage than SFCV and EFCV, because TFCV serves as a generalization for both algorithms.

Unfortunately, for datasets such as Seeds, Crabs, AIS, and Heart, performance improvement was not observed for any of the methods. However, the underlying mechanism remains unclear. In short, the power-based fuzzification controlled by \(m\) in SFCV and TFCV and the entropy-based fuzzification controlled by \(\lambda\) in EFCV and TFCV do not appear to provide differential advantages for enhancing clustering accuracy on these datasets. This suggests that achieving higher accuracy for such datasets may require a third type of fuzzification mechanism that is distinct from both the power-based and entropy-based approaches. Further, in the context of combining dimensionality reduction with fuzzy clustering, which is the central topic of this study, improving the accuracy is not limited to modifications of the fuzzification scheme. In this study, we restricted the dimensionality-reduction step to PCA and explored the effects of alternative fuzzification strategies under this framework. For these datasets, improved clustering performance may be obtained by adopting different dimensionality reduction techniques, such as probabilistic PCA. Investigating these approaches is an important direction for future research.