2.1. Unsupervised Breast Ultrasound Image Classification

BUS imaging plays a critical role in the early detection and classification of benign and malignant breast lesions. Traditionally, BUS image classification relies heavily on supervised learning methods in which CNNs are trained using large amounts of labeled data to identify lesions. However, obtaining labeled ultrasound data is time-consuming and expensive because it requires annotation by experienced radiologists. This challenge has motivated researchers to explore unsupervised learning methods to extract meaningful features from unlabeled data and reduce the dependence on labeled datasets.

Unsupervised learning has demonstrated immense potential across various medical image analysis tasks such as segmentation and anomaly detection. In the context of BUS image classification, unsupervised learning methods can leverage large volumes of unlabeled data to capture key structural and textural features that distinguish between benign and malignant lesions, without explicit supervision. Song and Kim ¹² proposed an unsupervised learning method based on convolutional autoencoders using a triple reconstruction strategy to generate different embedded space representations, thereby enhancing image discriminative ability and addressing the traditional reliance on large amounts of labeled data. Tmenova et al. ¹³ developed a novel unsupervised deep learning strategy for ultrasound that integrated key concepts from unsupervised deep spectral methods by combining spectral graph theory with deep learning approaches. This method employed self-supervised transformer features for spectral clustering and generated meaningful segments based on ultrasound-specific metrics, shape, and position priors to ensure semantic consistency across the dataset. The effectiveness of this approach was validated using three ultrasound datasets. Fan et al. ⁴ introduced a novel unsupervised diversified domain two-dimensional feature selection network that leveraged the similarities between medical images and used a reconstruction network with shared weights to extract features. The model included two adversarial learning modules that penalized domain feature distances, aligned feature spaces, and selected common features. The experimental results demonstrated significant improvements in the classification accuracy of the breast, thyroid, and endoscopic ultrasound images. Ru et al. ⁶ proposed a hybrid learning approach that integrated federated, spatial, and geometric learning by combining CNNs and graph neural networks. This method extracted features from the spatial and geometric structures of images, leveraging federated learning to learn features from multicenter data while preserving privacy. The experimental results demonstrated that this hybrid model performed exceptionally well across various datasets, achieving accuracy scores of 0.911, 0.871, and 0.767 for the BUSI, BUS, and OASBUD datasets, respectively.

2.2. Unsupervised Domain Adaptation

BUS images are not only time-consuming and expensive to acquire but also suffer from uneven data feature distribution owing to differences in the acquisition equipment and operator expertise. Although transfer learning enables models pretrained on natural images or other medical modalities to be fine-tuned on BUS images using unsupervised techniques, addressing the issue of imbalanced data distribution through standard fine-tuning is difficult. This challenge has led to the development of more effective deep transfer learning methods, specifically UDA, to mitigate this issue by aligning feature distributions across different domains, thereby enhancing the generalization ability of the model.

Tzeng et al. ¹⁴ proposed a deep domain confusion (DDC) model that learned domain-invariant features by minimizing the MMD ¹⁵ between two domains. Long et al. ¹⁶ extended this approach by introducing three adaptive layers into the DDC framework and measuring the domain discrepancy using MK-MMD, resulting in deep adaptation networks. With the advent of GANs, Ganin et al. ¹⁷ developed a domain adversarial neural network (DANN) model. The core of a DANN involved adversarial learning between the feature extractor, acting as a generator, and the domain discriminator that confuses the discriminator by extracting features from both the source and target domains. This enabled the model to extract domain-invariant features by balancing the generator and discriminator. Yu et al. ⁹ introduced a dynamic distribution adaptation into adversarial models using DAANs. Their work demonstrated that both the marginal and conditional distributions were mismatched in adversarial networks, and they addressed the issue of balancing the marginal and conditional distributions within the model. In the field of UDA for BUS imaging, Wei et al. ¹⁸ proposed a UDA approach aimed at autonomously generating and iteratively refining image masks to outline the region of interest for benign and malignant breast lesions. This method dynamically adjusted the generated pseudo-masks in response to evolving classification results, thereby improving breast cancer classification outcomes. This offered a practical solution for applications involving sparsely annotated data and enhanced the adaptability and performance of the model in clinical settings. Our team proposed an MK_DAAN ⁷. An MK_DAAN integrates adaptive layers into dynamic adversarial adaptation models and measures the distance between the source and target domain data using MK-MMD. This approach combines domain alignment methods from adaptive layers and adversarial learning, further aligning the feature distributions between the source and target domains to maintain consistency.

Although an MK_DAAN improves the alignment of feature distributions between the source and target domains, it overlooks the enhancements in the ability of the model to discriminate target features, which results in a suboptimal classification performance. To address this problem, we extend MK_DAAN by incorporating ALL and replace the distance measurement algorithm between the source and target domains with the deep CORAL algorithm. These modifications enhance both the ability of the model to discriminate target features and its capacity to align feature distributions between domains, ultimately improving the BUS image classification performance and cross-domain capability.

3.1. Adversarial-Learned Loss

The ALL method for domain adaptation was introduced by Chen et al. ¹⁰ in 2020 to address the lack of discriminability of target domain features in adversarial domain adaptation models. As shown in Fig. 1, adversarial-learned loss for domain adaptive (ALDA), as proposed by Chen et al., consists of a feature extractor acting as generator \(G\), domain discriminator \(D\), and classifier \(C\). The generator extracts features from both the source and target domain data, whereas the classifier predicts pseudo-labels based on the features generated by the generator. The domain discriminator generates a noise confusion matrix to correct the pseudo-labels predicted by the classifier that are subsequently used as training labels for the target domain data.

Through adversarial training between the generator and the domain discriminator, the noise vector output from the last layer of the domain discriminator is learned and used to construct a noise confusion matrix. This matrix is multiplied by the pseudo-labels to obtain the corrected label distributions. A corrected loss function is then constructed that serves as the loss function for the unlabeled target domain data.

The domain discriminator in ALDA differs from that in conventional adversarial domain adaptation methods. Instead of classifying whether the data features originate from the source or target domain, the domain discriminator in ALDA generates the corresponding noise vectors for both the source and target domains. Feature data \(G(x)\) extracted by the generator is fed into the domain discriminator and passed through a sigmoid layer to produce a noise vector \(\xi^{(x)} = \sigma(D(G(x)))\). Each component of noise vector \(\xi^{(x)}\) represents the probability that the pseudo-label matches the true label, \(\xi_{K}^{(x)} = P(y = k\mid \hat{y} = k, x)\), and noise vector \(\xi\) is then used to generate a noise confusion matrix \(\eta\).

For source domain data features \(G(x_{s})\), the domain discriminator minimizes the discrepancy between corrected label vector \(c^{(x_{s})}\) and true labels \(y_{s}\). The ALL function for the source domain is defined as follows:

For target domain data features \(G(x_t)\), the pseudo-labels are negated: \(u^{(\hat{y}_{t})} \in \mathbb{R}\), \(u^{(\hat{y}_{t})} = 0\) when \(k = \hat{y}_{t}\), and \(u^{(\hat{y}_{t})} = 1/(k-1)\) when \(k \ne \hat{y}_{t}\). The ALL function for the target domain is defined as follows:

The total ALL function can be expressed as follows:

The domain discriminator seeks to minimize this loss to differentiate between the source and target domain features, whereas the generator aims to maximize this loss to deceive the domain discriminator.

In addition, Chen et al. introduced a classification loss in the source domain for the domain discriminator, \(L_{\mathit{Reg}}(x_{s}, y_{s}) = L_{\mathit{CE}}(P_{D}^{(x_{s})}, y_{s})\), where \(P_{D}^{(x_{s})} = \mathit{softmax}(D(G(x_{s})))\) and \(L_{\mathit{CE}}\) represents the cross-entropy loss. The loss function of the domain discriminator becomes

After adversarial learning with confusion matrix \(\eta^{(x_{t})}\), the target domain data loss is constructed by assuming uniform noise and using a non-chain loss \(L_{\mathit{unh}}\) as the base loss function.

Thus, the loss functions for the generator and classifier are defined as follows:

3.2. Deep Correlation Alignment

In traditional deep learning and supervised transfer learning, the feature distributions of training and test data are typically assumed to be consistent. To address this issue, Sun et al. ¹⁹ proposed the CORAL algorithm that reduces the discrepancy between the source and target domain feature distributions by aligning their covariance matrices within deep neural networks, thereby improving the performance of the model in the target domain. The framework of the algorithm is illustrated in Fig. 2.

Building on the original CORAL algorithm, Sun et al. introduced nonlinear transformations, resulting in deep CORAL ¹¹. This method is seamlessly integrated into deep neural networks through a differentiable loss function, allowing end-to-end training. Deep CORAL enables the alignment of the second-order statistics of high-dimensional features while integrating with complex deep network architectures, thereby enhancing the flexibility of domain adaptation.

Suppose we have source domain data \(D_{S} = \{x_{i}\}\), \(x \in \mathbb{R}^{d}\), with labels \(L_{S} = \{y_{i}\}\), \(i \in (1, \dots, L)\), and target domain data \(D_{T} = \{u_{i}\}\), \(u \in \mathbb{R}^{d}\). The number of samples in the source and target domains are \(n_S\) and \(n_T\), respectively. Let \(D_{S}^{ij}\) and \(D_{T}^{ij}\) represent the \(j\)-th dimension of the \(i\)-th sample from the source and target domains, respectively, and \(C_{S}\) and \(C_{T}\) denote their respective covariance matrices. The deep CORAL loss is defined as the distance between the second-order statistics (variance) of the source and target domain features.

The covariance matrices for the source and target domains can be calculated as follows:

Sun et al. also considered the need to maintain a sufficient classification ability. Minimizing the classification loss alone can lead to overfitting in the source domain, thereby reducing the performance in the target domain. Conversely, minimizing only the deep CORAL loss may result in degenerate features. Therefore, a weighted combination of the classification and deep CORAL losses is used to balance both, leading to the final loss function.

Compared with MK-MMD that requires mapping features into an RKHS and typically involves kernel functions (for example, Gaussian kernel and multi-kernel) with relatively high computational complexity—particularly when using multiple kernels, deep CORAL aligns features by directly minimizing the Frobenius norm difference between the covariance matrices of the source and target domains. This makes deep CORAL a more lightweight and efficient option that is particularly suitable for resource-constrained environments.

3.3. Domain Adaptation Network Based on Adversarial-Learned Loss and Deep Correlation Alignment

In dynamic adversarial domain adaptation, the feature extractor serves as a generator to extract invariant features from both the source and target domains. These features are used to confuse the discriminator and hinder the identification of the data source, thus facilitating the fusion of interdomain feature distributions. Through adversarial training, a noise confusion matrix is learned that helps compute the loss of the target domain data, enhancing the ability of the model to discriminate the target domain features. The domain adaptation layer employs an adaptive function to map data from both domains into a higher-dimensional space, further aligning the data distributions. By combining the strengths of the ALL method and deep CORAL algorithm, we proposed a domain adaptation network model based on these two techniques. This model is applied to the classification of BUS images and demonstrates superior classification performance and cross-domain generalization under the conditions of limited annotations and data scarcity. The framework of the model is illustrated in Fig. 3.

(1) Feature extractor and classifier: The feature extractor, acting as a generator, extracts domain-invariant features from both the source and target domains to confuse the ability of the discriminator to distinguish the domain to which the features belong. The classifier classifies the source domain data to obtain \(L_{c}\) and generates pseudo-labels for the target domain data.

(2) Local domain discriminator: The local domain discriminator focuses on aligning the conditional distribution differences between the source and target domain data when the equipment, patient, and imaging angles are the same for downstream tasks. It also computes the corresponding loss, \(L_l\).

(3) Global domain discriminator: The global domain discriminator aligns the marginal distribution discrepancies in image data caused by variations in imaging devices, patients, and acquisition angles by distinguishing whether the extracted features originated from the source or target domain, and computes domain classification loss \(L_g\). This loss, combined with local domain discriminator loss \(L_l\), is weighted by a dynamic factor within the DAAN framework, resulting in the training loss, \(L_{\mathit{dn}}\), of the DAAN framework.

Based on the global domain discriminator in the DAAN framework, this study introduces an additional bottleneck layer and a softmax layer, enabling the discriminator to generate corresponding noise vectors for both the source and target domains while preserving its original functionality. Through adversarial training, these noise vectors are leveraged to learn a noise confusion matrix that is subsequently used to calibrate the pseudo-labels generated by the classifier for the target domain data. The framework then outputs both adversarial training loss \(L_{\mathit{adv}}(x_{s}, y_{s}, x_{t})\) and target domain loss \(L_{t}(x_{t}, L_{\mathit{unh}})\) obtained from the corrected pseudo-labels. Together, these components constitute the ALL, \(L_{\mathit{adl}}\).

(4) Adaptive layer: The adaptive layer computes the distance between the source and target domain data features extracted by the feature extractor using a deep CORAL algorithm. By adaptively aligning the feature distributions of the source and target domains, it ensures consistency between the feature distributions of the two domains and computes deep CORAL loss \(L_{\mathit{cor}}\).

Finally, the overall loss function of ALCOR_DA can be formulated as follows:

The model workflow of the domain adaptation network based on ALL and deep CORAL is illustrated in Fig. 4. Initially, the feature extractor is pretrained on the ImageNet dataset. Subsequently, labeled source-domain BUS images and unlabeled target-domain ultrasound images are simultaneously input into the feature extractor to obtain their respective data features. The feature extractor that acts as a generator undergoes adversarial training using a domain discriminator. The domain discriminator determines whether the data features produced by the generator originate from the source or target domain. Simultaneously, an adaptive layer constructed using deep CORAL is employed to align the feature distributions of the two domains. The classifier classifies the source domain data and generates pseudo-labels for the target domain data. In this process, the global domain discriminator aligns the marginal distribution differences between the data features of the domains, generates noise vectors, and converts them into noise confusion matrices to refine the pseudo-labels. The local domain discriminator aligns the conditional distribution differences of the data features. Finally, the model training concludes when equilibrium is reached between the generator and the domain discriminator.

4.1. Abbreviations and Acronyms

The datasets used in this experiment included two different BUS image datasets.

The first dataset was a publicly available BUS dataset from ultrasound cases that could be accessed at ultrasoundcases.info. After data augmentation through multi-angle rotation, random cropping, and horizontal flipping in PyTorch, the dataset contained 2,448 BUS images (1,292 benign and 1,156 malignant tumors). This dataset served as the source domain dataset for the UDA experiment.

The second dataset was provided by the UDIAT Diagnostic Center, authorized for use by Yap et al. ²⁰. After data augmentation through multi-angle rotation, random cropping, and horizontal flipping in PyTorch, the dataset contained 524 BUS images (436 benign and 88 malignant tumors). This dataset was used as the target domain dataset in the UDA experiment and was unannotated.

We evaluated the classification performance of the models using accuracy, precision, recall, F1-score, as well as receiver operating characteristic (ROC) and precision–recall (PR) curves.

4.2. Experimental Results and Analysis

In the experiments, ResNet50, pretrained on ImageNet (a dataset containing 1,000 categories and 1.28 million images), was used as the backbone network for all models. Regarding the parameter settings, the initial learning rate was set to 0.002, and the batch size was set to 4. Each experiment was run for 40 epochs, and each model underwent 20 experimental trials, with the final result being the average of these 20 trials.

Table 1 presents a performance comparison of the proposed ALCOR_DA model with four baseline models (DAAN, MK_DAAN, DAAN + COR, DAAN + ALL, and MK_DAAN + ALL) for BUS image classification. The experimental results indicated that ALCOR_DA achieved the best performance across all evaluation metrics, with an accuracy of 86.26%, a precision of 88.40%, a recall of 92.70%, and an F1-score of 0.9112. These results validated the effectiveness of integrating the ALL and the deep CORAL distance metric into the base model framework, significantly enhancing the classification performance.

The classification performance comparison between DAAN + COR, DAAN, and MK_DAAN demonstrated that incorporating an adaptive layer into the traditional DAAN framework effectively enhanced the alignment of the data feature distributions between the source and target domains. The experimental results robustly validated the effectiveness of the proposed improvements. Moreover, the deep CORAL algorithm exhibited superior sensitivity and robustness than MK-MMD in capturing geometric differences (for example, translation and shape variation) between the source and target domain distributions. This advantage was particularly pronounced when dealing with complex, high-dimensional distributions. Therefore, within the framework of adversarial networks, the deep CORAL algorithm demonstrated significantly stronger feature alignment capabilities.

The classification performance comparison between DAAN + ALL, MK_DAAN + ALL, DAAN, and MK_DAAN revealed that incorporating the ALL method led to performance improvements across all models for BUS image classification. This demonstrated that the ALL method effectively enhanced the discriminative ability of each model for the target domain features, thereby improving the classification accuracy for unlabeled data. Compared with the traditional DAAN model, ALCOR_DA achieved significant improvements in both classification accuracy and robustness. Additionally, it effectively strengthened the target domain feature discriminative capability of the MK_DAAN model that was previously proposed by the team, further enhancing the overall classification performance.

Figures 5 and 6 illustrate the ROC and PR curves, respectively, for the ALCOR_DA model and other baseline models in the BUS image classification task. The ROC curve showed that ALCOR_DA achieved the highest area under the curve (AUC) of 0.7454, indicating superior classification performance over the traditional DAAN and MK_DAAN models. The PR curve showed that ALCOR_DA exhibited the highest PR curve with an AUC value of 0.9229, demonstrating optimal precision and recall performance. These results suggested that the ALCOR_DA model that combined the ALL method and deep CORAL algorithm, effectively enhanced the ability of the model to capture features and handle domain shifts, thereby improving the classification performance.

4.3. Hyperparameter Sensitivity Analysis

We conducted a hyperparameter sensitivity analysis on ALCOR_DA to investigate the influence of the hyperparameter configurations on the model performance and identify the optimal settings. The batch size was varied within \(\{32, 16, 8, 4\}\), and the learning rate was explored within \(\{0.01, 0.001, 0.002, 0.0001\}\). Table 2 presents the experimental results obtained for identical random seeds.

As observed from the results, under a fixed learning rate of 0.001, reducing the batch size improved the overall model performance. This phenomenon could be attributed to the fact that smaller batch sizes introduced greater gradient estimation noise during each parameter update. This noise not only acted as an implicit regularizer that effectively mitigated overfitting but might also help the model escape sharp local minima and converge toward flatter regions with superior generalization capability.

When the batch size was fixed at four, varying the learning rate produced more complex and non-monotonic effects on the performance. With a relatively small learning rate, the model might suffer from slow convergence or become trapped in suboptimal solutions because of insufficient update steps. Conversely, a larger learning rate might bias the model toward predicting more positive samples, occasionally leading to misclassifications. When the batch size is 4 and the learning rate is 0.002, the model achieves the highest accuracy (Accuracy \(=\) 86.26%, Precision \(=\) 88.40%, Recall = 92.70%) while maintaining high precision and recall. In ultrasound image classification tasks, accuracy serves as the most critical metric for evaluating the overall diagnostic performance of a model, as it directly reflects the model’s comprehensive discriminative ability for both positive and negative samples, which is essential for the reliability of clinical decision-making.

4.4. Analysis of Loss Weight Balancing

To determine the optimal balance between the adversarial-learned and deep CORAL losses, weight parameter \(h\) was treated as a trainable parameter. To further assess the effect of \(h\) on the overall performance of ALCOR_DA, we compared the classification results for four different fixed values of \(h\) with those obtained using the adaptively learned trainable parameters. The experimental results are presented in Table 3.

Overall, the results demonstrated that the inclusion of weight parameter \(h\) was effective for balancing the adversarial-learned and deep CORAL losses and reflecting their relative contributions during the domain adaptation process. By comparing the results obtained with the maximum and minimum values of \(h\), we observed that when \(h\) was small, the deep CORAL loss dominated, and the overall performance improvement of the model was relatively limited; however, the model tended to cover a larger proportion of target samples to achieve distributional consistency, leading to higher recall. Conversely, when \(h\) was large, the precision increased but the recall decreased, and the overall accuracy did not reach the optimum value. This suggested that when ALL dominated, the model, despite its stronger discriminative capacity, failed to effectively transfer knowledge learned from the source domain to the target domain, resulting in the omission of certain difficult-to-classify target samples.

Unexpectedly, when we empirically set the weight to a fixed value of 0.6, the results showed that, compared with the adaptive convergence value of 0.64, the model achieved comparable accuracy and precision and obtained a higher recall. This finding validated the empirical insights summarized in earlier experiments, indicating their practical feasibility. A plausible explanation was that a slightly smaller h facilitated the coverage of more boundary or low-confidence samples in the target domain, thereby improving the recall.

In the context of breast tumor diagnosis, recall is typically of greater clinical importance because the cost of missing malignant cases far outweighs that of false positives. Identifying even one additional potentially malignant tumor provides patients with an increased chance of timely treatment and survival. Therefore, when \(h\) is set to 0.6, the performance of the model better aligns with practical diagnostic requirements—favoring the reduction of false negatives even at the expense of a certain number of false positives—that has significant clinical relevance in BUS-aided diagnosis.