2.1. Object Detection

Object detection, a cornerstone task in computer vision, has evolved significantly by introducing various innovative models and techniques. These methods are broadly categorized into one-stage and two-stage object detection methods.

Two-stage object detection: Two-stage methods focus primarily on accuracy and involve a preliminary step of generating candidate regions followed by classification and localization. The region-based convolutional neural networks (R-CNN) family is pivotal in this category. The original R-CNN model ¹⁷ uses a selective search to extract candidate regions and then employs CNNs for feature extraction, along with support vector machines for classification and bounding box regression. Advancing this approach, Faster R-CNN ¹⁸ introduces the region proposal network, which directly generates candidate boxes and employs ROI pooling to enhance feature extraction, significantly speeding up the detection process while maintaining high accuracy.

One-stage object detection: In contrast, one-stage methods simplify the object detection pipeline by eliminating the region proposal stage, aiming to perform object localization and classification in a single step. This approach is renowned for simplicity and efficiency. RetinaNet ¹⁹, a notable model in this category, addresses the challenge of class imbalance in object detection by introducing the focal loss function, which enhances both the accuracy and robustness of the detection. The YOLO series, including YOLOv3 and YOLOv4 ^20,21, epitomizes this category by converting the detection task into a regression problem, achieving real-time performance. The development within the YOLO series continues robustly, with newer models like YOLOv5, YOLOX ²², and YOLOv7 ¹ emerging, each introducing improvements in speed, accuracy, and adaptability to various detection scenarios.

2.2. Pooling Operation

Pooling operations play a crucial role in object detection. They help reduce the dimensionality of feature maps, thereby improving computational efficiency while preserving essential spatial hierarchies of features. Additionally, pooling aids in the optimization process by mitigating overfitting and enhancing the model’s translation invariance ²³.

In applying pooling operation, Hou et al. ²⁴ introduce a new strip pooling strategy that redefines spatial pooling using elongated kernels (such as \(1 \times N\) or \(N \times 1\)) and designs corresponding architectures to model long-range dependencies efficiently. Wu et al. ²⁵ introduce the pyramid pooling Transformer, an enhanced vision transformer that incorporates pyramid pooling into multi-head self-attention to reduce sequence length and effectively capture contextual features, demonstrating significant performance improvements across various vision tasks. Gao et al. ²⁶ propose a novel local importance-based pooling layer to enhance discriminative features during downsampling by learning adaptive importance weights from inputs, leading to consistent improvements in model performance on ImageNet and MS COCO datasets. Xu et al. ²⁷ propose a novel graph pooling technology, geometric pooling, which maintains negative-value nodes by measuring feature similarity, ensuring more representative information is retained during pooling.

2.3. Feature Fusion

Different levels of features contain unique information distributions. Shallow features provide high-resolution spatial structure information but are relatively weak in capturing global context. In contrast, deep features are rich in advanced semantic information that aids in accurate object localization, but due to the lower resolution, these features lack detailed spatial details of the object. Therefore, to comprehensively capture an object’s characteristics, an effective integration of shallow and deep features is necessary.

Chen et al. ²⁸ propose the high-resolution feature pyramid network, which improves the detection of small-scale objects and minimizes feature redundancy through advanced modules for feature alignment, fusion, and multi-scale prediction. Li et al. ²⁹ present a method called the two-way feature pyramid network, which enhances feature fusion by combining maps generated from different directions and resolutions. This design improves both low-level feature utilization and the reuse of backbone features, leading to better feature extraction. Hu et al. ³⁰ introduce the YOLO3-SM method, which incorporates a saliency map into YOLO to enhance feature fusion. The saliency map selects the grid cell containing the most salient part of the object rather than just the grid cell with the object’s center, allowing for more effective feature fusion and improved detection performance. Xu et al. ³¹ propose a method to locate multiple components in catenary systems by enhancing the YOLO-based framework, improving feature fusion through a novel “gather-and-distribute” mechanism, and incorporating Wasserstein distance loss to boost small object detection capabilities.

3.1. Overall Framework of YOLO-CF

The structure of YOLO-CF, as shown in Fig. 1, comprises three parts: the backbone, neck, and YOLO head. The backbone is the foundation of the entire network and is responsible for extracting high-level features from the input image, which are subsequently used for object detection. The neck primarily processes and optimizes the features extracted by the backbone. The YOLO head uses the feature maps processed by the neck to predict classes and bounding boxes ³². The following provides a detailed introduction to these three components of YOLO-CF.

Backbone: The left side of Fig. 1 shows the backbone structure. First, the input image is sliced and reorganized by focus operation to reduce the amount of calculation by reducing the image size. Then, the Res2Net operation extracts features at a fine-grained level to capture more detailed image information. Subsequently, the MFF module and HD module extract features at both coarse-grained and fine-grained levels and perform dimensionality reduction operations to extract more useful feature information. The final output of backbone is three feature maps of different scales, which are passed to the neck network for further processing and optimization. Through this structured approach, backbone efficiently captures and processes various feature details, thereby improving the overall performance of the YOLO-CF model.

In this work, Res2Net is adopted as the backbone because its hierarchical channel-splitting and feature reorganization mechanism enables fine-grained, multi-scale feature representation while maintaining low computational overhead. This makes it particularly suitable for real-time industrial detection scenarios that require both efficiency and accuracy. By contrast, Swin Transformer offers strong global modelling capabilities, but its self-attention structure introduces considerably higher computational and memory costs, rendering it impractical for deployment on resource-constrained industrial devices or edge platforms. The bidirectional feature pyramid network primarily enhances feature fusion at the neck stage, but does not directly improve fine-grained feature extraction within the backbone. This limits its ability to meet the refined representation requirements of this study. Taking these factors into account, Res2Net strikes a better balance between feature expressiveness, model complexity, and inference speed, aligning more closely with the overall design objectives of YOLO-CF.

Neck: The neck structure is shown in the middle of Fig. 1 and plays a vital role in the object detection network. It is mainly responsible for promoting feature fusion and information transmission. Based on the design principle of PANet ³³, this structure seamlessly integrates the bottom-up and top-down paths, effectively fusing feature information at different levels to enhance detection performance. In the neck structure, we use Res2Net and Res2Net block modules to enhance the representation of fine-grained features during the feature fusion process. In the bottom-level output part, the SPP ³⁴ module is added to handle the input image’s different scales and shapes effectively, further enhancing the network’s feature extraction capabilities.

Head: YOLO-CF uses the anchor-free YOLO head method, as shown on the right side of Fig. 1. This method achieves object detection by predicting the target’s center point and the offset of the bounding box, making the model more flexible and better able to adapt to targets of different sizes and scales. In addition, a dynamic head mechanism is introduced, which dynamically adjusts the structure of the detection head according to the size of the input image and the target, thereby improving the detection performance for targets across different scales.

In YOLO head, we add Res2Net and Res2Net block modules to provide a more prosperous and diverse feature representation for the final prediction, enhancing the model’s adaptability to object detection tasks. In the input part of YOLO head, after the classification network extracts features, the features are sent to the final classification and regression networks, respectively, which enhances the feature reuse capability and reduces the number of parameters.

3.2. Overall Framework of Res2Net and Res2Net Block Modules

To effectively extract features at a fine-grained level, YOLO-CF uses the Res2Net module. This module represents multi-scale features at a fine-grained level by constructing hierarchical residual-like connections within a single residual block. It increases the receptive field range of each network layer, as shown in Fig. 2(a). First, a \(1 \times 1\) convolution operation divides the features into \(s\) subsets, denoted as \(S_{i}\), where \(i \in \left\{1,2,\ldots,s\right\}\). In each subset, the number of channels is \(1/s\) of the original, and the feature map size is consistent with the original input. The feature subset \(S_{1}\) is directly output without any operation, while the remaining feature subsets \(S_{i}\) undergo a convolution operation with a 3 \(\times\) 3 kernel size and a dilation rate of \(d\), denoted as \({F}_{i}^{d}\). The output of \({F}_{i}^{d}\) is defined as \(O_{i}\). In this process, the feature subset \(S_{i}\) is added to \({F}_{i-1}^{d}\), and the result is sent to \({F}_{i}^{d}\) for further processing ³⁵. The output \(O_{i}\) can be expressed as:

Each 3 \(\times\) 3 convolution operation \({F}_{i}^{d}\) takes into account all preceding feature map subsets. Additionally, by applying a 3 \(\times\) 3 convolution with a dilation rate of \(d\), the output achieves a larger receptive field than \(S_{i}\). The output of the Res2Net module thus encompasses feature maps with varying numbers, sizes, and receptive fields.

A Res2Net block module is proposed, as illustrated in Fig. 2(b). Initially, a 1 \(\times\) 1 convolution operation is used to alter the dimension of the input feature map. Subsequently, the Res2Net module performs multi-scale feature extraction. Finally, another 1 \(\times\) 1 convolution operation is applied to expand the dimension. Additionally, a shortcut connection using a 1 \(\times\) 1 convolution operation links the input and output, dynamically adjusting the model’s complexity to ensure normal back-propagation of gradients.

3.3. MFF Module

This paper introduces a novel MFF module that revolutionizes feature representation at coarse and fine-grained levels, as depicted in Fig. 3. The initial step involves utilizing the structure of CSPNet to partition the input features into two segments. One segment directly employs Res2Net for finer-grained multi-scale feature extraction. After extracting multi-scale features through Res2Net, the other segment undergoes average pooling operations with different strides to provide multi-level feature representation.

In the average pooling branches with different strides, the Res2Net operations mitigate the aliasing effects caused by downsampling and upsampling. This allows for the interaction and integration of information across different scales and levels, similar to PANet’s top-down and bottom-up structures. The bottom-up pathway captures lower-level features and passes them to higher levels. In contrast, the top-down pathway leverages upper-level semantic information to guide and adjust the processing of lower-level features through feedback pathways, ensuring a comprehensive and effective feature representation.

Ultimately, the two segments of the CSPNet structure are merged to ensure the integration of information from different scales, thereby augmenting feature dimensions and significantly enhancing the model’s representation ability and accuracy. This innovative approach promises substantial benefits in computer vision and deep learning.

3.4. Hybrid Downsampling

In object detection tasks, downsampling is frequently used in feature map processing to reduce the number of parameters and computational costs. Standard methods include average pooling, max pooling, and convolution with a stride of 2. Each method has advantages and captures different types of feature information. To fully leverage the strengths of these three methods, this paper proposes a new downsampling module called “hybrid downsampling.” This module applies average pooling, max pooling, and convolution with a stride of 2 and then concatenates their outputs to form a comprehensive feature representation, as shown in Fig. 3.

Specifically, we consider \(X \in \mathbb{R}^{H \times W \times C}\) as the input feature, where \(H\), \(W\), and \(C\) represent the height, width, and channels, respectively. Max pooling is described by Eq. \(\eqref{eq:MP}\), average pooling by Eq. \(\eqref{eq:AvP}\), and HD by Eq. \(\eqref{eq:HD}\). This design offers several advantages. First, it enriches feature representation: average pooling smooths the feature map and suppresses noise, max pooling retains important edges and details, and convolution with a stride of 2 captures local spatial features. By combining these methods, the module generates a more diverse and comprehensive set of features. Second, it enhances information integration: concatenating the outputs of these different downsampling methods integrates the features captured by each method, enhancing the diversity and richness of the feature set. Finally, it improves robustness: since these downsampling methods have different sensitivities to image transformations, such as rotation and scaling, their combination improves the model’s robustness to such transformations, resulting in more stable performance across different scenarios. This combined downsampling approach significantly enhances the model’s feature extraction capabilities and accuracy.

3.5. Loss Function

The loss function of YOLO-CF is divided into three parts: classification loss, regression loss, and object loss, which can be expressed as:

The classification and object loss is calculated using binary cross entropy (BCE) loss. BCE is a widely used loss function for binary classification problems. The calculation equation is as follows:

YOLO-CF employs structured intersection over union (SIoU) as the loss function for the regression loss. SIoU is an improved version of the IoU loss function, designed to more accurately measure the overlap between the predicted and ground truth bounding boxes, particularly when targets have varying scales and shapes ³⁵, and can be expressed as:

The SIoU includes four parts, angle cost, distance cost, shape cost, and IoU cost, which can be formulated as:

The angular cost is calculated as follows:

The term \(C_{h}\) represents the height difference between the ground truth bounding box and the predicted bounding box’s center point. \(\sigma\) denotes the distance between the center points of the ground truth bounding box and the predicted bounding box. The coordinates \(b_{c_{x}}^{gt}\) and \(b_{c_{y}}^{gt}\) refer to the center point of the ground truth bounding box, while \(b_{c_{x}}\) and \(b_{c_{y}}\) are the center point coordinates of the predicted bounding box. It should be noted that the angle cost is zero when \(\alpha\) equals \({1/2}\) or 0. During training, if \(\alpha\) is less than \({1/4}\), the objective is to minimize \(\alpha\); otherwise, the goal is to minimize \(\beta\).

The distance cost is as follows:

\(C_{W}\) and \(C_{H}\) represent the width and height of the minimum bounding rectangle that encompasses both the ground truth and the predicted bounding boxes. The parameter \(\gamma\) prioritizes the distance value as the angle increases.

The shape cost is as follows:

The \((w,h)\) and \((w^{gt},h^{gt})\) correspond to the width and height of the predicted bounding box and the ground truth bounding box, respectively. The parameter \(\theta\) regulates the emphasis placed on the shape cost.

4.1. Implementation Details

All experiments use an i9-10900 CPU with a single NVIDIA GeForce RTX 3090 GPU. The operating system is Ubuntu 21.04, with CUDA version 11.4 and the GPU acceleration library cuDNN 8.2.4. To ensure the reproducibility and comparability of the results, all models discussed in this paper are reproduced using TensorFlow to replicate the official code.

The epoch is set to 300 in all experiments, and the batch size is set to 8. We employ the Mosaic and Mixup data augmentation techniques, a strategic choice that combines multiple images and mixed pixel values to generate new training samples. This approach enhances the robustness and generalization ability of the model, a key consideration in our experimental setup. Data augmentation is used for the first 210 epochs but is stopped for the last 90 epochs to prevent over-augmentation. The confidence threshold is set to 0.5 during evaluation, and the IoU threshold for non-maximum suppression is set to 0.3. All models are trained using the Adam optimizer with parameters \(\beta_{1}\) = 0.937 and \(\beta_{2}\) = 0.999. The learning rate is adjusted using a warm-up cosine annealing schedule, gradually increasing from 4e-6 to the initial setting of 1e-3 in the first five epochs and then decreasing according to a cosine curve, with a minimum learning rate of 1e-5 ³⁶.

4.2. Datasets

We have applied four datasets to measure the proposal’s performance.

GC10-DET: The GC10-DET public dataset comprises 2,294 images derived from actual industrial environments specifically tailored for detecting defects on metallic surfaces ³⁷. This dataset is categorized into ten different types of defects. It is divided as follows: 1,857 images are designated for training purposes, 207 are meant for validation, and 230 are set aside for testing.

PCB: The PCB public dataset includes 693 synthetic images designed explicitly for defect detection tasks on printed circuit boards (PCBs) ³⁸. These images are organized into six different defect categories. The distribution of images includes 560 for training, 63 for validation, and 70 for testing. Notably, the image defects are digitally created using Photoshop, a graphic editing tool developed by Adobe Systems.

Crack: The crack public dataset is an object detection dataset specifically annotated for detecting cracks in concrete ³⁹. It includes 996 images, all categorized under a single class. The dataset is divided into 879 images designated for training, 80 for validation, and 37 for testing.

NEU-DET: The NEU-DET public dataset includes 1,800 images that depict six different types of surface defects in hot rolled strip images ⁴⁰. The dataset is segmented into 1,458 images for training, 162 for validation, and 180 for testing.

Dish-20: The Dish-20 dataset comprises 506 images with 20 classes of dishes ³⁶. The dataset is divided into 409 images designated for training, 46 for validation, and 51 for testing.

4.3. Evaluation Metrics

This study uses precision, recall, average precision (AP), and mean average precision (mAP) as the main indicators to evaluate object detection performance.

Precision is defined as the proportion of positive samples among those predicted by the model. The calculation equation is as follows:

Recall is defined as the proportion of positive samples that are correctly predicted by the model as positive. The calculation equation is as follows:

AP measures the detection performance of a single category by calculating the area under the precision–recall curve. The calculation equation is:

The mAP is the average value of AP of all categories, which is used to measure the overall detection performance of the model on multiple categories, which is as follows:

In this study, the evaluation indicators in the COCO API are used to evaluate the performance of the object detection model ³⁵. These indicators include:

• \(\textit{AP}_{11}\) (AP at IoU from 0.5 to 0.95 in stride of 0.05).

• \(\textit{AP}_{50}\) (AP at IoU\(=0.5\)).

• \(\textit{AP}_{75}\) (AP at IoU\(=0.75\)).

• \(\textit{AP}_{s}\) (AP for small objects: area \(<\) 1024).

• \(\textit{AP}_{m}\) (AP for medium objects: 1024 \(<\) area \(<\) 9216).

• \(\textit{AP}_{l}\) (AP for large objects: area \(>\) 9216).

• \(AR_{s}\) (\(AR\) for small objects: area \(<\) 1024).

• \(AR_{m}\) (\(AR\) for medium objects: 1024 \(<\) area \(<\) 9216).

• \(AR_{l}\) (\(AR\) for large objects: area \(>\) 9216).

4.4. Performance Comparison

4.4.3. Comparison with SOTA Models on Other Public Datasets

To validate the generalization performance of YOLO-CF, we conduct extensive experiments on three public datasets: the PCB, the crack, the NEU-DET, and the Dish-20 public datasets, which are all introduced in Subsection 4.2, and the results are summarized in Table 2. Specifically, the PCB dataset uses inputs of 640 \(\times\) 640 \(\times\) 3, while both the crack and NEU-DET datasets use inputs of 416 \(\times\) 416 \(\times\) 3. Notably, Faster-RCNN uses inputs of 600 \(\times\) 600 \(\times\) 3 across all three datasets, EfficientDet uses inputs of \(512\times 512 \times3\) for the crack and NEU-DET datasets, RetinaNet uses inputs of 600 \(\times\) 600 \(\times\) 3 across all three datasets, and SSD uses inputs of 500 \(\times\) 500 \(\times\) 3 for the PCB dataset and 300 \(\times\) 300 \(\times\) 3 for the crack and NEU-DET datasets.

In the PCB public dataset, YOLO-CF achieves the highest mAP of 97.38%, surpassing the second-ranked YOLOX-X by 0.9 percentage points. However, in \(\textit{AP}_{11}\), YOLO-CF scored 0.485, 0.3 percentage points lower than YOLOX-X. In \(\textit{AP}_{50}\), YOLO-CF achieves an accuracy of 0.962, higher than YOLOX-X by 0.4 percentage points. In \(\textit{AP}_{75}\), YOLO-CF scored 0.379, 1.8 percentage points lower than YOLOX-X. Despite some fluctuations in accuracy compared to YOLOX-X, YOLO-CF’s overall accuracy exceeds other SOTA models. The YOLO-CF performs well on the PCB public dataset, showcasing the strong competitiveness and application potential in object detection tasks.

In the crack public dataset, YOLO-CF achieves the highest mAP at 84.72%, surpassing the second-place YOLOX-S by 0.61 percentage points. However, in \(\textit{AP}_{11}\), YOLO-CF achieves 0.514, 2 percentage points lower than YOLOX-X. In \(\textit{AP}_{50}\), YOLO-CF attained a precision of 0.782, 1.4 percentage points lower than YOLOX-X. In \(\textit{AP}_{75}\), YOLO-CF achieves a precision of 0.593, 4 percentage points lower than YOLOX-X. YOLO-CF is 1.1 percentage points lower than YOLOX-M in \(\textit{AP}_{50}\) compared to other models. The suboptimal precision in \(\textit{AP}_{75}\) indicates that YOLO-CF has shortcomings in accurately locating targets on the crack dataset, especially when higher IoU is required (such as in \(\textit{AP}_{75}\)). This suggests further optimizing the model’s feature extraction and bounding box regression capabilities to improve precision.

In the NEU-DET public dataset, YOLO-CF achieves the second-highest mAP at 73.35%, just 0.86 percentage points lower than the top-ranked Faster-RCNN (ResNet50). However, in \(\textit{AP}_{75}\), YOLO-CF achieved a precision of 0.419, 1.6 percentage points higher than Faster-RCNN (ResNet50). YOLO-CF outperformed all other SOTA models in mAP, \(\textit{AP}_{11}\), \(\textit{AP}_{50}\), and \(\textit{AP}_{75}\) compared to other models. This indicates that YOLO-CF demonstrates outstanding overall performance on the NEU-DET dataset, particularly excelling in high-precision target localization, showcasing significant advantages in feature extraction and bounding box regression capabilities.

In the Dish-20 dataset, YOLO-CF achieves the highest overall mAP, outperforming all compared SOTA models, as shown in Table 3. Specifically, YOLO-CF demonstrates strong detection accuracy across various dish categories characterized by diverse shapes, reflective surfaces, and frequent occlusions. In terms of \(\textit{AP}_{50}\) and \(\textit{AP}_{75}\), YOLO-CF consistently surpasses the baseline YOLO models, indicating its enhanced capability in both coarse-grained classification and fine-grained localization. These results confirm that YOLO-CF maintains excellent generalization performance on complex real-world scenarios and exhibits substantial advantages in robust feature representation and precise bounding box regression.

4.5. Discussion and Future Work

The experimental results obtained from multiple datasets, including GC10-DET, PCB, crack, NEU-DET, and Dish-20, demonstrate that YOLO-CF maintains stable detection performance under diverse conditions of texture, illumination, background complexity, and target appearance. These datasets differ substantially in visual characteristics, noise patterns, and scene variability, effectively simulating dynamic environmental factors such as lighting fluctuations, material variations, structural irregularities, and background interference. Across all datasets, YOLO-CF consistently delivers superior performance, indicating that its feature extraction and multi-scale fusion mechanisms are highly robust and adaptable to dynamic conditions, enabling reliable detection in real-world industrial scenarios.

Table 1 further shows that although some models possess higher parameters and FLOPs than YOLO-CF, their inference speeds are nonetheless faster. This suggests that YOLO-CF involves more data read and write operations, resulting in performance loss due to memory bandwidth constraints during data transfer between memory and computation units. Nevertheless, YOLO-CF achieves an inference speed of 36.68 fps, which is sufficient for most industrial surface defect detection applications. In practical production environments, medium-speed manufacturing lines typically require around 20–30 fps for stable online detection. In contrast, scenarios with higher throughput or stricter latency requirements benefit from speeds above 30 fps. Therefore, YOLO-CF’s current inference performance meets the real-time processing requirements of a wide range of industrial tasks, and it can be deployed on edge or resource-constrained devices.

Moreover, as illustrated in the last row of Fig. 4, the heatmap of the right target is not perfectly centered and exhibits two activation regions, suggesting that the feature extraction and fusion process still has room for refinement. In future work, we plan to optimize the MFF structure, enhance multi-scale feature extraction and fusion, and reduce redundant data read and write operations to further improve inference efficiency. Through these improvements, we aim to enhance both the accuracy and real-time performance of YOLO-CF in industrial applications.