Research Paper:
Exploring Prediction Confidence Capability of Deep Learning Architectures
Hengyi Li*,
, Qibo Xu*
, Aihui Wang*
, and Lin Meng**

*Zhongyuan University of Technology
No.41 Zhongyuan Road, Zhengzhou, Henan Province 450007, China
Corresponding author
**Ritsumeikan University
1-1-1 Nojihigashi, Kusatsu, Shiga 525-8577, Japan
With the ascent of deep learning (DL) across various domains, the evaluation of neural network architectures has been predominantly centered on prediction accuracy, whereas the confidence behavior of these architectures has received comparatively less attention. Prediction confidence provides important insights into the reliability and trustworthiness of DL systems, an essential dimension of model behavior that prediction accuracy alone fails to reflect. This paper presents a comprehensive empirical investigation of the prediction confidence capability of representative DL architectures on large-scale multi-class image classification. The study reveals systematic architecture-dependent confidence patterns: different design paradigms, network families, and scaling strategies, which give rise to distinct confidence profiles in terms of confidence level, dispersion, and their relation to model complexity. These findings establish a confidence-aware perspective for understanding DL architectures beyond conventional accuracy-centered evaluation. In addition, representative confidence calibration techniques are reviewed to situate the empirical analysis within the broader context of reliable DL. Overall, this study offers systematic evidence for analyzing the intrinsic confidence behavior of DL paradigms and provides valuable guidance for architecture design, optimization, and reliable deployment in practical applications.
- [1] L. Alzubaidi, J. Zhang, A. J. Humaidi, A. Al-Dujaili, Y. Duan, O. Al-Shamma, J. Santamaría, M. A. Fadhel, M. Al-Amidie, and L. Farhan, “Review of deep learning: Concepts, CNN architectures, challenges, applications, future directions,” J. of Big Data, Vol.8, Article No.53, 2021. https://doi.org/10.1186/s40537-021-00444-8
- [2] J. Ren, H. Li, A. Wang, K. Saho, and L. Meng, “Radar-based gait analysis by transformer-liked network for dementia diagnosis,” Biomedical Signal Processing and Control, Vol.91, Article No.105986, 2024. https://doi.org/10.1016/j.bspc.2024.105986
- [3] B. Lyu, H. Li, A. Tanaka, and L. Meng, “The early Japanese books reorganization by combining image processing and deep learning,” CAAI Trans. on Intelligence Technology, Vol.7, Issue 4, pp. 627-643, 2022. https://doi.org/10.1049/cit2.12104
- [4] H. Li and L. Meng, “Hardware-aware approach to deep neural network optimization,” Neurocomputing, Vol.559, Article No.126808, 2023. https://doi.org/10.1016/j.neucom.2023.126808
- [5] H. Li, X. Yue, and L. Meng, “Enhanced Mechanisms of Pooling and Channel Attention for Deep Learning Feature Maps,” PeerJ Computer Science, Vol.8, Article No.e1161, 2022. https://doi.org/10.7717/peerj-cs.1161
- [6] H. Li, Z. Wang, X. Yue, W. Wang, H. Tomiyama, and L. Meng, “An architecture-level analysis on deep learning models for low-impact computations,” Artificial Intelligence Review, Vol.56, pp. 1971-2010, 2023. https://doi.org/10.1007/s10462-022-10221-5
- [7] T. Su, Y. Zhang, X. Liang, and N. Ma, “Trajectory Planning Method for Fracture Reduction of Parallel Robots Based on DMP and APF,” IEEE Trans. on Automation Science and Engineering, Vol.23, pp. 4130-4141, 2026. https://doi.org/10.1109/TASE.2026.3653476
- [8] X. Liang, Y. Yan, W. Wang, T. Su, G. He, G. Li, and Z.-G. Hou, “Adaptive Human-Robot Interaction Torque Estimation With High Accuracy and Strong Tracking Ability for a Lower Limb Rehabilitation Robot,” IEEE/ASME Trans. on Mechatronics, Vol.29, Issue 6, pp. 4814-4825, 2024. https://doi.org/10.1109/TMECH.2024.3394491
- [9] S. Liu and W. Deng, “Very deep convolutional neural network based image classification using small training sample size,” 2015 3rd IAPR Asian Conf. on Pattern Recognition (ACPR), pp. 730-734, 2015. https://doi.org/10.1109/ACPR.2015.7486599
- [10] C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich, “Going deeper with convolutions,” 2015 IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2015. https://doi.org/10.1109/CVPR.2015.7298594
- [11] S. Ioffe and C. Szegedy, “Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift,” The 32nd Int. Conf. on Machine Learning (ICML), Vol.37, pp. 448-456, 2015.
- [12] C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, “Rethinking the Inception Architecture for Computer Vision,” 2016 IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 2818-2826, 2016. https://doi.org/10.1109/CVPR.2016.308
- [13] K. He, X. Zhang, S. Ren, and J. Sun, “Deep Residual Learning for Image Recognition,” 2016 IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 770-778, 2016. https://doi.org/10.1109/CVPR.2016.90
- [14] G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, “Densely Connected Convolutional Networks,” 2017 IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 2261-2269, 2017. https://doi.org/10.1109/CVPR.2017.243
- [15] M. Tan and Q. V. Le, “EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks,” Proc. of the 36th Int. Conf. on Machine Learning, Vol.97, pp. 6105-6114, 2019.
- [16] M. Tan and Q. V. Le, “EfficientNetV2: Smaller Models and Faster Training,” Proc. of the 38th Int. Conf. on Machine Learning, Vol.139, pp. 10096-10106, 2021.
- [17] J. Xu, Y. Pan, X. Pan, S. Hoi, Z. Yi, and Z. Xu, “RegNet: Self-Regulated Network for Image Classification,” IEEE Trans. on Neural Networks and Learning Systems, Vol.34, Issue 11, pp. 9562-9567, 2023. https://doi.org/10.1109/TNNLS.2022.3158966
- [18] Z. Liu, H. Mao, C.-Y. Wu, C. Feichtenhofer, T. Darrell, and S. Xie, “A ConvNet for the 2020s,” 2022 IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 11966-11976, 2022. https://doi.org/10.1109/CVPR52688.2022.01167
- [19] F. N. Iandola, S. Han, M. W. Moskewicz, K. Ashraf, W. J. Dally, and K. Keutzer, “Squeezenet: Alexnet-level accuracy with 50× fewer parameters and <0.5MB model size,” arXiv:1602.07360, 2016. https://doi.org/10.48550/arXiv.1602.07360
- [20] A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and H. Adam, “MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications,” arXiv:1704.04861, 2017. https://doi.org/10.48550/arXiv.1704.04861
- [21] M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen, “MobileNetV2: Inverted Residuals and Linear Bottlenecks,” 2018 IEEE/CVF Conf. on Computer Vision and Pattern Recognition, pp. 4510-4520, 2018. https://doi.org/10.1109/CVPR.2018.00474
- [22] X. Zhang, X. Zhou, M. Lin, and J. Sun, “ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices,” 2018 IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 6848-6856, 2018. https://doi.org/10.1109/CVPR.2018.00716
- [23] N. Ma et al., “ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design,” The 15th European Conf. on Computer Vision (ECCV), pp. 122-138, 2018. https://doi.org/10.1007/978-3-030-01264-9_8
- [24] M. Tan, B. Chen, R. Pang, V. Vasudevan, M. Sandler, A. Howard, and Q. V. Le, “MnasNet: Platform-Aware Neural Architecture Search for Mobile,” 2019 IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 2815-2823, 2019. https://doi.org/10.1109/CVPR.2019.00293
- [25] Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin, and B. Guo, “Swin Transformer: Hierarchical Vision Transformer using Shifted Windows,” 2021 IEEE/CVF Int. Conf. on Computer Vision (ICCV), pp. 9992-10002, 2021. https://doi.org/10.1109/ICCV48922.2021.00986
- [26] A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby, “An Image is Worth 16×16 Words: Transformers for Image Recognition at Scale,” Int. Conf. on Learning Representations, 2021.
- [27] Z. Tu, H. Talebi, H. Zhang, F. Yang, P. Milanfar, A. Bovik, and Y. Li, “MaxViT: Multi-axis Vision Transformer,” The 17th European Conf. on Computer Vision (ECCV 2022), pp. 459-479, 2022. https://doi.org/10.1007/978-3-031-20053-3_27
- [28] X. Xu, M. Xie, P. Miao, W. Qu, W. Xiao, H. Zhang, X. Liu, and T.-T. Wong, “Perceptual-Aware Sketch Simplification Based on Integrated VGG Layers,” IEEE Trans. on Visualization and Computer Graphics, Vol.27, Issue 1, pp. 178-189, 2021. https://doi.org/10.1109/TVCG.2019.2930512
- [29] H. Zhou, W. Chen, L. Cheng, D. Williams, C. W. De Silva, and M. Xia, “Reliable and Intelligent Fault Diagnosis With Evidential VGG Neural Networks,” IEEE Trans. on Instrumentation and Measurement, Vol.72, 2023. https://doi.org/10.1109/TIM.2023.3250308
- [30] S.-H. Wang, S. L. Fernandes, Z. Zhu, and Y.-D. Zhang, “AVNC: Attention-Based VGG-Style Network for COVID-19 Diagnosis by CBAM,” IEEE Sensors J., Vol.22, Issue 18, pp. 17431-17438, 2022. https://doi.org/10.1109/JSEN.2021.3062442
- [31] A. Sepas-Moghaddam, M. A. Haque, P. L. Correia, K. Nasrollahi, T. B. Moeslund, and F. Pereira, “A Double-Deep Spatio-Angular Learning Framework for Light Field-Based Face Recognition,” IEEE Trans. on Circuits and Systems for Video Technology, Vol.30, Issue 12, pp. 4496-4512, 2020. https://doi.org/10.1109/TCSVT.2019.2916669
- [32] X. Ren, H. Mosavat-Jahromi, L. Cai, and D. Kidston, “Spatio-Temporal Spectrum Load Prediction Using Convolutional Neural Network and ResNet,” IEEE Trans. on Cognitive Communications and Networking, Vol.8, Issue 2, pp. 502-513, 2022. https://doi.org/10.1109/TCCN.2021.3139030
- [33] P. Guo, H. Dai, Q. Yang, Q. Huang, and H. Yao, “An Improved Magnetic Tracking Approach Based on ResNet-LM Fusion Algorithm,” IEEE Trans. on Instrumentation and Measurement, Vol.71, 2022. https://doi.org/10.1109/TIM.2022.3216597
- [34] Z. Zhu, W. Zhai, H. Liu, J. Geng, M. Zhou, C. Ji, and G. Jia, “Juggler-ResNet: A Flexible and High-Speed ResNet Optimization Method for Intrusion Detection System in Software-Defined Industrial Networks,” IEEE Trans. on Industrial Informatics, Vol.18, Issue 6, pp. 4224-4233, 2022. https://doi.org/10.1109/TII.2021.3121783
- [35] D. O’Neill, B. Xue, and M. Zhang, “Evolutionary Neural Architecture Search for High-Dimensional Skip-Connection Structures on DenseNet Style Networks,” IEEE Trans. on Evolutionary Computation, Vol.25, Issue 6, pp. 1118-1132, 2021. https://doi.org/10.1109/TEVC.2021.3083315
- [36] F. Yuwei, L. Liejuan, H. Weihua, H. Guobin, H. Peijun, Z. Zhiyu, C. Chi, and W. Chuang, “Partial Discharge Pattern Recognition Method Based on Transfer Learning and DenseNet Model,” IEEE Trans. on Dielectrics and Electrical Insulation, Vol.30, Issue 3, pp. 1240-1246, 2023. https://doi.org/10.1109/TDEI.2023.3239032
- [37] L. Huang and C.-M. Pun, “Audio Replay Spoof Attack Detection by Joint Segment-Based Linear Filter Bank Feature Extraction and Attention-Enhanced DenseNet-BiLSTM Network,” IEEE/ACM Trans. on Audio, Speech, and Language Processing, Vol.28, pp. 1813-1825, 2020. https://doi.org/10.1109/TASLP.2020.2998870
- [38] N. K. Yadav, S. K. Singh, and S. R. Dubey, “MobileAR-GAN: MobileNet-Based Efficient Attentive Recurrent Generative Adversarial Network for Infrared-to-Visual Transformations,” IEEE Trans. on Instrumentation and Measurement, Vol.71, 2022. https://doi.org/10.1109/TIM.2022.3166202
- [39] J. Yang, Y. Wang, H. Zhao, and G. Gui, “MobileNet and Knowledge Distillation-Based Automatic Scenario Recognition Method in Vehicle-to-Vehicle Systems,” IEEE Trans. on Vehicular Technology, Vol.71, Issue 10, pp. 11006-11016, 2022. https://doi.org/10.1109/TVT.2022.3184994
- [40] T. Addabbo, A. Fort, M. Mugnaini, V. Vignoli, M. Intravaia, M. Tani, M. Bianchini, F. Scarselli, and B. T. Corradini, “Smart Gravimetric System for Enhanced Security of Accesses to Public Places Embedding a MobileNet Neural Network Classifier,” IEEE Trans. on Instrumentation and Measurement, Vol.71, 2022. https://doi.org/10.1109/TIM.2022.3162270
- [41] A. Lin, B. Chen, J. Xu, Z. Zhang, G. Lu, and D. Zhang, “DS-TransUNet: Dual Swin Transformer U-Net for Medical Image Segmentation,” IEEE Trans. on Instrumentation and Measurement, Vol.71, 2022. https://doi.org/10.1109/TIM.2022.3178991
- [42] X. Meng, Y. Yang, L. Wang, T. Wang, R. Li, and C. Zhang, “Class-Guided Swin Transformer for Semantic Segmentation of Remote Sensing Imagery,” IEEE Geoscience and Remote Sensing Letters, Vol.19, 2022. https://doi.org/10.1109/LGRS.2022.3215200
- [43] F. Sun, W. Zhou, L. Ye, and L. Yu, “Hierarchical Decoding Network Based on Swin Transformer for Detecting Salient Objects in RGB-T Images,” IEEE Signal Processing Letters, Vol.29, pp. 1714-1718, 2022. https://doi.org/10.1109/LSP.2022.3194843
- [44] C. Zhang, S. Bengio, M. Hardt, B. Recht, and O. Vinyals, “Understanding deep learning (still) requires rethinking generalization,” Commun. ACM, Vol.64, Issue 3, pp. 107-115, 2021. https://doi.org/10.1145/3446776
- [45] A. Kumar, S. Sarawagi, and U. Jain, “Trainable Calibration Measures for Neural Networks from Kernel Mean Embeddings,” Proc. of the 35th Int. Conf. on Machine Learning, Vol.80, pp. 2805-2814, 2018.
- [46] T. G. Dietterich, “Ensemble Methods in Machine Learning,” Int. Workshop on Multiple Classifier Systems (MCS 2000), 2000. https://doi.org/10.1007/3-540-45014-9_1
- [47] B. Lakshminarayanan, A. Pritzel, and C. Blundell, “Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles,” Proc. of the 31st Int. Conf. on Neural Information Processing Systems, pp. 6405-6416, 2017.
- [48] A. Mehrtash, W. M. Wells, C. M. Tempany, P. Abolmaesumi, and T. Kapur, “Confidence Calibration and Predictive Uncertainty Estimation for Deep Medical Image Segmentation,” IEEE Trans. on Medical Imaging, Vol.39, Issue 12, pp. 3868-3878, 2020. https://doi.org/10.1109/TMI.2020.3006437
- [49] J. C. Platt, “Probabilistic Outputs for Support Vector Machines and Comparisons to Regularized Likelihood Methods,” Advances in Large Margin Classifiers, Vol.10, No.3, pp. 61-74, 1999.
- [50] C. Guo, G. Pleiss, Y. Sun, and K. Q. Weinberger, “On Calibration of Modern Neural Networks,” Proc. of the 34th Int. Conf. on Machine Learning, Vol.70, pp. 1321-1330, 2017.
- [51] B. Zadrozny and C. Elkan, “Obtaining calibrated probability estimates from decision trees and naive Bayesian classifiers,” Proc. of the 18th Int. Conf. on Machine Learning, pp. 609-616, 2001.
- [52] B. Zadrozny and C. Elkan, “Transforming classifier scores into accurate multiclass probability estimates,” Proc. of the 8th ACM SIGKDD Int. Conf. on Knowledge Discovery and Data Mining, pp. 694-699, 2002. https://doi.org/10.1145/775047.775151
- [53] M. P. Naeini, G. Cooper, and M. Hauskrecht, “Obtaining Well Calibrated Probabilities Using Bayesian Binning,” Proc. of the 29th AAAI Conf. on Artificial Intelligence, pp. 2901-2907, 2015. https://doi.org/10.1609/aaai.v29i1.9602
- [54] G. Carneiro, L. Z. C. T. Pu, R. Singh, and A. Burt, “Deep learning uncertainty and confidence calibration for the five-class polyp classification from colonoscopy,” Medical Image Analysis, Vol.62, Article No.101653, 2020. https://doi.org/10.1016/j.media.2020.101653
- [55] D. Karimi and A. Gholipour, “Improving Calibration and Out-of-Distribution Detection in Deep Models for Medical Image Segmentation,” IEEE Trans. on Artificial Intelligence, Vol.4, Issue 2, pp. 383-397, 2023. https://doi.org/10.1109/TAI.2022.3159510
- [56] M. A. Haghpanah, M. T. Masouleh, and A. Kalhor, “Determining the trustworthiness of DNNs in classification tasks using generalized feature-based confidence metric,” Pattern Recognition, Vol.142, Article No.109683, 2023. https://doi.org/10.1016/j.patcog.2023.109683
- [57] G. Rossolini, A. Biondi, and G. Buttazzo, “Increasing the Confidence of Deep Neural Networks by Coverage Analysis,” IEEE Trans. on Software Engineering, Vol.49, Issue 2, pp. 802-815, 2023. https://doi.org/10.1109/TSE.2022.3163682
This article is published under a Creative Commons Attribution-NoDerivatives 4.0 Internationa License.