single-jc.php

JACIII Vol.30 No.6 pp. 1927-1940
(2026)

Research Paper:

Exploring Prediction Confidence Capability of Deep Learning Architectures

Hengyi Li*,† ORCID Icon, Qibo Xu* ORCID Icon, Aihui Wang* ORCID Icon, and Lin Meng** ORCID Icon

*Zhongyuan University of Technology
No.41 Zhongyuan Road, Zhengzhou, Henan Province 450007, China

†Corresponding author

**Ritsumeikan University
1-1-1 Nojihigashi, Kusatsu, Shiga 525-8577, Japan

Received:
February 3, 2026
Accepted:
July 13, 2026
Published:
November 20, 2026
Keywords:
deep learning, confidence, prediction, reliability, classification
Abstract

With the ascent of deep learning (DL) across various domains, the evaluation of neural network architectures has been predominantly centered on prediction accuracy, whereas the confidence behavior of these architectures has received comparatively less attention. Prediction confidence provides important insights into the reliability and trustworthiness of DL systems, an essential dimension of model behavior that prediction accuracy alone fails to reflect. This paper presents a comprehensive empirical investigation of the prediction confidence capability of representative DL architectures on large-scale multi-class image classification. The study reveals systematic architecture-dependent confidence patterns: different design paradigms, network families, and scaling strategies, which give rise to distinct confidence profiles in terms of confidence level, dispersion, and their relation to model complexity. These findings establish a confidence-aware perspective for understanding DL architectures beyond conventional accuracy-centered evaluation. In addition, representative confidence calibration techniques are reviewed to situate the empirical analysis within the broader context of reliable DL. Overall, this study offers systematic evidence for analyzing the intrinsic confidence behavior of DL paradigms and provides valuable guidance for architecture design, optimization, and reliable deployment in practical applications.

Cite this article as:
H. Li, Q. Xu, A. Wang, and L. Meng, “Exploring Prediction Confidence Capability of Deep Learning Architectures,” J. Adv. Comput. Intell. Intell. Inform., Vol.30 No.6, pp. 1927-1940, 2026.
Data files:
References
  1. [1] L. Alzubaidi, J. Zhang, A. J. Humaidi, A. Al-Dujaili, Y. Duan, O. Al-Shamma, J. Santamaría, M. A. Fadhel, M. Al-Amidie, and L. Farhan, “Review of deep learning: Concepts, CNN architectures, challenges, applications, future directions,” J. of Big Data, Vol.8, Article No.53, 2021. https://doi.org/10.1186/s40537-021-00444-8
  2. [2] J. Ren, H. Li, A. Wang, K. Saho, and L. Meng, “Radar-based gait analysis by transformer-liked network for dementia diagnosis,” Biomedical Signal Processing and Control, Vol.91, Article No.105986, 2024. https://doi.org/10.1016/j.bspc.2024.105986
  3. [3] B. Lyu, H. Li, A. Tanaka, and L. Meng, “The early Japanese books reorganization by combining image processing and deep learning,” CAAI Trans. on Intelligence Technology, Vol.7, Issue 4, pp. 627-643, 2022. https://doi.org/10.1049/cit2.12104
  4. [4] H. Li and L. Meng, “Hardware-aware approach to deep neural network optimization,” Neurocomputing, Vol.559, Article No.126808, 2023. https://doi.org/10.1016/j.neucom.2023.126808
  5. [5] H. Li, X. Yue, and L. Meng, “Enhanced Mechanisms of Pooling and Channel Attention for Deep Learning Feature Maps,” PeerJ Computer Science, Vol.8, Article No.e1161, 2022. https://doi.org/10.7717/peerj-cs.1161
  6. [6] H. Li, Z. Wang, X. Yue, W. Wang, H. Tomiyama, and L. Meng, “An architecture-level analysis on deep learning models for low-impact computations,” Artificial Intelligence Review, Vol.56, pp. 1971-2010, 2023. https://doi.org/10.1007/s10462-022-10221-5
  7. [7] T. Su, Y. Zhang, X. Liang, and N. Ma, “Trajectory Planning Method for Fracture Reduction of Parallel Robots Based on DMP and APF,” IEEE Trans. on Automation Science and Engineering, Vol.23, pp. 4130-4141, 2026. https://doi.org/10.1109/TASE.2026.3653476
  8. [8] X. Liang, Y. Yan, W. Wang, T. Su, G. He, G. Li, and Z.-G. Hou, “Adaptive Human-Robot Interaction Torque Estimation With High Accuracy and Strong Tracking Ability for a Lower Limb Rehabilitation Robot,” IEEE/ASME Trans. on Mechatronics, Vol.29, Issue 6, pp. 4814-4825, 2024. https://doi.org/10.1109/TMECH.2024.3394491
  9. [9] S. Liu and W. Deng, “Very deep convolutional neural network based image classification using small training sample size,” 2015 3rd IAPR Asian Conf. on Pattern Recognition (ACPR), pp. 730-734, 2015. https://doi.org/10.1109/ACPR.2015.7486599
  10. [10] C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich, “Going deeper with convolutions,” 2015 IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2015. https://doi.org/10.1109/CVPR.2015.7298594
  11. [11] S. Ioffe and C. Szegedy, “Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift,” The 32nd Int. Conf. on Machine Learning (ICML), Vol.37, pp. 448-456, 2015.
  12. [12] C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, “Rethinking the Inception Architecture for Computer Vision,” 2016 IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 2818-2826, 2016. https://doi.org/10.1109/CVPR.2016.308
  13. [13] K. He, X. Zhang, S. Ren, and J. Sun, “Deep Residual Learning for Image Recognition,” 2016 IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 770-778, 2016. https://doi.org/10.1109/CVPR.2016.90
  14. [14] G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, “Densely Connected Convolutional Networks,” 2017 IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 2261-2269, 2017. https://doi.org/10.1109/CVPR.2017.243
  15. [15] M. Tan and Q. V. Le, “EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks,” Proc. of the 36th Int. Conf. on Machine Learning, Vol.97, pp. 6105-6114, 2019.
  16. [16] M. Tan and Q. V. Le, “EfficientNetV2: Smaller Models and Faster Training,” Proc. of the 38th Int. Conf. on Machine Learning, Vol.139, pp. 10096-10106, 2021.
  17. [17] J. Xu, Y. Pan, X. Pan, S. Hoi, Z. Yi, and Z. Xu, “RegNet: Self-Regulated Network for Image Classification,” IEEE Trans. on Neural Networks and Learning Systems, Vol.34, Issue 11, pp. 9562-9567, 2023. https://doi.org/10.1109/TNNLS.2022.3158966
  18. [18] Z. Liu, H. Mao, C.-Y. Wu, C. Feichtenhofer, T. Darrell, and S. Xie, “A ConvNet for the 2020s,” 2022 IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 11966-11976, 2022. https://doi.org/10.1109/CVPR52688.2022.01167
  19. [19] F. N. Iandola, S. Han, M. W. Moskewicz, K. Ashraf, W. J. Dally, and K. Keutzer, “Squeezenet: Alexnet-level accuracy with 50× fewer parameters and <0.5MB model size,” arXiv:1602.07360, 2016. https://doi.org/10.48550/arXiv.1602.07360
  20. [20] A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and H. Adam, “MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications,” arXiv:1704.04861, 2017. https://doi.org/10.48550/arXiv.1704.04861
  21. [21] M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen, “MobileNetV2: Inverted Residuals and Linear Bottlenecks,” 2018 IEEE/CVF Conf. on Computer Vision and Pattern Recognition, pp. 4510-4520, 2018. https://doi.org/10.1109/CVPR.2018.00474
  22. [22] X. Zhang, X. Zhou, M. Lin, and J. Sun, “ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices,” 2018 IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 6848-6856, 2018. https://doi.org/10.1109/CVPR.2018.00716
  23. [23] N. Ma et al., “ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design,” The 15th European Conf. on Computer Vision (ECCV), pp. 122-138, 2018. https://doi.org/10.1007/978-3-030-01264-9_8
  24. [24] M. Tan, B. Chen, R. Pang, V. Vasudevan, M. Sandler, A. Howard, and Q. V. Le, “MnasNet: Platform-Aware Neural Architecture Search for Mobile,” 2019 IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 2815-2823, 2019. https://doi.org/10.1109/CVPR.2019.00293
  25. [25] Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin, and B. Guo, “Swin Transformer: Hierarchical Vision Transformer using Shifted Windows,” 2021 IEEE/CVF Int. Conf. on Computer Vision (ICCV), pp. 9992-10002, 2021. https://doi.org/10.1109/ICCV48922.2021.00986
  26. [26] A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby, “An Image is Worth 16×16 Words: Transformers for Image Recognition at Scale,” Int. Conf. on Learning Representations, 2021.
  27. [27] Z. Tu, H. Talebi, H. Zhang, F. Yang, P. Milanfar, A. Bovik, and Y. Li, “MaxViT: Multi-axis Vision Transformer,” The 17th European Conf. on Computer Vision (ECCV 2022), pp. 459-479, 2022. https://doi.org/10.1007/978-3-031-20053-3_27
  28. [28] X. Xu, M. Xie, P. Miao, W. Qu, W. Xiao, H. Zhang, X. Liu, and T.-T. Wong, “Perceptual-Aware Sketch Simplification Based on Integrated VGG Layers,” IEEE Trans. on Visualization and Computer Graphics, Vol.27, Issue 1, pp. 178-189, 2021. https://doi.org/10.1109/TVCG.2019.2930512
  29. [29] H. Zhou, W. Chen, L. Cheng, D. Williams, C. W. De Silva, and M. Xia, “Reliable and Intelligent Fault Diagnosis With Evidential VGG Neural Networks,” IEEE Trans. on Instrumentation and Measurement, Vol.72, 2023. https://doi.org/10.1109/TIM.2023.3250308
  30. [30] S.-H. Wang, S. L. Fernandes, Z. Zhu, and Y.-D. Zhang, “AVNC: Attention-Based VGG-Style Network for COVID-19 Diagnosis by CBAM,” IEEE Sensors J., Vol.22, Issue 18, pp. 17431-17438, 2022. https://doi.org/10.1109/JSEN.2021.3062442
  31. [31] A. Sepas-Moghaddam, M. A. Haque, P. L. Correia, K. Nasrollahi, T. B. Moeslund, and F. Pereira, “A Double-Deep Spatio-Angular Learning Framework for Light Field-Based Face Recognition,” IEEE Trans. on Circuits and Systems for Video Technology, Vol.30, Issue 12, pp. 4496-4512, 2020. https://doi.org/10.1109/TCSVT.2019.2916669
  32. [32] X. Ren, H. Mosavat-Jahromi, L. Cai, and D. Kidston, “Spatio-Temporal Spectrum Load Prediction Using Convolutional Neural Network and ResNet,” IEEE Trans. on Cognitive Communications and Networking, Vol.8, Issue 2, pp. 502-513, 2022. https://doi.org/10.1109/TCCN.2021.3139030
  33. [33] P. Guo, H. Dai, Q. Yang, Q. Huang, and H. Yao, “An Improved Magnetic Tracking Approach Based on ResNet-LM Fusion Algorithm,” IEEE Trans. on Instrumentation and Measurement, Vol.71, 2022. https://doi.org/10.1109/TIM.2022.3216597
  34. [34] Z. Zhu, W. Zhai, H. Liu, J. Geng, M. Zhou, C. Ji, and G. Jia, “Juggler-ResNet: A Flexible and High-Speed ResNet Optimization Method for Intrusion Detection System in Software-Defined Industrial Networks,” IEEE Trans. on Industrial Informatics, Vol.18, Issue 6, pp. 4224-4233, 2022. https://doi.org/10.1109/TII.2021.3121783
  35. [35] D. O’Neill, B. Xue, and M. Zhang, “Evolutionary Neural Architecture Search for High-Dimensional Skip-Connection Structures on DenseNet Style Networks,” IEEE Trans. on Evolutionary Computation, Vol.25, Issue 6, pp. 1118-1132, 2021. https://doi.org/10.1109/TEVC.2021.3083315
  36. [36] F. Yuwei, L. Liejuan, H. Weihua, H. Guobin, H. Peijun, Z. Zhiyu, C. Chi, and W. Chuang, “Partial Discharge Pattern Recognition Method Based on Transfer Learning and DenseNet Model,” IEEE Trans. on Dielectrics and Electrical Insulation, Vol.30, Issue 3, pp. 1240-1246, 2023. https://doi.org/10.1109/TDEI.2023.3239032
  37. [37] L. Huang and C.-M. Pun, “Audio Replay Spoof Attack Detection by Joint Segment-Based Linear Filter Bank Feature Extraction and Attention-Enhanced DenseNet-BiLSTM Network,” IEEE/ACM Trans. on Audio, Speech, and Language Processing, Vol.28, pp. 1813-1825, 2020. https://doi.org/10.1109/TASLP.2020.2998870
  38. [38] N. K. Yadav, S. K. Singh, and S. R. Dubey, “MobileAR-GAN: MobileNet-Based Efficient Attentive Recurrent Generative Adversarial Network for Infrared-to-Visual Transformations,” IEEE Trans. on Instrumentation and Measurement, Vol.71, 2022. https://doi.org/10.1109/TIM.2022.3166202
  39. [39] J. Yang, Y. Wang, H. Zhao, and G. Gui, “MobileNet and Knowledge Distillation-Based Automatic Scenario Recognition Method in Vehicle-to-Vehicle Systems,” IEEE Trans. on Vehicular Technology, Vol.71, Issue 10, pp. 11006-11016, 2022. https://doi.org/10.1109/TVT.2022.3184994
  40. [40] T. Addabbo, A. Fort, M. Mugnaini, V. Vignoli, M. Intravaia, M. Tani, M. Bianchini, F. Scarselli, and B. T. Corradini, “Smart Gravimetric System for Enhanced Security of Accesses to Public Places Embedding a MobileNet Neural Network Classifier,” IEEE Trans. on Instrumentation and Measurement, Vol.71, 2022. https://doi.org/10.1109/TIM.2022.3162270
  41. [41] A. Lin, B. Chen, J. Xu, Z. Zhang, G. Lu, and D. Zhang, “DS-TransUNet: Dual Swin Transformer U-Net for Medical Image Segmentation,” IEEE Trans. on Instrumentation and Measurement, Vol.71, 2022. https://doi.org/10.1109/TIM.2022.3178991
  42. [42] X. Meng, Y. Yang, L. Wang, T. Wang, R. Li, and C. Zhang, “Class-Guided Swin Transformer for Semantic Segmentation of Remote Sensing Imagery,” IEEE Geoscience and Remote Sensing Letters, Vol.19, 2022. https://doi.org/10.1109/LGRS.2022.3215200
  43. [43] F. Sun, W. Zhou, L. Ye, and L. Yu, “Hierarchical Decoding Network Based on Swin Transformer for Detecting Salient Objects in RGB-T Images,” IEEE Signal Processing Letters, Vol.29, pp. 1714-1718, 2022. https://doi.org/10.1109/LSP.2022.3194843
  44. [44] C. Zhang, S. Bengio, M. Hardt, B. Recht, and O. Vinyals, “Understanding deep learning (still) requires rethinking generalization,” Commun. ACM, Vol.64, Issue 3, pp. 107-115, 2021. https://doi.org/10.1145/3446776
  45. [45] A. Kumar, S. Sarawagi, and U. Jain, “Trainable Calibration Measures for Neural Networks from Kernel Mean Embeddings,” Proc. of the 35th Int. Conf. on Machine Learning, Vol.80, pp. 2805-2814, 2018.
  46. [46] T. G. Dietterich, “Ensemble Methods in Machine Learning,” Int. Workshop on Multiple Classifier Systems (MCS 2000), 2000. https://doi.org/10.1007/3-540-45014-9_1
  47. [47] B. Lakshminarayanan, A. Pritzel, and C. Blundell, “Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles,” Proc. of the 31st Int. Conf. on Neural Information Processing Systems, pp. 6405-6416, 2017.
  48. [48] A. Mehrtash, W. M. Wells, C. M. Tempany, P. Abolmaesumi, and T. Kapur, “Confidence Calibration and Predictive Uncertainty Estimation for Deep Medical Image Segmentation,” IEEE Trans. on Medical Imaging, Vol.39, Issue 12, pp. 3868-3878, 2020. https://doi.org/10.1109/TMI.2020.3006437
  49. [49] J. C. Platt, “Probabilistic Outputs for Support Vector Machines and Comparisons to Regularized Likelihood Methods,” Advances in Large Margin Classifiers, Vol.10, No.3, pp. 61-74, 1999.
  50. [50] C. Guo, G. Pleiss, Y. Sun, and K. Q. Weinberger, “On Calibration of Modern Neural Networks,” Proc. of the 34th Int. Conf. on Machine Learning, Vol.70, pp. 1321-1330, 2017.
  51. [51] B. Zadrozny and C. Elkan, “Obtaining calibrated probability estimates from decision trees and naive Bayesian classifiers,” Proc. of the 18th Int. Conf. on Machine Learning, pp. 609-616, 2001.
  52. [52] B. Zadrozny and C. Elkan, “Transforming classifier scores into accurate multiclass probability estimates,” Proc. of the 8th ACM SIGKDD Int. Conf. on Knowledge Discovery and Data Mining, pp. 694-699, 2002. https://doi.org/10.1145/775047.775151
  53. [53] M. P. Naeini, G. Cooper, and M. Hauskrecht, “Obtaining Well Calibrated Probabilities Using Bayesian Binning,” Proc. of the 29th AAAI Conf. on Artificial Intelligence, pp. 2901-2907, 2015. https://doi.org/10.1609/aaai.v29i1.9602
  54. [54] G. Carneiro, L. Z. C. T. Pu, R. Singh, and A. Burt, “Deep learning uncertainty and confidence calibration for the five-class polyp classification from colonoscopy,” Medical Image Analysis, Vol.62, Article No.101653, 2020. https://doi.org/10.1016/j.media.2020.101653
  55. [55] D. Karimi and A. Gholipour, “Improving Calibration and Out-of-Distribution Detection in Deep Models for Medical Image Segmentation,” IEEE Trans. on Artificial Intelligence, Vol.4, Issue 2, pp. 383-397, 2023. https://doi.org/10.1109/TAI.2022.3159510
  56. [56] M. A. Haghpanah, M. T. Masouleh, and A. Kalhor, “Determining the trustworthiness of DNNs in classification tasks using generalized feature-based confidence metric,” Pattern Recognition, Vol.142, Article No.109683, 2023. https://doi.org/10.1016/j.patcog.2023.109683
  57. [57] G. Rossolini, A. Biondi, and G. Buttazzo, “Increasing the Confidence of Deep Neural Networks by Coverage Analysis,” IEEE Trans. on Software Engineering, Vol.49, Issue 2, pp. 802-815, 2023. https://doi.org/10.1109/TSE.2022.3163682

*This site is desgined based on HTML5 and CSS3 for modern browsers, e.g. Chrome, Firefox, Safari, Edge, Opera.

Last updated on Nov. 20, 2026