Approximately 600,000 to 1,000,000 patients are diagnosed with rheumatoid arthritis (RA) in Japan. To provide appropriate treatment, it is necessary to accurately measure the progression of RA by diagnosing the disease several times a year. The modified total sharp score (mTSS) calculated from hand X-ray images is a standard diagnostic method for RA progression. However, this diagnostic method is time-consuming as the scores are rated at as many as 16 points per hand. Accordingly, in order to shorten the diagnosis time of RA patients and improve the quality of diagnosis, the development of computer-aided diagnosis (CAD) systems is expected. We have previously proposed a CAD system that can detect finger joint positions using a support vector machine and can estimate the mTSS using ridge regression. In this study, we propose a fully automatic detection method of RA score evaluation points in the carpal site from simple hand X-ray images using deep learning. The proposed method first segments the carpal site using deep learning. Next, the RA evaluation points are automatically determined from each segment based on prior knowledge. Experimental results on X-ray images of the hands of 140 patients with RA showed that the mTSS evaluation point at the carpal site could be detected with an average error of 25 pixels. This study enables the automatic detection of RA score evaluation points in the carpal site. In the diagnosis of RA, the time required for diagnosis can be reduced by automating the determination of diagnostic points by physician.

1. [1] H. Yamanaka, N. Sugiyama, E. Inoue, A. Taniguchi, and S. Momohara, “Estimates of the prevalence of and current treatment practices for rheumatoid arthritis in Japan using reimbursement data from health insurance societies and the IORRA cohort (I),” Modern Rheumatol, Vol.24, No.1, pp. 33-40, 2014.
2. [2] K. Raza, C. E. Buckley, M. Salmon, and C. D. Buckley, “Treating very early rheumatoid arthritis,” Best Practice & Research Clinical Rheumatology, Vol.20, No.5, pp. 849-863, 2006.
3. [3] M. F. Bakker, J. W. G. Jacobs, S. M. M. Verstappen, and J. W. J. Bijlsma, “Tight control in the treatment of rheumatoid arthritis: efficacy and feasibility,” Annals of the Rheumatic Diseases, Vol.66, No.Suppl 3, pp. iii56-iii60, 2007.
4. [4] E. W. St. Clair, D. M. F. M. van der Heijde, J. S. Smolen, R. N. Maini, J. M. Bothon, P. Emiry, E. Keystone, M. Schiff, J. R. Kalden, B. Wang, and K. Dewoody, “Combination of infliximab and methotrexate therapy for early rheumatoid arthritis: a randomized, controlled trial,” Arthritis and Rheumatism, Vol.50, No.11, pp. 3432-3443, 2004.
5. [5] F. C. Breedveld, M. H. Weisman, A. F. Kavanaugh, S. B. Cohen, K. Pavelka, R. van Vollebhoven, J. Sharp, J. L. Perez, and G. T. Spencer-Green, “The PREMIER study: A multicenter, randomized, double-blind clinical trial of combination therapy with adalimumab plus methotrexate versus methotrexate alone or adalimumab alone in patients with early, aggressive rheumatoid arthritis who had not had previo,” Arthritis and Rheumatism, Vol.54, No.1, pp. 26-37, 2006.
6. [6] D. M. van der Heijde, “Plain X-rays in rheumatoid arthritis: overview of scoring methods, their reliability and applicability,” Bailliere’s Clinical Rheumatology, Vol.10, No.3, pp. 435-453, 1996.
7. [7] Y. Huo, K. L. Vincken, D. M. van der Heijde, F. P. Lafeber, and M. A. Viergever, “Automatic quantification of radiographic finger joint space width of patients with early rheumatoid arthritis,” IEEE Trans. on Biomedical Engineering, Vol.63, No.10, pp. 2177-2186, 2016.
8. [8] K. Nakatsu, K. Morita, N. Yagi, and S. Kobashi, “Finger joint detection method in hand X-ray radiograph images using statistical shape model and support vector machine,” Proc. of the 2020 Int. Symp. on Community-Centric System (CcS), 2020.
9. [9] K. Morita, P. Chan, M. Nii, N. Nakagawa, and S. Kobashi, “Finger joint detection method for the automatic estimation of rheumatoid arthritis progression using machine learning,” Proc. of the 2018 IEEE Int. Conf. on Systems, Man, and Cybernetics (SMC), pp. 1315-1320, 2018.
10. [10] L. Su, X. Fu, X. Zhang, X. Cheng, Y. Ma, Y. Gan, and Q. Hu, “Delineation of carpal bones from hand X-ray images through prior model, and integration of region-based and boundary-based segmentations,” IEEE Access, Vol.6, pp. 19993-20008, 2018.
11. [11] L. K. Meng, A. Khalil, M. H. A. Nizar, M. K. Nisham, B. Pingguan-Murphy, Y. C. Hum, M. I. M. Salim, and K. W. Lai, “Carpal bone segmentation using fully convolutional neural network,” Current Medical Imaging Reviews, Vol.15, No.10, pp. 983-989, 2019.
12. [12] S. Ukil and J. M. Reinhardt, “Anatomy-Guided Lung Lobe Segmentation in X-Ray CT Images,” IEEE Trans. on Medical Imaging, Vol.28, No.2, pp. 202-214, 2009.
13. [13] D. H. Kim and T. MacKinnon, “Artificial intelligence in fracture detection: transfer learning from deep convolutional neural networks,” Clinical Radiology, Vol.73, No.5, pp. 439-445, 2018.
14. [14] Y. L. Thian, Y. Li, P. Jagmohan, D. Sia, V. E. Y. Chan, and R. T. Tan, “Convolutional neural networks for automated fracture detection and localization on wrist radiographs,” Radiology: Artificial Intelligence, Vol.1, No.1, e180001, 2019.
15. [15] J. H. Lee, Y. J. Kim, and K. G. Kim, “Bone age estimation using deep learning and hand X-ray images,” Biomedical Engineering Letters, Vol.10, No.3, pp. 323-331, 2020.
16. [16] O. Ronneberger, P. Fischer, and T. Brox, “U-Net: Convolutional Networks for Biomedical Image Segmentation,” Lecture Notes in Computer Science: Medical Image Computing and Computer-Assisted Intervention (MICCAI 2015), Vol.9351, pp. 234-241, 2015.
17. [17] R. L. Drake, A. W. Vogi, and A. W. M. Mitchell, “Gray’s Anatomy for Students,” Churchill Livingstone, 2015.
18. [18] W. R. Crum, O. Camara, and D. L. G. Hill, “Generalized Overlap Measures for Evaluation and Validation in Medical Image Analysis,” IEEE Trans. on Medical Imaging, Vol.25, No.11, pp. 1451-1461, 2006.
19. [19] C. H. Sudre, W. Li, T. Vercauteren, S. Ourselin, and M. J. Cardoso, “Generalised Dice Overlap as a Deep Learning Loss Function for Highly Unbalanced Segmentations,” Int. Workshop on Deep Learn. Med. Image Anal. Multimodal Learn. Clin. Decis. Support, pp. 240-248, 2017.
20. [20] L. R. Dice, “Measures of the amount of ecologic association between species,” Ecology, Vol.26, No.2, pp. 297-302, 1945.
21. [21] J. Roerdink and A. Meijster, “The Watershed Transform: Definitions, Algorithms and Parallelization Strategies,” Fundamenta Informaticae, Vol.41, Nos.1-2, pp. 187-228, 2000.