Approximately half of the patients with spinal cord injuries in Japan have a cervical spinal cord injury. Owing to the trunk dysfunction, patients with high-level spinal cord injuries have particular difficulty when searching for or picking up objects from the floor. Recently, welfare robot arms have been developed to help such individuals increase self-reliance. In this study, we propose an operating system that includes an eye-in-hand system with a touchscreen interface for grasping objects from the floor and delivering them to the individual. In the proposed method, the visual information of the target object is shown on a touchscreen interface. The patient specifies the target position for the robot arm by drawing a line on the target object on the interface. We conducted an experiment to compare the proposed interface with an on-screen joystick to demonstrate the proposed system’s efficiency and its ability to reduce physical burden. The results show that the proposed method is both quicker to use and effectively reduces the physical burden on the user compared to the conventional method.

1. [1] Cabinet Office, “Annual Report on Government Measures for Persons with Disabilities,” p. 213, 2021 (in Japanese).
2. [2] Cabinet Office, “Annual Report on Government Measures for Persons with Disabilities,” p. 1, 2011 (in Japanese).
3. [3] H. Sakai, T. Ueta, T. Maeda, and K. Shiba, “Survey for Spinal Cord Injuries in Fukuoka,” Sogo Rehabilitation, Vol.36, No.10, pp. 969-972, 2008 (in Japanese).
4. [4] K. Shibazaki, “National Spinal Cord Injury Registration Statistics 2002.1–12,” The J. of the Japan Medical Society of Spinal Cord Lesion, Vol.18, pp. 271-274, 2005 (in Japanese).
5. [5] F. Tajima, Y. Kamijo, and Y. Nishimura, “Basis of Rehabilitation in Patients with Spinal Cord Injury,” Spinal Surgery, Vol.30, No.1, pp. 58-67, 2016 (in Japanese).
6. [6] K. Nagata, Y. Wakita, N. Yamanobe, and N. Ando, “Development of RT-Component for Assistive Robotic Arm and Man-Machine Interface Using OpenRTM-aist,” Trans. of the Japan Society of Mechanical Engineers, Series C, Vol.76, No.766, pp. 1419-1426, 2010 (in Japanese).
7. [7] V. Maheu, P. S. Archambault, J. Frappier, and F. Routhier, “Evaluation of the JACO Robotic Arm: Clinico-Economic Study for Powered Wheelchair Users with Upper-Extremity Disabilities,” 2011 IEEE Int. Conf. on Rehabilitation Robotics, 2011. https://doi.org/10.1109/ICORR.2011.5975397
8. [8] W. Yoon, Y. Wakita, and N. Yamanobe, “Self-Reliance Support by RAPUDA, a Robot Arm that Supports the Lives of People with Disabilities in the Upper Limbs,” J. of the Society of Biomechanisms, Vol.37, No.2, pp. 100-104, 2013 (in Japanese).
9. [9] J. Vogel et al., “An Ecosystem for Heterogeneous Robotic Assistants in Caregiving: Core Functionalities and Use Cases,” IEEE Robotics & Automation Magazine, Vol.28, No.3, pp. 12-28, 2021.
10. [10] J. Yamaguchi et al., “Clinical Evaluation of Assistive Robotic Arm: Efficacy Evaluation Using Psychological Estimation Scale,” Proc. of the 59th Annual Conf. of JSSD, 2012 (in Japanese). https://doi.org/10.11247/jssd.59.0_150
11. [11] C.-S. Chung et al., “Performance Evaluation of a Mobile Touchscreen Interface for Assistive Robotic Manipulators: A Pilot Study,” Top Spinal Cord Injury Rehabilitation, Vol.23, No.2, pp. 131-139, 2017.
12. [12] National Rehabilitation Center for Persons with Disabilities, “Electric Wheelchair Compatibility & Operation Practice Manual,” p. 8, 2006 (in Japanese).
13. [13] T. Matsumaru, A. I. Septiana, and K. Horiuchi, “Three-Dimensional Aerial Image Interface, 3DAII,” J. Robot. Mechatron., Vol.31, No.5, pp. 657-670, 2019.
14. [14] A. Namiki, “Multi-Eye Vision Robot,” J. of the Robotics Society of Japan, Vol.40, No.5, pp. 387-392, 2022 (in Japanese).
15. [15] S. Hashimoto, A. Ishida, M. Imani, and T. Igarashi, “TouchMe: An Augmented Reality Interface for Remote Robot Control,” J. Robot. Mechatron., Vol.25, No.3, pp. 529-537, 2013.
16. [16] B. J. F. Driessen, T. K. T. Kate, F. Liefhebber, A. H. G. Versluis, and J. A. v. Woerden, “Collaborative Control of the Manus Manipulator,” Universal Access in the Information Society, Vol.4, No.2, pp. 165-173, 2005.
17. [17] P. Hebert et al., “Supervised Remote Robot with Guided Autonomy and Teleoperation (SURROGATE): A Framework for Whole-Body Manipulation,” 2015 IEEE Int. Conf. on Robotics and Automation (ICRA), pp. 5509-5516, 2015.
18. [18] J. Soares, A. Vale, and R. Ventura, “A Multi-Purpose Rescue Vehicle and a Human-Robot Interface Architecture for Remote Assistance in ITER,” Fusion Engineering and Design, Vols.98-99, pp. 1656-1659, 2015.
19. [19] K. M. Tsui, D.-J. Kim, A. Behal, D. Kontak, and H. A. Yanco, “‘I want that’: Human-in-the-Loop Control of a Wheelchair-Mounted Robotic Arm,” Applied Bionics and Biomechanics, Vol.8, No.1, pp. 127-147, 2011.
20. [20] H. Fujii, K. Sugimoto, A. Yamashita, and H. Asama, “Online Half Diminished-Reality Imaging Using Multiple RGB-D Sensors for Remote Control Robot,” J. of the Japan Society for Precision Engineering, Vol.81, No.12, pp. 1185-1192, 2015 (in Japanese).
21. [21] H. Jiang, T. Zhang, J. P. Wachs, and B. S. Duerstock, “Enhanced Control of a Wheelchair-Mounted Robotic Manipulator Using 3-D Vision and Multimodal Interaction,” Computer Vision and Image Understanding, Vol.149, pp. 21-31, 2016.
22. [22] D. Ding, B. Styler, C.-S. Chung, and A. Hourtet, “Development of a Vision-Guided Shared-Control System for Assistive Robotic Manipulators,” Sensors, Vol.22, No.12, Article No.4351, 2022.
23. [23] H. Tanaka et al., “Visual Marker System for Autonomous Object Handling by Assistive Robotic Arm,” J. Robot. Mechatron., Vol.23, No.4, pp. 484-493, 2011.
24. [24] Y. Miyawaki et al., “Interface for Operating Electric Wheelchair Without Gazing Operation Screen,” The Proc. of 68th Conf. of Tokai Branch, Article No.519, 2019 (in Japanese). https://doi.org/10.1299/jsmetokai.2019.68.519