JRM Vol.26 No.6 pp. 704-717
doi: 10.20965/jrm.2014.p0704


Image-Projecting Desktop Arm Trainer for Hand-Eye Coordination Training

Takafumi Matsumaru, Yang Liu, Yi Jiang, and Chuankai Dai

Graduate School of Information, Production and Systems, Waseda University, 2-7 Hibikino, Wakamatsu-ku, Kitakyushu 808-0135, Japan

April 24, 2014
October 20, 2014
December 20, 2014
upper-extremity exercise, hand-eye coordination training, image-projecting desktop arm trainer (IDAT), step-on interface (SOI), human-robot interaction (HRI) design


This paper presents a novel arm-training system, known as the image-projecting desktop arm trainer (IDAT), which is aimed at hand-eye coordination training. The projector displays an exercise image on a desktop in front of a seated patient, and the scanning range finder measures the behavior of the patient as he/she performs the exercise. IDAT is non-invasive and does not constrain the patient. Its efficiency is based on the voluntary movements of the patient, although it offers neither the physical assistance nor tactile feedback of some conventional systems. Three kinds of training content have been developed: “mole-hitting,” “balloon-bursting,” and “fish-catching.” These games were designed for training hand-eye coordination in different directions. A patient and/or medical professional can set a suitable training level, that is, the training time, speed of movement of the objects, and number of objects to appear at any one time, based on the patient’s condition and ability. A questionnaire survey was carried out to evaluate IDAT-3, and the results showed that it was highly acclaimed in terms of user-friendliness, fun, and usefulness.

Cite this article as:
T. Matsumaru, Y. Liu, Y. Jiang, and C. Dai, “Image-Projecting Desktop Arm Trainer for Hand-Eye Coordination Training,” J. Robot. Mechatron., Vol.26, No.6, pp. 704-717, 2014.
Data files:
  1. [1] B. R. Brewer, S. K. McDowell, and L. C. Worthen-Chaudhari, “Poststroke Upper Extremity Rehabilitation,” A Review of Robotic Systems and Clinical Results, Topics in Stroke Rehabilitation, Vol.14, No.6, pp. 22-44, 2007.
  2. [2] G. Kwakkel, B. J. Kollen, and H. I. Krebs, “Effects of Robot-Assisted Therapy on Upper Limb Recovery after Stroke,” A Systematic Review, Neurorehabilitation and Neural Repair, Vol.22, No.2, pp. 11-21, 2008.
  3. [3] A. C. Lo, P. D. Guarino, L. G. Richards, J. K. Haselkorn, G. F. Wittenberg, D. G. Federman, R. J. Ringer, T. H. Wagner, H. I. Krebs, B. T. Volpe, C. T. Bever, Jr., D. M. Bravata, P. W. Duncan, B. H. Corn, A. D.Maffucci, S. E. Nadeau, S. S. Conroy, J. M. Powell, G. D. Huang, and P. Peduzzi, “Robot-Assisted Therapy for Long-Term Upper-Limb Impairment after Stroke,” New England J. of Medicine, Vol.362, No.19, pp. 1772-1783, 2010.
  4. [4] The Japan Stroke Society, “Guidelines for Stroke Therapy 2009,” The Japan Stroke Society, 2009 (in Japanese).
  5. [5] B. H. Dobkin, “Rehabilitation after Stroke,” The New England J. of Medicine, Vol.352, No.16, pp. 1677-1684, 2005.
  6. [6] G. Barling and J. Barling, “Sling Suspension Therapy,” Trafford Publishing, 2002.
  7. [7] T. Matsumaru and K. Akai, “Functions of Mobile-Robot Step-On Interface,” J. of Robotics and Mechatronics, Vol.21, No.2, pp. 267-276, 2009.
  8. [8] M. Kikawada, W. Saito, and T. Matsumaru, “Operation Method of Human-Friendly Robot (10th report) – Development of Touchgame Application using Step-on Interface –,” 2011 JSME Conf. on Robotics and Mechatronics (ROBOMEC 2011 in Okayama), 1P1-H01, 2011 (in Japanese).
  9. [9] Y. Liu, Y. Jiang, and T. Matsumaru, “Development of Imageprojective Desktop Arm Trainer,” IDAT. 2012 IEEE/SICE Int. Symp on System Integration (SII), pp. 355-360, 2012.
  10. [10] T. Matsumaru, Y. Jiang, and Y. Liu, “Image-projective Desktop Arm Trainer IDAT for Therapy,” The 22nd IEEE Int. Symp on Robot and Human Interactive Communication (ROMAN 2013), pp. 501-506, 2013.
  11. [11] J. E. Speich and J. Rosen, “Medical Robotics,” Encyclopedia of Biomaterials and Biomedical Engineering, Marcel Dekker Inc., pp. 983-993, 2004.
  12. [12] N. Hogan, H. I. Krebs, J. Charnnarong, P. Srikrishna, and A. Sharon, “MIT-MANUS: a workstation for manual therapy and training I,” Proc. IEEE Int. Workshop on Robot and Human Communication 1992 (RO-MAN 1992), pp. 161-165, 1992.
  13. [13] M. L. Aisen, H. I. Krebs, N. Hogan, F. McDowell, and B. T. Volpe, “The effect of robot-assisted therapy and rehabilitative training on motor recovery following stroke,” Archives of Neurology, Vol.54, No.4, pp. 443-446, 1997.
  14. [14] C. G. Burgar, P. S. Lum, P. C. Shor, and H. F. M. Van der Loos, “Development of robots for rehabilitation therapy,” the Palo Alto VA/ Stanford experience, J. of Rehabilitation Research and Development, Vol.37, No.6, pp. 663-673, 2000.
  15. [15] P. S. Lum, C. G. Burgar, M. V. der Loos, P. C. Shor, M. Majmundar, and R. Yap, “MIME robotic device for upper-limb neurorehabilitation in subacute stroke subjects: A follow-up study,” J. of Rehabilitation Research and Development, Vol.43, No.5, pp. 631-642, 2006.
  16. [16] D. J. Reinkensmeyer, J. P. A. Dewald, and W. Z. Rymer, “Guidancebased quantification or arm impairment following brain injury: a pilot study,” IEEE Trans. Rehabilitation Engineering, Vol.7, No.1, pp. 1-11, 1999.
  17. [17] D. J. Reinkensmeyer, L. E. Kahn, M. Averbuch, A. McKenna-Cole, B. D. Schmit, and W. Z. Rymer, “Understanding and treating arm movement impairment after chronic brain injury: Progress with the ARM Guide,” J. Rehabilitation Research and Development, Vol.37, No.6, pp. 653-662, 2000.
  18. [18] R. Loureiro, F. Amirabdollahian, M. Topping, B. Driessen, and W. Harwin, “Upper Limb Robot Mediated Stroke Therapy-GENTLE/s Approach,” Autonomous Robots, Vol.15, No.1, pp. 35-51, 2003.
  19. [19] F. Amirabdollahian, R. Loureiro, E. Gradwell, C. Collin, W. Harwin, and G. Johnson, “Multivariate analysis of the Fugl-Meyer outcome measures assessing the effectiveness of GENTLE/S robotmediated stroke therapy,” J. of NeuroEngineering and Rehabilitation, Vol.4, pp. 4, 2007.
  20. [20] T. Nef, M. Mihelj, G. Colombo, and R. Riener, “ARMin – robot for rehabilitation of the upper extremities,” Proc. 2006 IEEE Int. Conf. on Robotics and Automation (ICRA 2006), pp. 3152-3157, 2006.
  21. [21] M. Mihelj, T. Nef, and R. Riener, “ARMin – Toward a six DoF upper limb rehabilitation robot,” Proc. the First IEEE/RAS-EMBS Int. Conf. on Biomedical Robotics and Biomechatronics (BioRob 2006), pp. 1154-1159, 2006.
  22. [22] M. Guidali, A. Duschau-Wicke, S. Broggi, V. Klamroth-Marganska, T. Nef, and R. Riener, “A robotic system to train activities of daily living in a virtual environment. Medical & Biological Engineering & Computing,” Vol.49, No.10, pp. 1213-1223, 2011.
  23. [23] E. B. Brokaw, T. Murray, T. Nef, and P. S. Lum, “Retraining of interjoint arm coordination after stroke using robot-assisted timeindependent functional training,” J. of Rehabilitation Research and Development, Vol.48, No.4, pp. 299-316, 2011.
  24. [24] I. Treger, S.Faran, and H. Ring, “Robot-assisted therapy for neuromuscular training of sub-acute stroke patients a feasibility study,” European J. of physical and Rehabilitation Medicine (Europa Medicophysica), Vol.44, No.4, pp. 431-435, 2008.
  25. [25] S. Faran, O. Einav, D. Yoeli, M. Kerzhner, D. Geva, G. Magnazi, S. van Kaick, and K. H. Mauritz, “Reo Assessment to Guide the ReoGo Therapy: Reliability and Validity of Novel Robotic Scores,” Virtual Rehabilitation Int. Conf. 2009.
  26. [26] M. Terashima and S. Sakane, “A Human-Robot Interface Using an Extended Digital Desk,” Proc. IEEE Int. Conf. on Robotics and Automation 1999 (ICRA ’99), Vol.4, pp. 2874-2880, 1999.
  27. [27] S. Sato and S. Sakane, “A human-robot interface using an interactive hand pointer that projects a mark in the real work space,” Proc. IEEE Int. Conf. on Robotics and Automation 2000 (ICRA ’00), Vol.1, pp. 589-595, 2000.
  28. [28] T.Machino, S. Iwaki, H. Kawata, Y. Yanagihara, Y. Nanjo, and K. Shimokura, “Remote-Collaboration System using Mobile Robot with Camera and Projector,” Proc. IEEE Int. Conf. on Robotics and Automation, 2006 (ICRA 2006), pp. 4063-4068, 2006.
  29. [29] Y. Nakamura, T. Machino, M. Motegi, Y. Iwata, T. Miyamoto, S. Iwaki, S. Muto, and K. Shimokura, “Framework and service allocation for network robot platform and execution of interdependent services,” Robotics and Autonomous Systems, Vol.56, No.10, pp. 793-797, 2008.
  30. [30] G. Reinhart, W. Vogl, and I. Kresse, “Projection-based User Interface for Industrial Robots,” IEEE Symp on Virtual Environments, Human-Computer Interfaces and Measurement Systems 2007 (VECIMS 2007), pp. 67-71, 2007.
  31. [31] G. Reinhart, U. Munzert, and W. Vogl, “A programming system for robot-based remote-laser-welding with conventional optics,” CIRP Annals – Manufacturing Technology, Vol.57, No.1, pp. 37-40, 2008.
  32. [32] K. Hosoi, V. N. Dao, A. Mori, and M. Sugimoto, “VisiCon: a robot control interface for visualizing manipulation using a handheld projector,” Proc. of Int. Conf. on Advances in Computer Entertainment Technology (ACE 2007), pp. 99-106, 2007.
  33. [33] K. Hosoi, V. N. Dao, A. Mori, and M. Sugimoto, “CoGAME: Manipulation using a Handheld Projector,” 34th Int. Conf. and Exhibition on Computer Graphics and Interactive Techniques (SIGGRAPH 2007) Emerging Technologies, No.2, 2007.
  34. [34] T. Matsumaru, “Mobile Robot with Preliminary-announcement and Display Function of Forthcoming Motion using Projection Equipment,” 15th IEEE Int. Symp. on Robot and Human Interactive Communication (RO-MAN 06), pp. 443-450, 2006.
  35. [35] J. Park and G. J. Kim, “Robots with projectors: an alternative to anthropomorphic HRI,” Proc. of 4th ACM/IEEE Int. Conf. on Human robot interaction (HRI 2009), pp. 221-222, 2009.
  36. [36] T. Matsumaru, “A Characteristics Measurement of Twodimensional Range Scanner and its Application,” The Open Automation and Control Systems J., Vol.2, pp. 21-30, 2009.
  37. [37] T. Matsumaru and K. Akai, “Step-on interface on mobile robot to operate by stepping on projected button,” The Open Automation and Control Systems J., Vol.2, pp. 85-95, 2009.
  38. [38] T. Matsumaru, Y. Horiuchi, K. Akai, and Y. Ito, “Truly-Tender-Tailed Tag-Playing Robot through Friendly Amusing Mobile Function,” J. of Robotics and Mechatronics, Vol.22, No.3, pp. 301-307, 2010.
  39. [39] T. Matsumaru, W. Saito, and Y. Ito, “User-robot interaction based on mobile robot step-on interface,” Trans of the Virtual Reality Society of Japan, Vol.15, No.3, pp. 335-345, 2010 (in Japanese).
  40. [40] T. Matsumaru, “Friendly Amusing Mobile Function for Human-Robot Interaction,” In Proc. 19th IEEE Int. Symp on Robot and Human Interactive Communication (RO-MAN 10), pp. 88-93, 2010.
  41. [41] S. Takasugi, M. Suzuki, Y. Kawamura, S. Ishii, S. Okada, T. Nejime, I. Kawano, T. Kamishima, K. Masumoto, and Y. Iwamoto,“12-month intervention of playing arcade games significantly enhanced body balance and reaction time in the elderly women,” Int. Symp on Preventing Falls and Fractures in Older Persons (ISFR Symposium), 2004.
  42. [42] I. Kawano, S. Takasugi, T. Nejime, T. Kamishima, and Y. Iwamoto, “Research and Development of An Arcade Game Machine that Activates Body and Brain Functions,” 15th Int. Congress of the World Confederation for Physical Therapy (World Physical Therapy 2007), pp. 26-19, 2007.
  43. [43] K. Ishihara, K. Oogomori, M. Nagamachi, and S. Ishihara, “An electromechanical whack-a-mole-type game for older people,” Gerontechnology. Vol.9, No.2, pp. 294. 2010.
  44. [44] K. Ogomori, M. Nagamachi, K. Ishihara, S. Ishihara, and M. Kohchi, “Requirements for a Cognitive Training Game for Elderly or Disabled People,” 2011 Int. Conf on Biometrics and Kansei Engineering (ICBAKE), pp. 150-154, 2011.
  45. [45] M. Annett, F. Anderson, D. Goertzen, J. Halton, Q. Ranson, W. F. Bischof, and P. Boulanger, “Using a Multi-touch Tabletop for Upper Extremity Motor Rehabilitation,” Proc. of 21st Annual Conf. of the Australian Computer-Human Interaction Special Interest Group (CHISIG) of the Human Factors and Ergonomics Society of Australia (HFESA) (OZCHI 2009- Design: Open 24/7), pp. 261-264, 2009.
  46. [46] A. Dunne, S. Do-Lenh, G. O. Laighin, C. Shen, and P. Bonato, “Upper Extremity Rehabilitation of Children with Cerebral Palsy using Accelerometer Feedback on a Multitouch Display,” Proc. of 32nd Annual Int. Conf. of IEEE Engineering in Medicine and Biology Society (EMBC’10), pp. 1751-1754, 2010.
  47. [47] D. Jack, R. Boian, A. Merians, M. Tremaine, G. Burdea, S. Adamovich, M. Recce, and H. Poizner, “Virtual reality enhanced stroke rehabilitation,” IEEE Trans. Neural Systems and Rehabilitation Engineering, Vol.9, No.3, pp. 308-318, 2001.
  48. [48] L. F. Lucca, “Virtual reality and motor rehabilitation of the upper limb after stroke: a generation of progress,” J. of Rehabilitation Medicine, Vol.41, No.12, pp. 1003-1006, 2009.
  49. [49] A. S. Merians, D. Jack, R. Boian, M. Tremaine, G. C. Burdea, S. V. Adamovich, M. Recce, and H. Poizner, “Virtual Reality-Augmented Rehabilitation for Patients Following Stroke,” Physical Therapy, Vol.82, No.9, pp. 898-915, 2002.
  50. [50] M. S. Cameirao, S. B. Badia, E. D. Oller, and P. F. M. J. Verschure, “Neurorehabilitation using the virtual reality based Rehabilitation Gaming System: methodology, design, psychometrics, usability and validation,” J. of NeuroEngineering and Rehabilitation, Vol.7, pp. 48, 2010.
  51. [51] M. King, L. Hale, A. Pekkari, M. Persson, M. Gregorsson, and M. Nilsson, “An affordable, computerised, table-based exercise system for stroke survivors,” Disability and Rehabilitation: Assistive Technology, Vol.5, No.4, pp. 288-293, 2010.
  52. [52] M. King, J. Hijmans, M. Sampson, J. Satherley, N. McMillan, and L. Hale, “Bilateral movement training with computer games for stroke rehabilitation,” Proc of 4th Int. Convention on Rehabilitation Engineering & Assistive Technology (iCREATe ’10), B1-2, 2010.
  53. [53] M. Pirovano, R. Mainetti, G. Baud-Bovy, P. L. Lanzi, and N. A. Borghese, “Self-Adaptive Games for Rehabilitation at Home,” 2012 IEEE Conf. on Computational Intelligence and Games (CIG), pp. 179-186, 2012.
  54. [54] D. Norman, “The Psychology of Everyday Things,” Basic Books, 1988.
  55. [55] C. Crawford, “The Art of Computer Game Design: Reflections of A Master Game Designer,” Osborne/Mcgraw-Hill, 1984.
  56. [56] Ministry of Economy, Trade and Industry, “Product Safety Risk Assessment, 1st edition,” Ministry of Economy, Trade and Industry, 2012 (in Japanese).
  57. [57] Y. Jiang, Y. Liu, and T. Matsumaru, “Appling Infrared Radiation Image Sensor to Step-on Interface: Touched Point Detection and Tracking,” 2012 IEEE/SICE Int. Symposium on System Integration (SII), pp. 752-757, 2012.

*This site is desgined based on HTML5 and CSS3 for modern browsers, e.g. Chrome, Firefox, Safari, Edge, IE9,10,11, Opera.

Last updated on Nov. 12, 2018