This study proposes a human tracking algorithm that combines vision-based object detection and body heat tracking. The small autonomous mobile robot developed in this study is equipped with an ultracompact camera and three thermal array sensors. We propose a hybrid human tracking method based on camera images and body heat information. In particular, the proposed machine-learning-based image recognition method employs Faster Objects, More Objects, an object detection algorithm optimized for resource-constrained edge devices such as microcontrollers. This algorithm generates an inference model focusing on the clothing features of the target person, which is implemented in the robot control system. Furthermore, a human-robot distance estimation model is derived to calculate the tracking distance between the target person and the robot using a monocular camera. Experiments demonstrated that the proposed inference model enables robust tracking even when the target is partially occluded by another person. Additionally, the body-heat-distribution-based tracking method employs the summation of multiple thermal pixels to improve tracking performance. In this study, tracking performance was improved by emphasizing the target region in the thermal image using four-pixel, nine-pixel, and one-column pixel summation methods. The experimental results demonstrate that the four-pixel summation method, which minimizes fluctuations in the robot posture angle during motion, is the most suitable for the proposed tracking algorithm. The proposed algorithm is effective for hybrid control in environments where illumination conditions change abruptly between bright and dark areas.

1. [1] R. Tasaki, M. Kitazaki, J. Miura, and K. Terashima, “Prototype design of medical round supporting robot ‘Terapio’,” IEEE Int. Conf. Robotics and Automation, pp. 829-834, 2015. https://doi.org/10.1109/ICRA.2015.7139274
2. [2] S. Mahajan and C. M. Vidhyapathi, “Design of a medical assistant robot,” IEEE Int. Conf. Recent Trends in Electronics, Information & Communication Technology, pp. 877-881, 2017. https://doi.org/10.1109/RTEICT.2017.8256723
3. [3] H. Yoo, D. Kim, J. Sohn, K. Lee, and C. Kim, “Development of a worker-following robot system: Worker position estimation and motion control under measurement uncertainty,” Machines, Vol.11, No.3, Article No.366, 2023. https://doi.org/10.3390/machines11030366
4. [4] A. Eirale, M. Martini, and M. Chiaberge, “Human following and guidance by autonomous mobile robots: A comprehensive review,” IEEE Access, Vol.13, pp. 42214-42253, 2025. https://doi.org/10.1109/ACCESS.2025.3548134
5. [5] Y. Min, J. Shui, S. Liu, J. Wang, and Z. Guo, “Human target detection and tracking algorithm based on improved YOLOv7,” 2023 IEEE 13th Int. Conf. CYBER Technology in Automation, Control, and Intelligent Systems, pp. 297-302, 2023. https://doi.org/10.1109/CYBER59472.2023.10256546
6. [6] Y. Zhao, Y. Gao, Q. Sun, Y. Tian, L. Mao, and F. Gao, “A real-time low-computation cost human-following framework in outdoor environment for legged robots,” Robotics and Autonomous Systems, Vol.146, Article No.103899, 2021. https://doi.org/10.1016/j.robot.2021.103899
7. [7] A. Farooq and M. Imran, “Innovating vending machine through object detection for currency recognition and enhanced cash handling,” 7th Int. Symp. Multidisciplinary Studies and Innovative Technologies, 2023. https://doi.org/10.1109/ISMSIT58785.2023.10304989
8. [8] Q. M. Nguyen, V. T. Le, M. N. Lai, and H. B. Vo, “Resource-constrained intelligent trap: Fruit flies surveillance framework with TinyML integration,” 10th Int. Conf. Communications and Electronics, pp. 415-420, 2024. https://doi.org/10.1109/ICCE62051.2024.10634657
9. [9] R. Alaa, H. Al-Libawy, and E. A. Hussein, “Low-cost blind spot detection system based on lite object detection algorithm and limited resources hardware,” 4th Int. Conf. Science and Information Technology in Smart Administration, pp. 469-474, 2024. https://doi.org/10.1109/ICSINTESA62455.2024.10747868
10. [10] H. V. L. B. Silva, F. A. P. de Figueiredo, and S. B. Mafra, “Performance evaluation of edge computing object detection models for maritime surveillance on a Raspberry Pi,” 2024 IEEE Latin-American Conf. on Communications (LATINCOM), 2024. https://doi.org/10.1109/LATINCOM62985.2024.10770682
11. [11] J. L. R. Ronda et al.  “Real-time path hole detection for motorcycle driver safety assistance using Faster Object More Object (FOMO) algorithm,” TENCON IEEE Region 10 Conf., pp. 1746-1750, 2024. https://doi.org/10.1109/TENCON61640.2024.10902942
12. [12] F. G. F. Rocha, H. V. L. B. Silva, R. B. Vimieiro, and F. A. P. de Figueiredo, “Defect detection in printed circuit boards based on EdgeML and computer vision,” Int. Conf. Intelligent Cybernetics Technology & Applications, pp. 573-577, 2024. https://doi.org/10.1109/ICICYTA64807.2024.10913437
13. [13] N. Arul, V. Chaitanya, S. Deepana, S. Gowsalya, and P. Lathasree, “Design and development of autonomous ball collecting machine using FOMO algorithm,” Int. Conf. Computing and Communication Technologies, 2025. https://doi.org/10.1109/ICCCT63501.2025.11019447
14. [14] K. Li, Y. Wang, and Z. Hu, “Improved YOLOv7 for small object detection algorithm based on attention and dynamic convolution,” Applied Sciences, Vol.13, No.16, Article No.9316, 2023. https://doi.org/10.3390/app13169316
15. [15] Q. Su, Y. Zhang, P. Wang, G. Liu, L. Zhu, and W. Li, “Research on human following technology of robot based on ECO-HC,” Int. Conf. Mechatronics Technology and Intelligent Manufacturing, pp. 347-351, 2023. https://doi.org/10.1109/ICMTIM58873.2023.10246498
16. [16] T. Doi, A. Mizuta, and K. Nagumo, “Harmful animal detection using visual information for wire-type mobile robots,” J. Robot. Mechatron., Vol.37, No.3, pp. 742-751, 2025. https://doi.org/10.20965/jrm.2025.p0742
17. [17] M. Akiyama, H. Ninomiya, and T. Saito, “Method to achieve high speed and high recognition rate of goal from long distance for CanSat,” J. Robot. Mechatron., Vol.35, No.1, pp. 194-205, 2023. https://doi.org/10.20965/jrm.2023.p0194
18. [18] M. A. K. Niloy et al.  “Critical design and control issues of indoor autonomous mobile robots: A review,” IEEE Access, Vol.9, pp. 35338-35370, 2021. https://doi.org/10.1109/ACCESS.2021.3062557
19. [19] S. H. Tsai, L. H. Kao, H. Y. Lin, T. C. Lin, Y. L. Song, and L. M. Chang, “A sensor fusion based nonholonomic wheeled mobile robot for tracking control,” Sensors, Vol.20, No.24, Article No.7055, 2020. https://doi.org/10.3390/s20247055
20. [20] M. B. Alatise and G. P. Hancke, “A review on challenges of autonomous mobile robot and sensor fusion methods,” IEEE Access, Vol.8, pp. 39830-39846, 2020. https://doi.org/10.1109/ACCESS.2020.2975643
21. [21] S. A. Ahmed, A. V. Topalov, N. G. Shakev, and V. L. Popov, “Model-free detection and following of moving objects by an omnidirectional mobile robot using 2D range data,” IFAC-PapersOnLine, Vol.51, Issue 22, pp. 226-231, 2018. https://doi.org/10.1016/j.ifacol.2018.11.546
22. [22] B. Shubha and V. V. D. Shastrimath, “Real-time occupancy detection system using low-resolution thermopile array sensor for indoor environment,” IEEE Access, Vol.10, pp. 130981-130995, 2022. https://doi.org/10.1109/ACCESS.2022.3229895
23. [23] M. Gochoo, T. H. Tan, T. Batjargal, O. Seredin, and S. C. Huang, “Device-free non-privacy invasive indoor human posture recognition using low-resolution infrared sensor-based wireless sensor networks and DCNN,” IEEE Int. Conf. Systems, Man, and Cybernetics, pp. 2311-2316, 2018. https://doi.org/10.1109/SMC.2018.00397
24. [24] A. Géczy, R. D. Jorge Melgar, A. Bonyar, and G. Harsanyi, “Passenger detection in cars with small form-factor IR sensors (Grid-eye),” 2020 IEEE 8th Electronics System-Integration Technology Conf., 2020. https://doi.org/10.1109/ESTC48849.2020.9229693
25. [25] A. F. Alraeesi, H. F. Kharbash, J. S. Alghfeli, S. S. Alsaedi, and M. Gochoo, “Privacy-preserved social distancing system using low-resolution thermal sensors and deep learning,” 2021 IEEE Int. Conf. Systems, Man, and Cybernetics, pp. 66-71, 2021. https://doi.org/10.1109/SMC52423.2021.9659292
26. [26] M. I. Rusydi, A. Novira, T. Nakagome, J. Muguro, R. Nakajima, W. Njeri, K. Matsushita, and M. Sasaki, “Autonomous movement control of coaxial mobile robot based on aspect ratio of human face for public relation activity using stereo thermal camera,” J. of Robotics and Control, Vol.3, Issue 3, pp. 361-373, 2022. https://doi.org/10.18196/jrc.v3i3.14750
27. [27] N. Bharathiraja, R. B. Indhuja, P. R. A. Krishnan, S. Anandhan, and S. Hariprasad, “Real-time fall detection using ESP32 and AMG8833 thermal sensor: A non-wearable approach for enhanced safety,” 2023 Second Int. Conf. Augmented Intelligence and Sustainable Systems, pp. 1732-1736, 2023. https://doi.org/10.1109/ICAISS58487.2023.10250598
28. [28] H. Garg, A. K. Bhartee, A. Rai, M. Kumar, and A. Dhakrey, “A review of object detection algorithms for autonomous vehicles: Trends and developments,” 5th Int. Conf. Advances in Computing, Communication Control and Networking, pp. 1173-1181, 2023. https://doi.org/10.1109/ICAC3N60023.2023.10541773
29. [29] J. Da Silva, T. Flores, S. Junior, and I. Silva, “TinyML-based pothole detection: A comparative analysis of YOLO and FOMO model performance,” 2023 IEEE Latin American Conf. Computational Intelligence (LA-CCI), 2023. https://doi.org/10.1109/LA-CCI58595.2023.10409357
30. [30] T. Fukushima, K. Itoya, and T. Inoue, “WiFi-communicated cooperative platoon running of multiple mobile robots using image machine learning with FOMO,” IEEE Int. Conf. Real-time Computing and Robotics, TuB2(2), 2025. https://doi.org/10.1109/RCAR65431.2025.11139496
31. [31] T. Inoue, Y. Okazaki, and K. Itoya, “Person following algorithm with pixel-area addition method of thermal sensors for autonomous mobile robots,” 10th Int. Conf. Control, Automation and Robotics, pp. 77-82, 2024. https://doi.org/10.1109/ICCAR61844.2024.10569955
32. [32] T. Inoue and Y. Okazaki, “Human-following control of omnidirectional autonomous mobile robot by integrating sensor information,” 23rd Int. Conf. Control, Automation and Systems, pp. 1055-1059, 2023. https://doi.org/10.23919/ICCAS59377.2023.10317018
33. [33] K. Itoya and T. Inoue, “Image machine-learning based person following under sequential occlusions,” Int. Conf. Control, Automation and Systems, WeAT2.4, 2025.