Top > JACIII > Design and Implementation of a Nonverbal Information Analysis ...

single-jc.php

JACIII Vol.30 No.4 pp. 1025-1032
(2026)
doi: 10.20965/jaciii.2026.p1025

Research Paper:

Views over last 60 days: 72

Design and Implementation of a Nonverbal Information Analysis System

Jinseok Woo*,† ORCID Icon, Chifuyu Matsumoto**, and Yuka Sone**

*Department of Mechanical Engineering, School of Engineering, Tokyo University of Technology
1404-1 Katakuramachi, Hachioji, Tokyo 192-0982, Japan

†Corresponding author

**Sustainable Engineering Program, Graduate School of Engineering, Tokyo University of Technology
1404-1 Katakuramachi, Hachioji, Tokyo 192-0982, Japan

Received:
December 2, 2025
Accepted:
January 27, 2026
Published:
July 20, 2026
Creative Commons License
Keywords:
nonverbal information, gesture analysis, human system interaction, user behaviors
Abstract

In recent years, achieving sustainable development goals has required of personalized support systems that can adapt to diverse users through natural human–system interactions. Understanding the nonverbal information of users is essential for facilitating these interactions. In this study, we focus on human gestures as a fundamental nonverbal modality that conveys user emotions and intentions. Therefore, we propose a gesture analysis system based on the relative positions of human joints and arm orientations. Using an RGB-D camera, we acquire human skeletal information and develop a gesture recognition system. Within this system, two analytical approaches are investigated: dynamic time warping-based classification method and a neural network-based classification methods. Through experimental evaluation of a small-scale dataset, we analyze the behavioral characteristics and recognition tendencies of each approach under controlled conditions. In addition, we present several cases demonstrating the effectiveness of the proposed system and discuss its applicability.

Nonverbal analysis using RGB-D camera

Nonverbal analysis using RGB-D camera

Close full text
Cite this article as:
J. Woo, C. Matsumoto, and Y. Sone, “Design and Implementation of a Nonverbal Information Analysis System,” J. Adv. Comput. Intell. Intell. Inform., Vol.30 No.4, pp. 1025-1032, 2026.
Data files:

1. Introduction

In recent years, with the advancement of robotics technology, robots have been actively used in various fields ranging from industrial applications to everyday life. In the industrial fields, robots are replacing humans in performing simple and harsh or dangerous tasks 1. Additionally, robots are also playing an active role in situations that require direct interactions with people such as indoor meal delivery and guidance services 2. Robots that interact with people are expected to not only perform simple actions but also coexist in the same space as humans and provide services adapted to human behaviors 3. Therefore, robots must interact like humans to achieve these human-adapted services. For example, when providing guidance, they should clearly indicate directions, or when retrieving an object, their movements should effectively convey their intent to the recipient 4.

The interactions between users and systems in living spaces take various forms and play crucial roles in daily life. For example, pressing a physical button can turn lights on or off, and operating a remote control with multiple buttons allows users to adjust the room temperature or change television channels, making interactions more complex 5. Additionally, touchscreen interfaces on smartphones and tablets serve as one of the most familiar examples of human–system interactions, enabling users to operate devices by directly touching the screen with their fingers. By eliminating physical buttons, these systems allow for intuitive operations while offering multifunctionality. Similarly, current robotic systems are being developed to provide simple services in response to user actions.

However, merely reacting to user actions is insufficient for robots interacting with humans to provide high-quality services 6. To enhance these services, robotic systems must be capable of more complex interactions, such as independently observing the state of users and taking appropriate actions or analyzing the intentions of users to deliver the desired service. To observe the state of users, a system must be developed that enables robots to effectively measure and assess human conditions 7. Elements that represent the state of a person include auditory information such as conversations, tactile information such as touch interactions, and visual information such as gestures and facial expressions 8. Among these, gestures are particularly significant as they serve as an indicator of the emotional state of a user 9,10. Based on this background, we considered that if robots could understand human conditions and recognize user emotional states, they could provide better services (Fig. 1).

Accordingly, we propose a gesture recognition system to analyze the upper-body gestures of users and provide services that support their daily activities. Furthermore, this study aims to develop a system capable of estimating the state of users based on nonverbal behaviors by proposing an integrated analysis system for gesture expressions. The remainder of this paper is organized as follows. Section 2 describes the content of nonverbal communication and research direction. Section 3 introduces a user analysis system that uses a dynamic time warping (DTW)-based classification method. Section 4 introduces a user analysis system that uses a neural network-based classification method. Section 5 describes the results of our experiments and conclusions, and future work is presented in Section 6.

figure

Fig. 1. Understanding users to deliver a quality service.

2. Development of a System for Estimating the User Status

Since our previous study, we have developed various systems to support users. For example, we have developed systems for supporting users, such as reviewing a heart rate-based support system while proposing a mobility system 11, and applying measurement systems to analyze user gazes and behaviors 12,13,14,15,16. Here, we review the need for user gesture analysis and explore ways to use human gestures in support systems. Therefore, this study focuses on user gestures.

figure

Fig. 2. Measuring sensor and the measurement example.

2.1. Development of a Measurement System

To recognize user gestures, we used Azure Kinect DK 17, an RGB-D camera developed by Microsoft (hereafter referred to as “camera”). Skeletal recognition was performed based on the depth information obtained from the camera, as shown in Fig. 2. The measurement environment using the sensor is illustrated in Fig. 3. To visualize the recognized skeleton and store skeletal coordinates, we developed a simulation environment using Unity 18. As illustrated in Fig. 4, skeletal coordinates were recorded at 0.06-s intervals for gesture analysis. At this stage, the skeletal coordinates were stored as two-dimensional data from the frontal perspective of the camera, rather than in three dimensions.

figure

Fig. 3. Measurement environment for the experiment.

figure

Fig. 4. Wrist coordinate-based estimation method.

2.2. Analysis System for User Gestures

According to the cognitive linguistic approach by Tomasello, humans have cooperative motivations 19. Early humans developed by cooperating in various ways for shared purposes. In this process, humans used gestures to coordinate cooperative activities, eventually leading to the emergence of a language for communication. Consequently, humans shared common goals and intentionally shared information with others. Similarly, modern humans learn from infancy to communicate their intentions through hand and foot gestures that help them express their needs, ask for help, and cooperate with others. Therefore, human gestures have important meaning, and studying them is essential to understand human nature. Based on this theoretical background, this study aims to develop a system to analyze user gestures as a means to understand the intentional and collaborative aspects of human behaviors. Accordingly, this section presents the system and architecture proposed in this study for user analysis.

2.3. Designing Gestures for Skeletal-Based Gesture Recognition

Gestures play an important role in communications involving speech such as conversations. For example, gestures used during conversations can be categorized into “representational gestures” that are directly related to speech and “beat gestures” that are not directly related to the content of speech but express their emphasis or intensity 9. Representational gestures involve movements that depict abstract concepts or concretely express their size, shape, or direction. Examples include hand movements indicating “something large” or pointing gestures that indicate a “rightward direction.” These gestures serve as supplements to verbal expressions and contribute to the interpretation of the communicative intent of the speaker. By contrast, beat gestures represent the rhythm and emphasis of speech, conveying important parts of the utterance and fluctuations in emotions. The amplitude of these gestures reflects the intensity of the emotional expression.

These gestures play an important role in human communication by conveying intentions, emphasis, and affective information beyond the spoken language. Among various gesture categories, gestures such as “waving hand,” “indicating size,” and “folding arms” considered in this study are frequently observed in daily interactions and are associated with distinct communicative functions. In particular, waving hands are commonly used to attract attention or initiate interactions, indicating that size gestures express quantitative or comparative intent, and folding arms are often linked to affective or attitudinal states. Therefore, these gestures were selected as representative examples to investigate the manner in which different types of nonverbal information could be captured and analyzed using skeletal features under controlled conditions. Based on the aforementioned considerations and to prioritize gestures that are both frequently used in everyday interactions and fundamental in the early stages of research, this study developed a system for classifying the three gesture types, as shown in Fig. 5. The “waving hand” gesture, as shown in Fig. 5(a), is a representational gesture consisting of a movement where the arm transitions from a bent position (①) to an extended position (②), repeated twice.

figure

Fig. 5. Gestures used for user analysis.

The “indicate size” gesture, as shown in Fig. 5(b), is also a representational gesture but specifically conveys a concrete indication. It involves moving the hand from a position near the head (①) in a circular motion down to the chest (②). The “folding arms” gesture, as shown in Fig. 5(c), is a beat gesture consisting of a continuous movement where the hands transition from being placed on the knees (①) to being tucked under the armpits (②).

3. User Analysis System Using the DTW Method

User gestures can be described as temporal sequences of motion features. To examine the characteristics of sequential data, DTW was applied to fixed-length gesture segments. Additionally, a neural network-based multiclass classifier was introduced as a reference model to explore the feasibility of learning-based classification under the same experimental constraints.

3.1. DTW

DTW is a method used to calculate the distance between each point of two time-series data by brute force and determine their similarity 20. If two time-series data points are represented as \(S = {s_{1}, s_{2}, \ldots, s_{n}}\) and \(T = {t_{1}, t_{2}, \ldots, t_{m}}\), the similarity to be calculated is \(\textit{DTW}(S, T)\), and matrix \(D\) used to compute \(\textit{DTW}(S, T)\) is

\begin{align} D= \begin{bmatrix} d_{0,0} & \cdots & d_{0,m} \\ \vdots & \ddots & \vdots\\ d_{n,0} & \cdots & d_{n,m} \end{bmatrix} . \end{align}
Then, each element of matrix \(D\) is
\begin{align} D_{0,0}&= 0,~ D_{i,0}= \infty, ~ D_{0,j}= \infty,\\ \end{align}
\begin{align} D_{i,j}&= \delta(i,j) + \min \begin{cases} D_{i-1,j-1}, \\ D_{i-1,j}, \\ D_{i,j-1}, \end{cases}\\ \end{align}
\begin{align} \delta(i,j)&= \left|s_i-t_j\right|, \notag\\ (i&=0,1,2, \ldots, n; ~~ j=0,1,2, \ldots, m). \end{align}
Each element of matrix \(D\) is represented as \(\delta(i,j)\); \(\delta(i,j)\) represents the absolute error between the points of \(S\) and \(T\). The value of \(\textit{DTW}(S, T)\) is obtained at \(D_{n,m}\). Furthermore, the alignment of points between \(S\) and \(T\) can be obtained by tracing from \(D_{n,m}\) back to \(D_{0,0}\).

3.2. DTW Barycenter Averaging (DBA)

To calculate the average of three or more time-series data, the DTW DBA algorithm was used 21. When three time-series data existed, one was randomly selected from the data group as a time series \(A = \{a_{1}, a_{2}, \ldots, a_{n}\}\) that served as the initial value for calculating the barycenter. The remaining time-series data were denoted as \(S = \{s_{1}, s_{2}, \ldots, s_{m}\}\) and \(T = \{t_{1}, t_{2}, \ldots, t_{l}\}\). Next, DTW was performed between \(A\), \(S\), and \(T\), and the barycenter \(\{b_{1}, b_{2}, \ldots, b_{n}\}\) was calculated for each point in \(A\) and the corresponding points in \(S\) and \(T\). Data \(B\), formed by connecting the barycenter points, became the barycenter time series of the data group. Using \(B\) as the new initial value and repeating the same procedure, the true barycenter was approached.

3.3. Time-Series \(k\)-Means

We used the \(k\)-means algorithm to cluster the gestures 22. In standard \(k\)-means, the distance between each data point and the centroid is calculated, and the data points are assigned to the cluster with the closest centroid. In DTW-based time-series \(k\)-means, DTW was applied between the centroid of each cluster and each time-series instance. Then, each time series was assigned to the cluster with the smallest DTW distance. Additionally, the centroid of each cluster was computed using DBA. This procedure enabled clustering based on the temporal similarity of time-series patterns rather than point-wise distance measures.

3.4. Gesture Recognition Analysis Using the DTW-Based Model

Based on the clustering results, validation data were assigned to gesture classes according to their similarity to the cluster centroids. The validation dataset was independent of the clustering data and consisted of five recordings per gesture. The number of clusters in the \(k\)-means was set to three using the elbow method 23.

figure

Fig. 6. Clustering results for each gesture.

For validation, the centroid of each cluster \(B_i=\{b_1,b_2,…,b_n\}\) was calculated using DBA. Then, DTW was applied between each centroid and test data \(V_i=\{v_1,v_2,\ldots,v_n\}\). The similarity, \(\textit{DTW}(B_i,V)\), was computed, and total similarity \(S\) with each cluster was calculated using Eq. 5. Then, prediction probability \(P(B_i,V)\) was obtained using Eq. 6. Here, \(i\) denotes the index of the cluster, and because three gestures were considered, \(i=0,1,2\).

\begin{align} S&= \sum_{i=0}^2 \textit{DTW}\left(B_i,V\right) , \label{eq:s}\\ \end{align}
\begin{align} P\left(B_i,V\right)&= \frac{\displaystyle\frac{S}{\textit{DTW}\left(B_i,V\right)}}{\displaystyle\sum_{i=0}^2 \displaystyle\frac{S}{\textit{DTW}\left(B_i,V\right)}}. \label{eq:p} \end{align}
The clustering results are shown in Fig. 6, where each cluster is color-coded. Yellow corresponds to the “waving hand” cluster, light green to “indicating size,” and blue to “folding arms.” The predicted probabilities ranged from approximately \(60\%\) to \(70\%\) across the three gesture classes, with “waving hand” and “indicating size” showing relatively higher values than “folding arms.” These values reflected the relative confidence levels within the clustering results under the current experimental conditions.

4. Consideration of Methods for Gesture Classification

This section presents a gesture classification system designed to operate under constrained conditions, with a focus on minimizing the handling of personal information and enabling offline use.

4.1. System Architecture of the User Analysis Model

figure

Fig. 7. Architecture of the neural network-based gesture classification model, illustrating the input layer, six hidden layers with ReLU activation, and the output layer with softmax activation.

The neural network-based gesture classification model architecture is shown in Fig. 7. The architecture consisted of an input layer with 180 nodes, six hidden layers, and an output layer corresponding to the three gesture classes. Each gesture was recorded 15 times, resulting in 45 samples. The input layer received fixed-length feature vectors derived from the wrist coordinate displacements over a predefined time window.

The purpose of explicitly defining the input, hidden, and output layers in this architecture was to reproduce the implementation. The present configuration was adopted to illustrate the manner in which fixed-length skeletal features were processed within a neural network-based gesture classification system under controlled experimental conditions. Additionally, the architectural design, as shown in Fig. 7, was used as a reference configuration for comparative analysis, and the reported classification behavior was interpreted using the dataset size and experimental constraints, as described in Section 5. Each hidden layer in this architecture used the ReLU activation function to model the nonlinear relationships between the input features, and the output layer applied the softmax function to generate the probability distributions for each class. This architecture was designed to examine the feasibility of applying a standard feedforward neural network to structured gesture data under controlled input conditions, rather than to optimize the model complexity or generalization performance.

The input layer processed the images captured at 30 fps using a camera. From these camera images, skeleton estimation was performed using MediaPipe 24. The estimated wrist coordinates were saved for analysis. The input layer of the model received the displacement of the wrist coordinates. Each gesture was recorded in 3-s segments, with MediaPipe operating at 30 fps. Because two values, \(\Delta x\) and \(\Delta y\), were used, the number of nodes in the input layer was calculated as \(3 \times 30 \times 2 = 180\). Each gesture was recorded 15 times and used for analysis. In this case, the activation functions used were the ReLU function (Eq. 7) 25 and the softmax (Eq. 8) 26.

\begin{align} f\left(x_i\right)&= \begin{cases} 0~~ \left(x_i < 0\right), \\ x_i~~ \left(x_i \geq 0\right), \end{cases} \label{eq:relu}\\ \end{align}
\begin{align} f\left(x_i\right)&= \frac{\exp\left(x_i\right)}{\displaystyle\sum_{j=1}^n \exp\left(x_j\right)}. \label{eq:softmax} \end{align}
Here, \(x_i\) represents the value of the node, and \(n\) denotes the number of nodes in the layer. The loss was calculated using the cross-entropy error (Eq. 9). ADAM was used as the optimization method 27.
\begin{equation} E= -\sum_{k} T_k \log y_k. \label{eq:adam} \end{equation}
Here, \(k\) represents the number of categories, \(T_k\) denotes the true value (zero or one), and \(y_k\) denotes the predicted probability.

4.2. System Overview for Gesture Analysis

To examine the gesture classification, multiclass classification was performed using a neural network trained using the backpropagation method. PyTorch version 2.2.2 was used for the implementation. The specifications of the PC are presented in Table 1.

Table 1. Hardware specifications of the PC.

figure

5. Experimental Evaluation

This section analyzes the observations derived from the aforementioned experiments.

5.1. Analysis of Neural Network-Based Gesture Classification

The input layer periodically captured images using a camera, and skeletal estimation was performed on these images using MediaPipe (Fig. 8(a)). Then, the estimated wrist coordinates were recorded for gesture analysis (Fig. 8(b)). At this stage, the wrist coordinates were defined with the top-right corner of the captured image as the origin, resulting in an inverted graph when visualized, as shown in Fig. 8(c). Finally, the model was trained using the architecture shown in Fig. 8(d).

figure

Fig. 8. Overview of the processing pipeline from pose estimation to arm coordinate features.

For the design of the hidden layers, as shown in Fig. 7, this study compared the number of layers and, as shown in Fig. 9, constructed the system using a six-layer structure with low noise and stability.

figure

Fig. 9. Changes in the training loss owing to differences in the number of layers.

figure

Fig. 10. Comparison of the number of training iterations until convergence when changing the batch size.

The batch size, that was the size of each group when the dataset was divided into multiple groups, was set to 5 because it resulted in the fastest convergence (Fig. 10).

Regarding the probabilities predicted by the neural network model, relatively consistent confidence patterns were observed across the three gesture classes in the evaluated dataset. These observations indicated that the predefined gesture categories were distinguishable under the controlled experimental conditions considered in this study.

5.2. Comparative Discussion of the Two Recognition Approaches

DTW focused on the shape of the time-series data and calculated the distance between each point in the time series through an exhaustive search to determine similarity. This allowed for computation even when the lengths of the time series differed. For the same reason, DTW was less affected by variations in the gesture movement speed. However, in the current method using \(k\)-means, factors such as the size of the gesture and the location where the gesture was performed (for example, waving the hand above the head versus near the chest) could lower the similarity, leading to a decrease in the recognition accuracy.

However, the neural network-based approach allowed the model to incorporate multiple feature dimensions related to the gesture size and spatial location that could provide flexibility under controlled variations in these factors. However, because the input dimensionality was fixed, this approach was not directly applicable to time-series data of varying lengths without additional preprocessing or architectural modifications. Therefore, although differences existed between the two methods, DTW served as an effective analytical approach for handling variations in the time-series length. In contrast, for data with relatively similar structural patterns, a neural network-based analysis could be considered a suitable option under controlled experimental conditions.

5.3. Implications and Limitations of Each System

5.3.1. DTW-Based System

To use human gesture information, this study proposed a gesture recognition method based on clustering using time-series \(k\)-means. Three types of gestures were considered for recognition: waving hand, indicating size, and folding arms. For each gesture, the resulting clusters were used to classify the test data representing one among the three gestures by outputting probabilistic values for the cluster assignment.

The experimental results indicated that the recognition accuracy for each gesture exceeded \(60\%\). The proposed method could successfully discriminate between the gestures; however, because only three types of gestures were considered, experiments had to be conducted based on a more diverse set of gestures. Furthermore, the recognition accuracy for user gestures must be enhanced in future studies. These limitations should be addressed in future studies.

5.3.2. Neural Network-Based System

A system capable of multiclass classification using a neural network was developed to improve the gesture recognition accuracy. Using this system, consistent class separation was observed within a small-scale pilot dataset. However, because only three gesture types were considered and the training dataset consisted of 45 samples, the present results should be interpreted as preliminary observations. Further investigation is required to assess the scalability and robustness when both the dataset size and gesture diversity are increased.

5.3.3. Manual Segmentation and System Extensibility

In this study, gesture segments were manually defined with a fixed duration to isolate the gesture recognition performance from temporal detection errors. This design choice was adopted to focus the analysis on feature representation and classifier behaviors without introducing additional uncertainty arising from the gesture boundary estimation. By decoupling segmentation from recognition, the comparative characteristics of DTW-based clustering and neural network-based classification could be examined under controlled conditions.

Nevertheless, we acknowledge that manual segmentation limits direct applicability in real-world human–system interaction scenarios in which gestures occur continuously and with variable durations. Several algorithmic extensions can be considered to address this limitation. One feasible approach is gesture spotting using sliding-window analysis combined with similarity thresholds, in which DTW can naturally handle variable-length sequences 28. Another approach involves detecting motion onset and offset based on velocity or acceleration changes in joint trajectories, thereby enabling automatic temporal segmentation prior to recognition 29. For neural network-based methods, variable-length inputs may be handled by incorporating temporal models such as recurrent neural networks 30 or attention-based architectures 31, that can process sequential data without fixed-length constraints.

6. Conclusion and Future Works

This study proposed a gesture analysis system for capturing human nonverbal behavioral cues using skeletal information derived from joint positions and arm orientations. Focusing on three representative gesture types—waving hand, indicating size, and folding arms—we implemented and examined two analytical approaches: DTW-based time-series clustering and neural network-based multiclass classification. Through a small-scale experimental evaluation, we observed that both approaches could distinguish among the selected gestures under controlled conditions. These observations highlighted the complementary characteristics of the two methods rather than the definitive performance superiority. Given the limited number of gesture classes and training samples, the reported results should be interpreted within this constrained scope.

Future work will focus on expanding the gesture set, increasing the dataset diversity, and incorporating automatic segmentation techniques to enable continuous gesture recognition. Furthermore, integrating additional nonverbal modalities such as facial expressions will be an important step toward more comprehensively estimating user states and developing context-aware human–system interaction systems.

Acknowledgments

This work was supported by JSPS KAKENHI (Grant-in-Aid for Early-Career Scientists) (Grant Number 23K17261).

References
  1. [1] C. Wong, E. Yang, X.-T. Yan, and D. Gu, “An overview of robotics and autonomous systems for harsh environments,” 2017 23rd Int. Conf. on Automation and Computing (ICAC), 2017. https://doi.org/10.23919/IConAC.2017.8082020
  2. [2] S. Yamamoto, J. Woo, W. H. Chin, K. Matsumura, and N. Kubota, “Interactive information support by robot partners based on informationally structured space,” J. Robot. Mechatron., Vol.32, No.1, pp. 236-243, 2020. https://doi.org/10.20965/jrm.2020.p0236
  3. [3] R. Jahanmahin, S. Masoud, J. Rickli, and A. Djuric, “Human-robot interactions in manufacturing: A survey of human behavior modeling,” Robotics and Computer-Integrated Manufacturing, Vol.78, Article No.102404, 2022. https://doi.org/10.1016/j.rcim.2022.102404
  4. [4] T. Kanda, M. Kamasima, M. Imai, T. Ono, D. Sakamoto, H. Ishiguro, and Y. Anzai, “A humanoid robot that pretends to listen to route guidance from a human,” Autonomous Robots, Vol.22, pp. 87-100, 2007. https://doi.org/10.1007/s10514-006-9007-6
  5. [5] B. B. Lonkar, A. Kuthe, R. Shrivastava, and P. Charde, “Design and implement smart home appliances controller using IOT,” Information Systems and Management Science: Conf. Proc. of 3rd Int. Conf. on Information Systems and Management Science (ISMS) 2020, pp. 105-117, 2022. https://doi.org/10.1007/978-3-030-86223-7_11
  6. [6] S. K. Roy, G. Singh, R. L. Gruner, B. L. Dey, S. Shabnam, S. S. Muhammad, and M. Quaddus, “Relationship quality in customer-service robot interactions in Industry 5.0: An analysis of value recipes,” Information Systems Frontiers, 2023. https://doi.org/10.1007/s10796-023-10445-y
  7. [7] Y. Wang, H. Wu, G. Tian, G. Liu, F. Lu, and Y. Wang, “Fast search strategy for robots in dynamic home environment,” J. Adv. Comput. Intell. Intell. Inform., Vol.26, No.3, pp. 315-324, 2022. https://doi.org/10.20965/jaciii.2022.p0315
  8. [8] P. Bull and J. P. Doody, “Gesture and body movement,” J. A. Hall and M. L. Knapp (Eds.), “Nonverbal Communication,” pp. 205-228, 2013. https://doi.org/10.1515/9783110238150.205
  9. [9] S. Kita, “Why do people make gesture?,” Cognitive Studies, Vol.7, No.1, pp. 9-21, 2000 (in Japanese). https://doi.org/10.11225/jcss.7.9
  10. [10] W.-F. Hsieh, E. Sato-Shimokawara, and T. Yamaguchi, “Investigation of robot expression style in human-robot interaction,” J. Robot. Mechatron., Vol.32, No.1, pp. 224-235, 2020. https://doi.org/10.20965/jrm.2020.p0224
  11. [11] J. Woo, T. Sato, and Y. Ohyama, “Development of a human-centric system using an IoT-based socially embedded robot partner,” J. Robot. Mechatron., Vol.35, No.3, pp. 859-866, 2023. https://doi.org/10.20965/jrm.2023.p0859
  12. [12] C. Matsumoto, Y. Sone, J. Woo, and Y. Ohyama, “Development of a gesture analysis system for human system interaction,” 2024 Joint 13th Int. Conf. on Soft Computing and Intelligent Systems and 25th Int. Symp. on Advanced Intelligent Systems (SCIS&ISIS), 2024. https://doi.org/10.1109/SCISISIS61014.2024.10760070
  13. [13] Y. Sone, C. Matsumoto, J. Woo, and Y. Ohyama, “Development of a control support system for smart homes using the analysis of user interests based on mixed reality,” ROBOMECH J., Vol.12, Article No.2, 2025. https://doi.org/10.1186/s40648-024-00288-w
  14. [14] J. Woo and J. Hu, “System for analyzing user interest based on eye gaze responses to enhance empathy with users,” J. Adv. Comput. Intell. Intell. Inform., Vol.29, No.3, pp. 641-648, 2025. https://doi.org/10.20965/jaciii.2025.p0641
  15. [15] J. Woo and J. Hu, “System for analyzing user interest based on eye gaze responses to enhance empathy with users,” J. Adv. Comput. Intell. Intell. Inform., Vol.29, No.3, pp. 641-648, 2025. https://doi.org/10.20965/jaciii.2025.p0641
  16. [16] Y. Sone and J. Woo, “Design of a human-centric robotic system for user support based on gaze information,” J. Adv. Comput. Intell. Intell. Inform., Vol.29, No.4, pp. 796-802, 2025. https://doi.org/10.20965/jaciii.2025.p0796
  17. [17] M. Servi, A. Profili, R. Furferi, and Y. Volpe, “Comparative evaluation of Intel RealSense D415, D435i, D455 and Microsoft Azure Kinect DK Sensors for 3D vision applications,” IEEE Access, Vol.12, pp. 111311-111321, 2024. https://doi.org/10.1109/ACCESS.2024.3441238
  18. [18] A. Hussain, H. Shakeel, F. Hussain, N. Uddin, and T. L. Ghouri, “Unity game development engine: A technical survey,” Univ. Sindh J. Inf. Commun. Technol., Vol.4, No.2, pp. 73-81, 2020.
  19. [19] M. Tomasello, “Origins of Human Communication,” MIT Press, 2010.
  20. [20] M. Müller, “Dynamic time warping,” Information Retrieval for Music and Motion, pp. 69-84, 2007. https://doi.org/10.1007/978-3-540-74048-3_4
  21. [21] F. Petitjean, A. Ketterlin, and P. Gançarski, “A global averaging method for dynamic time warping, with applications to clustering,” Pattern Recognition, Vol.44, No.3, pp. 678-693, 2011. https://doi.org/10.1016/j.patcog.2010.09.013
  22. [22] X. Huang, Y. Ye, L. Xiong, R. Y. Lau, N. Jiang, and S. Wang, “Time series k-means: A new k-means type smooth subspace clustering for time series data,” Information Sciences, Vol.367, pp. 1-13, 2016. https://doi.org/10.1016/j.ins.2016.05.040
  23. [23] M. Cui et al., “Introduction to the k-means clustering algorithm based on the elbow method,” Accounting, Auditing and Finance, Vol.1, No.1, pp. 5-8, 2020. https://dx.doi.org/10.23977/accaf.2020.010102
  24. [24] C. Lugaresi, J. Tang, H. Nash, C. McClanahan, E. Uboweja, M. Hays, F. Zhang, C.-L. Chang, M. G. Yong, J. Lee et al., “MediaPipe: A framework for building perception pipelines,” arXiv:1906.08172, 2019. https://doi.org/10.48550/arXiv.1906.08172
  25. [25] J. Liu, L. Zhong, J. Wickramasuriya, and V. Vasudevan, “uWave: Accelerometer-based personalized gesture recognition and its applications,” Pervasive and Mobile Computing, Vol.5, No.6, pp. 657-675, 2009. https://doi.org/10.1016/j.pmcj.2009.07.007
  26. [26] X. Zhang, X. Chen, Y. Li, V. Lantz, K. Wang, and J. Yang, “A framework for hand gesture recognition based on accelerometer and emg sensors,” IEEE Trans. on Systems, Man, and Cybernetics – Part A: Systems and Humans, Vol.41, No.6, pp. 1064-1076, 2011. https://doi.org/10.1109/TSMCA.2011.2116004
  27. [27] F. Farhadi-Niaki, J. Gerroir, A. Arya, S. A. Etemad, R. Laganiére, P. Payeur, and R. Biddle, “Usability study of static/dynamic gestures and haptic input as interfaces to 3D games,” Proc. of the 6th Int. Conf. on Advances in Computer-Human Interactions, pp. 315-323, 2013.
  28. [28] Z. Tang, J. Ma, W. Liu, P. Zhang, and F. Lv, “Sliding window dynamic time-series warping-based ultrasonic guided wave temperature compensation and defect monitoring method for turnout rail foot,” IEEE Trans. on Ultrasonics, Ferroelectrics, and Frequency Control, Vol.69, No.9, pp. 2681-2695, 2022. https://doi.org/10.1109/TUFFC.2022.3195933
  29. [29] J. Lee, T. Ahmed, T. Rikakis, and P. Turaga, “Automatic temporal segmentation for post-stroke rehabilitation: A keypoint detection and temporal segmentation approach for small datasets,” Proc. of the Winter Conf. on Applications of Computer Vision, pp. 21-29, 2025. https://doi.org/10.1109/WACVW65960.2025.00008
  30. [30] H. Liu, A. Panahi, D. Andrews, and A. Nelson, “An fpga-based upper-limb rehabilitation device for gesture recognition and motion evaluation using multi-task recurrent neural networks,” IEEE Sensors J., Vol.22, No.4, pp. 3605-3615, 2022. https://doi.org/10.1109/JSEN.2022.3141659
  31. [31] E. Rahimian, S. Zabihi, A. Asif, D. Farina, S. F. Atashzar, and A. Mohammadi, “Hand gesture recognition using temporal convolutions and attention mechanism,” 2022-2022 IEEE Int. Conf. on Acoustics, Speech and Signal Processing (ICASSP), pp. 1196-1200, 2022. https://doi.org/10.1109/ICASSP43922.2022.9746174

Creative Commons License  This article is published under a Creative Commons Attribution-NoDerivatives 4.0 Internationa License.

*This site is desgined based on HTML5 and CSS3 for modern browsers, e.g. Chrome, Firefox, Safari, Edge, Opera.

Last updated on Jul. 19, 2026