1. Introduction

Tactile Internet uses haptic devices to transmit tactile sensations ^1,2. With the development of the internet and the availability of sufficient bandwidth, the performance of haptic devices is considered important because device specifications directly determine the comfort of wearing and the tactile sensations that can be transmitted. Various haptic devices using pressure ^3,4,5,6,7, vibration ^8,9,10, etc. have been investigated and developed. However, conventional rigid exoskeletons are heavy and uncomfortable. Therefore, research on haptic devices that use soft robots is being actively conducted ^{11,12,13,14,15,16,17,18,19,20}.

Funabot is a soft actuator device ^21,22,23. The Funabot consists of a base fabric and McKibben-type pneumatic artificial muscles (EM20, s-muscle Co., Ltd., Japan) fixed to the base fabric. It is flexible and freely deformable and can be used as a haptic device. By specializing in Funabot for Grabbed tactile, we developed Funabot-Grab ²⁴, which can control the distribution with force distribution sensors. The Funabot-Grab can independently control the tightening force around the arm in the longitudinal direction, with seven channels. The width of each channel is 2.2 cm. Previous studies have shown that it is possible to achieve a tightening force of 7.9 N for each channel by using a model arm and that the error from the command value is kept to within 0.21 N. On the other hand, there was room for improvement in the response performance, and no experiments were conducted with subjects wearing the device.

This study investigates the feasibility of a tactile internet system through experiments involving human subjects. As a preliminary step toward a full-scale experiment with human subjects, we conducted a tactile internet experiment with three subjects involved in the research.

As a result of the experiment, the subjects were able to reproduce the peaks at the locations where they were grabbed within an error of one channel (2.2 cm). In addition, all the subjects were able to perceive the movement of the locations where they were grabbed.

However, several issues remain to be resolved. Unlike the models used in a previous study ²⁴, the actual arm is difficult to wear the device ideally because the shape of a subjects’ arm varies from person to person, with differences in hardness and length.

In the future, we will consider a mechanism that allows all the subjects to wear the device under the same conditions, and we will increase the channel density to smoothen out continuous locational changes.

2. Problem Statement

Of the two remaining issues from the previous study ²⁴, namely, room for improvement in response performance and experiments conducted with subjects wearing the device, this study aims to investigate the feasibility of tactile internet via experiments with human subjects. The feasibility of tactile internet with this device was verified by transmitting the grabbed tactile to multiple subjects who had no prior experience wearing the device and by assessing how accurately the subjects perceived the location and size of the transmitted tactile sensation. We also verified whether the force-tracking performance achieved in a previous study ²⁴ when the device was attached to a mannequin could be similarly reproduced when the device was attached to a human. The verification methods and their validity are discussed in Section 4.

The following will not be considered in the experiments, based on the position taken in this study. First, the following experiments were conducted as a preliminary verification before employing the device in a specific application; therefore, transmission delays were not considered. Second, mechanical improvements were made to facilitate the attachment and detachment of the device for multiple subjects; however, no changes were made to the control performance compared to the previous version ²⁴. The details of the device configuration are described in the next section.

3. Device

The operating principle and control method are the same as those in previous versions ²⁴, but the design has been changed to make it easier to attach and detach for use in experiments with human subjects. Fig. 1 shows the design overview and appearance of the actual device. The \(12 \times 7\) cell force distribution sensor was covered with cloth, and artificial muscles were sewn onto the inner (the face that comes into contact with the sensor) of the outer side cloth. There are seven channels of artificial muscles every 2.2 cm, and the tightening force around the arm can be controlled independently in the direction of the upper arm lengthwise. The outer Velcro tape made it easy to wrap around the arm and attach to it. A numerical version of the SR sensor (Sumitomo Riko Company Limited ^a) was used as the force distribution sensor. The specifications of the sensors are listed in Table 1. For further details, please refer to the previous paper ²⁴.

Figure 2 shows the appearance of wearing. The artificial muscles are sewn on the inside; therefore, the participants could not see them moving during the experiment.

4. Experiment

The experiment was conducted to confirm whether Funabot-Grab could transmit the grabbed tactile between the experimenter and the subject, and whether the subject could perceive it.

The experiment involved the transmission of both static and dynamic tactile sensations. Using the two-point discrimination capacity as a criterion, the experiment examined whether the difference in the peak force applied by the subject to the mannequin sensor compared with the force read by the reader was small or significantly larger than the two-point discrimination capability of the upper arm (20 mm ²⁵). The spatial control resolution of the device used in the experiments was based on the measurement resolution of the sensor. Because the sensor resolution is 2.2 cm wide per channel, if the peak error is within one channel, it means that the tactile position can be recognized within the range of two-point discrimination ability in humans, relative to the control resolution of the device. For dynamic tactile transmission, a questionnaire was administered to determine whether the subject felt the sensation continuously. The tracking performance of the tactile sensation of the device provided to the subject in relation to the tactile input from the reader was evaluated based on whether the output of the device tracked the input to the reader within a tracking error of 0.21 N, achieved in a previous study.

The Ethics Committee of the Graduate School of Engineering, Nagoya University, approved the experiments (Approval No.22-31).

4.1. System Architecture

Figure 3 shows an overview of the system used in the experiment. The reader-follower system enables the transmission of tactile sensations. On the leader side, the sensor is wrapped around the arm model to generate a command distribution. On the follower side, the subject wore the Funabot-Grab, which consisted of an actuator and force distribution sensor. On the follower side, there was a mannequin with a force distribution sensor wrapped around the arm. By using the mannequin as the subject, the location and strength of the sensation felt by the subject can be reproduced on the mannequin’s sensors, allowing for quantitative evaluation of the tactile transmission performance. \({\symbfcal{F}}_{\textbf{\textit{tactile}}}\) represents the tactile sensation that the experimenter intends to transmit, and \({\symbfcal{F}}_{\textbf{\textit{tactile}}}\) is measured as \({\boldsymbol{F}}_{\textbf{\textit{ref}}}\). \({\boldsymbol{F}}_{\textbf{\textit{send}}}\) and \({\boldsymbol{F}}_{\textbf{\textit{sensor}}}\) are the force transmitted by the Funabot-Grab and the measured value of \({\boldsymbol{F}}_{\textbf{\textit{send}}}\), respectively. \({\boldsymbol{F}}_{\textbf{\textit{error}}}\) is the deviation between \({\boldsymbol{F}}_{\textbf{\textit{ref}}}\) and \({\boldsymbol{F}}_{\textbf{\textit{sensor}}}\). Applied air pressure to the artificial muscles was controlled using a PID controller. \(K_p\), \(K_i\), and \(K_d\) are the gain parameters. \({\boldsymbol{P}}_{\textbf{\textit{ref}}}\) is the air pressure command sent to the electro-pneumatic regulator. The regulator can measure and feedback the air pressure as \({\boldsymbol{P}}_{\textbf{\textit{sensor}}}\) is applied to each artificial muscle.

\(\symbfcal{F}\) is a two-dimensional vector and \(\boldsymbol{F}\) is a one-dimensional vector. According to the design of the artificial muscle orientation of the Funabot-Grab, the degree of control freedom is limited by the tightening force in the circumferential direction of the arm. Therefore, \(\symbfcal{F}\) degenerates in the longitudinal direction of the arm, and the tightening forces in each position (channel of the artificial muscle) are controlled independently. Each element of \({\boldsymbol{F}}_{\textbf{\textit{ref}}}\), the vector, is calculated by summing the contact forces of the corresponding channels in \(\symbfcal{F}\) (Fig. 4).

4.3. Results and Discussion

Throughout the experiment, all the patterns were tried at once in random order but are presented separately in the results and discussion as the static patterns (Top, Middle, Bottom, and Whole) and TB-Slide.

4.3.1. Static Pattern

Figure 6 shows the results of the Top, Middle, Bottom, and Whole static patterns. The \(x\)-axis represents the tightening force, and the \(y\)-axis represents the channel (1: Top, 7: Bottom) of each graph. The top, middle, and bottom rows show the results for Subject A, Subject B, and Subject C, respectively. Each column shows the results for the Top, Middle, Bottom, and Whole patterns from left to right. The labels in the three colors indicate the power of the sender \({\boldsymbol{F}}_{\textbf{\textit{ref}}}\) (blue), the power exerted by the Funabot-Grab \({\boldsymbol{F}}_{\textbf{\textit{sensor}}}\) (orange), and the subject’s impression as measured by the mannequin (gray).

In explaining the experimental conditions, it was stated that the contact force was managed by the difference from the initial mounting state; however, a negative value meant that the sensors floated to the initial condition. Because floating is irrelevant to the verification of tactile transmission, it was rounded to 0 in the results and discussion.

First, focusing on blue and orange, it is clear that in many cases, the force applied by the device on the follower side cannot track the command value.

The mean absolute error (standard deviation) of orange relative to blue for each channel was 1.8 N (3.0 N) for Subject A, 1.2 N (1.1 N) for Subject B, and 1.8 N (2.8 N) for Subject C, all of which were larger errors than 0.21 N observed in a previous study.

On the other hand, the mean absolute error (standard deviation) of gray relative to orange for each subject was 0.59 N (0.93 N) for Subject A, 1.7 N (3.0 N) for Subject B, and 0.75 N (1.5 N) for Subject C. Gray, representing the subjects’ perceptions, reproduced the actual force applied to the subjects (orange) with an average error of less than 1 N, except for Subject B.

For position, the orange and gray peaks were consistent for the Top, Middle, Bottom, and Whole, except for Subject B’s Middle and Whole, but the error was only one channel (2.2 cm) off.

Excluding Middle and Whole, for which the peak force has shifted error, the mean absolute error (standard deviation) of orange relative to blue for each channel in the Subject B was 0.89 N (0.89 N), and the mean absolute error (standard deviation) of gray relative to orange was 0.93 N (1.7 N). Therefore, the tracking performance of the device still had an error greater than 0.21 N, but without considering the position error, the force reproduction relative to the force actually applied to Subject B was comparable to that of other subjects.

On the other hand, focusing on the Whole, the distribution of gray tended to be narrower than that of orange. In other words, the subjects perceived a tactile sensation over a wider area than when the force was applied.

These results led to the following hypotheses. First, regarding the transmission of position, the perception of the force position remained within the initial target peak error of 2 cm, indicating that the device performed as expected within its capabilities. For example, although Subject B accurately perceived the position at the Top and Bottom, the subject misperceived the position or perceived a narrower range in the Middle and Whole. This suggests that the fabric on the device surface or the subject’s clothing may have acted as a filter, thus reducing the stimulus range.

Regarding force transmission, the force-tracking performance was inferior to those reported in previous studies. Furthermore, in many cases, the blue-orange error was larger than the orange-gray error, indicating that the device’s control performance was inferior to the human perceptual resolution. While it is not essential to consider the magnitude of the orange-gray error in the three subjects in this study because it is related to individual perceptual ability and fine motor skills, the fact that the blue-orange error was greater than the orange-gray error is sufficient to clearly support the conclusion that the force-tracking performance of the device is insufficient.

The failure of the force exerted by the device to track the commanded value may be due to the instability of the attachment of the device to the arm. Unlike the plastic arm model used in a previous study ²⁴, the surface shape of the human body is complex and soft. If there is looseness in the initial attachment, a tightening force cannot be fully exerted. Therefore, it was assumed that the command value was excessively large and could not be tracked. The fact that orange is smaller than blue in Fig. 6 supports this hypothesis.

If the aforementioned challenges can be overcome and the device can fundamentally exert greater force, it is expected that subjects will be able to perceive the tactile sensations more clearly and that the positional transmission performance will improve, at least for relatively large forces.

The next challenge is to consider how to design a mounting mechanism that ensures stable attachment without degrading the force-tracking performance, even when worn by a human, and then proceed with considerations regarding control, such as gain design.

4.3.2. TB-Slide Pattern

The results for Subjects A and B are discussed in this paper because Subject C did not show any appreciable force on the sensor, although a tactile sensation was felt. Figs. 7 and 8 show the results of the TB-Slide patterns of Subjects A and B, respectively. The \(x\)-axis represents time [s], and the \(y\)-axis represents tightening force [N]. Each row shows the results for each channel (Ch. 1: top, Ch. 7: bottom). The color legend is the same as in Fig. 6.

For both subjects, the latency for orange was within approximately 1 s of that for blue, indicating that the tactile sensation was transmitted with the same level of latency as that in a previous study ²⁴. This delay in response can be attributed to the physical characteristics of the pneumatic artificial muscles.

In the TB-Slide pattern, the force moves from the shoulder to the elbow over approximately 4 s (blue and orange). On the other hand, the subject’s perceptions (gray) had a delay of approximately 2 s at Channel 1, the starting point of the movement, while at Channel 7, the end point of the movement, the timing was consistent with that of the orange, without delay. The delay at the starting point is the time taken to perceive the tactile sensation and reproduce the force felt on the mannequin. The fact that the timing is in sync at the endpoint of the movement is the result of the subject perceiving the continuous movement of the location where the force is being applied and its speed, and trying to catch up. This shows that it is possible to translate the force movement using this device. However, according to the impressions of the subjects, all three answered that they were able to recognize that the location where the force was being applied was moving but that it was discontinuous. This is most likely due to the channel width of 2.2 cm, which is close to the 2 cm two-point discrimination ability of the upper arm.

From the above, it was suggested that it is possible to present continuous force movement using this device; however, it was also clarified that the channel density of the artificial muscle needs to be increased to make the transmission feel smooth.

5. Conclusion

The tactile internet is a trend in the haptic field. Previous research has shown that the Funabot-Grab can transmit grabbed tactile signals. We conducted a preliminary experiment with the subjects involved prior to the full-scale experiment.

As a result of the experiment, the subjects were able to reproduce the peaks at the locations where they were grabbed within an error of one channel (2.2 cm). In addition, all the subjects were able to perceive the movement of the locations where they were grabbed.

However, owing to the instability of the device attachment to the arm, it was not possible to achieve the tactile reproduction performance shown in a previous study. However, it was suggested that coarse channel spacing relative to the arm’s two-point discrimination threshold likely caused the discontinuous sensation.

In the future, we will consider a mechanism that allows all the subjects to wear the device under the same conditions, and we will increase the channel density to smooth continuous positional changes.

Furthermore, while various studies are being conducted on pressure distribution sensors ^26,27, the availability of sensors with higher spatial resolution at lower prices will certainly open new avenues for consideration.

If these challenges are overcome and the control performance of the device improves, further considerations can be made regarding their applications. For example, it can be used to transmit stimuli for remote rehabilitation, where real-time performance is not a priority. However, for purposes where real-time performance is important, such as motion instruction, further consideration of the response performance, which was excluded from the current challenges considered, will be necessary.