2.1. Configuration of SVA-M

The objective of this research is to build a VR environment that allows the perception of the surface texture of a virtual object through the incorporation of tracing. To achieve this goal, we propose a method for integrating a haptic display capable of presenting the surface texture of virtual objects and a sensor for measuring hand movements for tracing into a single device. Thus, this study developed a soft vibration actuator mouse (SVA-M) by integrating an SVA, a haptic display with a high spatial resolution, into a computer mouse. Fig. 1(a) shows the external appearance of SVA-M. The total weight of the SVA-M, including the embedded magnets, is approximately 10 g, making it sufficiently lightweight for handheld use. The SVA was positioned at the center for haptic presentation. Vibrations can be perceived by placing the pad of a finger, such as the index finger, on the SVA. The vibration was generated in the normal direction relative to the skin surface, whereas the actuator motion itself occurred in the horizontal direction along the skin. Fig. 1(b) shows the structure of the integrated SVA. Four soft tubes arranged in parallel vibrate due to electromagnetic force; these tubes can vibrate independently, enabling high-density haptic presentation ¹². The number of the parallel actuators was determined to cover the contact area of the fingertip during normal mouse operation, with four actuators selected as an appropriate balance between coverage area and compact device size. The magnet size (\(10 \times 10 \times 10\) mm) was chosen to maximize magnetic force while remaining sufficiently small to be integrated into the mouse housing. N35 grade neodymium magnets were used in this study. According to the manufacturer datasheet, the magnet has a surface flux density of 488.6 mT and a coercivity of at least 868 kA/m. Thinner tubes were preferred because reducing the tube mass increases the achievable vibration amplitude; thus, the tube diameter was minimized while maintaining stable operation. The actuator was constructed using silicone tubes and liquid metal, which provide high flexibility and allow large mechanical deformation. Each tube had an outer diameter of 1.0 mm and an inner diameter of 0.5 mm, and the length of the portion driven by the electromagnetic force and presented to the fingertip was 30 mm. The liquid metal (ZZGS22011, Zairyo-ya.com) is an alloy of gallium, indium, and tin, with a melting point of \(-\)19°C, remaining liquid at room temperature. The resonance frequency of the SVA can be adjusted by changing the tube tension; the tension was set such that resonance occurred near 120 Hz, corresponding to a frequency range to which humans are particularly sensitive. For stable current control, a 5 \(\Omega\) resistor was connected in series with the actuator, and the driving current reached approximately 1 A under a circuit voltage of 5 V. Thus the total power consumption of the driving circuit was approximately 5 W. In addition, because the voltage applied to the SVA itself is lower than 0.2 V even at maximum drive, the device poses little risk of electrical shock. The SVA-M can also function as a conventional mouse by implementing left and right clicks. To move the cursor, an optical mouse sensor SPCP168A was installed on the bottom surface of the SVA-M. Fig. 2 shows the SVA-M in use. When the user grips the SVA-M and scans over a real mouse pad, the vibration generated by the haptic display changes according to the movement and the presented virtual texture, thereby providing feedback corresponding to the surface texture of a virtual object. The SVA-M comprised commercially available components and a previously validated SVA ¹². Therefore, the objective of this study was not to evaluate the fundamental performance of each individual component, but to investigate the effectiveness of the integrated mouse-type haptic display system.

2.2. SVA-M System

The SVA-M presents haptics that correspond to hand movements and the haptic texture being traced. Fig. 3 shows the system diagram of the SVA-M. The SVA-M transmits the movement information measured by the optical sensor to the PC. The movement information operates the mouse cursor on the screen. The frequency presented by the SVA is determined by the cursor movement speed and the pixel value directly beneath it, and a signal is transmitted to the driver. The driver consists of a microcontroller (Arduino Uno), a DC motor driver, and the SVA. The PC sends a control signal to the microcontroller, which outputs a frequency-variable square wave current of up to 1 A from the attached DC motor driver. This square-wave current drives the SVA and presents haptics to the fingertips.

3.1. Overview

In this experiment, we used the developed SVA-M to evaluate the spatial resolution when a person moved their hand to obtain tactile perception. Specifically, we demonstrated that a user of the SVA-M can distinguish between haptic textures of different coarseness by tracing them.

3.2. Experimental Design

In this experiment, we prepared the haptic textures in the form of black and white vertical stripe patterns, as shown in Fig. 4, and displayed them on a PC screen. To focus on the spatial perception of texture through tracing, all tubes of the SVA were driven in phase and treated as a single vibration source. The SVA vibrated at 120 Hz when the cursor was on the black part of the haptic texture and stopped vibrating when it was on the white part. The operational range of the cursor corresponds only to white or black pixels (corresponding to image pixels). In this experiment, a ratio was set such that a physical movement of 40 mm of the SVA-M corresponded to a 1000-pixel movement on the screen. Therefore, one pixel on the screen corresponded to a physical distance of 0.04 mm. The haptic textures used in the experiment, as shown in Fig. 4, had stripe widths of 0.04 mm (1 pixel), 0.08 mm (2 pixels), 0.16 mm (4 pixels), 0.32 mm (8 pixels), 0.64 mm (16 pixels), and 1.28 mm (32 pixels). Hereafter, the texture with a stripe width of 0.04 mm is treated as the finest texture, and the 1.28 mm texture as the coarsest. A stripe width of 1 pixel is close to the diameter of a human hair, and 32 pixels is close to the diameter of an AWG16 coated wire.

Before starting the experiment, participants practiced operating the SVA-M. In the practice, participants traced all the haptic textures to be presented in the experiment while visually confirming them using the SVA-M. After the participants became sufficiently accustomed to the SVA-M, a texture discrimination task was performed. During the task, participants were asked to trace the haptic textures using the SVA-M and evaluate their coarseness without visually confirming them. Throughout the experiment, no restrictions were placed on the tracing method; the participants traced the haptic textures as they would a real object. The experimental task adopted a paired comparison format in which participants touched two randomly displayed haptic textures with the SVA-M in one trial and judged which was finer. The specific procedure is as follows. In Step 1, the experimenter randomly selected and presented one of the six textures, asking the participant to touch it for 5 s. In Step 2, a white texture was presented for 1 s as a period of no stimulation. In Step 3, a texture different from that in Step 1 was presented for 5 s for the participant to touch. In Step 4, the screen was again presented as white, and the participant was asked to verbally state which of the two test textures presented in Steps 1 and 3 was perceived as finer. This task was conducted for 30 pairs formed by permutations of the six textures, with four trials for each pair, totaling 120 trials. In addition to the participant responses, the cursor trajectory was recorded as coordinate data during the experiment.

We employed five male participants (aged 19–23 years) with no visual, sensory, or motor impairments. All experimental procedures were performed in accordance with the ethical guidelines of the authors’ affiliated institutions. Prior to the experiment, we used the ethics review checklist of the institutions to confirm that formal ethical review approval was not required, as the study involved minimal risk and did not collect sensitive personal information.

3.3. Results

We compared the results for each type of coarseness. The average discrimination rate was 0.66 and the standard deviation was 0.11. Table 1 shows the results of the comparison with the discrimination rate (the percentage of times the coarseness was correctly identified) for each pair. A value closer to 1 indicates that the distinction in coarseness was easier, and 0.5 means no distinction was made. According to these results, the discrimination rate was 0.83 or higher for pairs with the 1.28 mm stripe width texture, 0.63 or higher for pairs with the 0.64 mm stripe width texture, and averaged about 0.58 for pairs with the 0.32 mm stripe width texture. The discrimination rate for pairs combining 0.16 mm, 0.08 mm, and 0.04 mm stripe width textures was 0.5 or lower for all. Next, to examine the perceived coarseness of textures across all participants, we calculated \(Z\)-scores using a method equivalent to Case V of Thurstone’s paired comparison method ¹³. Fig. 5 shows the plots of the calculated \(Z\)-scores. The \(Z\)-score indicates the relative intensity of the coarseness; a smaller value indicates that the texture was perceived as coarser. For convenience, a correction was made by setting the value of the 32-pixel texture, which was perceived as the coarsest, to 0.

Figure 5 shows that the order of the perceived fineness matched the actual order for textures 0.16 mm, 0.32 mm, 0.64 mm, and 1.28 mm or more. Conversely, for the 0.04 mm, 0.08 mm, and 0.16 mm textures, the order of the \(Z\)-score and the order of the stripe width were reversed, and the difference between the \(Z\)-scores was also small. These results suggest that when comparing textures of 0.16 mm or greater, there is a tendency to perceive the correct order of coarseness; however, discrimination becomes difficult for finer textures. We confirmed the internal consistency of the calculated \(Z\)-scores. Internal consistency refers to the degree to which the paired comparison judgments made by participants were consistent and free from contradictions in the comparisons between stimuli. Mosteller ¹⁴ proposed a framework for verifying the agreement between the observed and estimated rates through a goodness-of-fit test using the paired comparison method. In this study, we calculated the chi-square value by comparing the expected discrimination rate derived from the \(Z\)-score, with the measured discrimination rate to assess the agreement. We obtained \(\chi^2=1.97\). The rejection threshold for 15 degrees of freedom is approximately 25.00, and 1.97 being below this value, implies that there is no significant difference. Therefore, the deviation between the observed and expected values was not statistically significant, and internal consistency in the \(Z\)-scores was confirmed.

The average movement speed of the cursor during the experiment was approximately 240 pixels/s. This implied a hand movement of approximately 10 mm/s. This tracing speed was slower than that reported in previous studies on unconstrained tactile exploration ^15,16. Considering the movement speed based on the refresh rate of the optical display (60 Hz), it is understood that an average movement of 4 pixels/frame occurred.

4.1. Coarseness Perception by SVA-M

The experimental results suggest that textures with a coarseness of 0.16 mm or more (4 pixels or more) have a significantly high discrimination rate under tracing conditions, indicating that this could be the threshold for spatial resolution affecting the perception of texture coarseness. This result was also supported by the consistency between the order of the texture stripe widths and the order of perceived coarseness (\(Z\)-score). On the other hand, the discrimination rate remained at 0.50 or below for pairs that combined textures finer than 0.16 mm, and the order of the \(Z\)-scores was reversed within this finer range. This is likely the result of aliasing, as shown in Fig. 6. Aliasing occurs when the output result differs from the original owing to insufficient sampling speed. In this experiment, the SVA itself was driven at 120 Hz; therefore, the device was capable of reproducing the vibration on–off switching required for the textures used. However, the pixel value beneath the cursor was sampled at a screen refresh rate of 60 Hz, which limited the spatial sampling of the displayed texture. At the average cursor speed of 240 pixels/s, a texture with a stripe width of 0.16 mm (4 pixels) produces vibration on–off switching at approximately 30 Hz, which remains within the limit imposed by the 60 Hz refresh rate. In contrast, textures finer than 0.16 mm exceed this limit, so the corresponding on–off switching cannot be reproduced correctly under the present screen configuration. This explains why discrimination failed for pairs containing textures finer than 0.16 mm, whereas the 0.16 mm texture could still be discriminated from coarser textures. These results indicate that the limitation arises from the screen refresh rate rather than the presentation capability of the SVA itself. It should, therefore, be clarified how the 0.16 mm value should be interpreted. This value represents the spatial resolution at which texture differences become reliably discriminable under the present operating conditions, namely, the reduced tracing speed associated with operating the device in a standard 60 Hz display environment. This is neither an intrinsic constant of human tactile perception, nor a fixed specification of the proposed device. The underlying cause was not specific to the SVA-M. The limiting factor is the spatial sampling of the displayed texture at the refresh rate of the screen. Therefore, a comparable resolution limit is expected for any mouse-type haptic display that samples a screen-rendered texture at a display refresh rate at similar tracing speeds. In this sense, the present result is better understood not as a device-specific specification but as a practical criterion for designing haptic textures in screen-coupled, mouse-type haptic displays intended for everyday use.

Although aliasing may reduce the discriminability of fine textures, the participants did not show a tendency to slow their tracing speed to explore textures in greater detail. This result suggests that, when designing haptic textures for natural tracing, it is more practical to adopt a spatial resolution around the perceivable threshold of 0.16 mm rather than a finer one. Designing textures finer than this level may not improve perceptual quality during free exploration because users may not consciously perceive such fine differences while naturally tracing the surface.

The standard deviation of the discrimination rate across participants was 0.11, indicating individual differences in absolute performance. However, given that internal consistency was confirmed through Thurstone’s paired comparison method, it can be considered that the participants shared a common perceptual tendency: coarser textures were easier to discriminate, whereas finer textures were more difficult, despite the variability in individual discrimination rates.

Based on these results, it was shown that when performing realistic tracing with the SVA-M under the reduced tracing speed imposed by device operation in a standard 60 Hz display environment, differences in coarseness are easily perceived if a haptic change occurs with hand movements of 0.16 mm or more. This result suggests that even when exploratory behavior such as tracing is reproduced virtually, the spatial change of the object can be perceived. Furthermore, these findings provide guidelines for selecting an appropriate resolution for designing haptic textures.

In addition, the average tracing speed observed in this experiment was lower than that reported in previous studies on unconstrained tactile exploration. One possible reason for this is that the SVA-M is heavier and more difficult to manipulate than direct fingertip tracing on a real surface. Therefore, reducing the weight and improving the maneuverability of the device may contribute to a more natural tracing behavior and more realistic texture perception.

4.2. Limitations

The above results have two limitations. First, what was evaluated in this experiment was the user’s perception when tracing virtual objects with the SVA-M, and not only the spatial resolution of the texture that the device can present but also the scan speed must be considered. The spatial resolution of the haptic texture refers to the minimum texture coarseness that can be presented by the device. In other words, the discrimination rate may improve, depending on the user’s tracing speed. Although the experimental conditions in this study indicated that textures finer than 0.16 mm could not be discriminated, they might be distinguishable by tracing slowly without time restrictions. Thus, the perceivable texture resolution of the device changes significantly depending on the scan speed. This study evaluated the spatial resolution of a presentable texture at a typical tracing speed. Another limitation is that the discrimination rate can change depending on the screen refresh rate. As discussed in Section 4.1, aliasing is likely to have influenced the textures that could not be discriminated in this experiment. In an environment with a higher screen refresh rate, even finer textures can be discriminated.

4.3. Future Works

In this study, all the SVA tubes were driven in phase using a single vibration frequency to evaluate the perceivable spatial resolution of the haptic textures. However, the SVA can independently control multiple actuators at a high density. Therefore, future studies should include the presentation of multiple vibration frequencies below the two-point discrimination threshold to reproduce more complex tactile sensations.

In addition, future studies could investigate haptic rendering methods that reproduce tactile perception directly from visual information such as images and 3D models. However, it should be noted that in the present system, haptic stimuli are rendered from the image shown on the screen; that is, the SVA-M functions as a tactile counterpart to a visual display, reconstructing texture from the displayed image. As discussed in Section 4.1, this makes the achievable spatial resolution bounded by the sampling of the screen-rendered image. Therefore, a promising direction is to render haptic stimuli directly from physical surface data rather than from a displayed image, so that the presentation is no longer constrained by the screen refresh rate. Such a data-driven rendering approach can provide finer tactile details than the image-based rendering used in this study. By combining high-density multipoint haptic presentation with rendering techniques, a more realistic tactile exploration in virtual environments can be achieved.