2.1. Surface Distortion Mapping System

Figure 1 presents the system architecture proposed in this study. In this system, a human operator scans the inspection surface using a tactile sensor, and the regions of detected minute surface distortions are displayed on a monitor. The actual implementation of the system is shown in Fig. 2. The system estimates the sensor’s position upon detecting a distortion and plots a marker at the corresponding coordinates. Moreover, the system incorporates a human-in-the-loop framework, in which the operator visually confirms the plotted markers and performs focused scanning of the surrounding area, thereby increasing the probability of defect detection. For surface distortion detection, we adopted the wavelet transform method proposed by Tanaka et al. ². Additionally, a camera-based marker tracking system is employed to estimate the sensor position during detection. Details of the tactile sensors utilized in this study are described in Section 2.2.

2.2. Proposed Sensor

In this study, the sensor used in Ando et al. ⁹ is treated as the conventional sensor for the purposes of this research. The conventional sensor is presented in Fig. 3. It consists of a flexible sheet and numerous pins, with strain gauges (KC-60-120-A1-11L1M2R, Kyowa Electronic Instruments Co., Ltd., Japan) attached to its surface. The strain gauges are only bonded where they contact the sheet; they are not adhered to the pins. The main limitations of this sensor include its susceptibility to tearing during fabrication and measurement, as well as the difficulty of manually holding it during operation due to its non-flat contact surface. Fig. 4 shows the newly developed sensor designed to address these issues. This sensor features a structure in which the strain gauge is sandwiched between the amplification structure from above and below, with adhesion occurring only at the sensor edges. It employs a strain gauge with a gauge length of 60 mm, and the sensor dimensions are 13 mm \(\times\) 80 mm.

One limitation of conventional sensors is their tendency to tear during fabrication or measurement. This issue is attributed to the large number of bonding surfaces between the sheet and the strain gauge, which makes the gauge susceptible to damage under deformation. To address this problem, the bonding area was restricted to the sensor edges, thereby reducing the total bonding area compared with that of conventional sensors and improving resistance to tearing during both fabrication and measurement. In addition, the difficulty of manually holding the sensor was noted. This problem was also addressed by sandwiching the strain gauge between two layers of the amplification structure. Consequently, this study employs the proposed sensor for detecting surface distortions.

3.1. Validation of the Amplification Effect

Figure 5 illustrates the experimental system for measuring surface distortion. The convex profile has a diameter of \(\phi\)4 mm and a height of 50 μm. In the experiment, a tactile sensor is used to scan both convex and non-convex surfaces. The evaluation method involves calculating the difference between the maximum value and the value measured 0.025 s prior to the maximum value, and then analyzing that difference. To verify the amplification effect, experiments were conducted using three types of sensors: a strain gauge-only sensor (Fig. 6), a conventional sensor, and the proposed sensor. The unit used for the strain measurements obtained from the sensor is μ\(\epsilon\), which corresponds to a strain magnitude of \(10^{-6}\). The experimental results are presented in Figs. 7–9, which display graphs of scans over a flat surface and over a surface with a convex profile, respectively. Additionally, to evaluate the variation in sensor output values between trials, a total of 10 trials were conducted under identical conditions. The results for the difference between the maximum value obtained from all 10 trials and the value measured 0.025 s prior are presented in Table 1.

The result graph confirms that the range from distortion detection to the maximum value is approximately 7.08 μ\(\epsilon\) for the strain gauge-only sensor, approximately 16.3 μ\(\epsilon\) for the conventional sensor, and approximately 53.8 μ\(\epsilon\) for the proposed sensor. As the sensor values vary as each sensor passes over convex surface features, the results demonstrate that the strain gauge-based sensor can reliably detect minute convex shapes on an object’s surface. Moreover, the results indicate that the sensor incorporating the amplification structure produces larger strain values than the sensor using only the strain gauge. This suggests that the proposed sensor is amplified by approximately 7.6 times through the integrated amplification structure. Additionally, comparison with the output of the conventional sensor confirms that the proposed sensor produces a larger output. As presented in Table 1, the proposed sensor exhibits a higher average output value and a smaller standard deviation compared to the other two sensors. This demonstrates that the proposed sensor is capable of generating high-output signals with enhanced reproducibility while effectively suppressing inter-trial variability, even for minute convex shapes with a height of 50 μm.

3.2. Improved Sensitivity Through Signal Superposition

Small-scale features, such as micro-convexities, induce localized deformation in only a portion of the strain gauge, potentially reducing sensitivity and causing signals to be obscured by noise. Therefore, multiple strain gauges were used, and their signals were combined to improve sensitivity. In the experiment, scanning of the micro-convexity was performed using the proposed sensor, which consists of two strain gauges stacked as presented in Fig. 10, and the combined signal was subsequently analyzed. The test specimens and the evaluation methodology are identical to those in Section 3.1. Additionally, as in Section 3.1, 10 experimental trials were conducted to evaluate inter-trial variability. Moreover, to verify the noise suppression effect achieved by summing the outputs of the stacked strain gauges, the root mean square (RMS) of the output signal was calculated and compared. Specifically, the RMS of a signal artificially amplified to twice the output level of a single strain gauge was calculated and compared with the RMS of a signal obtained by summing the outputs of two individual strain gauges.

The experimental results are shown in Figs. 11 and 12, as well as Tables 2 and 3. The graphs clearly show a distinct peak around 0.35 s. Although the variation in each strain is relatively small, at approximately 34 μ\(\epsilon\), combining the signals yields an output of approximately 68 μ\(\epsilon\) without being obscured in noise. Table 2 indicates that the output of each strain gauge exhibits low variation and is stable. Table 3 demonstrates that the RMS value of the signal obtained by adding the outputs of two gauges is significantly smaller than the RMS value of the signal obtained by doubling the output of a single gauge. This demonstrates that the proposed method, in contrast to simple electrical amplification, is effective in isolating minute strain variations from noise. This further indicates that the sensor sensitivity is enhanced through the combination of signals from two strain gauges.

3.3. Position Detection Using Markers

This study aims to visualize minute convex features using the proposed sensor. However, such visualization cannot be achieved without accurately determining their spatial locations, making position detection essential. Current systems only detect the presence of micro-convex features without accurately localizing them, leaving their exact positions unknown. Therefore, position detection of micro-convex features was performed using markers, as presented in Fig. 13. The test specimen was the same as that described in Section 3.1. In the experiment, the \(y\)-coordinate of the micro-convexity feature was detected using markers upon detection. We collected 20 \(y\)-coordinate measurements for evaluation.

The experimental results showed that the average estimated position was 124.6 mm, with a standard deviation of 5.5 mm, compared to the actual value of 120 mm. Moreover, the dimensionless error, obtained by dividing the error from the true value by the gauge length of 60 mm, was 7.2%. These results indicate that the position estimation achieved satisfactory accuracy. Therefore, the proposed sensor’s surface distortion localization capability enables position estimation using multiple markers.

3.4. Narrowing Down Estimated Positions Based on Existence Probability

In Section 3.2, we performed position detection using markers. However, as shown in Fig. 14, the estimated micro-convex positions reflect the geometry of the sensor, necessitating further refinement of the estimated positions. Therefore, we constructed a system that detects micro-scale surface distortions multiple times to increase their probability of existence and refine the estimated positions. Here, we propose the following update law to compute the probability \(P_n\) of surface distortion at position \(\pmb{p} \in \mathcal{S} \subset \Re^2\) at the \(n\)-th detection by the sensor:

We further prove, by mathematical induction, that the unity condition of the existence probability is preserved for all detection steps \(n \ge 0\), that is,

(i) Unity condition at the initial stage (\(n = 0\))

At the initial state, the probability is assumed to be uniformly distributed over the entire region of the camera image \(\mathcal{S}\). Integrating \(P_0(\pmb{p})\) over the entire domain \(\mathcal{S}\) yields:

(ii) Unity condition assumption at the \((n-1)\)-th detection

We assume that the unity condition of the probability at the \((n-1)\)-th detection step holds, i.e., \(\int_{\mathcal{S}} P_{n-1}(\pmb{p}) d\pmb{p} = 1\).

(iii) Unity condition at the \(n\)-th detection

Integrating both sides of the update law over the entire domain \(\mathcal{S}\) yields

From (i), (ii), and (iii) above, it has been proven by mathematical induction that the unity condition of the probability distribution is preserved for all \(n \ge 0\).

The proposed update rule \(P_n(\pmb{p})\) functions not as a mathematically independent convergence algorithm, but rather as part of an interactive exploration process involving human operation. The update rule accumulates past scan data and visualizes the current estimated state. The actual identification (narrowing down) of the defect location depends on the operator’s active scanning strategy, guided by this probability distribution \(P_n(\pmb{p})\). The update rule emphasizes regions with significant signal responses. The operator then concentrates the scanning process on these regions, thereby accumulating additional signal components. This iterative feedback mechanism ensures improved final positional accuracy.

The experimental method involves the comparison of the estimated detection positions obtained when scanning using the proposed system with those obtained when scanning while relying solely on the output waveform. In addition, to demonstrate the effectiveness of this system, we compare the estimated detection positions obtained from a single scanning pass. The inspection surface is presented in Fig. 15, and the experimental system is presented in Fig. 16.

For the micro-surface convex feature, a paper with a thickness of 0.4 mm and an outer diameter of 4.5 mm was used and sandwiched between two transparent sheets, each with a thickness of 1.5 mm. Its position was defined at an \(x\)-coordinate of 120 mm and a \(y\)-coordinate of 70 mm from the origin of the reference marker. Two sheets of paper were inserted between the two transparent sheets to prevent the micro-protrusions from being visible to the naked eye. To verify the effectiveness of the proposed system, experiments were conducted according to the following procedure. First, within a single trial, sensor scanning was repeated a total of four times to progressively narrow down the estimated defect location. This location estimation process using four scans was treated as one set, and a total of four independent trials were performed. Table 4 presents the mean and standard deviation of the final detection results obtained in each of these four trials (i.e., the estimated location after completion of the fourth scan in each trial). Thus, the reported results correspond to the mean and standard deviation of the fourth scan. For the single-scan condition, 10 sets were performed, and the results are presented in Table 5.

Figure 17 illustrates photographs of probability changes during four scans using the proposed system. The probability of existence is represented by overlaying a color intensity map onto the image. The results suggest that, when scanning based solely on the output waveform, deviations from the true value were approximately 4.2 mm in the \(x\)-direction and 10.4 mm in the \(y\)-direction. When scanning using a single pass, the deviation from the true value was approximately 3.9 mm in the \(x\)-direction and 13.8 mm in the \(y\)-direction. In contrast, when scanning using the proposed system, the deviation from the true value was reduced to 0.8 mm in the \(x\)-direction and 5.7 mm in the \(y\)-direction. These results demonstrate that the proposed system enables more accurate estimation of the position of micro-scale surface distortions. The results also demonstrate that the proposed system reduces variability, as evidenced by a lower standard deviation.

4.1. Factors Enabling the Proposed Sensor to Achieve Greater Output Compared to Conventional Sensors

One factor contributing to the increased output of the proposed sensor compared to the conventional sensor is attributed to improved stability during manual operation. From the result graphs shown in Figs. 8 and 9, the initial strain value is approximately 450 μ\(\epsilon\) for the proposed sensor, compared to approximately 200 μ\(\epsilon\) for the conventional sensor. This is considered to be the result of the strain value increasing because the sensor is being properly suppressed. Second, the pins may have also contributed to the amplification effect. While conventional sensors bond the strain gauge to the amplification structure using adhesive, the proposed sensor fixes the strain gauges with pins. This design enables the pins to deform along with the amplification structure during deformation, potentially producing greater deformation than conventional sensors. These findings indicate that the proposed sensor may have a higher output response than the conventional sensor.

4.2. Factors Contributing to Improved Accuracy When Introducing Probability-Based Filtering

One key factor contributing to the improved accuracy is the visual feedback provided regarding the estimated position. In this technique, the location of the surface distortion is obscured by the paper surface, making it impossible to visually identify it directly. However, when the sensor detects surface distortion, its estimated position is displayed in real time. This display likely serves as an effective indicator for determining the subsequent scanning direction.

Furthermore, the accumulation of multiple detection results progressively narrows the candidate range for the estimated position, thereby constraining the possible scanning directions. This is also considered to contribute significantly to the improved accuracy. In contrast, when scanning based solely on waveform information, although the detection timing can be identified, it is necessary to interpret the instantaneous waveform change and determine the exact position on the inspection plane where the detection occurred. This process is inherently ambiguous and fails to sufficiently constrain the estimated location range. Consequently, it likely results in deviations of the estimated mean position from the true value and increases the variability of the measured values.

Therefore, it can be concluded that the method of visualizing the estimated surface strain location and progressively narrowing the estimated range through multiple scans effectively enhances the overall positional accuracy.