2.1. Concept

The proposed method bridges the temporal-scale gap between high-speed sensors and conventional robots through three separate processes: sensor measurement, information extraction, and motion planning (Fig. 1). Each high-speed sensor independently measures the target phenomena. The information extraction process utilizes high-speed measurement data to extract task-relevant meta-information. The motion planning process generates feasible robot trajectories based on the extracted information and robot constraints.

Our prior study ¹² demonstrated this concept but lacked a systematic framework for multi-sensor information integration. This study employed MVDC as an integrated framework.

2.2. Multisensor Information Integration Framework Based on MVDC

The MVDC proposed by Arita et al. ¹³ generalizes series admittance-impedance control ¹⁴ by introducing the concept of virtual force. The framework treats various sensor outputs uniformly as virtual forces that drive series-connected virtual dynamics (mass–damper–spring models). Each layer responds to virtual forces while treating the previous layer’s output as its equilibrium state, thereby enabling the dynamic integration of information from multiple sensors. The framework offers three key advantages: 1) no switching of control laws, 2) an independent parameter design for each layer, and 3) high modularity for sensor addition. The parameters for each layer can be designed independently, allowing the separate parameterization of the influence of each sensor based on its characteristics. Through a series connection, the influences from all layers accumulate, and the final output reflects their integrated effect. In this study, we extended the applicability of MVDC from motion control to sensor information integration, where the state variables represent task-relevant meta-information rather than physical quantities.

Figure 2 shows the proposed multisensor information integration framework. Multiple sensors measure the target phenomena independently and asynchronously transmit the outputs to the information extraction process. Each sensor output is extracted into task-relevant feature values and then into virtual forces using virtual force generators. These virtual forces drive the respective virtual dynamics (mass–damper–spring models with parameters \(M_i\), \(D_i\), and \(K_i\)). The virtual dynamics are connected in series, where the output of the \(n\)-th layer becomes the integrated value, which is then utilized in the motion planning process to generate robot trajectories. The gains of the virtual force generators and the parameters of the virtual dynamics are designed by considering the measurement characteristics of each sensor (sampling rate and noise level), required response speed, and convergence stability.

The key advantage of this framework is that, by using the concept of virtual force, any sensor output can be handled uniformly, enabling system expansion by simply adding new sensors without modifying the existing virtual dynamics, thus achieving high modularity. Furthermore, as each virtual dynamic can be designed independently, information extraction can be tailored to the sensor characteristics and task requirements by individually adjusting the parameters of each layer. When multiple feature values are necessary for a task, this can be addressed by treating the feature values and virtual forces as vectors or matrices and designing virtual dynamics as multidimensional systems. Regarding the design of MVDC virtual dynamics parameters (mass, damper coefficient, and spring constant), adjustment methods for mass–damper–spring systems in control engineering have been studied for several years, and the existing knowledge can be utilized. By applying established methods, such as parameter design based on critical damping conditions and time constants, adjustment according to sensor characteristics and task requirements can be performed relatively easily, which is an important advantage that enhances the practicality of the proposed method. In our previous approach ¹², sensor integration required manually designed switching logic with predetermined priorities. The MVDC framework eliminates this limitation by automatically integrating sensor information through virtual dynamics.

3.1. System Configuration

The system was constructed by attaching a gripper (2F-85, Robotiq), a high-speed streaming camera (INFINICAM UC-1, Photron), a proximity sensor module, and an analog input USB I/O unit (AIO-1608GY-USB, CONTEC) to an industrial robotic arm (UR5e, Universal Robots).

First, a high-speed camera was used to discover the target phenomena and enable the robot to approach them. However, high-speed cameras inherently involve a tradeoff between resolution and processing time. Maintaining a high resolution during high-speed measurements restricts the field of view. To address this issue, we adopted a hand–eye system to secure a flexible measurement area. As the focal length of the high-speed camera is fixed, the measurement becomes difficult when the robot begins to approach the phenomenon.

Proximity sensors were introduced to perform secondary measurements after the robot approached the target. To accurately measure objects passing through the gripper, we adopted infrared reflective photoreflectors (RPR-220, ROHM), which are compact, have a high temporal resolution, and are suitable for distributed placement on the gripper surface. The design of the proximity sensor module is based on the NSPS proposed by Hasegawa et al. ¹, which achieves a high-speed response and measures object center position by connecting multiple sensor elements through a resistor network. The proximity sensor module consists of two parts: a sensor element unit (six RPR-220 units arranged at intervals of 10 mm) and an analog computing circuit unit, which are mounted on the gripper palm and robot wrist, respectively. The computing circuit adds and distributes the currents from each phototransistor and calculates the voltage difference at both ends \(V_1 - V_2\) as the first moment of the photocurrent distribution, thereby measuring the centroid of the object relative to the sensor center. Therefore, an I/O unit was used to sample \(V_1\) and \(V_2\). Fig. 3 shows the end-effector configuration.

3.2. Case Study: Grasping a Pendulum Object

To validate the effectiveness of the proposed method, a case study was conducted in which a conventional industrial robot (UR5e) grasped a pendulum. This task was designed to exemplify the problem setting introduced in Section 1, which inherently requires high-speed sensors to capture the spatiotemporal characteristics of the phenomenon, and the limited response speed of the robot creates a temporal-scale gap that makes direct feedback control impractical.

Specifically, the accurate prediction of the pendulum motion requires detecting the moment when the object passes through its equilibrium point (lowest point), where the velocity changes the most steeply. Sensors with low temporal resolution cannot capture this instantaneous change. With insufficient sampling rates, the object moves significantly between discrete measurements, causing aliasing where peak detection occurs at positions spatiotemporally different from the true timing, leading to significant prediction errors. In contrast, the high-speed camera (1 kHz) and proximity sensor (1 kHz) used in this study could accurately detect these critical moments.

Notably, the proposed method does not rely on the dynamic pendulum system model. Instead, it operates solely on the assumption that the object motion has a constant period. Therefore, the method remains applicable regardless of variations in physical parameters, such as mass, string length, and initial angle, as these values are not assumed to be prior knowledge but are captured implicitly through continuous high-speed measurements. However, the maximum joint velocity of the robot (90°/s) and gripper closing time (approximately 0.44 s) make it difficult to track pendulum motion through direct feedback. Therefore, accurate prediction requires extracting temporal meta-information (periods) from high-speed sensor data and predicting an object’s future arrival time based on this information. Generating motion plans that consider both the predicted future state and robot constraints enables a slow robot to handle fast phenomena, which is the aim of the proposed method.

To maximize the challenge and demonstrate the capability of the method, we intentionally set the grasping point to the equilibrium position where the object velocity is the highest and the tolerance for prediction errors is minimal. The pendulum system provides a structurally simple, yet sufficiently dynamic target for method validation. Despite its simplicity, the task encompassed the core elements of the proposed method: meta-information extraction through high-speed sensing, the integration of heterogeneous sensor information via MVDC, and the generation of feasible trajectories under robot dynamic constraints.

We designed the following three processes to apply the proposed framework to pendulum grasping.

Measurement Process

A high-speed camera and proximity sensor independently measured the pendulum position and passage timing at 1 kHz.

Information Extraction Process

Each sensor independently extracted the period information (\(T_{\mathrm{period}}^{\mathrm{cam}}\), \(T_{\mathrm{period}}^{\mathrm{prox}}\)) from its measurements. The extracted period predictions were integrated using MVDC to obtain the final predicted period.

Motion Planning Process

The feasible grasping time and the trajectory of the motion planning process were determined considering the predicted period and robot constraints (maximum velocity 90°/s, maximum acceleration 90°/s\(^2\), and gripper closing time 0.44 s).

The following sections describe the implementation of these processes.

4.1. Period Information Acquisition by Sensors

The high-speed camera performs binary processing and contour detection on the acquired images to track the center of the pendulum. To simplify the motion analysis, the pendulum motion was assumed to be two-dimensional, and the depth distance was estimated using the ball radius and calibration based on the known ball diameter. By dynamically optimizing the region of interest based on the predicted velocity of the object, the computational load can be reduced while achieving high-speed processing. From the time-series data of the \(y\)-coordinates of the detected object, the local maxima (lowest points) were detected to identify when the pendulum passed through the equilibrium point, and the half-period of the pendulum \(T_{\mathrm{period}}^{\mathrm{cam}}\) was calculated from the time difference between the consecutive maxima.

The proximity sensor module sampled the value \(V_1 - V_2\) representing the object centroid relative to the sensor center, at 1 kHz. When the object passes through the gripper, \(V_1 - V_2\) changes its sign from positive to negative (or from negative to positive). By detecting this moment of sign change (i.e., the zero-crossing point), the time at which the object passes through the sensor center is accurately identified. The half-period of the pendulum \(T_{\mathrm{period}}^{\mathrm{prox}}\) was calculated from the time interval between consecutive zero crossings.

The two sensors operate at different spatial scales because of the hand-eye camera configuration. A high-speed camera was mounted on the end effector of the robot with its focal length fixed for distant observation, enabling the discovery of wide-range phenomena and initial period estimation while the robot was at its resting position. However, as described above, once the robot begins approaching the grasping point, the reduced distance to the object causes it to fall outside the depth-of-focus of the camera, rendering camera measurement impossible during the approach and grasping phases. Consequently, because of the limited speed of the robot, camera-based period estimation must be completed well before the grasping execution, as the robot requires time to approach the grasping point. Over this interval, prediction errors accumulate through multiple pendulum cycles, thereby contributing to grasping failure. The proximity sensor directly addressed this fundamental limitation. It was mounted on the gripper itself and measured the passage timing of the object at the grasping point as the pendulum swung through the gripper. This provided a final period of re-estimation immediately before the grasping execution at the precise moment and location where the camera was unable to operate, thereby mitigating the cumulative prediction errors of the long-term camera estimate.

4.2. Period Information Integration Using MVDC

A two-layer virtual dynamics model was constructed to predict the pendulum motion period. The purpose of these virtual dynamics is to integrate period information from multiple sensors and output a predicted period \(\omicron_2\), which is used to determine the grasp timing.

Each layer is represented by a mass–damper–spring model, where \(\omicron_i\) represents the predicted period of the \(i\)-th layer. The virtual dynamics operate as follows. When a sensor input (measured period) differs from the current predicted value \(\omicron_i\), this difference acts as a virtual force that drives the predicted value toward the sensor input. Through this mechanism, the predicted period was continuously updated and converged to a stable value as the virtual dynamics evolved over time.

The virtual dynamics described below do not represent a physical model of the pendulum system. Instead, they serve as a mathematical framework for integrating sensor information. The state variables \(\omicron_i\) represent temporal meta-information, specifically the estimated period for determining the grasping timing. This conceptually differs from physical quantities, such as position or force, as addressed in conventional MVDC research. Virtual dynamics do not generate new meta-information; instead, they integrate already extracted meta-information from multiple sensors and converge it into a single predicted value. This extension expands the applicability of MVDC from motion control to sensor information integration.

The first layer uses the pendulum period \(T_{\mathrm{period}}^{\mathrm{cam}}\) obtained from the high-speed camera as the input, whereas the second layer uses the pendulum period \(T_{\mathrm{period}}^{\mathrm{prox}}\) obtained from the proximity sensor as the input. The virtual dynamics of each layer are described by the following differential equations.

The series-connected structures expressed in Eqs. \(\eqref{eq:eq1}\) and \(\eqref{eq:eq2}\) enable flexible sensor integration independent of the measurement order and timing. The coupling term \(\ddot{\omicron}_1\) in Eq. \(\eqref{eq:eq2}\) allows the first-layer estimate to influence the second-layer estimate. When the camera information arrives first, \(\omicron_1\) converges to \(T_{\mathrm{period}}^{\mathrm{cam}}\) and propagates to \(\omicron_2\) through this coupling. When the proximity sensor information arrives, \(\omicron_2\) updates toward \(T_{\mathrm{period}}^{\mathrm{prox}}\) while incorporating the camera information. In general, whenever a new measurement arrives from either sensor, the estimate is updated automatically at that moment, regardless of the order in which the sensors provide the information. This design eliminates the need for manual switching logic, which depends on which sensor first provides the data.

At equilibrium (when \(\ddot{\omicron}_1 = \dot{\omicron}_1 = 0\) and \(\ddot{\omicron}_2 = \dot{\omicron}_2 = 0\)), Eq. \(\eqref{eq:eq1}\) yields \(\omicron_1 = T_{\mathrm{period}}^{\mathrm{cam}}\) and Eq. \(\eqref{eq:eq2}\) yields \(\omicron_2 = T_{\mathrm{period}}^{\mathrm{prox}}\), indicating that the second-layer output converges to the proximity sensor measurement. The virtual dynamics parameters of each layer were set as \(M_1=M_2=0.1\), \(D_1=D_2=2.8\), \(G_1=G_2=40.0\), with a numerical integration step size of \(\Delta t= 0.01\) s, which corresponds to the lower limit achievable on standard Windows OS. In practice, the actual update interval is approximately 15–16 ms (62–65 Hz) owing to the OS thread-scheduling overhead. Despite this lower MVDC update rate, the high-speed information required by the proposed method was fully utilized. The system comprised four independent operating parallel threads: a camera thread (1 kHz), proximity sensor thread (1 kHz), MVDC thread (100 Hz), and robot control thread. Period extraction was performed entirely within the dedicated sensor threads at 1 kHz, and the resulting period estimates were passed to the MVDC thread for each detection event. Therefore, the integration step size \(\Delta t= 0.01\) s for MVDC is independent of the sensor sampling rate. The mass–damper–spring parameters \(M_i\), \(D_i\), \(G_i\) were designed based on a well-established damping ratio \(\zeta = 1/\sqrt{2}\) (approximately 0.707), which is known to achieve the fastest convergence while allowing a moderate overshoot. As the predicted period \(\omicron_2\) determines the grasp timing rather than the robot tracking trajectories, transient overshoots are acceptable, making this damping ratio suitable for period prediction.

4.3. Determination of Gripper Grasp Timing

Algorithm 1 shows the pseudocode for determining the grasping timing based on MVDC. The MVDC thread continuously updates the predicted period \(\omicron_2\) and determines the earliest feasible grasping cycle \(N\) after the current time based on the first valid input time \(t_{\mathrm{first}}\). The start time of the grasping action is calculated as \(t_{\mathrm{grasp}} = t_{\mathrm{first}} + N \cdot \omicron_2 - T_{\mathrm{gr}}\), which is the predicted \(N\)-th equilibrium passage time minus the gripper response time \(T_{\mathrm{gr}}\).

When new sensor measurements become available (i.e., when \(T_{\mathrm{period}}^{\mathrm{cam}}\) or \(T_{\mathrm{period}}^{\mathrm{prox}}\) is updated), the virtual dynamics respond and the predicted period \(\omicron_2\) changes dynamically, necessitating convergence detection. In this method, the most recent \(W_{\mathrm{size}}\) predicted periods were monitored, and convergence was determined when the maximum relative deviation from the mean value fell below a threshold \(\epsilon_{\mathrm{conv}}\). In these experiments, \(W_{\mathrm{size}}=10\) and \(\epsilon_{\mathrm{conv}}=1\%\) were used. Convergence detection was enabled when a new measurement was obtained; after the first convergence following the camera measurement and the second convergence following the proximity sensor measurement, the integration of information from both sensors was confirmed.

4.4. Robot Trajectory Generation and Grasp Execution

In trajectory planning, smooth trajectories are generated using fifth-order polynomials in joint space. The trajectory was designed to satisfy the kinematic constraints of the robot, including the joint position range, maximum velocity, and maximum acceleration. The trajectory was divided into two stages: approach and grasping. In the approach stage, to avoid collision between the robot and pendulum trajectories, an intermediate position with a height offset added to the target position of the grasping stage was used. This intermediate position allowed the gripper to enter from below the pendulum trajectory, thereby enabling the object to pass safely between the gripper fingers. In the grasping stage, the robot moved from an intermediate position to the grasping position. When the start time of the grasping action \(t_{\mathrm{grasp}}\) determined by the MVDC (Section 4.3) was sent to the gripper, it waited until the specified time and then executed the grasping action.

5.1. Experimental Setting

Figure 4 shows the configuration of the experimental system. To physically realize the two-dimensional planar pendulum motion assumed in the task setting, a pendulum system was constructed using a pin joint with ball bearings as the rotation center and a rigid aluminum shaft connecting the joint to the pendulum object. This mechanical design constrained the pendulum motion to a single vertical plane. The high-speed camera was set to a frame rate of 1000 fps, a shutter speed of 1/2000 s, and a resolution of 1246\(\times\)1008. The proximity sensor module was sampled at 1 kHz. The gripper was operated at a fingertip velocity of 100 mm/s and a grasping force of 200 N. The kinematic constraints of the robot included a maximum joint range of motion of 180°, a maximum joint velocity of 90°/s, and a maximum joint acceleration of 90°/s\(^2\).

5.2. Results

Figure 5 shows the snapshots of the pendulum-grasping experiment. After releasing the pendulum object (Fig. 5(a)), the high-speed camera detected the first (Fig. 5(b)) and second equilibrium passages (Fig. 5(c)), from which the half-period \(T_{\mathrm{period}}^{\mathrm{cam}}\) was calculated and input to the first layer of the MVDC. Once the period information from the camera was obtained, the robot began moving toward the grasping position (Fig. 5(d)) and ascended via an intermediate position (Fig. 5(e)). When the robot approached the grasping position, the proximity sensor module detected the equilibrium passage (Fig. 5(f)), and the half-period \(T_{\mathrm{period}}^{\mathrm{prox}}\) was calculated and input to the second layer of the MVDC. After the convergence of the integrated period prediction from both sensors, the gripper closing action was executed at the predicted grasping timing (Fig. 5(g)), and grasping at the equilibrium point (lowest point) was successful (Fig. 5(h)).

Figure 6 shows the timeline of each thread referenced to the start time of the program. The four independent threads (camera, proximity sensor, MVDC, and robot control) operated asynchronously and were time-synchronized through unified time management. After the start of the program, system initialization and robot initial posture setting were performed, and approximately 20 s later, the pendulum object was manually released to begin the experiment. The operation of each thread is explained below, along with a quantitative analysis of the experimental data in chronological order.

In Fig. 6(a), the object position in the image coordinates was sampled at 1000 fps, and the half-period \(T_{\mathrm{period}}^{\mathrm{cam}}\) was calculated by peak detection. In the experiment, the first peak detection after the pendulum release was performed at 21.051 s, capturing the moment when the pendulum passed through the lowest point (equilibrium point). The second peak was detected at 21.677 s (0.626 s after the first), and the camera-based half-period \(T_{\mathrm{period}}^{\mathrm{cam}} = 0.626\) s was calculated from this time difference. After estimating the period from these two peaks, the robot began moving toward the grasping position, and the subsequent peaks (third and later) were not used for the period calculation. As the high-speed camera was mounted on the end effector of the robot, it moved simultaneously with the robot during the approach phase, causing the relative position between the camera and the object to change. Consequently, the image coordinates of the subsequent peaks were recorded at different heights, not because the trajectory of the object changed, but because the camera itself was moving. This is an unavoidable phenomenon in hand-eye systems and does not affect the period estimation accuracy because the period has already been calculated from the first two peaks acquired while the robot is stationary.

In Fig. 6(c), information from different sensors was integrated through two-layer virtual dynamics. When the period information obtained from the camera was input to the first layer of the MVDC (red line), period prediction began through virtual dynamics. At this point, the predicted grasping time was calculated to be 26.059 s (8th cycle).

In Fig. 6(b), the voltage signals \(V_1 - V_2\) (blue line) and \(V_1 + V_2\) (red line) were measured at 1 kHz, and the half-period \(T_{\mathrm{period}}^{\mathrm{prox}}\) was calculated using zero-crossing detection. In this experiment, the initial peak detection was performed at 25.941 s. This time was 0.118 s earlier than the predicted passage time of 26.059 s based on the camera-only prediction, indicating that the camera-alone period prediction could not accurately capture the actual pendulum motion. The time difference from the first camera peak was 4.891 s, and from this time difference and the camera period of 0.626 s, the pendulum was predicted to have completed approximately 7.813 oscillations (\(4.891/0.626=7.813\) s). Using the nearest integer number of oscillations (8), the proximity-sensor-based period was calculated as \(T_{\mathrm{period}}^{\mathrm{prox}} = 4.891/8 = 0.611\) s.

When this period information was input to the second layer of the MVDC (purple line), the camera and proximity sensor information was integrated and converged at 26.120 s. At this point, the predicted period was 0.615 s with a variation rate of 0.893%, satisfying the convergence criterion (below 1%). Upon detecting the second convergence, the MVDC thread sent a command for a grasping time of 26.163 s to the gripper control thread.

In Fig. 6(d), the gripper position trajectories (\(X\): red, \(Y\): green, \(Z\): blue) are shown, along with the grasping start and completion times. After waiting 0.043 s until the specified time, the gripper closing action began at 26.163 s (purple dashed line) and grasping was completed at 26.603 s, approximately 0.44 s later (orange dashed line).

Thus, the proposed method realized pendulum grasping through the following processes: (1) initial period prediction from global measurement by the camera (0.626 s), (2) accurate period prediction integrating local measurement by the proximity sensor (0.611 s \(\to\) 0.615 s), (3) convergence determination and optimal grasping timing determination by the MVDC (26.163 s), (4) successful grasping through accurate timing control. Compared with the camera-only prediction (0.626 s), integrating the proximity sensor improved the period prediction accuracy (0.615 s), enabling a more accurate prediction of grasping timing.

Comparative experiments were conducted over multiple trials to evaluate the robustness and effectiveness of the proposed method under varying initial conditions. We performed 20 trials for both the visual-only approach (camera-based period estimation) and proposed method (MVDC-based sensor integration). The pendulum was manually released from different initial angles during each trial to ensure diverse initial conditions.

The experimental results were as follows: the visual-only approach achieved 8 successes out of 20 trials (40% success rate), whereas the proposed method achieved 12 successes out of 20 trials (60% success rate). The proposed method demonstrated a 50% improvement in the success rate compared with the visual-only approaches. This improvement validates the effectiveness of integrating heterogeneous sensor information using an MVDC framework.

6.1. Effectiveness of Proposed Framework

First, we discuss the effectiveness of the information extraction process using the MVDC. As shown in Fig. 6(c), the period information from the sensors with different characteristics (high-speed camera and proximity sensor) was integrated through two-layer virtual dynamics and converged to the final period prediction. This result demonstrates that the MVDC can asynchronously integrate heterogeneous sensor information with different spatial scales and measurement principles. In particular, combining global measurement by a camera and local measurement by a proximity sensor has the potential to achieve a more robust period prediction than using a single sensor. The virtual dynamic architecture of MVDC achieves this continuous integration without explicit switching logic, rather than manually specifying when to transition from camera-based to proximity-sensor-based information; the series-connected virtual dynamics naturally blend the two contributions as sensor inputs arrive asynchronously.

Subsequently, we compared the proposed method with the conventional approaches. In conventional high-speed robot systems ¹⁰, the direct feedback control of dynamic objects is possible owing to the high-speed responsiveness of the robot. However, in this study, we used a conventional industrial robot (a maximum joint velocity of 90°/s), which made it difficult to respond to dynamic phenomena using direct feedback control. The proposed method overcomes this temporal-scale gap by extracting period information from detailed time-series data using high-speed sensors and predicting the optimal grasping timing based on this information. As shown in Fig. 5(h), the successful grasping at the equilibrium point, which is the most difficult position, is an important achievement that demonstrates the effectiveness of the proposed method. Note that the focus of conventional high-speed sensor studies is different from that in this study. Conventional approaches focus on realizing high-speed robot motions that can directly exploit fast sensors, whereas this study focused on methods that effectively utilize high-speed sensor information for robots with limited motion speeds. Given these fundamentally different objectives and problem settings, a direct performance comparison of the two approaches is difficult.

6.2. Applicability and Generalizability

The proposed framework is not limited to the specific task of pendulum grasping and can be applied to a wider range of high-speed tasks. It is important to extract meta-information about target phenomena from detailed time-series data using high-speed sensors and generate motion plans for conventional industrial robots based on that information. For example, this framework may be applicable to various tasks in which there is a temporal-scale gap between the sensors and robots, such as assembly work on vibrating objects or collaborative work with high-speed moving objects.

Importantly, the meta-information extracted by the proposed method is not necessarily used only for the future prediction of the target phenomena. It is also applicable for adaptively determining the state and actions of a robot based on phenomena observed in time-series information. For example, in coating work, by measuring liquid droplet bounce behavior with a high-speed sensor and predicting the thickness distribution of the liquid film from the time-series information, adaptive action planning according to the work state becomes possible, such as coating thinner areas more thickly. Thus, the proposed framework is flexible and applicable not only to future-prediction-type tasks but also to tasks requiring state prediction and adaptive control based on phenomenon observation.

From a practical perspective, the high-speed camera and proximity sensor used in this study have undergone miniaturization and cost reduction owing to recent technological advances, and the barriers to their introduction into industrial robot systems are decreasing. By applying the proposed method to existing industrial robots, the ability to respond to high-speed phenomena can be improved without accelerating the robot body itself, which is important from the perspective of the effective utilization of existing equipment.

6.3. Limitations and Future Challenges

In this case study, we selected the period characteristic based on the prior knowledge of the nature of the phenomenon, assuming that pendulum motion is a periodic phenomenon. For more complex aperiodic or unknown phenomena, it is difficult to identify which meta-information is necessary for task execution, and the selection of meta-information itself requires task-specific expertise. Developing methods for discovering and selecting the required meta-information is an important future challenge. However, once the appropriate meta-information is identified, the information processing design for extracting it (feature calculation and MVDC parameter adjustment) can be performed relatively systematically.

In addition, both sensors were operated at a sampling rate of 1 kHz. As the MVDC is designed to accept sensor inputs asynchronously through virtual dynamics, the framework is theoretically applicable to sensors with different sampling rates. The experimental validation of this multirate sensor integration capability is a future challenge.