Research Paper:
A Feature Fusion Method for IMU and sEMG Signals Based on Modal Reliability Weighting
Juan Zhao*1,*2
, Qin Peng*1,*3,*4
, Jie Chen*5,*6
, Seiichi Kawata*1,*3,*4
, and Jinhua She*7,

*1School of Artificial Intelligence and Automation, China University of Geosciences
No.388 Lumo Road, Hongshan District, Wuhan, Hubei 430074, China
*2Engineering Innovation Training Center, China University of Geosciences
No.388 Lumo Road, Hongshan District, Wuhan, Hubei 430074, China
*3Hubei Key Laboratory of Advanced Control and Intelligent Automation for Complex Systems
No.388 Lumo Road, Hongshan District, Wuhan, Hubei 430074, China
*4Engineering Research Center of Intelligent Technology for Geo-Exploration, Ministry of Education
No.300 Lumo Road, Hongshan District, Wuhan, Hubei 430074, China
*5Xiangjiang Laboratory, Changsha
Changsha, Hunan 410000, China
*6School of Computer Science, Hunan University of Technology and Business
No.569 Yuelu Avenue, Changsha, Hunan 410205, China
*7School of Engineering, Tokyo University of Technology
1404-1 Katakuramachi, Hachioji, Tokyo 192-0982, Japan
Corresponding author
Gait analysis has become a critical means of evaluating motor function by analyzing kinematic and physiological information during human locomotion. Inertial measurement unit (IMU) and surface electromyography (sEMG) are extensively utilized in gait data acquisition and gait analysis for their compact size and ease of wear. However, the single-modality sensors exhibit certain limitations in information representation and features of different modalities are often utilized unevenly during multimodal data fusion, which affect the robustness of fusion features. This paper presents a feature fusion method for IMU and sEMG signals based on modal reliability weighting. The method analyzes the statistical stability of features of different modalities to evaluate their reliability quantitatively and allocates fusion weights adaptively to achieve weighted feature fusion. Experimental results demonstrate that the presented method outperforms other feature fusion methods, such as equal fusion and single-modality analysis, in terms of classification accuracy and feature stability in the activity classification task. It also provides a promising and practical strategy for multimodal gait analysis and rehabilitation assessment.
1. Introduction
Gait is one of the most fundamental and common forms of human movement. It serves as a clear external indicator of an individual’s motor ability, neuromuscular function, and overall health and contains a wealth of kinematic and physiological information 1. Objective and quantitative analysis and the evaluation of gait data are of great significance in disease screening and the formulation of rehabilitation programs in certain fields, such as rehabilitation medicine and human locomotion analysis 2.
With the advancements in wearable sensor technology, gait analysis based on wearable sensor data has gradually become a hot and prominent research topic. In particular, inertial measurement unit (IMU) and surface electromyography (sEMG) are widely utilized in gait data acquisition and analysis for their low cost, compact size, and ease of wear. IMU typically consists of accelerometers and gyroscopes, which records kinematic information, such as acceleration, angular velocity, and gait rhythm continuously during walking, thereby intuitively reflecting lower-limb movement states and overall gait patterns 3. sEMG measures electromyographic activity signals on the skin surface and reveals muscle activation sequences, coordination patterns, and control strategies at the neuromuscular level, providing important physiological evidence for understanding the mechanisms underlying gait generation 4. Although IMU and sEMG signals possess distinct advantages in gait analysis, single-modality data often captures only one aspect of gait. Multi-source data fusion is an effective method for enhancing gait analysis performance. By fusing information from different modalities at the data, feature, or decision level, the limitations of single-modality data are compensated for to a certain extent, thereby improving the robustness and generalizability of the system. Among these fusion levels, feature fusion has garnered widespread attention in multimodal gait analysis since it preserves comprehensive modal information while being flexible and computationally efficient 5. Consequently, utilizing complementary information obtained from different sensors to construct a more comprehensive and robust gait feature representation has become an important research topic in gait analysis.
Owing to differences in sensor type, environmental conditions, and other factors, the stability and reliability of IMU and sEMG signals vary significantly in practical applications 6. Although IMU signals typically exhibit high stability, they are susceptible to drift and cumulative errors under complex motion conditions. On the other hand, sEMG signals are highly sensitive to muscle activity, but more prone to noise interference and individual variations 7. During feature fusion, the reliability of different modalities needs to be modeled properly instead of performing equal weight fusion. It is difficult to fully utilize the complementary advantages of multi-source or multimodal information. Consequently, fusion performance should be enhanced to adjust the fusion weight of each modality adaptively based on the statistical characteristics of features at the feature-level fusion stage.
To address the aforementioned problems, this paper presents a feature fusion method for IMU and sEMG signals based on modal reliability weighting. It evaluates the stability of features obtained from each modality quantitatively from the statistical perspective and adaptively allocates fusion weights to achieve weighted fusion of multimodal features based on modality stability. Experimental results demonstrate that the method exhibits high effectiveness and stability in the construction of multimodal gait features. This provides an interpretable and practically applicable fusion method for real-world applications, such as gait analysis and rehabilitation assessment.
The remainder of this paper is organized as follows. In Section 2, relevant research on multimodal gait analysis and feature fusion methods is reviewed. In Section 3, the IMU and sEMG feature fusion method based on modal reliability weighting is detailed. In Section 4, the experimental design and results are discussed. Finally, the paper is concluded with a summary and potential directions of future research are presented in Section 5.
2. Related Works
As the two most commonly utilized wearable sensors, IMU and sEMG characterize gait from the perspectives of kinematics and neuromuscular activity respectively, providing complementary sources of information for gait analysis. This section presents a systematic review of relevant research focusing on gait signals, features, and feature fusion methods to provide a theoretical foundation for the subsequent method and experimental design.
IMU captures linear acceleration and angular velocity data continuously during human locomotion. Relatively unaffected by environmental noise, the IMU signal exhibits good stability and engineering feasibility for practical applications 8. Conventional methods to IMU feature extraction analyze data across time-domain, frequency-domain, and time-frequency domains. Time-domain features, including mean, standard deviation (SD), peak value, and so on, describe the amplitude distribution and overall fluctuation characteristics of signals. Their calculation and direct extraction from raw data are straightforward and intuitive without requiring additional signal transformations. They are suitable for a variety of multi-source gait signals, such as plantar pressure and sEMG, and have been widely utilized in gait cycle analysis and motion recognition 9. Frequency-domain features transform raw signals into the frequency space by utilizing methods, such as the Fourier transform and power spectral density estimation, which characterize the periodicity, rhythmicity, and energy distribution of signals and reflect gait cycle stability and movement rhythm regularity. Furthermore, some research has introduced time-frequency analysis methods, such as wavelet transform and empirical mode decomposition to capture non-stationary gait characteristics. Yoon et al. acquired data utilizing a gait analysis system based on wearable IMUs 10. The participants walked at a steady speed in a controlled environment, while angular velocity, acceleration, and magnetic field data from each sensor were synchronously recorded for time-domain and frequency-domain feature extraction.
sEMG signal records muscular electrical activity on the skin surface, reflecting the nervous system’s control strategies over muscles and muscle coordination mechanisms and serving as an essential source of physiological information for gait analysis 11. Features from sEMG signal are similarly extracted from both time-domain and frequency-domain perspectives. Time-domain features characterize muscle activation intensity, whereas frequency-domain features are commonly adopted to analyze muscle fatigue and changes in activation patterns. De et al. preprocessed the collected back-muscle sEMG signals utilizing band-pass filtering, artifact removal, and denoising 12. Then, they extracted four temporal correlation features based on a fourth-order autoregressive model, 64 time-frequency domain features based on a four-level discrete wavelet transform and six basic statistical features, such as the mean ultimately obtaining a 74-dimensional sEMG feature vector. Zhou et al. conducted sEMG signal recognition for four lower-limb movements, namely stair ascent, stair descent, level walking, and squatting in 20 subjects 13. They extracted and standardized time-domain and frequency-domain features from the preprocessed sEMG signals. The time-domain features included the mean absolute value (MAV) and root mean square (RMS) while the frequency-domain features included mean power frequency extracted utilizing the power spectral density function. In addition, some research has integrated time-frequency analysis with nonlinear features to capture local frequency variations, transient energy distributions, and burst characteristics of signals at different time scales simultaneously, enhancing the representation of complex muscle activities 14.
Currently, there has been an increasing focus on multimodal fusion of IMU and sEMG in gait analysis and human activity recognition. Fusing features from IMU and sEMG has been demonstrated to improve analysis and recognition performance compared with the single-modality features. Representative methods primarily include feature concatenation, deep neural network-based feature learning, or attention mechanism-based multimodal fusion. Zhou et al. acquired lower-limb signals utilizing multi-channel sEMG and IMU sensors 15. They then extracted eight-dimensional time-domain and entropy features from sEMG and mean and extreme value features from IMU and then concatenated two sets of engineered feature vectors into a high-dimensional feature set, which was transmitted to classifiers to perform activity recognition. Li et al. acquired five-channel sEMG and IMU data during ankle movement and extracted six types of time-domain and energy features from the IMU for the classification of motion intensity labels 16. They extracted two types of modality-specific features and performed cross-modal deep feature fusion via bidirectional cross-attention mechanisms. Although these methods leverage the complementary information between IMU and sEMG features effectively, most of them concatenate features directly or allocate fusion weights randomly without considering the statistical reliability of different modalities.
To exploit the multi-source signals, researchers have presented a variety of multi-source signals methods. Depending on the fusion level, current methods are categorized as data-level, feature-level, and decision-level fusion 17. Feature-level fusion has been widely applied in gait analysis since it preserves information representation capability while maintaining relatively simple implementation and high computational efficiency. Consequently, feature-level fusion has become one of the most widely adopted methods for IMU and sEMG signals fusion. In early research, feature fusion was predominantly performed utilizing linear and statistical rule-based methods, such as feature concatenation and statistical pooling, whereby features were combined directly according to dimensionality or statistical rules. Wang et al. extracted linear and nonlinear features from EMG signals and concatenated standardized features of different types directly to form high-dimensional feature matrices, which were ultimately input into a neural network for motion recognition 18. Nweke et al. extracted and selected features obtained from accelerometers and gyroscopes 19. They then directly concatenated the selected feature vectors to achieve fusion forming a unified high-dimensional feature representation that was subsequently transmitted to a classifier for activity recognition. Feature concatenation exhibits strong compatibility and combines features of different types and dimensions flexibly without losing the original distribution characteristics of linear and nonlinear features. However, as the number of feature types increases, high-dimensional matrices introduce redundant information, thereby increasing the burden of model training and risk of overfitting. Such methods are simple to implement and retain all modality feature information, assuming that different modalities are equally important in the fusion process. The effectiveness of the fusion method relies heavily on feature quality. However, it fails to account for variations in signal quality and statistical characteristics across different modalities, which may introduce redundant information and affect the stability and discriminative capability of the fusion features.
In response to the limitations of fixed-weight fusion methods, research began to focus on modal reliability or uncertainty modeling and design adaptive weighted fusion methods accordingly. Such methods usually evaluate the credibility of different modalities from the perspectives of statistical characteristics, noise levels, or model uncertainty and adjust their contribution ratios dynamically during the fusion process 20. Weight design based on variance, signal-to-noise ratio, or error distribution characterizes modality stability quantitatively without introducing complex model structure. Gu presented an improved multi-sensor data fusion method that combines the jackknife resampling technique with adaptive weighting. First, the fusion was accomplished by utilizing the resampling to compensate for data bias variance, then allocating weights dynamically based on the inverse of the variance of each sensor’s estimates and finally performing a weighted summation 21. The algorithm adjusted the fusion strategy adaptively according to the real-time characteristics of the sensor data enhancing the accuracy and robustness of the fusion method. Alteriis et al. analyzed the raw measurement data from the redundant IMU sensors, utilizing the Allan variance to extract two core noise parameters: bias instability and random walk 22. Subsequently, they utilized the reciprocal of the product of these noise parameters for each sensor as the fusion weight with less noise, resulting in higher weights. Finally, they fused three-axis raw data for acceleration, angular velocity, and magnetic field utilizing a weighted average formula to obtain the final fused measurement values. In contrast to the methods that introduce attention mechanisms or gating structures into deep learning, weighted fusion based on statistical reliability exhibits lower computational cost and higher interpretability. By introducing modal reliability modeling at the feature level, it is possible to suppress the negative impact of unstable modalities on the overall features while maintaining fusion flexibility, thereby enhancing the stability and effectiveness of the fusion features.
In summary, IMU and sEMG signals provide complementary kinematic and physiological information for gait analysis. Research have demonstrated the effectiveness of multimodal fusion of IMU and sEMG features. However, current methods still rely on direct feature concatenation or empirically fixed fusion weights, which cannot leverage the statistical reliability differences between feature modalities sufficiently. In the following section, we present a feature fusion method for IMU and sEMG signals based on modal reliability weighting.
3. Method
Considering the discrepancies among the features of different modalities in statistical stability and signal reliability during feature fusion, this paper presents a feature fusion method based on modal reliability weighting. The method evaluates the reliability of different modality features from the perspective of statistical stability and allocates fusion weights adaptively based on modal reliability to construct a more robust multimodal feature representation. In contrast to conventional feature-level fusion methods, which often rely on simple feature concatenation or fixed-weight fusion, the weighted fusion method quantifies modal reliability through statistical fluctuation and performs adaptive weighting to improve the contribution of the modality with superior stability to the fusion process.
3.1. Weighted Fusion Method
In gait analysis, IMU records acceleration and angular velocity describing the kinematic characteristics of gait, whereas sEMG reflects the neuromuscular activation patterns from a physiological perspective. Accordingly, fusing the IMU and sEMG data to some extent compensates for the limitations of single-modality information. However, owing to the differences in sampling mechanisms and noise characteristics between the two types of sensors, the signals usually exhibit significant differences in stability and statistical distribution. IMU signal usually exhibits relatively high temporal stability but may be subjected to drift and cumulative errors. In contrast, although sEMG signal can reflect muscle activity information, it is more susceptible to factors, such as electrode placement, skin impedance and individual variability. As a consequence, if the fixed-weight fusion is adopted for multimodal feature fusion simply, the modality with poorer stability may exert an excessively large impact on the fusion results, thereby reducing the overall feature quality.
In this context, this paper presents a feature fusion method based on modal reliability weighting in response to these problems. The workflow of the method mainly comprises the following steps: data preprocessing, feature construction, modal reliability estimation, and weighted feature fusion (Fig. 1). Specifically, the method begins by extracting statistical features from IMU and sEMG signals respectively, thereby constructing the initial feature representations for each modality. Subsequently, the overall stability of each modality is evaluated quantitatively by calculating the variance of the modality features across samples. Given that feature variance reflects the degree of fluctuation in the distribution of features across different samples, the reciprocal of feature variance is adopted as a metric for modal reliability. On this basis, fusion weights are then derived to perform weighted feature fusion, enabling the adaptive adjustment of the contribution level of each modality feature. The method explicitly incorporates modality stability information within the feature-level fusion process without introducing a complex model structure or additional learning parameters. Moreover, it maintains implementation simplicity and engineering deployability and enhances the statistical stability and discriminative performance of the fusion features effectively.

Fig. 1. Workflow of the feature fusion method based on modal reliability weighting.
3.2. Data Preprocessing and Feature Construction
In multimodal gait analysis, raw signals from sensors usually exhibit distinct orders of magnitude and value scales across different modalities. Performing feature fusion directly may result in modalities with larger numerical ranges dominating the fusion process, thereby compromising the fusion performance. Therefore, it is necessary to preprocess and standardize the raw data before feature extraction and feature fusion.
We assume that the matrix obtained by reading any single sample file is defined as \({X \in \mathbb{R}^{N \times D}}\), where \(N\) represents the number of time sampling points of the sample and \(D\) represents the total number of channels. If feature fusion is performed directly on raw data, large-scale features tend to dominate the fusion results. Accordingly, to eliminate the differences in physical dimensions and numerical ranges among the various channels, the signal of each channel is standardized separately utilizing \(Z\)-score standardization
Following the initial sample standardization, the statistical features are extracted from the modal signal of each sample. We assume that a standardized single-modality signal is represented by an \({N\times D}\) matrix, where \(N\) represents the number of temporal sampling points and \(D\) represents the number of channels. The following eight types of statistical features are extracted channel by channel (Table 1): Mean, Median, Maximum (Max), Minimum (Min), Variance, Root mean square (RMS), Peak-to-peak value (PTP), and Mean absolute value (MAV). Taking the sequence of the \(j\)-th channel, represented by \(y_{j}\) as an example, the mathematical definitions of the eight types of statistical features are listed in Table 1.
Through applying these preprocessing operations, the time series for each channel is converted into a set of statistical features. By concatenating different features across channels, we obtain a single-modality feature vector
| Feature | Mathematical equation |
| Mean | \(\mu_{j} = \displaystyle \dfrac{1}{N} \sum_{t=1}^{N} y_{j}(t)\) |
| Median | \(y_{j}^{\text{med}} = \displaystyle \operatorname*{median}_{t} \bigl(y_{j}(t)\bigr)\) |
| Maximum | \(y_{j}^{\text{max}} = \displaystyle \max_{t} \bigl(y_{j}(t)\bigr)\) |
| Minimum | \(y_{j}^{\text{min}} = \displaystyle \min_{t} \bigl(y_{j}(t))\) |
| Variance | \(\operatorname{Var}\bigl(y_{j}\bigr) = \displaystyle \dfrac{1}{N} \sum_{t=1}^{N} \bigl(y_{j}(t)-\mu_{j}\bigr)^{2}\) |
| RMS | \(\mathrm{RMS}_{j} = \sqrt{\displaystyle \dfrac{1}{N} \sum_{t=1}^{N} y^{2}_{j}(t)}\) |
| PTP | \(\mathrm{PTP}_{j} = y_{j}^{\text{max}} - y_{j}^{\text{min}}\) |
| MAV | \(\mathrm{MAV}_{j} = \displaystyle \dfrac{1}{N} \sum_{t=1}^{N} \left\lvert y_{j}(t) \right\rvert\) |
By repeating the aforementioned process for all samples, feature matrices for the IMU and sEMG modality are constructed, thereby providing the foundational input for weighted feature fusion based on modal reliability.
3.3. Modal Reliability Estimation
During multimodal feature fusion, different modalities typically exhibit significant differences in statistical stability and signal noise. If we simply adopt the fixed-weight feature fusion method, the modality with higher noise may exert an excessive impact on the fusion results, thereby reducing the stability of the overall feature representation. In this paper, the reliabilities of different modalities are quantified utilizing statistical methods and modality weights for feature fusion are estimated and calculated accordingly. Specifically, we utilize the cross-sample variance to characterize the degree of fluctuation in modality features across the sample set. We assume that the feature matrix of the IMU is defined as \({F_{\mathrm{imu}} \in \mathbb{R}^{M \times D_{\mathrm{imu}}}}\), where \(M\) represents the number of samples and \(D_{\mathrm{imu}}\) represents the dimensionality of \(F_{\mathrm{imu}}\). The feature vector of the \(k\)-th dimension is represented by \(F_{\mathrm{imu}}^{(k)}\). Therefore, the variance of this feature along the sample dimension is
In order to obtain the stability indicator of the modality, the variances of all feature dimensions are averaged to derive the overall variance of \(F_{\mathrm{imu}}\).
As a result, we follow the same procedure to derive the overall variance of \(F_{\mathrm{semg}}\).
The indicator reflects the degree of fluctuation in modality features across different samples. Generally, a smaller variance indicates that the features are more stable and the modal reliability is higher, whereas a larger variance suggests that the modality features are more strongly affected by noise or individual differences.
3.4. Weighted Feature Fusion
Based on the statistical stability indicators of the IMU and sEMG modality features, we design a weighted fusion method in which fusion weights are allocated according to modality stability such that the modality with higher stability is allocated a larger fusion weight.
First, according to the feature variances of each modality, the initial weights are separately calculated:
Since the IMU modality contains substantially more feature dimensions than the sEMG modality, the direct application of inverse-variance weighting further reduces the contribution of sEMG features during the fusion process. However, despite their relatively small dimensionality, sEMG features provide complementary physiological information regarding muscle activation that cannot be obtained based solely on IMU signals. Therefore, an enhancement coefficient \(\alpha\) (\({\alpha>1}\)) is introduced into the weight calculation process to increase the contribution of the sEMG modality moderately while maintaining the modal reliability weighting. If \(\alpha\) is too small, the contribution of the sEMG modality remains highly insufficient to complement the IMU features effectively. Conversely, an excessively large value may amplify inherently unstable sEMG features and reduce the robustness of the final fusion representation. Therefore, after introducing \(\alpha\), the adjusted \(\tilde{\omega}'_{\mathrm{semg}}\) is
Finally, the modality weights are normalized and the final fusion weights of the two modality features are
In the meantime, the fusion weights of the two modality features after normalization satisfy the following constraint:
Following the acquisition of the final modality weights, we perform the weighted feature fusion and obtain the weighted fusion feature vector
The fusion method adjusts the fusion weights adaptively according to the statistical stability of features obtained from different modalities. This process enhances the stability and discriminative capability of the fusion features while maintaining a simple methodological structure and achieving an effective fusion of multimodal features.
The presented method in this paper achieves the adaptive fusion of IMU and sEMG multimodal gait features without relying on a complex model structure. It provides a more stable and discriminative feature representation for subsequent gait analysis and activity classification tasks.
4. Experiments and Results
We conducted experiments on the public human activity dataset “HuGaDB” and performed comparative analyses of different fusion methods to validate the effectiveness of the feature fusion method based on modal reliability weighting. The experiments consist of three parts. First, by analyzing the variance distribution of fusion features across samples, the stability of features under different fusion methods is compared. Second, by further analyzing the contribution ratios of different modalities within fusion features, the impact of the weighted fusion method on the utilization of multimodal information is evaluated. Finally, by constructing an activity state classification task, the performances of different feature construction methods are validated. Based on this multi-perspective experimental analysis, we comprehensively validate the effectiveness of fusion methods in multimodal gait feature construction.
4.1. Experiment Settings
The dataset “HuGaDB” contains multimodal signal data acquired from the human body under different activity states utilizing wearable sensor systems comprising IMU and sEMG sensors 23. IMU records acceleration and angular velocity during human locomotion, whereas sEMG records electrical activity of muscles during motion. The dataset contains action sample files from 18 participants (14 males and 4 females) with variations in age and body shape to ensure the diversity in experimental data. It also covers 12 categories of daily human activities including walking, running, going up, going down, standing, bicycling, and other movements. Each sample file consists of multi-channel observations at multiple time sampling points. Each data file contains 39 signal columns, of which the first 36 columns are IMU channels, the 37th and 38th columns are sEMG channels and the last column corresponds to the activity label information.
Four typical activities: walking, running, bicycling, and standing are selected as the experimental subjects. Eight statistical measures (Table 1) are extracted from each signal channel to construct fixed-dimensional feature vectors for feature extraction. For the IMU modality, the dataset contains IMU data from 36 channels resulting in a feature matrix with \({36\times 8=288}\) dimensions. For the sEMG modality, the dataset contains data from 2 channels resulting in a feature matrix with \({2\times 8=16}\) dimensions. Following the aforementioned process, single-modality feature matrices are constructed for IMU and sEMG.
To validate the effectiveness of the presented method comprehensively, we compare four feature fusion methods under the same experimental settings, classifier, and evaluation metrics:
-
(1)
IMU-only: Only utilizing IMU features;
-
(2)
sEMG-only: Only utilizing sEMG features;
-
(3)
Equal fusion: Fixed-weighted feature fusion, the fusion weights of IMU and sEMG features are both 0.5;
-
(4)
Weighted fusion: Weighted feature fusion based on modal reliability weighting presented in this paper.
Specifically, IMU-only and sEMG-only are considered as single-modality baselines with equal fusion as the conventional fixed-weight fusion method and weighted fusion as the presented fusion method. Utilizing the logistic regression (LR) classifier for all comparison methods ensures that the performance differences are mainly attributed to the feature fusion methods rather than the classifier. These methods are selected to provide representative single-modality and conventional feature fusion baselines under a unified experimental framework, enabling a fair evaluation of the effectiveness of the weighted method while minimizing the impact of other experimental factors.
4.2. Implementation of Weighted Feature Fusion
After completing the construction of single-modality features, we evaluate the statistical stability corresponding to different modalities. According to the method presented in Section 3, the overall variances of the IMU and sEMG features are first calculated utilizing Eqs. (4) and (5) respectively, which are then utilized as quantitative indicators of modality stability. Subsequently, fusion weights are allocated based on the modality with higher stability, where the modality with higher stability receives a larger weight. Accordingly, the inverse of the variance is adopted as the initial weight in accordance with Eqs. (6) and (7). However, considering the number of sEMG signal channels is small and its feature dimensionality is significantly lower than that of IMU feature, an enhancement coefficient \(\alpha\) is introduced to compensate for the imbalance in modality contribution. In this experiment, the enhancement coefficient is set to \({\alpha=4}\) in Eq. (8), ensuring a reasonable balance between enhancing the complementary contribution of the sEMG modality and maintaining the modal reliability weighting. Following that, the modality weights are standardized utilizing Eqs. (9) and (10) and the resulting feature fusion weights of the two modalities are obtained as \({\omega_{\mathrm{imu}}=0.68}\) and \({\omega_{\mathrm{semg}}=0.32}\) respectively. Finally, the fusion feature vector \(F_{\mathrm{fusion}}\) is constructed utilizing Eq. (12). In this way, the fusion weights are adjusted adaptively based on the statistical stability of features from different modalities, thereby achieving effective multimodal feature fusion while maintaining a simple algorithmic structure.
In order to evaluate the stability of the fusion features, we conduct a statistical analysis of the variance distribution of different feature dimensions across the samples. We assume that the fusion feature vector is defined as \({F\in \mathbb{R}^{M \times D}}\), where \(M\) represents the number of samples and \(D\) represents the dimensionality of the features. For the \(k\)-th feature dimension of the fusion features, its variance across all samples is
On this basis, we conduct statistical analysis on the variance across all feature dimensions and calculate three representative statistical indicators, namely the mean, median, and SD values of the feature variance, so as to further describe the distribution characteristics of the overall feature variance. The corresponding calculation equations of these indicators are
The contributions of the IMU and sEMG modality in the fusion characteristics are analyzed quantitatively by the energy ratio indicator to quantify the contribution ratio of different modalities in the fusion characteristics. Feature energy is defined as the sum of the squares of the modality feature vector, which is utilized to describe the overall amplitude of that modality within the fusion features.
Based on the aforementioned definition, the feature energy of the IMU modality is
As a result, we follow the same procedure to derive the feature energy of the sEMG modality
Therefore, the contribution ratio of IMU modality in the fusion features is
4.3. Validation of Classification Performance
We conduct a multiclass activity classification experiment based on the “HuGaDB” dataset to further validate the practical effectiveness of the weighted fusion method in downstream tasks. Considering the representativeness of gait patterns and the availability of recorded data, four representative activities, walking, running, bicycling, and standing are selected as classification targets. Although the numbers of samples differ between these activities, they provide representative motion patterns for evaluating the weighted fusion method. In terms of the classification model, LR is adopted as the baseline classifier. To improve the model generalizability, feature standardization is performed during the training process and five-fold cross-validation with random shuffling is adopted for model evaluation 24. The dataset is partitioned at the sample level rather than the subject level, while preserving the class distribution in each fold.
The specific procedure of the classification experiment is described below. First, all samples are categorized into five mutually exclusive subsets according to the class distribution. Within each fold, four subsets are utilized for training and the remaining subset is utilized for testing. Following feature standardization, the LR classifier is trained and evaluated. Given the uneven sample sizes across the selected activities, the LR classifier is implemented with balanced class weights to reduce the impact of class imbalance during the training process. Finally, the mean and SD of the performance metrics over the five folds are reported as the final results.
Four metrics, Accuracy, Precision, Recall, and F1-score, are adopted to evaluate the classification performance quantitatively. Accuracy represents the proportion of correctly classified samples among all samples, measuring the overall classification performance. Precision refers to the proportion of the samples predicted as a specific class that truly belong to that class, thus indicating the precision of the classification results. Recall characterizes the proportion of samples from a certain class that are correctly identified, reflecting the model’s capacity to recognize samples of that class. The corresponding calculation equations of these metrics are
F1-score is defined as the harmonic mean of Precision and Recall. It comprehensively reflects the overall performance of the classification model and provides a balanced performance indicator. The calculation equation of F1-score is
For multiclass classification, the F1-score is calculated for each class and the arithmetic mean of these F1-scores is adopted to obtain the Macro-F1. This metric is capable of reducing the impact of class imbalance on the overall evaluation results and better reflecting the comprehensive model performance across all categories. The calculation equation of Macro-F1 is
4.4. Results and Results Analysis
We validate the weighted fusion method by multiple groups of experiments, yielding several sets of experimental results. The mean of feature variance measures the overall degree of feature fluctuation. The median robustly reflects the overall level of feature variance and the SD describes the dispersion of the variance distribution across different feature dimensions. Table 2 presents the statistical results of feature variance under different fusion methods. As is evident from Table 2, the weighted fusion method exhibits a slightly higher mean and median feature variance than the equal fusion method, while exhibiting a lower SD of feature variance. This suggests that the weighted fusion method does not simply minimise the overall feature variance but adjusts the contributions of different modalities to obtain a more consistent distribution of feature variability. Therefore, the statistical results presented in Table 2 primarily characterize the distribution of the fusion features. The effectiveness of the weighted fusion method is further evaluated in terms of the modality contribution analysis and classification performance.
| Fusion method | Mean of feature variance | Median of feature variance | SD of feature variance |
| Equal fusion | 0.115 | \(1.031\times10^{-5}\) | 0.341 |
| Weighted fusion | 0.159 | \(1.902\times10^{-5}\) | 0.336 |
The modality contribution ratio quantifies the relative contributions of the IMU and sEMG modality to the fusion features. This helps determine whether the fusion result is overly biased towards one modality, thereby providing a basis for evaluating the rationality of the weighted fusion method. A bar chart illustrating the modality contribution ratio of the IMU under different fusion methods is obtained via statistical analysis of the modality contribution ratio of all samples (Fig. 2). In the weighted fusion method, the mean contribution ratio of the IMU modality in the fusion features is 0.973, which is higher than that in equal fusion method (0.893). Moreover, its SD is 0.017, which is smaller than that of the equal fusion method (0.060). The results indicate that modal reliability weighting allocates a larger contribution to the IMU modality with higher stability. Nevertheless, the small contribution ratio of sEMG does not suggest that its information is negligible, as it provides complementary muscle activation cues that are not available from IMU. The method reallocates modality information rather than eliminating it: IMU provides the dominant kinematic information and sEMG supplements it with neuromuscular information. This contributes to a more discriminative fusion feature and helps to improve the final classification performance.
We perform an activity state classification experiment based on multimodal fusion features to further validate the effectiveness of the weighted fusion method in practical tasks. Four typical activities are selected as classification targets and classification experiments are conducted utilizing different feature construction methods to evaluate the performance of the fusion features in an activity classification task. In terms of classification performance, Fig. 3 and Table 3 compare the performances of different feature construction methods in the classification task. The results demonstrate that among the single-modality methods, the Accuracy of IMU-only is 0.967 and that of sEMG-only is 0.772, both of which are lower than those of the multimodal fusion methods. This indicates that single-modality features exhibit certain limitations, whereas multimodal fusion provides a much more comprehensive feature representation and significantly achieves better classification performance. Among the fusion methods, the weighted fusion method achieves a mean Accuracy of 0.988 and a mean Macro-F1 of 0.989, both of which exceed those of the fixed equal fusion method with Accuracy and Macro-F1 of 0.968 and 0.966 respectively. Furthermore, the two SDs are basically identical, indicating that the weighted fusion method exhibits clear advantages in terms of overall classification performance.
These results indicate that compared to single-modality features, multimodal fusion features leverage the data information obtained from the IMU and sEMG more effectively, thereby yielding a more comprehensive representation of motion characteristics. Compared to the equal fusion method, the weighted fusion method evaluates the stability of different modality features quantitatively based on statistical analysis. It then adjusts the modality weights adaptively based on the stability. In this way, the contribution ratio of each modality is allocated more rationally during the fusion process. The weighted fusion method consistently outperforms other methods in terms of Accuracy, Macro-F1, and other evaluation metrics. Combined with the analysis presented in Table 2 and Fig. 2, these results demonstrate that the weighted fusion method produces a more reliable multimodal feature representation and ultimately improves classification performance.

Fig. 2. IMU modality contribution ratio under different methods.

Fig. 3. Comparison of classification performance metrics under different methods.
| Fusion method | Mean of Accuracy | SD of Accuracy | Mean of Macro-F1 | SD of Macro-F1 |
| IMU-only | 0.967 | 0.013 | 0.965 | 0.015 |
| sEMG-only | 0.772 | 0.048 | 0.712 | 0.058 |
| Equal fusion | 0.968 | 0.014 | 0.966 | 0.015 |
| Weighted fusion | 0.988 | 0.013 | 0.989 | 0.014 |
5. Conclusions
This paper presents a feature fusion method for IMU and sEMG signals based on modal reliability weighting. By quantitatively evaluating the statistical stability of modality features, the method allocates fusion weights adaptively to achieve effective multimodal feature fusion without introducing a complex model structure. The main contributions of this paper are as follows:
-
(1)
The concept of modal reliability is introduced and a modality stability quantification method is constructed utilizing overall variance as an indicator by analyzing the statistical fluctuation of IMU and sEMG features across the sample dimensions, providing a data basis for the determination of fusion weights.
-
(2)
A feature fusion method based on modal reliability weighting is presented. It allocates fusion weights according to modality stability such that the modality with higher stability is allocated a larger fusion weight. The method achieves adaptive weighted fusion of multimodal features without introducing complex model structure, thereby enhancing the expressive power and stability of the fusion features.
-
(3)
Different feature construction methods are systematically validated by combining statistical analysis of fusion features with classification experiments. The experimental results reveal that the method outperforms other methods in terms of feature distribution balance and classification accuracy, thereby validating its effectiveness and engineering feasibility in multimodal gait feature construction.
Despite the aforementioned contributions, several limitations should be acknowledged. First, the sources of experimental data are relatively limited. Second, the features adopted in this paper are primarily statistical and the fusion weights rely on static statistical features without considering dynamic variations during the gait process. Third, the method is evaluated utilizing representative baseline methods within a unified experimental framework, while comparisons with a broader range of feature fusion methods require further investigation. Future work will incorporate gait data from more diverse sources and explore diverse features for further validation. In addition, we will expand the experimental evaluation by including comparisons with more feature fusion methods to further validate the generalizability of the method. Furthermore, we intend to introduce machine learning algorithms to establish a dynamic weight adjustment mechanism that enables fusion weights to adapt to the changes in movement states and signal quality. The method is also expected to be applied to rehabilitation assessment and motion monitoring to further enhance its practical utility.
Acknowledgments
This work was supported in part by the Major Program of Xiangjiang Laboratory under Grant 25XJB01; the National Science Foundation of Hubei Province, China, under Grant 2020CFA031; Fundamental Research Funds for the Central Universities, China University of Geosciences (Wuhan), under Grant CUG240635; the “CUG Scholar” Scientific Research Funds at China University of Geosciences (Wuhan) under Grant 2022029; and JSPS (Japan Society for the Promotion of Science) KAKENHI under Grant 24K03325.
- [1] G. Cicirelli et al., “Human gait analysis in neurodegenerative diseases: A review,” IEEE J. Biomed. Health Inform., Vol.26, No.1, pp. 229-242, 2022. https://doi.org/10.1109/JBHI.2021.3092875
- [2] A. A. Carmo, A. F. R. Kleiner, P. H. Lobo da Costa, and R. M. L. Barros, “Three-dimensional kinematic analysis of upper and lower limb motion during gait of post-stroke patients,” Braz. J. Med. Biol. Res., Vol.45, No.6, pp. 537-545, 2012. https://doi.org/10.1590/S0100-879X2012007500051
- [3] M. Bonanno et al., “Gait analysis in neurorehabilitation: From research to clinical practice,” Bioengineering, Vol.10, No.7, Article No.785, 2023. https://doi.org/10.3390/bioengineering10070785
- [4] V. B. Semwal, N. Gaud, P. Lalwani, V. Bijalwan, and A. K. Alok, “Pattern identification of different human joints for different human walking styles using inertial measurement unit (IMU) sensor,” Artif. Intell. Rev., Vol.55, No.2, pp. 1149-1169, 2022. https://doi.org/10.1007/s10462-021-09979-x
- [5] X. Ru, N. Gu, H. Shang, and H. Zhang, “MEMS inertial sensor calibration technology: Current status and future trends,” Micromachines, Vol.13, No.6, Article No.879, 2022. https://doi.org/10.3390/mi13060879
- [6] S. R. Estévez, J. Mallah, D. Kazieczko, C. Tang, and L. G. Occhipinti, “Deep learning for motion classification in ankle exoskeletons using surface EMG and IMU signals,” Sci. Rep., Vol.15, Article No.38242, 2025. https://doi.org/10.1038/s41598-025-22103-1
- [7] Y. Chen et al., “Biomechanical monitoring of exercise fatigue using wearable devices: A review,” Bioengineering, Vol.13, No.1, Article No.13, 2026. https://doi.org/10.3390/bioengineering13010013
- [8] S. M. Sid’El Moctar, I. Rida, and S. Boudaoud, “Comprehensive review of feature extraction techniques for sEMG signal classification: From handcrafted features to deep learning approaches,” IRBM, Vol.45, No.6, Article No.100866, 2024. https://doi.org/10.1016/j.irbm.2024.100866
- [9] Y. Lu, J. Zhu, W. Chen, and X. Ma, “An IMU-based real-time gait detection method for intelligent control of knee assistive devices,” IEEE Trans. Instrum. Meas., Vol.72, Article No.2532309, 2023. https://doi.org/10.1109/TIM.2023.3329222
- [10] D. H. Yoon et al., “Inertial measurement unit sensor-based gait analysis in adults and older adults: A cross-sectional study,” Gait Posture, Vol.107, pp. 212-217, 2024. https://doi.org/10.1016/j.gaitpost.2023.10.006
- [11] A. Vijayvargiya, B. Singh, R. Kumar, and J. M. R. S. Tavares, “Human lower limb activity recognition techniques, databases, challenges and its applications using sEMG signal: An overview,” Biomed. Eng. Lett., Vol.12, No.4, pp. 343-358, 2022. https://doi.org/10.1007/s13534-022-00236-w
- [12] S. De, P. Mukherjee, and A. H. Roy, “GLEAM: A multimodal deep learning framework for chronic lower back pain detection using EEG and sEMG signals,” Comput. Biol. Med., Vol.189, Article No.109928, 2025. https://doi.org/10.1016/j.compbiomed.2025.109928
- [13] Z. Zhou et al., “Lower limb motion recognition based on sEMG and CNN-TL fusion model,” Sensors, Vol.24, No.21, Article No.7087, 2024. https://doi.org/10.3390/s24217087
- [14] Q. Hu et al., “EA-Net: Edge-aware network for brain structure segmentation via decoupled high and low frequency features,” Comput. Biol. Med., Vol.150, Article No.106139, 2022. https://doi.org/10.1016/j.compbiomed.2022.106139
- [15] B. Zhou et al., “Non-invasive dual attention TCN for electromyography and motion data fusion in lower limb ambulation prediction,” J. Neural Eng., Vol.19, No.4, Article No.046051, 2022. https://doi.org/10.1088/1741-2552/ac89b4
- [16] C. Li et al., “TTCLNet: Ankle foot motion classification model based on wireless wearable sEMG and IMU sensors,” Sens. Actuators A: Phys., Vol.408, Article No.118042, 2026. https://doi.org/10.1016/j.sna.2026.118042
- [17] H. Feng et al., “Security of target recognition for UAV forestry remote sensing based on multi-source data fusion transformer framework,” Inf. Fusion, Vol.112, Article No.102555, 2024. https://doi.org/10.1016/j.inffus.2024.102555
- [18] J.-Y. Wang, Y.-H. Dai, and X.-X. Si, “Feature layer fusion of linear features and empirical mode decomposition of human EMG signal,” J. Electron. Sci. Technol., Vol.20, No.3, Article No.100169, 2022. https://doi.org/10.1016/j.jnlest.2022.100169
- [19] H. F. Nweke, Y. W. Teh, G. Mujtaba, U. R. Alo, and M. A. Al-garadi, “Multi-sensor fusion based on multiple classifier systems for human activity identification,” Hum.-Cent. Comput. Inf. Sci., Vol.9, No.1, Article No.34, 2019. https://doi.org/10.1186/s13673-019-0194-5
- [20] S. Chaabene, A. Boudaya, B. Bouaziz, and L. Chaari, “An overview of methods and techniques in multimodal data fusion with application to healthcare,” Int. J. Data Sci. Anal., Vol.20, No.4, pp. 3093-3117, 2025. https://doi.org/10.1007/s41060-025-00715-0
- [21] M. Gu, “Improved Kalman filtering and adaptive weighted fusion algorithms for enhanced multi-sensor data fusion in precision measurement,” Informatica, Vol.49, No.10, pp. 85-100, 2025. https://doi.org/10.31449/inf.v49i10.7122
- [22] G. de Alteriis et al., “Innovative fusion strategy for MEMS redundant-IMU exploiting custom 3D components,” Sensors, Vol.23, No.5, Article No.2508, 2023. https://doi.org/10.3390/s23052508
- [23] R. Chereshnev and A. Kertész-Farkas, “HuGaDB: Human gait database for activity recognition from wearable inertial sensor networks,” Proc. 6th Int. Conf. Anal. Images Soc. Netw. Texts, pp. 131-141, 2017. https://doi.org/10.1007/978-3-319-73013-4_12
- [24] V. K. Verma, K. Saxena, and U. Banodha, “Analysis effect of K values used in K fold cross validation for enhancing performance of machine learning model with decision tree,” Proc. 13th Int. Adv. Comput. Conf., Part 1, pp. 374-396, 2023. https://doi.org/10.1007/978-3-031-56700-1_30
This article is published under a Creative Commons Attribution-NoDerivatives 4.0 Internationa License.