CN116129464A - Human body upper limb posture estimation method based on progressive unscented Kalman filter network - Google Patents

Abstract

A method for estimating the posture of human upper limbs based on progressive unscented Kalman filter network, comprising: step 1) building a set of acquisition system, obtaining angle information of human upper limb motion joints through a visual capture system, and using a myoelectric wristband to monitor the movement of the forearm Collect the EMG signals during exercise; step 2) establish a model between the sEMG signal and the joint angle of the upper limb; step 3) construct an unscented Kalman filter system with parameters estimated by LSTM, and use the LSTM neural network to learn the measurement of the system Equation h( ), system noise Q _k and measurement noise R _k ; with UT variation, weighted statistical linear regression is used to approximate the posterior mean of the nonlinear equation

and variance P _k|k‑1 ; Step 4) According to UT transformation and measurement noise R _k , obtain the one-step prediction of the measured variable

Variance P _zz,k|k-1 and cross-covariance P _xz,k|k-1 ; Step 5) Calculate the Kalman gain according to the calculation steps of UKF, and calculate the Kalman gain according to the measured value z _k at time k. state estimation

and estimated variance P _k|k .

Description

A method for human upper limb posture estimation based on progressive unscented Kalman filter network

Technical Field

The invention belongs to the field of human body posture estimation, in particular to a method for estimating human upper limb posture based on a progressive unscented Kalman filter network.

Background Art

With the continuous maturity and development of human-computer interaction technology based on bioelectric signals, this technology has been widely used in prosthetic control, clinical medicine, virtual reality technology, robotics and sports biomechanics. Surface electromyography (sEMG) is a superimposed electrical signal formed on the surface of human skin by the motor unit action potential (MUAP) of the movement-related muscle propagating along the direction of the muscle fiber. This signal reflects the muscle contraction state that causes limb movement, contains rich information such as muscle contraction force and joint torque, and can decode the direct movement-related intention from it. It is widely used in human posture estimation and human behavior analysis. Compared with the recognition of discrete limb movements, continuous motion estimation based on sEMG signals can extract more human motion information and is applied to intelligent prostheses, medical rehabilitation robots and motion evaluation.

At present, there are two common methods to achieve continuous joint motion estimation based on sEMG. The first method is to establish a joint dynamics model with sEMG as input in combination with muscle physiological mechanics, and then calculate continuous quantities such as joint torque, angular acceleration, and angular velocity. The advantage of this method is that the established model can explain the process of motion generation. The most widely used muscle force model is the Hill model, which is a physiological phenomenological model that contains multiple physiological parameters that cannot be directly measured. Therefore, it is necessary to identify effective parameters through preliminary experiments, which also limits its application in practice. The second method is to directly establish a regression model that associates sEMG and continuous joint motion. The advantage of this method is that the modeling process is direct and the use of sEMG is not restricted. The most widely used direct modeling methods are divided into two types. One is a method based on deep learning, which uses deep neural networks to establish a regression model that associates sEMG and joint motion. It is usually necessary to preprocess the sEMG signal first, and then use the processed signal to train the neural network to finally obtain the corresponding regression model. However, the method based on deep learning is a data-driven network, and some important prior knowledge is inevitably ignored in the modeling process. Moreover, a large number of samples are required as training data for deep neural network training. As a non-stationary signal, sEMG has individual differences, muscle fatigue and other differences or interferences in the acquisition stage, resulting in significantly different distributions between the training set and the test set, which ultimately leads to inaccurate regression models. Another method is to use a time series filter to improve regression accuracy under the premise of small sample capacity, represented by Kalman filtering (KF). In real systems, KF measurement functions and state functions are often difficult to obtain directly, and artificially designed measurement models and state models are often rough approximations of complex models, which reduces the performance of KF. Especially in the task of using sEMG to estimate continuous human motion, limb posture and joint angle do not follow a simple motion model, and the physical relationship between sEMG generated by muscle contraction and limb posture cannot be directly obtained. In this case, KF modeling becomes very difficult. Moreover, the linear state space model represented by KF does not adequately describe the nonlinear relationship between sEMG and joint motion, and its generalization ability needs to be further improved.

To overcome these limitations, some researchers have tried to use fully connected neural networks or long short-term memory to learn motion models directly from training data. The model obtained by this method can combine the prior knowledge in KF with the motion model learned by the deep neural network from the data, thereby improving the accuracy of continuous human motion estimation with a smaller data set. However, KF is a filtering method for linear systems and has limited ability to describe nonlinear models. Even various types of progressive Kalman methods have limited ability to describe nonlinear models of deep neural networks, which also affects the accuracy and robustness of the model. In summary, among the existing methods for continuous joint motion estimation based on sEMG, there is no Kalman filter network that can fully and robustly describe nonlinear models.

Therefore, the present invention proposes a method for estimating human upper limb posture based on a progressive unscented Kalman filter network, which can more fully describe the nonlinearity of sEMG and joint motion when using a smaller data set, and improve the accuracy and robustness in estimating continuous motion of the human body using sEMG signals.

Summary of the invention

In order to overcome the problem of difficulty in Kalman filter modeling and the problem of inaccurate deep neural network regression model in the task of estimating continuous motion of the human body using sEMG, the present invention provides a method for estimating the posture of human upper limbs based on an asymptotic unscented Kalman filter network, which effectively improves the accuracy and robustness of continuous motion estimation of the human body by fusion of deep learning and asymptotic unscented Kalman filter.

The technical solution adopted by the present invention to solve the technical problem is:

A method for estimating the posture of human upper limbs based on a progressive unscented Kalman filter network, the method comprising the following steps:

Step 1) Data collection: Build a collection system to obtain the angle information of the upper limb motion joints of the human body through a visual capture system, and use an electromyographic bracelet to collect the electromyographic signals of the forearm during movement;

Step 2) Establish a model between sEMG signals and upper limb joint angles:

x _k =f(x _k-1 )+w _k-1 (1)

z _k =h(x _k )+v _k (2)

和方差P_k|k-1。Step 3) An unscented Kalman filter system with LSTM estimated parameters was constructed, and the LSTM neural network was used to learn the system's measurement equation h(·), system noise Q _k and measurement noise R _k ; UT changes were adopted, and the posterior mean of the nonlinear equation was approximated by weighted statistical linear regression

And the variance is P _k|k-1 .

方差P_zz，k|k-1以及互协方差P_xz，k|k-1。Step 4) Obtain the one-step prediction of the measured variable based on the UT transformation and the measurement noise R _k

Variance _Pzz,k|k-1 and cross-covariance _Pxz,k|k-1 .

和估计方差P_k|k。Step 5) Calculate the Kalman gain according to the calculation steps of UKF, and find the state estimate at time k based on the measured value z _k at time k

and the estimated variance P _k|k .

Furthermore, in the step 1), the A, B, and C represent the numbers of the joint points of the human elbow, wherein AB represents the upper arm of the experimenter, BC represents the forearm of the experimenter, and the joint angle of the upper limb elbow joint is the angle θ.

Furthermore, in step 1), the visual capture system is composed of 12 cameras, and the sampling frequency of the myoelectric bracelet is 200 Hz.

Furthermore, in the step 2), the x _k and z _k are respectively the n-dimensional upper limb joint state vector at time k and the m-dimensional sEMG signal measurement vector.

In the step 2), the f(·) and h(·) are the nonlinear state equation and measurement equation of the system.

In the step 2), the w _k and v _k are system noise and measurement noise with zero mean, respectively, and the corresponding covariances Q _k and R _k are Gaussian white noises that are uncorrelated with each other.

Furthermore, in the step 3), the unscented Kalman filter system with LSTM estimated parameters is constructed, and the UKF partial model is learned using the LSTM neural network to directly obtain the measurement noise, system noise and measurement function from the training data.

方差P_zz，k|k-1以及互协方差P_xz，k|k-1。Furthermore, in step 4), the measurement equation h(·) is directly replaced by an LSTM _h(·) module, and then the measurement noise R _k obtained by UT transformation and LSTM _R is used to obtain the one-step prediction of the measurement variable.

Variance _Pzz,k|k-1 and cross-covariance _Pxz,k|k-1 .

和估计方差P_k|k。Furthermore, in step 5), a progressive UKF filtering method with adaptive measurement update is added to the UKF to improve the stability of the system; the covariance of the measurement noise is artificially increased to solve the contradiction between stability and filtering accuracy, and the state estimation at time k is obtained according to the measured value z _k at time k.

and the estimated variance P _k|k .

The present invention is based on the UKF calculation graph and uses the LSTM model to obtain the unknown parameters of the UKF. At the same time, the progressive filtering method is used to solve the Kalman gain instability problem caused by adding LSTM to UKF. The model combines the advantages of UKF and LSTM, can be applied in nonlinear systems and obtain higher regression accuracy from smaller data sets, and is better than other learning-based KF models in upper limb joint angle estimation based on sEMG. The present invention provides a method for upper limb posture estimation based on a progressive unscented Kalman filter network, which effectively improves the accuracy and robustness of upper limb posture estimation.

The beneficial effects of the present invention are mainly manifested in: providing a human body posture estimation method based on progressive unscented Kalman filter network fusion, aiming at the problem that it is very difficult to use Kalman filtering to model limb posture and joint angle in the task of estimating continuous motion of the human body using _sEMG , using a long short-term memory neural network to directly learn the motion model from training data, and combining a progressive UKF filtering method with adaptive measurement update to solve the problem of Kalman gain instability, thereby improving the accuracy and robustness of upper limb posture estimation.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a flow chart of upper limb posture estimation based on LSTM-pUKF fusion of the present invention.

FIG2 is an algorithm framework diagram of upper limb posture fusion estimation based on LSTM-pUKF of the present invention.

Figure 3a is a module network diagram of LSTM _Q of the present invention, Figure 3b is a module network diagram of LSTM _R , and Figure 3c is a module network diagram of LSTM _h .

DETAILED DESCRIPTION

The present invention will be further described below in conjunction with the accompanying drawings.

Referring to FIG. 1 , FIG. 2 , and FIG. 3 , a method for estimating the posture of human upper limbs using a progressive unscented Kalman filter network is provided, and the method comprises the following steps:

Step 1) Data collection: Build a collection system, which can obtain the angle information of the upper limb motion joints of the human body through the visual capture system, so that the electromyographic collection bracelet can collect the electromyographic signals of the forearm during the movement;

Step 2) Establish a model between sEMG signals and upper limb joint angles:

x _k =f(x _k-1 )+w _k-1 (1)

z _k =h(x _k )+v _k (2)

和方差P_k|k-1。Step 3) An unscented Kalman filter system with LSTM estimated parameters was constructed. The LSTM neural network was used to learn the system's measurement equation h(·), system noise Q _k and measurement noise R _k . The UT was varied and weighted statistical linear regression was used to approximate the posterior mean of the nonlinear equation.

And the variance is P _k|k-1 .

方差P_zz，k|k-1以及互协方差P_xz，k|k-1。Step 4) Obtain the one-step prediction of the measured variable based on the UT transformation and the measurement noise R _k

Variance _Pzz,k|k-1 and cross-covariance _Pxz,k|k-1 .

和估计方差P_k|k。Step 5) Calculate the Kalman gain according to the calculation steps of UKF, and find the state estimate at time k based on the measured value z _k at time k

and the estimated variance P _k|k .

The flowchart of human upper limb posture estimation based on progressive unscented Kalman filter network fusion is shown in Figure 1. First, build an acquisition system, use an electromyographic bracelet to obtain the electromyographic signal during the movement of the forearm, and use a motion capture system to synchronously collect the upper limb elbow joint angle data. Experiments were conducted on multiple different testers, ten sets of data were collected for each specified action, and then five sets of exercises were performed with random posture angles. There was enough rest time between each experiment to prevent muscle fatigue. The model between sEMG signals and upper limb joint angles is established as follows:

x _k =f(x _k-1 )+w _k-1 (1)

z _k =h(x _k )+v _k (2)

和

分别为k时刻n维上肢关节状态向量和m维的sEMG信号量测向量，f(·)和h(·)为系统非线性状态方程和量测方程，系统噪声w_k和量测噪声v_k分别是均值为零，协方差为Q_k和R_k的互不相关的高斯白噪声。in,

and

are the n-dimensional upper limb joint state vector and the m-dimensional sEMG signal measurement vector at time k, respectively. f(·) and h(·) are the nonlinear state equation and measurement equation of the system. The system noise w _k and measurement noise v _k are independent Gaussian white noises with zero mean and covariances Q _k and R _k , respectively.

Secondly, an unscented Kalman filter system with LSTM parameter estimation is constructed. In the model between the sEMG signal and the upper limb joint angle, the state equation f(·) is known, and the measurement equation h(·), system noise Q _k and measurement noise R _k are unknown. Since the state noise covariance matrix Q _k and the measurement noise covariance matrix R _k are difficult to estimate, LSTM is used to learn Q _k and R _k directly from the state vector and noise vector:

是k-1时刻LSTM_Q输出的隐藏单元，

是k-1时刻LSTM_R输出的隐藏单元。在所有网络的训练中，首先对预处理后的_sEMG数据进行均方根欠采样，然后进行归一化处理；将归一化后的数据集按照7：3划分作为训练集、测试集；将70％的训练集数据输入网络模型进行训练，使用30％的测试集数据验证模型有效性。在网络初始化阶段，对于所有LSTMcell单元，使用均匀分布[0.01，0.01]初始化所有LSTM权重矩阵，使用Xavier初始化所有全连接层权重矩阵。除LSTM遗忘偏置外，所有偏置均初始化为零。初始学习率设为0.01，然后通过ADAM优化器在32个大小的batch中进行100个epoch的训练。通过R²和均方根误差(RMSE)评估网络性能。R²表示估计结果与真实状态的相关性，RMSE表示状态估计和测量之间的在幅值差异。Among them, LSTM _Q and LSTM _R represent the LSTM modules used to learn Q _k and R _k respectively, x _k-1 is the n-dimensional sEMG signal state vector at time k, z _k is the m-dimensional sEMG signal measurement vector at time k,

is the hidden unit output by LSTM _Q at time k-1,

is the hidden unit of LSTM _R output at time k-1. In the training of all networks, the preprocessed _s EMG data was firstly undersampled by RMS and then normalized; the normalized data set was divided into training set and test set according to 7:3; 70% of the training set data was input into the network model for training, and 30% of the test set data was used to verify the effectiveness of the model. In the network initialization stage, for all LSTM cell units, all LSTM weight matrices were initialized using uniform distribution [0.01, 0.01], and all fully connected layer weight matrices were initialized using Xavier. All biases were initialized to zero except the LSTM forget bias. The initial learning rate was set to 0.01, and then the training was performed for 100 epochs in batch size 32 by ADAM optimizer. The network performance was evaluated by R ² and root mean square error (RMSE). R ² represents the correlation between the estimated result and the true state, and RMSE represents the difference in amplitude between the state estimate and the measurement.

Then, according to the core idea of the UKF algorithm, a symmetric sampling strategy is adopted to use 2n+1 Sigma sampling points {χ′ _i }, i = 1, 2, …, 2n, to approximate the posterior mean and variance of the nonlinear state equation. The obtained _2n +1 sampling points are:

集带入非线性状态方程得到：Sigma sampling point

Substituting the set into the nonlinear state equation, we get:

The corresponding mean weight _Wim and variance _weight ^{Wic for} symmetric sampling are:

Among them, K is the Sigma sampling point spacing ratio factor of the random variable x, and n is the dimension of the variable x. The mean and variance of the one-step state prediction can be obtained by weighted statistical linear regression technology:

和预测方差P_k|k-1代入采样公式得到新的采样点集

和相应的均值权值W_i ^(m)和方差权值W_i ^(c) According to the calculated state prediction estimate

Substitute the predicted variance P _k|k-1 into the sampling formula to obtain a new set of sampling points

and the corresponding mean weights _Wi ^(m) and variance weights _Wi ^(c)

Next, we directly replace the measurement equation h(·) with an LSTM _h(·) module:

方差P_zz，k|k-1、互协方差P_xz，k|k-1以及卡尔曼增益：According to the measurement noise R _k obtained by UT transformation and LSTM _R learning, the one-step prediction of the measurement variable is further obtained

Variance _Pzz,k|k-1 , cross-covariance _Pxz,k|k-1 and Kalman gain:

K _k =P _xz,k ( _Pzz,k|k-1 ) ^-1 (16)

Wherein, N means that the measurement noise covariance increases to N times each time the progressive measurement is updated, and N steps are required to realize the measurement update, so as to achieve the purpose of adaptive measurement update and progressive filtering, thereby improving the stability of the system and the accuracy of filtering.

和估计方差P_k|k：Finally, according to the measured value zk at time k, the state estimate at time k is obtained

And the estimated variance P _k|k :

Claims (6)

1. A human body upper limb posture estimation method based on a progressive unscented Kalman filter network is characterized by comprising the following steps of: the method comprises the following steps:

step 1) data acquisition: building a set of acquisition system, acquiring angle information of a human upper limb movement joint through a vision capturing system, and acquiring myoelectric signals of the forearm in the movement process by using a myoelectric bracelet;

step 2) establishing a model between the sEMG signals and the upper limb joint angles:

x _k ＝f(x _k-1 )+w _k-1 (1)

z _k ＝h(x _k )+v _k (2)

step 3) constructing a unscented Kalman filtering system of the LSTM estimated parameters, and learning a measurement equation h (·) and system noise Q of the system by using the LSTM neural network _k And measuring noise R _k The method comprises the steps of carrying out a first treatment on the surface of the Approximation of nonlinear equations using weighted statistical linear regression with UT variationPosterior mean of (2)

Sum of variances P _k|k-1 ；

Step 4) transforming and measuring noise R according to UT _k One-step prediction of measured variables

Variance P _zz,k|k-1 Cross covariance P _xz,k|k-1 ；

Step 5) calculating Kalman gain according to the UKF calculation step, and measuring value z according to k time _k Determining a state estimate at time k

And estimated variance P _k|k 。

2. The method for estimating the posture of the upper limb of the human body based on the progressive unscented kalman filter network as defined in claim 1, wherein the method comprises the following steps of: in the step 1), A, B, C is used to represent the label of the elbow joint point of the human body, wherein AB is the big arm of the experimenter, BC is the forearm of the experimenter, and the joint angle of the elbow joint of the upper limb is the included angle theta.

3. The method for estimating the posture of the upper limb of the human body based on the progressive unscented kalman filter network as defined in claim 1, wherein the method comprises the following steps of: in the step 1), the visual capturing system consists of 12 cameras, and the sampling frequency of the myoelectric bracelet is 200Hz.

4. The human upper limb posture estimation method based on the progressive unscented kalman filter network as defined in claim 1 or 2, wherein: in the step 2) of the above-mentioned process,

and->

sEMG signal measurement vectors of the n-dimensional upper limb joint state vector and the m-dimensional upper limb joint state vector at the moment k respectively, wherein f (·) and h (·) are a system nonlinear state equation and a measurement equation, and w _k And v _k System noise and measurement noise with mean value of zero respectively, and corresponding covariance Q _k And R is _k Is uncorrelated gaussian white noise.

5. The method for estimating the posture of the upper limb of the human body based on the progressive unscented kalman filter network as defined in claim 1, wherein the method comprises the following steps of: in the step 3), the unscented kalman filter system of the LSTM estimation parameters learns the UKF partial model by using the LSTM neural network, and directly obtains the measurement noise, the system noise and the measurement function from the training data.

6. The method for estimating the posture of the upper limb of the human body based on the progressive unscented kalman filter network as defined in claim 1, wherein the method comprises the following steps of: in the step 4), a progressive UKF filtering method with self-adaptive measurement updating is added into the UKF, so that the stability of the system is improved; manually increasing covariance of measurement noise to solve contradiction between stability and filtering precision according to measurement value z at k moment _k The state estimation and estimation variance at time k are obtained.

Images