CN108392207B - An Action Recognition Method Based on Gesture Labels - Google Patents

Abstract

The present invention provides an action recognition method based on gesture labels, which abstracts action recognition into gesture recognition, abstracts gestures into gesture labels based on the relative position method of key nodes, and finds out what human beings have by comparing the posture changes of humans in a certain period of time. The method reduces the difficulty of building a template library, greatly reduces the speed and operation requirements of action recognition, and improves the versatility of action recognition for recognizing individuals. The method has important application value in the fields of human-computer interaction, virtual reality, video surveillance, and motion feature analysis.

Description

An Action Recognition Method Based on Gesture Labels

technical field

The invention belongs to the technical field of action recognition, and relates to an action recognition method based on gesture tags.

Background technique

Action recognition has been a hot research topic in recent years. The existing research results in the field of action recognition have been applied to various fields such as civil air defense and security, research on human life habits, human-computer interaction, and virtual reality, and have produced great positive effects. The traditional action recognition directly analyzes the image (including video, several photos, etc.) through the technical methods related to image processing, and finally realizes the action recognition through the steps of image segmentation, feature extraction, action feature extraction, action feature classification and so on. Although the existing action recognition methods have made great progress, there are still some problems, such as the huge amount of calculation; the action feature database is not easy to establish, and professionals need to input materials; Accuracy will be greatly reduced and so on.

SUMMARY OF THE INVENTION

Aiming at the problems existing in the prior art, the purpose of the present invention is to provide an action recognition method based on gesture tags, which solves the problems of large amount of calculation, difficulty in establishing a template library and poor generality of the template library in the existing motion identification technology. question.

In order to achieve the above object, the present invention adopts the following technical solutions:

A method for decomposing actions into pose labels, including the following steps:

Step 1, using the skeleton tracking device to obtain the position data of the key node of the human body torso action at each moment, the position data of the key node is the data in the coordinate system of the skeleton tracking device; the key node includes at least the key node HEAD, the key node SHOULDER CENTER, key node SPINE, key node HIP CENTER, key node SHOULDE RIGHT, key node SHOULDER LEFT, key node ELBOW RIGHT, key node ELBOW LEFT, key node WRIST RIGHT, key node WRIST LEFT, key node HAND RIGHT, key node HAND LEFT, key node HIP RIGHT, key node HIP LEFT, key node KNIEE RIGHT, key node KNIEE LEFT, key node ANIKLE RIGHT, key node ANIKLE LEFT, key node FOOT RIGHT, key node FOOT LEFT;

Step 2: Convert the position data of the key nodes at each moment obtained in step 1 into the position data of the key nodes under the morphological coordinate system; the morphological coordinate system takes the facing direction of the human torso as the positive direction of the Z axis. , take the morphological upper end of the human body as the positive direction of the Y-axis, take the left direction of the person as the positive direction of the X-axis, and take the key node HIPCENTER as the origin;

Step 3, using the position data of the key nodes under the morphological coordinate system obtained in step 2 at each moment to obtain the posture label at each moment, the posture label includes the main body posture label GL _body , the left forelimb posture label GL _lf , right forelimb pose label _GLrf , left hindlimb pose label _GLlb and right hindlimb pose label _GLrb .

Optionally, the method for obtaining the body pose label GL _body in the step 3 is as follows:

Select the coordinate value with the largest absolute value among X _F , Y _F and Z _F , and find the value of the GL _body corresponding to the interval to which the coordinate value belongs, which is the value of the subject attitude label GL _body . The formula used is as follows:

为关键节点HEAD与关键节点HIP CENTER在骨骼追踪设备坐标系下形成的向量；Among them, X _F , Y _F and Z _F are the coordinates of the three coordinate axes of the unit vector F respectively; the unit vector

It is the vector formed by the key node HEAD and the key node HIP CENTER in the coordinate system of the bone tracking device;

The method for obtaining the left forelimb posture label GL _lf , the right forelimb posture label GL _rf , the left hind limb posture label GL _lb and the right hind limb posture label GL _rb in the step 3 is as follows:

The four attitude labels all include three key nodes, which are denoted as key node 1, key node 2 and key node 3, and the three key nodes included in the left forelimb attitude label GL _lf are ELBOW LEFT, WRIST LEFT and HANDLEFT respectively, For the right forelimb pose label GL _rf , the three key nodes included are KNIEE LEFT, ANIKLE LEFT and FOOT LEFT, respectively, for the left hindlimb pose label GL _lb , the three key nodes included are ELBOW LEFT, WRIST LEFT and HAND LEFT, respectively. The three key nodes included in the attitude label GL _rb are KNIEE LEFT, ANIKLE LEFT and FOOT LEFT.

The data of the three key nodes in the morphological coordinate system are respectively represented by (X ₁ , Y ₁ , Z ₁ ) (X ₂ , Y ₂ , Z ₂ ) (X ₃ , Y ₃ , Z ₃ ); the above four Attitude labels include height label G1, orientation label G2 and curl label G3;

Among them, the calculation method of the height label G1 is as follows:

G1=(g ₁ +g ₂ +g ₃ )/3 rounding, where,

Among them, n=1, 2, 3, Y _H is the Y-axis coordinate in the morphological coordinate system of the key node HEAD, and Y _HC is the Y-axis coordinate in the morphological coordinate system of the key node SHOULDER CENTER;

The method of obtaining the orientation label G2 is as follows:

Count the symbols of the X-axis and Z-axis coordinates of key node 1, key node 2 and key node 3, and use the following formula to find the bearing label G2:

The method of obtaining the curled label G3 is as follows:

According to the key node 1, the key node 2 and the key node 3, the key node 4 is introduced, and the distances D ₁ , D ₂ , D ₃ between the key node 1, the key node 2 and the key node 3 and the key node 4 are calculated respectively; for Left forelimb pose label GL _lf , key node 4 is SHOULDER LEFT, for right fore limb pose label GL _rf , key node 4 is SHOULDER RIGHT, for left hind limb pose label GL _lb , key node 4 is HIP LEFT, for right hind limb pose label GL _rb , the key node 4 is HIP RIGHT;

The value method of the curled label G3 adopts the following formula:

The present invention also provides a method for obtaining an action template library, comprising the following steps:

Step 1, make standard actions multiple times, decompose the standard actions made each time into attitude labels at each moment; select the attitude label at the initial moment as the starting frame attitude label, and select the attitude label at the termination moment as the ending frame attitude label; Take the standard action made for the first time as the comparison standard action, and the standard action made in the other times as the reference standard action; The corresponding termination frame pose label is used as the termination frame comparison pose label;

The described standard action that is made each time is decomposed into the attitude label under each moment is the attitude label obtained according to the method of claim 1;

Step 2, find the initial frame similarity coefficient group, the specific method is as follows:

Calculate the similarity Sl1(A) _n of each attribute of the starting frame attitude label of multiple reference standard actions and the starting frame contrast attitude label respectively, and the formula used is as follows:

Sl1(A) _n = A _n ×z1 _n ÷ln (n∈Z, n∈[1, 13], when n=1, 4, 7, 10, 13, ln takes 5, and the rest take 3)

Among them, An is the initialized similarity coefficient value, An = 1, _{n is} the serial number of the attribute, the serial number from 1 to 13 is the body pose label GL _body of the pose label, and the height label in the left forelimb pose label GL _lf G1, orientation label G2 in left forelimb pose label GL _lf , curl label G3 in left forelimb pose label GL _lf , height label G1 in left hind limb pose label GL _lb , orientation label G2 in left hind limb pose label GL _lb , Curl tag G3 in left hind limb pose tag GL _lb , height tag G1 in right forelimb pose tag GL _rf , orientation tag G2 in right forelimb pose tag GL _rf , curl tag G3 in right forelimb pose tag GL _rf , right hind limb pose tag GL The height label G1 of _rb , the orientation label G2 of the right hind limb posture label GL _rb , the curl label G3 of the right hind limb posture label GL _rb ; z1 _n is the corresponding attribute of the posture label of the starting frame of the reference standard action and the starting frame comparison of the posture label the absolute value of the difference;

For each attribute n, the next largest value in the similarity S11(A) _n calculated for the starting frame pose labels of multiple reference standard actions is selected as the similarity coefficient value A1 _n under the attribute. The similarity coefficient value A1 _n corresponding to each attribute n forms the initial frame similarity coefficient group A _star ={A1 _n ,n∈Z,n=1,2,...,13};

Step 3, find the similarity coefficient group of the termination frame, and the specific method is as follows:

Calculate the similarity Sl2(A) _n of each attribute of the terminal frame attitude label of multiple reference standard actions and the terminal frame contrast attitude label respectively, and the formula used is as follows:

Sl2(A) _n = A _n ×z2 _n ÷ln (n∈Z, n∈[1, 13], when n=1, 4, 7, 10, 13, ln takes 5, the rest take 3)

Wherein, z2 _n is the absolute value of the difference between the corresponding attribute of the termination frame gesture tag of the reference standard action and the termination frame contrast gesture tag;

For each attribute n, select the second largest value in the similarity S12(A) _n calculated for the termination frame pose labels of multiple reference standard actions as the similarity coefficient value A2n under the attribute; each attribute _n corresponds to The similarity coefficient value of A2 _n forms the termination frame similarity coefficient group A _stop ={A2 _n ,n∈Z,n=1,2,...,13};

Step 4, according to the method of steps 1-3 for a plurality of standard actions, obtain the initial frame similarity coefficient group and the termination frame similarity coefficient group corresponding to each standard action, the initial frame similarity coefficient group corresponding to all standard actions and The set of similarity coefficients of termination frames forms an action template library.

The present invention also provides an action recognition method based on gesture tags, comprising the following steps:

Step 1, for the action to be recognized, the action to be recognized is decomposed into gesture labels at each moment; the decomposed action to be recognized into gesture labels at each moment is the gesture obtained by the method according to claim 1 Label;

Step 2, select a certain standard action in the action template library, calculate the termination frame pose label obtained in step 1, and the similarity SL(B) _n of each attribute between the termination frame pose label of the selected standard action, denote The attitude label of the termination frame is the attitude label of the t-th frame, and the formula used is as follows:

SL(B) _n = A1 _n ×z3 _n ÷ln (n∈Z, n∈[1, 13], when n=1, 4, 7, 10, 13, ln takes 5, the rest take 3)

Wherein, z3 _n is the absolute value of the difference between the corresponding attributes of the termination frame pose tag obtained in step 1 and the termination frame pose tag of the selected standard action;

To calculate the overall similarity S(B) between the pose label of the termination frame obtained in step 1 and the pose label of the selected standard action termination frame, the formula used is as follows:

Step 3, if the overall similarity S(B) is greater than the set threshold MAXBLUR, then go back to Step 2; otherwise, go to Step 4;

Step 4: Calculate the similarity SL(C) _n of each attribute between the last frame pose label of the end frame pose label and the selected standard action starting frame pose label, and record the last frame pose label as the t-th 1 frame pose label, the formula used is as follows:

SL(C) _n = A2 _n ×z4 _n ÷ln (n∈Z, n∈[1, 13], when n=1, 4, 7, 10, 13, ln takes 5, the rest take 3)

Wherein, z4 _n is the absolute value of the difference between the previous frame pose label and the corresponding attribute of the selected standard action starting frame pose label;

To calculate the overall similarity S(C) between the pose label of the previous frame and the pose label of the selected standard action starting frame, the formula used is as follows:

Step 5, if the overall similarity S(C) is less than the set threshold MAXBLUR, then the action to be identified is consistent with the selected standard action; if the overall similarity S(C) is greater than the set threshold MAXBLUR, then return to step 4 to process the object. The t-1 frame pose label is replaced with the t-2 frame pose label, until the processing object is the first frame pose label, and the overall similarity S(C) is greater than the set threshold MAXBLUR, then return to step 2.

Compared with the prior art, the present invention has the following technical effects: the method abstracts action recognition as gesture recognition, abstracts gestures into gesture labels based on the relative position method of key nodes, and finds out human beings by comparing the posture changes of humans over a certain period of time. The method reduces the difficulty of establishing a template library, greatly reduces the speed and operation requirements of action recognition, and improves the versatility of action recognition for recognizing individuals. The method has important application value in the fields of human-computer interaction, virtual reality, video surveillance, and motion feature analysis.

The solution of the present invention will be further explained and described in detail below in conjunction with the accompanying drawings and embodiments.

Description of drawings

FIG. 1 is a schematic diagram of the coordinate system of the bone tracking device used in the present invention.

FIG. 2 is a schematic diagram of the positions of twenty key skeleton nodes obtained by the present invention.

Detailed ways

The present invention provides a method for decomposing actions into gesture labels, comprising the following steps:

Step 1, using a bone tracking device to obtain position data of a key node of the action of the human body torso, where the position data of the key node is data in the coordinate system of the bone tracking device. Among them, the bone tracking device can use Kinect, which uses Kinect to obtain the key node data of the action according to a certain frequency. The position data of the key node represents the position of the 20 bone nodes specific to the bone. The node name and serial number of the key node are shown in the following table. :

Among them, the coordinate system of the skeleton tracking device takes the device camera as the origin, the forward direction of the camera is the positive direction of the Z axis, the opposite direction of gravity is the positive direction of the Y axis, the left direction of the camera is the positive direction of the X axis, and the unit length is 1 meter. The coordinate system of the bone tracking device is a static coordinate system.

Step 2: Convert the position data of the key nodes at each moment obtained in step 1 into the position data of the key nodes under the morphological coordinate system respectively; the formula used is as follows:

Among them, (x,y,z)=(XX _HC , YY _HC , ZZ _HC ) represents the coordinates of the vector between any key node NODE in the coordinate system of the bone tracking device obtained in step 1, (X, Y, Z ) represents the position data of the key node NODE, (X _HC , Y _HC , Z _HC ) represents the position data of the key node HIPCENTER; α, β and γ are the coordinates of each coordinate axis in the morphological coordinate system relative to the coordinate system of the bone tracking device the rotation angle.

Then the position data of the key nodes in the morphological coordinate system is (x', y', z').

In this morphological coordinate system, the facing direction of the human torso is the positive direction of the Z-axis, the morphological upper end of the human torso is the positive direction of the Y-axis, the left direction of the human body is the positive direction of the X-axis, and the key node HIP CENTER is the positive direction of the X-axis. origin.

The morphological upper end of the human torso refers to the morphological upper end of the human body starting from the head and extending downward and outward along the body. For example, when a person is standing upright, his hands are naturally drooping, and the three parts of the left shoulder, left elbow, and left hand are as follows: the left shoulder is the morphological upper end of the left elbow, and the left elbow is the morphological upper end of the left hand.

Step 3: Find the main body pose label GL _body , the left fore limb pose label GL _lf , the right fore limb pose label GL _rf , the left hind limb pose label GL _lb and the right hind limb pose label GL _rb at each moment.

Specifically, in yet another embodiment, the method for determining the facing direction of the human torso and the morphological upper end direction of the human torso in step 2 is as follows:

The position data of the key node SHOULDER RIGHT obtained in step 1 is (X _SR , Y _SR , Z _SR ), the position data of the key node SHOULDER LEFT is (X _SL , Y _SL , Z _SL ), the position of the key node HIP CENTER The data is (X _HC , Y _HC , Z _HC ), and the three key nodes can determine a plane, which is the plane where the human body is located.

其中，The normal vector of the plane where the human torso lies

in,

Find the vector of the key node HEAD and the key node HIP CENTER in the coordinate system of the bone tracking device

乘以

若取值为正，

取正号，若取值为负，

取负号。

的方向即为人体躯干面对方向，

的方向即为人体躯干的形态学上端方向。Since the head of the Kinect device is always tilted forward, the

multiply by

If the value is positive,

Take a positive sign, if the value is negative,

Take a negative sign.

The direction is the direction of the human torso facing,

The direction is the morphological upper direction of the human torso.

Specifically, the method for obtaining the body pose label GL _body is as follows:

令单位向量

Find the vector of the key node HEAD and the key node HIP CENTER in the coordinate system of the bone tracking device

Let the unit vector

Select the coordinate value with the largest absolute value among X _F , Y _F and Z _F , and find the value of the GL _body corresponding to the interval to which the coordinate value belongs, which is the value of the subject attitude label GL _body . The formula used is as follows:

X_F，Y_F和Z_F中一个为0，且另外两个的值相等时，求得相等的两个值为

则X_F，Y_F和Z_F中的最大值大于

Since F is a unit vector, then

When one of X _F , Y _F and Z _F is 0, and the other two are equal, the two equal values are obtained

then the maximum value of X _F , Y _F and Z _F is greater than

The methods for obtaining the left forelimb pose label GL _lf , the right fore limb pose label GL _rf , the left hind limb pose label GL _lb and the right hind limb pose label GL _rb are as follows:

The above four attitude labels all include three key nodes, denoted as key node 1, key node 2 and key node 3. For the left forelimb pose label GL _lf , the three key nodes included are ELBOW LEFT, WRIST LEFT and HANDLEFT, respectively. The three key nodes included in the right forelimb pose label GL _rf are KNIEE LEFT, ANIKLE LEFT and FOOT LEFT respectively, and the three key nodes included in the left hindlimb pose label GL _lb are ELBOW LEFT, WRIST LEFT and HAND LEFT respectively, the right hindlimb pose The three key nodes included in the label GL _rb are KNIEE LEFT, ANIKLE LEFT and FOOT LEFT.

The data of the three key nodes in the morphological coordinate system are respectively represented by (X ₁ , Y ₁ , Z ₁ ) (X ₂ , Y ₂ , Z ₂ ) (X ₃ , Y ₃ , Z ₃ ); the above four The attitude labels all include a height label G1, an orientation label G2 and a curl label G3.

Among them, the calculation method of the height label G1 is as follows:

G1=(g ₁ +g ₂ +g ₃ )/3 rounded up, the smaller the value of G1, the closer the position of the part is to the upper end of the morphology. in:

Among them, n=1, 2, 3, Y _H is the Y-axis coordinate in the morphological coordinate system of the key node HEAD, Y _HC is the Y-axis coordinate in the morphological coordinate system of the key node SHOULDER CENTER, and Y _H > Y _hc .

The method of obtaining the orientation label G2 is as follows:

Count the symbols of the X-axis and Z-axis coordinates of key node 1, key node 2 and key node 3, and use the following formula to find the bearing label G2:

The method of obtaining the curled label G3 is as follows:

According to the key node 1, the key node 2 and the key node 3, the key node 4 is introduced, and the distances D ₁ , D ₂ , D ₃ between the key node 1, the key node 2 and the key node 3 and the key node 4 are calculated respectively. For the left forelimb pose label GL _lf , the key node 4 is SHOULDER LEFT, for the right fore limb pose label GL _rf , the key node 4 is SHOULDER RIGHT, for the left hind limb pose label GL _lb , the key node 4 is HIP LEFT, for the right hind limb pose label GL _rb , the key node 4 is HIP RIGHT.

The value method of the curled label G3 adopts the following formula:

Another aspect of the present invention provides a method for obtaining an action template library, comprising the following steps:

Step 1, make standard actions multiple times, and decompose the standard actions made each time into pose labels at each moment according to the above-mentioned method of decomposing actions into posture labels; select the posture label at the initial moment as the initial frame posture label, and select The pose label at the end time is the end frame pose label. The standard action performed the first time is the comparison standard action, and the standard action performed the other time is the reference standard action. The starting frame pose label corresponding to the standard action is compared as the starting frame comparing pose label, and the ending frame pose label corresponding to the standard action performed for the first time is used as the ending frame comparing pose label.

Step 2, find the initial frame similarity coefficient group, the specific method is as follows:

Calculate the similarity Sl1(A) _n of each attribute of the starting frame attitude label of multiple reference standard actions and the starting frame contrast attitude label respectively, and the formula used is as follows:

Sl1(A) _n =A _n ×z1 _n ÷ln(n∈Z,n∈[1,13], when n=1,4,7,10,13, ln takes 5, the rest take 3) (6)

Among them, An is the initialized similarity coefficient value, An = 1, _{n is} the serial number of the attribute, the serial number from 1 to 13 is the body pose label GL _body of the pose label, and the height label in the left forelimb pose label GL _lf G1, orientation label G2 in left forelimb pose label GL _lf , curl label G3 in left forelimb pose label GL _lf , height label G1 in left hind limb pose label GL _lb , orientation label G2 in left hind limb pose label GL _lb , Curl tag G3 in left hind limb pose tag GL _lb , height tag G1 in right forelimb pose tag GL _rf , orientation tag G2 in right forelimb pose tag GL _rf , curl tag G3 in right forelimb pose tag GL _rf , right hind limb pose tag GL The height label G1 of _rb , the orientation label G2 of the right hind limb posture label GL _rb , the curl label G3 of the right hind limb posture label GL _rb ; z1 _n is the corresponding attribute of the posture label of the starting frame of the reference standard action and the starting frame comparison of the posture label The absolute value of the difference, such as z1 ₁ is the absolute value of the difference between the main gesture label GL _body in the initial frame gesture label of the reference standard action and the main gesture label GL _body in the initial frame contrast gesture label.

For each attribute n, the next largest value in the similarity S11(A) _n calculated for the starting frame pose labels of multiple reference standard actions is selected as the similarity coefficient value A1 _n under the attribute. The similarity coefficient value A1 _n corresponding to each attribute n forms an initial frame similarity coefficient group A _star ={A1 _n ,n∈Z,n=1,2,...,13}.

Step 3, find the similarity coefficient group of the termination frame, and the specific method is as follows:

Calculate the similarity S12(A) _n of each attribute of the termination frame pose labels of multiple reference standard actions and the termination frame contrast pose labels respectively, and the formula used is as follows;

Sl2(A) _n =A _n ×z2 _n ÷ln(n∈Z,n∈[1,13], when n=1,4,7,10,13, ln takes 5, the rest take 3) (7)

Among them, z2 _n is the absolute value of the difference between the attitude label of the termination frame of the reference standard action and the corresponding attribute of the termination frame compared with the attitude label, for example, z2 ₁ is the reference to the termination frame of the standard action. The pose label GL _body and the termination frame The absolute value of the difference of the body pose label GL _body against the pose label.

For each attribute n, the next largest value in the similarity S12(A) _n calculated for the terminal frame pose labels of multiple reference standard actions is selected as the similarity coefficient value A2 _n under the attribute. The similarity coefficient value A2 _n corresponding to each attribute n forms a termination frame similarity coefficient group A _stop ={A2 _n ,n∈Z,n=1,2,...,13}.

Step 4, according to the method of steps 1-3 for a plurality of standard actions, obtain the initial frame similarity coefficient group and the termination frame similarity coefficient group corresponding to each standard action, the initial frame similarity coefficient group corresponding to all standard actions and The set of similarity coefficients of termination frames forms an action template library.

A third aspect of the present invention provides an action recognition method, comprising the following steps:

Step 1: For the action to be recognized, the action to be recognized is decomposed into the gesture label at each moment according to the above method of decomposing the action into gesture labels.

Step 2, select a certain standard action in the action template library, calculate the termination frame pose label obtained in step 1, and the similarity SL(B) _n of each attribute between the termination frame pose label of the selected standard action, denote The attitude label of the termination frame is the attitude label of the t-th frame, and the formula used is as follows:

SL(B) _n = A1 _n ×z3 _n ÷ln (n∈Z, n∈[1, 13], when n=1, 4, 7, 10, 13, ln takes 5, the rest take 3)

(8)

Wherein, z3 _n is the absolute value of the difference between the corresponding attributes of the termination frame pose tag obtained in step 1 and the termination frame pose tag of the selected standard action.

To calculate the overall similarity S(B) between the pose label of the termination frame obtained in step 1 and the pose label of the selected standard action termination frame, the formula used is as follows:

Step 3, if the overall similarity S(B) is greater than the set threshold MAXBLUR, then go back to Step 2; otherwise, go to Step 4;

Step 4: Calculate the similarity SL(C) _n of each attribute between the last frame pose label of the end frame pose label and the selected standard action starting frame pose label, and record the last frame pose label as the t-th 1 frame pose label, the formula used is as follows:

SL(C) _n = A2 _n ×z4 _n ÷ln(n∈Z, n∈[1, 13], when n=1, 4, 7, 10, 13, ln takes 5, the rest take 3) (10)

Among them, z4 _n is the absolute value of the difference between the previous frame pose label and the corresponding attribute of the selected standard action starting frame pose label.

To calculate the overall similarity S(C) between the pose label of the previous frame and the pose label of the selected standard action starting frame, the formula used is as follows:

Step 5, if the overall similarity S(C) is less than the set threshold MAXBLUR, then the action to be identified is consistent with the selected standard action; if the overall similarity S(C) is greater than the set threshold MAXBLUR, then return to step 4 to process the object. The t-1 frame pose label is replaced with the t-2 frame pose label, until the processing object is the first frame pose label, and the overall similarity S(C) is greater than the set threshold MAXBLUR, then return to step 2. MAXBLUR represents the fuzzy degree of the action matching algorithm, and the value is 0.25-0.05.

Example

Action recognition using traditional methods:

A single Kinect is used, and when the recognition action is a right-hand salute, a template library is established using the traditional method. Tester a is 173cm tall and weighs 60kg, tester b is 191cm tall and weighs 100kg, and tester c is 181cm tall and weighs 80kg. The first 50 samples are entered by tester a, and the samples with sample serial numbers 51-80 are entered by tester b. It takes about 2 minutes for each sample to be entered. The action to be entered is selected during the entry, and the entryr stands in front of the device 1.5 Mi Chu makes a right-hand salute, the sample library and test point are all 20 skeleton nodes of Kinect

During the test, the testers are required to stand 1.5 meters in front of the equipment and do their best to comply with the standard actions. Every time a new sample is entered to optimize the template library, each person does a right-hand salute ten times, and the recognition results are counted. The statistics of the recognition results are shown in Table 1:

Table 1

Number of samples 20 30 40 50 60 70 80 tester a 70% 80% 90% 100% 90% 80% 80% tester b 0% 30% 30% 30% 70% 70% 90% tester c 30% 40% 40% 40% 50% 50% 50%

It can be seen from the test results that when tester a is both a tester and a sample entryr, with the increase of the number of entries, the recognition success rate increases significantly, and the final recognition success rate reaches 100% when the number of samples is 50, while the other The success rate of the tester is basically unchanged. When the tester b is used as the sample entry person, the success rate of the tester b is greatly improved, but the success rate of the tester a is reduced. Because tester c did not participate in the entry, the recognition success rate was low, but as the number of samples increased, the success rate increased. Test 1 took a total of 4 hours and 20 minutes.

Action recognition using the method of the present invention:

The equipment is a single Kinect, and the recognized actions are salute with right hand and waving with both hands. This method is used to build gesture label library, action-pose library, all 20 nodes are used, and it takes 30 minutes to build template library, including six actions: standing , left hand raised, right hand raised, both hands raised, left arm salute, right arm salute. Among them, the raising of the right hand is similar to the salute of the right arm, and the raising of both hands is a complex movement of three postures. Tester a is 173cm tall and weighs 60kg, tester b is 191cm tall and weighs 100kg, tester c is 181cm tall and weighs 80kg, which is consistent with test 1.

During the test, the testers are required to stand 1.5 meters in front of the equipment and perform standard movements as much as possible. Each person salutes with his right hand and waves his hands ten times. The template library is not updated throughout the test, so there is no need for each person to do multiple rounds of movements. , the statistical identification results. The statistics of the recognition results are shown in Table 2:

Table 2

Among them, tester c was mistakenly identified as a right-hand high raise for a salute with his right arm, and a right-hand high-raise was wrongly recognized as a high-handed gesture, which was related to the relevant action settings in the action-stance library.

The overall test had a much higher recognition success rate than Test 1, and had good success rates for all three testers of different body sizes. The overall test took 1 hour and 10 minutes in total, and the recognition actions were richer and more difficult than Test 1.

It shows that this method has good versatility for testers, and the input (design) of the template library is simpler and more convenient.

Claims (3)

1. A method of decomposing an action into gesture tags, comprising the steps of:

step 1, adopting skeleton tracking equipment to obtain position data of key nodes of human trunk actions at each moment, wherein the position data of the key nodes are data in a skeleton tracking equipment coordinate system; the key nodes at least comprise a key node HEAD, a key node SHOULDER CENTER, a key node SPINE, a key node HIP CENTER, a key node SHOULDE RIGHT, a key node SHOULDER LEFT, a key node ELBOW RIGHT, a key node ELBOW LEFT, a key node WRIST RIGHT, a key node WRIST LEFT, a key node HAND RIGHT, a key node HANDLEFT, a key node HIPRIGHT, a key node HIPLEFT, a key node KNIEE RIGHT, a key node KNIEE LEFT, a key node ANIKLE RIGHT, a key node ANIKLE LEFT, a key node FOOT RIGHT and a key node FOOT LEFT;

step 2, respectively converting the position data of the key nodes at each moment obtained in the step 1 into position data of the key nodes in a morphological coordinate system; the morphological coordinate system takes the facing direction of the human body trunk as the positive direction of a Z axis, the direction of the morphological upper end of the human body trunk as the positive direction of a Y axis, the left direction of the human body as the positive direction of an X axis and the key node HIP CENTER as the origin;

step 3, respectively obtaining attitude labels at each moment by using the position data of the key nodes under the morphological coordinate system at each moment, which are obtained in the step 2, wherein the attitude labels comprise main body attitude labels GL_bodyLeft forelimb posture label GL_1fRight front limb posture label GL_rfLeft hind limb posture label GL_1bAnd right hind limb posture label GL_rb；

The subject posture label GL in the step 3_bodyThe method for obtaining (1) is as follows:

selecting X_F，Y_FAnd Z_FFinding the coordinate value with the maximum absolute value, and finding the GL corresponding to the interval to which the coordinate value belongs_bodyIs the subject posture label GL_bodyThe following formula is adopted:

wherein, X_F，Y_FAnd Z_FCoordinates of 3 coordinate axes of the unit vector F respectively; unit vector

A vector formed by the key node HEAD and the key node HIP CENTER in the skeleton tracking device coordinate system;

the left forelimb posture label GL in the step 3_1fRight front limb posture label GL_rfLeft hind limb posture label GL_1bAnd right hind limb posture label GL_rbThe method for obtaining (1) is as follows:

the four posture labels all comprise three key nodes which are marked as a key node 1, a key node 2 and a key node 3, and the posture label GL of the left forelimb_1fThree key nodes included are ELBOW LEFT, WRIST LEFT and HAND LEFT, respectively, for the right forelimb posture label GL_rfThree key nodes included are KNIEE LEFT, ANIKLE LEFT and FOOT LEFT, respectively, for LEFT hind limb posture label GL_1bThe three key nodes are ELBOW LEFT, WRIST LEFT and HAND LEFT, right hind limb posture label GL_rbThe three key nodes are KNIEE LEFT, ANIKLE LEFT and FOOT LEFT respectively;

the data of the three key nodes in the morphological coordinate system are respectively used as (X)₁，Y₁，Z₁)(X₂，Y₂，Z₂)(X₃，Y₃，Z₃) Represents; the four attitude tags each include height tag G1, orientation tag G2, and curl tag G3;

the height label G1 is obtained by the following method:

G1＝(g₁+g₂+g₃) And/3, rounding, wherein,

wherein n is 1,2,3, Y_HIs a Y-axis coordinate in the morphological coordinate system of the key node HEAD_HCThe Y-axis coordinate of the key node SHOULDER CENTER under the morphological coordinate system;

the orientation label G2 is obtained as follows:

counting the symbols of the X-axis coordinate and the Z-axis coordinate of the key node 1, the key node 2 and the key node 3, and solving an orientation label G2 by adopting the following formula:

the method for obtaining the crimp label G3 is as follows:

introducing a key node 4 according to the key node 1, the key node 2 and the key node 3, respectively calculating the distances D between the key node 1, the key node 2 and the key node 3 and the key node 4₁，D₂，D₃(ii) a For left forelimb posture label GL_1fThe key node 4 is SHOULDER LEFT, for the right forelimb posture label GL_rfThe key node 4 is SHOULDER RIGHT, for left hind limb posture label GL_1bThe key node 4 is HIPLEFT, and the right hind limb posture label GL_rbThe key node 4 is HIPRIGHT;

the value of the curl label G3 is given by the following formula:

2. a method for obtaining an action template library, comprising the steps of:

step 1, making standard actions for multiple times, and decomposing the standard actions made each time into attitude tags at each moment; selecting the attitude tag at the initial moment as an initial frame attitude tag, and selecting the attitude tag at the termination moment as a termination frame attitude tag; taking the standard action made for the first time as a comparison standard action, and taking the standard actions made for other times as reference standard actions; comparing the initial frame posture label corresponding to the standard action as an initial frame comparison posture label, and using the termination frame posture label corresponding to the first made standard action as a termination frame comparison posture label;

the standard action made each time is decomposed into attitude tags at various moments, wherein the attitude tags are obtained according to the method of claim 1;

step 2, solving an initial frame similarity coefficient group, wherein the specific method is as follows:

respectively calculating the similarity sl1(A) of each attribute of the start frame attitude tag and the start frame contrast attitude tags of a plurality of reference standard actions_nThe formula used is as follows:

Sl1(A)_n＝A_n×z1_n÷ln(n∈Z，n∈[1，13]when n is 1,4,7,10,13, ln is 5, and the rest is 3)

Wherein A is_nFor initialized similarity coefficient values, A_nN denotes the serial number of the attribute, and the serial numbers 1 to 13 are the body posture labels GL of the posture labels, respectively_bodyLeft forelimb posture label GL_1fHeight label G1 in (1), left forelimb posture label GL_1fMiddle orientation label G2, left forelimb posture label GL_1fMiddle curl tag G3, left hind limb posture tag GL_1bHeight label G1 in, left hind limb posture label GL_1bMiddle orientation label G2, left hind limb posture label GL_1bMiddle curl label G3, right front limb posture label GL_rfHeight label G1, right front limb posture label GL_rfOrientation label G2, right forelimb posture label GL_rfThe curl label G3, the right hind limb posture label GL_rbHeight label G1, right hind limb posture label GL_rbOrientation label G2, right hind limb posture label GL_rbCrimp label G3; z1_nThe absolute value of the difference value of the corresponding attributes of the initial frame attitude tag and the initial frame contrast attitude tag of the reference standard action;

for each attribute n, selecting similarity Sl1(A) calculated according to the starting frame attitude labels of a plurality of reference standard actions_nThe second largest value in (b) is taken as the similarity coefficient value A1 under the attribute_nThe similarity coefficient value A1 corresponding to each attribute n_nForming an initial frame similarity coefficient set A_star＝{A1_n，n∈Z，n＝1，2，...，13}；

And step 3, solving a similarity coefficient group of the termination frame, wherein the specific method comprises the following steps:

respectively calculating the similarity Sl2(A) of each attribute of the terminal frame attitude tag and the terminal frame contrast attitude tags of a plurality of reference standard actions_nThe formula used is as follows:

Sl2(A)_n＝A_n×z2_n÷ln(n∈Z，n∈[1，13]when n is 1,4,7,10,13, ln is 5, and the rest is 3)

Wherein, z2_nThe absolute value of the difference between the corresponding attributes of the terminating frame pose tag and the terminating frame contrast pose tag for the reference standard action;

selecting similarity Sl2(A) calculated according to the attitude labels of the termination frames of a plurality of reference standard actions for each attribute n_nThe second largest value in (b) is taken as the similarity coefficient value A2 under the attribute_n(ii) a The similarity coefficient value A2 corresponding to each attribute n_nForming a group A of termination frame similarity coefficients_stop＝{A2_n，n∈Z，n＝1，2，...，13}；

And 4, aiming at a plurality of standard actions, obtaining an initial frame similarity coefficient group and an ending frame similarity coefficient group corresponding to each standard action according to the method in the step 1-3, wherein the initial frame similarity coefficient group and the ending frame similarity coefficient group corresponding to all the standard actions form an action template library.

3. A motion recognition method based on attitude tags is characterized by comprising the following steps:

step 1, decomposing the action to be recognized into attitude tags at each moment aiming at the action to be recognized; the gesture label for decomposing the action to be recognized into each moment is obtained according to the method of claim 1;

step 2, selecting a certain standard action in the action template library, calculating the similarity SL (B) n of each attribute between the termination frame attitude tag obtained in the step 1 and the termination frame attitude tag of the selected standard action, and recording the termination frame attitude tag as a tth frame attitude tag, wherein the adopted formula is as follows:

SL(B)_n＝A1_n×z3_n÷ln(n∈Z，n∈[1，13]when n is 1,4,7,10,13, ln is 5, and the rest is 3)

Wherein, z3_nSelecting similarity Sl1(A) calculated for a plurality of initial frame attitude tags of reference standard actions for the absolute value of the difference between the attributes corresponding to the end frame attitude tag obtained in step 1 and the end frame attitude tag of the selected standard action_nThe second largest value in (b) is taken as the similarity coefficient value A1 under the attribute_n；

Calculating the overall similarity S (B) between the attitude tag of the termination frame obtained in the step 1 and the attitude tag of the termination frame of the selected standard action, wherein the formula is as follows:

step 3, if the overall similarity S (B) is greater than the set threshold MAXBLUR, returning to the step 2; otherwise, executing step 4;

step 4, calculating the similarity SL (C) of each attribute between the previous frame attitude tag of the ending frame attitude tag and the starting frame attitude tag of the selected standard action_nAnd recording a frame attitude tag as a t-1 frame attitude tag, wherein the adopted formula is as follows:

SL(C)_n＝A2_n×z4_n÷ln(n∈Z，n∈[1，13]when n is 1,4,7,10,13, ln is 5, and the rest is 3)

Wherein, z4_nSelecting similarity Sl2(A) calculated for a plurality of end frame attitude tags of reference standard actions for the absolute value of the difference between the corresponding attributes of the previous frame attitude tag and the start frame attitude tag of the selected standard action_nThe second largest value in (b) is taken as the similarity coefficient value A2 under the attribute_n；

Calculating the overall similarity S (C) between the pose label of the last frame and the pose label of the initial frame of the selected standard action by the following formula:

step 5, if the overall similarity S (C) is less than a set threshold MAXBLUR, the action to be recognized is consistent with the selected standard action; and if the overall similarity S (C) is greater than the set threshold MAXBLUR, returning to the step 4, replacing the posture label of the t-1 frame of the processed object with the posture label of the t-2 frame of the processed object until the overall similarity S (C) is greater than the set threshold MAXBLUR when the processed object is the posture label of the first frame, and returning to the step 2.

Images