CN117653997A

CN117653997A - A method and system for boxing detection and attribute value calculation based on IMU

Info

Publication number: CN117653997A
Application number: CN202310922042.XA
Authority: CN
Inventors: 王悦磊; 王宁
Original assignee: Ningyu Shanghai Technology Co ltd
Current assignee: Ningyu Shanghai Technology Co ltd
Priority date: 2023-07-26
Filing date: 2023-07-26
Publication date: 2024-03-08

Abstract

This application is a method and system for boxing detection and attribute value calculation based on IMU. This method first generates a set of punching candidate areas based on the input IMU raw data; and then uses a neural network to identify the candidate areas. The punching method of punching; if the alternative area punching method is identified, the integral algorithm is further used to calculate the attribute value information of the punching area; finally, the punching method and attribute value information of the punching area are output. This application method solves the problems of high difficulty in boxing evaluation boxing method identification, difficulty in quantifying attribute values, lack of technology, and low accuracy.

Description

Boxing method detection and attribute value calculation method and system based on IMU

Technical Field

The application relates to the field of data processing and action recognition of IMU equipment, in particular to a method and a system for recognizing and acquiring attribute values of boxing actions.

Background

In the boxing action recognition field, the fine actions can be captured mainly through optical capturing in the market at present, but the application range of the boxing action recognition device is limited due to the reasons of high price, inconvenient carrying, complex equipment operation, large influence of color and light and the like; in the aspect of obtaining boxing property value information, most instruments such as a force measuring table and a photoelectric door are adopted at the present stage, and most of the instruments are laboratory equipment, so that accurate numerical values can be obtained, but the problems that the carrying is inconvenient, the result cannot be calculated in real time and the like are also caused.

The IMU device can detect and measure acceleration and rotation in real time, and the boxing method and attribute values of the boxing can be accurately calculated through the numerical values and the method. Meanwhile, the IMU equipment can be integrated into the wearable equipment, and through data interaction with software, the boxing method and the attribute value of the boxing can be calculated and displayed in real time without dividing the field and time, and the wearing and the use of athletes are facilitated.

Disclosure of Invention

The boxing method comprises the steps of determining a boxing attribute value of a boxing machine, determining a boxing attribute value of the boxing machine, and calculating the boxing attribute value of the boxing machine.

The application adopts the following technical scheme:

a boxing action recognition method, comprising the steps of:

step 1: collecting boxing action data by using IMU equipment fixed on the wrist, and preprocessing the data to obtain a preprocessed boxing alternative area;

step 2: utilizing the accelerometer, euler angle and rotation matrix change and the neural network to identify the fist alternative area data, judging a fist method, and taking the fist alternative area as an information area of the identification attribute value if the fist alternative area data is identified as a correct fist method;

step 3: if the fist alternative area identified in the step 2 is the correct fist making method, calculating fist attribute information by using an integral algorithm;

step 4: and outputting the boxing data and the boxing attribute information obtained in the step 2 and the step 3.

In addition, the application also relates to a boxing method and a computing system of attribute values thereof, comprising:

the boxing alternative region generation module is used for receiving data input by the IMU equipment and generating a group of boxing alternative regions;

the boxing detection module is used for judging the change of the accelerometer, the Euler angle and the rotation matrix, identifying what boxing is the boxing alternative area generated by the boxing alternative area generation module by utilizing a neural network, and taking the boxing alternative area information as boxing information if the boxing is correct;

the attribute value calculation module is used for calculating the boxing attribute value information of the data of the boxing alternative area when the boxing identification module identifies the boxing method;

the output module is used for outputting the boxing method and the attribute value information of the boxing obtained by the boxing identification module and the attribute value calculation module;

according to the method and the device, the boxing alternative area is identified through the neural network, a high-accuracy identification result can be obtained, information of the boxing attribute value is calculated through an integral algorithm, and a quantized result can be accurately obtained. By utilizing the recognition and calculation method, the problems of high recognition difficulty, large attribute value quantization difficulty, less technology and low accuracy of the boxing method in the prior art can be solved;

drawings

FIG. 1 is a flow chart of a boxing method detection and attribute value calculation method in the present application;

FIG. 2 is a flow chart of a method of determining an alternative region of a fist in the present application;

FIG. 3 is a schematic illustration of a set of punch data visualization and alternative areas in the present application;

FIG. 4 is a schematic diagram of a neural network used in a boxing method of the present application;

FIG. 5 is a flow chart of a punch attribute value calculation sequence in the present application;

FIG. 6 is a schematic diagram of the output results of a method for fist identification and punch attribute values in the present application;

fig. 7 is an overall frame diagram of a boxing attribute value recognition system in the present application.

Detailed Description

In order that the above objects and advantages of the present application will be readily understood, a more particular description of the invention will be rendered by reference to specific embodiments that are illustrated in the appended drawings.

Example noun terms designed in the present application are explained first.

IMU: (Inertial Measurement Unit ), sensors for mainly detecting and measuring acceleration and rotational motion, including accelerometers and gyroscopes (angular velocity meters), wherein the accelerometers can obtain acceleration in each axis direction, and the gyroscopes can obtain angular velocities of each axis, thereby determining angle information. Some inertial measurement units also include magnetometers, from which ambient magnetic field information can be obtained.

A punch method recognition and attribute value calculation thereof will be explained in detail in the present application with reference to fig. 1 to 6. As shown in fig. 1, the method specifically comprises the following steps:

step 1: in the IMU data which is input, a group of fist making candidate areas are generated, wherein the fist making candidate areas actually refer to related IMU data information when a fist making action to be identified is made, and in the subsequent processing process, the fist making method to be identified is determined by judging the fist making candidate area information.

Athletes typically perform a continuous punch action, and therefore, collect information on the punch candidate area, including:

determining an action interval of a fist alternative area;

collecting boxing to be identified in the action interval;

the punch action interval specifically indicates a time period from a punch starting posture to a punch ending posture, namely a time period from a punch withdrawing posture to a punch finishing posture.

After the boxing action zone is determined, the boxing action is considered to be completed, and the boxing action information in the zone can be acquired through the IMU equipment worn on the wrist.

The IMU device can acquire information of the triaxial accelerometer and the triaxial gyroscope in motion.

In practical application, as shown in fig. 2, the action interval of the punch candidate region is generated by the collected IMU raw data, the action gesture recognition is performed by calculating whether the acceleration of the region meets the threshold and the variation of the euler angle and the rotation matrix, whether the punch is correct is judged by neural network recognition, and the correct punch region is finally output.

Step 2: and (3) identifying whether the punch candidate area generated in the step (1) is a correct punch or not by using an accelerometer, an Euler angle, a rotation matrix and a neural network.

Step 21, firstly, the fist-making candidate area in step 1 is transferred into the candidate area preliminary determination function in step 22, if yes, the data is input into the neural network in a specified format to make fist-making determination, and the step further comprises:

step 22: and calculating whether the value of the acceleration y direction of the received region meets a threshold value or not, simultaneously solving the posture of the original data of the IMU through the Euler angle and the rotation matrix, returning to receive the next alternative region if the condition is not met, and transmitting the region into the neural network to perform boxing recognition if the condition is met.

Step 23: a neural network is constructed. The neural network may be used to determine whether the punch candidate area in step 22 is a correct punch. The structure of the neural network is shown in fig. 4. The network is connected with 1 flattening layer, 1 DropOut layer and 2 fully connected layers in sequence.

The input of the flattening layer is inertial navigation data original data, the IMU equipment transmits one piece of data at a time, the period comprises 9 data of values of two accelerometers and one gyroscope in three directions, and in an alternative area, 30 pieces of data before and after the position corresponding to the maximum acceleration value in the acceleration y direction are taken to form an array with the shape of (61, 9). The flattening layer outputs the set of data as a feature vector of length 549.

The input of the full-connection layer is the characteristic vector output by the flattening layer or the last full-connection layer, and after the input characteristic vector is multiplied by the connection weight of the full-connection layer, a fixed-length characteristic vector or a final prediction result is output. The output lengths of the two full connection layers are 64,3 respectively.

The network is trained by using a plurality of data of the punch area calibrated by the boxing method, and the neural network constructed in the step 23 is trained by a back propagation algorithm, so as to obtain network parameters. The network parameters of the neural network, including the connection weight of the full connection layer, are obtained through training by the method.

And (3) training the obtained network parameters through the step (23) to obtain a neural network which is input into a boxing region and can judge the boxing method of the region.

Through the neural network in step 23, it is identified whether the punch candidate area in step 22 is a correct punch action. If the punch is correctly made, the area information is outputted as a correct punch area.

As shown in fig. 3, IMU data is output in a continuous data form, and according to screening of the punch region, punch original information of each section can be obtained.

The specific punch generating area adopts the following modes:

firstly judging whether the acceleration meets a threshold value or not through the received alternative area, wherein the judgment basis is that the punch moves rapidly, the corresponding acceleration direction changes greatly, then the acceleration changes of the starting gesture and the ending gesture of the punch approach zero, further judging the action interval of one punch, and the interval of the punch can be seen obviously according to the visual result in the figure 3.

In the practical application process, the boxing recognition model needs to be trained in advance, and the training boxing model also needs corresponding model training data, so that the model method further comprises the following steps:

acquiring boxing action information and boxing action type by using IMU equipment;

processing the acquired punch information into unified punch gesture information;

extracting the action characteristic information of the fist from the fist action information and the fist gesture information;

training a boxing recognition model according to the boxing action characteristic information and the boxing action type;

the specific operation is that the information of the punch action and the type of the punch action are known information respectively, and the data can be marked in a manual marking mode. For example, after the boxing player wears the IMU device, technical actions of straight boxing, boxing and boxing are sequentially executed, and meanwhile, the IMU device acquires corresponding boxing action information, and the boxing action types are straight boxing, boxing and boxing in sequence.

After the boxing action information is obtained, the boxing action characteristic information and the boxing action type can be formed into a training sample set, and the training sample set is used for training a boxing recognition model.

Specifically, training a boxing recognition model according to the boxing action feature information and the boxing action type, including:

inputting the boxing action characteristic information into a boxing recognition model;

obtaining a predicted boxing action type output by the boxing recognition model;

calculating a model loss value through a back propagation algorithm according to the predicted boxing action type and the boxing action type;

and adjusting model parameters of the boxing recognition model based on the model loss value, and continuously training the boxing recognition model until training conditions are reached.

In the training of an actual boxing recognition model, a plurality of training sample sets exist, each training sample set comprises boxing motion characteristic information and boxing motion types corresponding to the boxing motion information, specifically, the boxing motion characteristic information is input into the boxing recognition model to be trained for training, and the boxing recognition model responds to the boxing motion characteristic information to output the boxing motion types.

In the training process, the predicted boxing action type is predicted by an untrained boxing recognition model, and the predicted boxing action type is different from the actual boxing action type to a certain extent, so that comparison is needed by predicting the boxing action type and the actual boxing action type, specifically, the model loss value is calculated by predicting the boxing action type and the actual boxing action type, and in the application, we select a multi-classification loss function to calculate the loss value.

After the loss value is calculated, the model loss value can be transmitted to the boxing recognition model through a back transmission algorithm, parameters of each layer of the boxing recognition model are adjusted, training of the current batch is finished, next boxing sample data sets of the next batch are needed to be used, and training of the model is continued until model training stopping conditions are reached.

In the present application, the model training stop condition includes that the loss value is smaller than a preset threshold value for 5 consecutive times and/or that the model training round reaches a preset round.

The boxing identification model in the application is completed through training of a large number of boxing action samples, so that the boxing identification model is more accurate, the error is smaller, the identification error rate is reduced, and the user experience is improved.

Finally, extracting parameters of each layer of the model according to a weight file generated by the boxing identification model, reproducing the model through C++, and finally implanting reproduced codes into a calculation module of the IMU equipment, so that boxing identification of original boxing information acquired by the IMU can be completed in real time.

And step 3, if the punch area is identified as the correct punch in the step 2, calculating the punch attribute value information by further utilizing an integral algorithm. Wherein the attribute values include punch speed, strength, power, acceleration, action time. The steps further include:

and step 31, receiving the punch area data determined in the step 2.

Step 32 calculates the punch speed using an integral algorithm.

Wherein the output result V is the speed of punch, t is the interval time between two sampling points, (x) _k 、y _k 、z _k ) Acceleration values at a certain moment in different directions are output for the IMU device.

Step 33, taking out the triaxial acceleration at the moment corresponding to the maximum acceleration value returned by the accelerometer in the fist area to calculate the total acceleration, specifically:

wherein a is _X 、a _Y 、a _Z The values of the triaxial accelerations, respectively.

Step 34, calculating the boxing force by using Newton's second law, specifically:

F＝ma

where m is the weight of the user's arm, if the weight of the arm is unknown, then by weight calculation, where male arm weight typically accounts for 0.057 of the weight, female arm weight typically accounts for 0.0497 of the weight, and a is the total acceleration.

Step 35, taking out the fist starting index to the maximum acceleration value index to calculate the fist action time, specifically:

T＝(t _max -t _start )

wherein (t) _max 、t _start ) The time corresponding to the maximum acceleration value and the punch starting time are respectively set.

Step 36, calculating to obtain the power of the punch through a dynamic formula, wherein the power is specifically:

P＝FV

wherein F and V are the punch strength and speed calculated in the foregoing.

And 37, integrating the boxing method and the attribute values and outputting the integrated boxing method and the integrated attribute values, specifically, communicating with mobile terminal equipment through a Bluetooth protocol, sending boxing information to the background, and displaying the information through output equipment such as a display screen, a touch screen and the like. Fig. 6 is a schematic diagram of a boxing method and attribute values corresponding to one-time boxing data.

In practical cases, the application provides a system for detecting boxing and identifying attribute values thereof, the structure of which is shown in fig. 7, and the system specifically comprises the following modules:

and the punch candidate region generation module is used for generating a group of punch candidate regions in the original data transmitted by the received IMU.

As shown in fig. 3, the IMU device outputs as continuous data, which may or may not include a punch.

And the boxing method identification module is used for judging boxing methods of the boxing alternative areas.

Specifically, the boxing method identification module is divided into two steps of judgment:

firstly, judging whether a boxing action is performed or not through the accelerometer and the change of the Euler angle and the rotation matrix;

and secondly, inputting the alternative area into the neural network in the step 23, and judging whether the area is a correct punch or not.

And if the two steps meet the conditions, performing a third step, and calculating the attribute value.

And the attribute value calculation module calculates and obtains the attribute of the punch through an integral algorithm, the Newton second theorem and a dynamic formula, wherein the attribute comprises punch speed, strength, power, acceleration and action time.

And the output module is used for communicating with the mobile terminal equipment through a Bluetooth protocol after the calculation of the attribute value is completed, sending the punch information to the background and displaying the information through output equipment such as a display screen, a touch screen and the like.

The above disclosure is only intended to assist in understanding the methods and core ideas of the present disclosure; meanwhile, as those skilled in the art will have variations in the specific embodiments and application scope in light of the ideas of the present disclosure, the present disclosure should not be construed as being limited to the above description.

Claims

1. A method of boxing detection and attribute value calculation based on IMU, characterized in that the method includes:

Step 1: Generate a set of punching candidate areas from the input IMU data;

Step 2: Use the alternative area judgment function to determine whether the punch alternative area is sufficient for punching. If so, use a neural network to identify whether the punch alternative area is a correct punch. If so, the punch alternative will be used. The regional information is used as a set of punching information and returned to the neural network to identify the punching method;

Step 3: If the punch candidate area identified in step 2 is the correct punch, use the integral algorithm to calculate the punch attribute value information;

Step 4: Output the punching method and punching attribute value information obtained in steps 2 and 3;

Among them, the punching candidate areas in step 1 include:

If it is detected that the acceleration value returned by the IMU exceeds the threshold, the point corresponding to the maximum acceleration value in the interval is taken. The starting time of the punching candidate area is the second zero point before the maximum value point, and the end time is the maximum value point. after the first zero point.

2. The method of claim 1, wherein step 2 further includes:

Step 21: Receive initial punching candidate areas;

Step 22: Calculate whether the value of the acceleration in the y direction meets the threshold based on the received area. At the same time, the original data of the IMU needs to be solved for the attitude through the Euler angle and rotation matrix. If the conditions are not met, return to receive the next alternative. area, if satisfied, the area will be passed to the neural network for boxing recognition;

Step 23: Build a neural network;

Step 24: Use several training data that calibrate punching candidate areas to train the network and obtain network parameters;

Step 25: Using the network parameters trained in step 24, obtain a neural network used to determine the punching method of the punching candidate area in step 22, and use the neural network to identify the punching method.

3. The method of claim 2, wherein the neural network in step 23 includes:

There are 1 flattening layer, 1 Dropout layer, and 2 fully connected layers connected in sequence. Among them, the flattening layer is used to flatten the punching candidate area data into one-dimensional data, and the Dropout layer is used to discard part of the data during training. , to improve the robustness of the neural network, the fully connected layer is used to output a fixed-length feature vector or result.

4. The method of claim 1, wherein step 3 calculates the punch attribute value, and the attribute value is calculated using the following formula:

The output results (V, a, F, P, T) are boxing-related attribute values, V is the speed of the punch, a is the total acceleration, F is the power of the punch, P is the power of the punch, and T is the completion Punch action time. t is the interval time between two sampling points, (x _k , y _k , z _k ) is the acceleration value at a certain moment in different directions output by the IMU device. _a , women’s arm weight generally accounts for 0.0497 of body weight. (t _max , t _start ) are the time corresponding to the value of maximum acceleration and the starting time of punching respectively.

5. A system for IMU-based boxing detection and attribute value identification, characterized in that the system includes:

The boxing alternative area generation module is used to receive data input from the IMU device and generate a set of punching alternative areas;

The boxing method detection module is used to use Euler angles and rotation matrices to determine whether the punching alternative area meets the punching action posture. If it is satisfied, a neural network is used to identify the boxing method generated by the boxing method alternative area generation module. If the punching method is correct, the punching candidate area information will be used as the punching information;

The indicator calculation module is used to calculate the punch attribute value information on the data in the boxing method alternative area when the punching method detection module identifies the punching method;

The output module is used to output the punching method and punching attribute value information obtained by the punching method detection module and the index calculation module;

Among them, the area generated in the boxing alternative area generation module is:

The formula for calculating the attribute value of the candidate area is:

6. The system of claim 5, wherein the boxing detection module further includes:

Neural network building blocks, used to build a neural network;

The neural network training module is used to train the network using several data that calibrate punching methods to obtain network parameters;

The punching method judgment module is used to obtain a neural network used to determine the punching method of the punching alternative area in the punching alternative area generation module through the network parameters trained by the neural network training module, and use the neural network to identify Boxing information.

7. The system according to claim 6, characterized in that the neural network constructed by the neural network building module: has 1 flattening layer, 1 Dropout layer and 2 fully connected layers connected in sequence, wherein the flattening layer is used for The punching candidate area data is flattened into one-dimensional data. The Dropout layer is used to discard part of the data during training to improve the robustness of the neural network. The fully connected layer is used to output a fixed-length feature vector or result.