Disclosure of Invention
Aiming at the defects or the improvement requirements of the prior art, the invention provides a motion identification method based on an environment background radio frequency signal, and aims to solve the technical problems that the existing motion detection and identification technology depends on a specific signal source, a receiving end and a transmitting end need to be in seamless cooperation, and an identification area is too small.
In order to achieve the above object, the present invention provides a method for identifying motion based on an environmental background wireless radio frequency signal, comprising:
s1, collecting multi-channel multi-band wireless radio frequency signals from an environment background where a moving target to be identified is located;
s2, motion characteristic extraction:
s2.1, extracting and separating a dynamic frequency shift component and a static frequency shift component in the acquired signal, and filtering the static frequency shift component and environmental noise;
s2.2, extracting a principal component sequence of the filtered signal, and further analyzing and extracting time-frequency characteristics of the principal component sequence;
s2.3, selecting and arranging time-frequency characteristics of the acquired signals to form a motion characteristic time-frequency graph to be identified;
s3, adopting a trained motion discriminator to discriminate the extracted motion characteristic time-frequency graph to obtain the probability distribution of the time-frequency graph belonging to the known motion type; when the maximum probability is larger than a preset threshold corresponding to a certain known motion, judging that the motion to be identified belongs to a known motion type, and determining the specific motion type to which the motion belongs; and when the probability of the time-frequency graph corresponding to all the known motion types is lower than a preset threshold value, judging that the motion characteristics to be identified corresponding to the time-frequency graph belong to unknown motion types.
Further, principal component analysis is used to extract the principal component sequence from the filtered signal.
Further, before performing step S2, the method further includes:
respectively extracting high correlation sequences in the collected multiband multipath wireless radio frequency signal samples: setting a plurality of sliding windows with determined sizes for the wireless radio frequency signals of each frequency band, enabling the sliding windows to slide according to a time sequence, and calculating the correlation among signal sequences contained in the windows; extracting a signal corresponding to a time sequence with the correlation larger than a set threshold;
the signals in the extracted sequence are sent to step S2.
Further, before performing step S3, the method further includes:
calculating the relative position of a moving object and a receiving end for acquiring a wireless radio frequency signal;
the motion speed of the object is calculated by using the correlation between the frequency shift and the speed, and the motion speed and the motion time-frequency feature map are merged and input into the motion discriminator described in step S3.
Further, the method further comprises:
and S4, clustering the motion characteristic time-frequency graphs corresponding to all the unknown motion types judged in the step S3.
Further, the method further comprises:
and S5, determining the number of the classes of the unknown motions according to the clustering result, and labeling each class of the unknown motions obtained after clustering by adopting a pseudo label.
Further, the method further comprises:
s6, evaluating the clustering performance effect, and when the clustering performance effect is higher than a set threshold value, taking the pseudo label of the unknown motion category as a new known label and updating the new known label to a known motion category label list;
s7, retraining the motion judger according to the new known motion category list, and repeatedly executing the steps S3-S7.
In general, the above technical solutions contemplated by the present invention can achieve the following advantageous effects compared to the prior art.
(1) The invention utilizes the attribute that the motion of an object can change the transmission path and other related characteristics of radio-electromagnetic waves in space, combines the advantages that environment background radio frequency signals (including Wi-Fi, LTE, 4G, 5G and the like) are ubiquitous and no additional signal emission sources are required to be arranged, gets rid of the dependence on a specific signal source and the surrounding environment in motion identification, greatly widens the recognizable area of motion, and is beneficial to further exploring the application potential of the radio signals in the motion identification field.
(2) The method comprises the steps of training a motion discriminator capable of identifying a known motion type by using label data by using clustering and self-learning technologies, extracting unknown motion type samples in acquired data, clustering the unknown motion type samples and marking the unknown motion type samples into a new motion type, realizing self-labeling and self-learning of the unknown motion type by circulating the steps, breaking the limitation of the prior art in identifying the motion type and an application scene, and greatly exploring the application potential of a wireless signal in the motion identification field.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. In addition, the technical features involved in the embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.
In order to meet the demand of modern people on high-speed and convenient data services, wireless radio frequency signals in the environmental backgrounds of 4G, LTE, 5G and the like are widely deployed, and the wireless radio frequency signals are brought to great convenience for people and are also tightly integrated into daily life. The invention provides a motion identification method based on an environment background radio frequency signal by utilizing the property that the basic parameters of the radio signal can be changed by the motion of an object on a radio signal propagation path and combining the advantage of wide coverage of the environment background radio frequency signal and the strong autonomous learning ability of self-learning and other technologies, so that the dependence on a signal source, a receiver position, an identifiable action type and the surrounding environment in motion identification is eliminated, an identifiable region and an environment are greatly expanded, and classification and identification can be automatically carried out aiming at unknown motion types.
The traditional motion identification method based on wireless signals depends on specific signals, and mainly comprises WiFi, frequency modulation continuous waves, sound waves, millimeter waves and the like. Although the signals in these methods are various and have different propagation characteristics, they all require the transmitting end to transmit fixed coded information to ensure that the identification accuracy is not reduced due to the variation of the signal source. Meanwhile, the transmitting end and the receiving end need to perform strict time synchronization operation, so that the receiving end can accurately calculate information such as the time of flight (tof) (time of flight) of the signal.
The invention chooses to utilize wireless radio frequency signals in the environment background such as 4G, LTE, 5G, etc., although the information transmitted by the signals is various and random, and various different environmental shelters in the space also present great challenges to the ability of the identification method to cross the environment domain. However, by analyzing and utilizing the frame structure characteristics of the signals, the motion characteristic extraction route is skillfully selected, so that the motion identification based on the wireless radio frequency signals of the environment background becomes possible.
First, a Primary Synchronization Signal (PSS) of a fixed transmission sequence at a fixed time interval is present in the frame structure of a radio frequency Signal in an environmental background such as 4G, LTE or 5G. The main synchronization signal in the wireless signal is extracted through a correlation detection method, so that the clock synchronization precision requirement of a receiving end can be reduced according to a fixed time interval, and the influence of the difference of transmission information on the motion identification accuracy can be avoided by using a fixed transmission sequence. Meanwhile, by extracting the dynamic Doppler frequency shift characteristics caused by the movement and filtering the environmental noise, the influence of static environmental obstacles on the identification result can be easily avoided, the correlation between the acquired ToF information and the shape and the position of the environmental obstacles does not need to be considered, and the movement identification of the cross-environment domain is realized.
The invention provides a motion identification method based on an environment background wireless radio frequency signal, which comprises the following steps:
s1, collecting multi-channel multi-band wireless radio frequency signals from an environment background where a moving target to be identified is located;
s2, motion characteristic extraction:
s2.1, separating a dynamic frequency shift component from a static frequency shift component in the acquired signal, and filtering the static frequency shift component and environmental noise; specifically, after conjugate multiplication is carried out on the obtained multipath signals pairwise, a band-pass filter is used for filtering, low-frequency signal static offset components, high-frequency out-of-band noise and other useless components are removed, the low-frequency cut-off frequency can be selected within the range of 0.5-5Hz, and the high-frequency cut-off frequency is higher than the frequency shift caused by the target maximum speed;
s2.2, extracting a principal component sequence of the filtered signal, and further extracting time-frequency characteristics of the principal component sequence; firstly, performing principal component analysis on the filtered signal to extract a principal component sequence of the signal; selecting short-time Fourier transform to adapt to non-stable environment background signals in the process of performing time-frequency transform on the extracted principal component sequence; in order to prevent spectrum leakage, a proper window function (such as a gaussian window function) needs to be selected to select a time sequence length in the process of performing short-time fourier transform; and finally, arranging the frequency spectrums within a specified time (0.5-5S) in time sequence to obtain a motion characteristic diagram to be identified.
S3, adopting a trained motion discriminator to discriminate the motion characteristic diagram to be recognized to obtain the probability distribution of the motion belonging to the known motion type; when the maximum probability is larger than a preset threshold corresponding to a certain known motion, judging that the motion to be identified belongs to a known motion type, and determining the specific motion type to which the motion belongs; and when all the probabilities are lower than the preset threshold corresponding to the known motion, judging that the motion to be identified belongs to the unknown motion category. The motion discriminator uses the characteristic diagram of the known motion with the label to carry out supervised learning so as to extract and summarize the characteristics of the characteristic diagram of the motion on time and space, and the parameters of the motion discriminator are adjusted by the extracted and summarized characteristics, so that the motion discriminator has high discrimination accuracy on the type of the known motion.
The motion discriminator consists of a hybrid neural network and an unknown motion discriminator. A hybrid Neural network consisting of CNN (volumetric Neural networks) plus RNN (Current Neural network) learns the frequency shift and time characteristics of known actions. And selecting a Softmax function in the last layer of the network, and calculating to obtain the probability distribution of the label corresponding to the sample.
For example, five gesture samples including up-down hand swinging, left-right hand swinging, front-back push-pull, clapping and drawing circle are divided into two groups; one group is a known motion type group and comprises three gestures of swinging up and down, swinging left and right, and pushing and pulling back and forth; the other group is an unknown motion type group and comprises two gestures of clapping and circling. Firstly, a motion discriminator is trained by using motion samples of a known motion type group with labels, all five motion samples without labels are input into the motion discriminator, and the motion discriminator calculates the probability that each motion sample belongs to three known motion types. Setting a probability threshold (such as 0.7), wherein if the maximum probability value of the sample belonging to the three known motion types is more than 70%, the sample is judged to belong to the known motion; if the maximum probability value of the sample belonging to the three known motion types is less than 70%, the sample is judged to belong to two unknown motions of clapping and circling.
When the feature classification of the input unknown motion is needed, the method of the invention further comprises the following steps: clustering all the unknown motion characteristic graphs determined in the step S3; according to the clustering result, the number of classes of unknown motions is determined, auxiliary labels for distinguishing motion classes are labeled on different classes of unknown motions, and the complete method flow refers to fig. 2.
In order to avoid the negative influence of the randomness and the difference of transmission information on the motion recognition accuracy and reduce the requirement of the clock synchronization precision of a receiving end, the invention extracts main synchronization signals existing in frame structures of wireless radio frequency signals in environment backgrounds such as 4G, LTE and 5G by utilizing a related sequence detection technology, and is characterized by having a fixed time interval and a fixed transmission sequence. The fixed transmission sequence can avoid the negative influence of randomness and difference of transmission information on the motion recognition accuracy, and the fixed time interval (millisecond level) can reduce the clock synchronization precision requirement of a receiving end from a nanosecond level to a millisecond level, so that the system cost is greatly reduced.
Based on the above analysis, the method of the present invention may further include the following processing before performing step S2, and the complete method flow refers to fig. 3:
respectively extracting high correlation sequences in the collected multiband multipath wireless radio frequency signal samples: setting a plurality of sliding windows with determined sizes for the wireless radio frequency signals of each frequency band, enabling the sliding windows to slide according to a time sequence, and calculating the correlation among signal sequences contained in the windows; extracting a signal corresponding to a time sequence with the correlation larger than a correlation threshold value; wherein, the value of the correlation is between [0,1], the closer the value is to 1, the higher the correlation is represented, and the closer the value is to 0, the lower the correlation is represented. In the present embodiment, the correlation threshold is set to 0.9.
The signals in the extracted sequence are sent to step S2.
In order to further improve the motion recognition accuracy, the method may further include the following processing before executing step S3:
the signal is reflected to the receiving end by the moving object, the relative Angle between the receiving end and the signal reflected by the moving object is calculated by adopting an Angle-of-Arrival (AOA) positioning algorithm, and then the relative position between the moving object and the receiving end can be obtained by a triangulation method;
then, the motion speed v of the object can be obtained by using a frequency shift-speed conversion formula:
wherein f isdIs a frequency shift quantity, lambda is the wavelength of a wireless signal, and cos theta is a cosine value of an included angle between the moving speed direction and a connecting line of a moving object and a receiving end;
the moving speed direction can be determined according to the moving direction of the moving object; and the motion speed and the motion characteristic map are merged and input to the motion discriminator in step S3.
When the self-adaptive identification capability of the input unknown motion is required, the method of the invention also utilizes clustering and self-learning technology to realize self-labeling and self-learning of the unknown motion type, and specifically comprises the following steps, and the complete method flow refers to the figure 4:
unknown motion samples are first clustered, optionally using a Density-Based Clustering method with Noise (DBSCAN), in hopes of finding motion feature clusters of arbitrary shape and finding invalid motion samples. Then the clustering performance effect is evaluated, optionally by calculating a contour Coefficient (SC) to evaluate the clustering effect:
firstly, calculating the contour coefficient of a single motion sample i, and assuming that the average distance between the sample i and other samples in a cluster where the sample i is located is a (i) and the average distance between the sample i and other cluster samples is b (i), the contour coefficient S (i) of the sample i is:
the overall contour coefficient SC of the cluster is then calculated:
wherein, N represents the number of unknown motion samples for clustering. The value of the contour coefficient is between [ -1,1], the closer the value to 1, the better the clustering performance is represented, and the closer the value to-1, the worse the clustering performance is represented. In this embodiment, when the contour coefficient is greater than or equal to 0.6, the clustering effect is considered to be better, at this time, pseudo labels of corresponding categories are marked on different types of clustered unknown motion samples, and if the contour coefficient is less than 0.6, the clustering effect is considered to be poor, at this time, the unknown motion samples are re-clustered. Meanwhile, the pseudo label is used as a new known label and is updated to a known motion category label list. Finally, the motion judger is retrained based on the new list of known motion classes, and steps S3-S7 are repeatedly performed.
Self-learning means that the system has the ability to improve the algorithm according to experience in its own run. In this embodiment, the motion discriminator firstly distinguishes the known motion sample from the unknown motion sample, and when the unknown motion sample is clustered and the auxiliary label mark is completed, the motion discriminator can be trained again by using the unknown motion sample, so as to improve the algorithm in the operation process.
It will be understood by those skilled in the art that the foregoing is only a preferred embodiment of the present invention, and is not intended to limit the invention, and that any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the scope of the present invention.