CN114580473A - Radar-based human behavior identification method and system - Google Patents

Abstract

The invention provides a human behavior identification method and a system based on radar, which comprises the following steps: a signal acquisition step: the radar wave beam radiates a detected human body to obtain a baseband signal of the radar; a signal processing step: preprocessing the baseband signals, extracting the characteristics of the preprocessed baseband signals to obtain characteristic indexes of human behaviors, and classifying the human behaviors. The invention is suitable for the human behavior recognition of all-time, non-contact and no privacy disclosure risk, and can recognize: the total 6 common human body behaviors of walking-falling, standing-falling, normal walking, standing-swinging, standing-sitting, walking-sitting and the like solve the problems of difficult resolution and error resolution of similar actions such as falling, sitting and the like, and can realize high-precision recognition effect of small sample size based on feature driving; and training the model by using a small amount of sample data as a training set to obtain a high-efficiency and high-precision training model and realize high-accuracy behavior recognition.

Description

Radar-based human behavior recognition method and system

technical field

The present invention relates to the technical field of human behavior recognition, and in particular, to a radar-based human behavior recognition method and system. In particular, it is preferably related to a method and system for indoor human behavior recognition based on millimeter wave radar.

Background technique

In recent years, human activity recognition technology has become more and more important in our daily life. For example, falls are considered to be more important human activity information. Studies have shown that nearly 1/3 of the elderly will fall every year, which seriously affects the quality of life of the elderly. By observing other human behavior information for a long time, it is possible to count its development laws and analyze the physical and mental health of the human body.

Human behavior recognition technology can be roughly divided into contact and non-contact according to the type of sensor. Among them, the contact sensor is mainly an acceleration sensor. Non-contact sensors mainly include cameras, radars, and microphones.

The Chinese invention patent document with publication number CN113869189A discloses a method, system, equipment and medium for human behavior recognition, which belongs to the field of data retrieval. Video features, acceleration features and angular velocity features related to human behavior recognition in video sequences, acceleration signals and angular velocity signals; perform multi-sensor signal fusion processing on the cyclic matrix formed by the acceleration features and the cyclic matrix formed by the angular velocity features to obtain the inertial sensor fusion features vector; perform dual-modal fusion based on Tucker decomposition of inertial sensor fusion feature vectors and video features to obtain fused behavior features; input the fused behavior features into the classifier for human behavior recognition to predict and output human actions.

In view of the above-mentioned related technologies, the inventor believes that when using a touch sensor, the touch sensor needs to be worn for a long time, so it is easy to be damaged, and the sensor is easy to lose, and the false alarm rate of the sensor is high. When using a non-contact sensor, the camera and microphone in the non-contact sensor are likely to cause privacy leakage, and the camera has high requirements on lighting conditions and cannot be used in environments with weak lighting conditions. The anti-interference of the microphone is poor and easy to use. received environmental interference.

SUMMARY OF THE INVENTION

In view of the defects in the prior art, the purpose of the present invention is to provide a radar-based human behavior recognition method and system.

A radar-based human behavior recognition method provided according to the present invention includes the following steps:

Signal acquisition steps: the radar beam radiates the human body under test to obtain the baseband signal of the radar;

Signal processing step: preprocessing the baseband signal, extracting the features of the preprocessed baseband signal, obtaining the characteristic index of human behavior, and classifying the human behavior according to the characteristic index of human behavior.

Preferably, the signal acquisition step includes the following steps:

Signal generation steps: generate radar input signal;

Splitting step: dividing the radar input signal into a first radar input signal and a second radar input signal;

The step of sending and receiving: transmitting the first radar input signal, the first radar input signal encounters the reflection of the human body, and receives the reflected first radar input signal;

Frequency mixing step: mixing the second radar input signal and the reflected first radar input signal to obtain the baseband signal.

Preferably, the method further includes an analog-to-digital conversion step: converting the baseband signal into a digital signal;

In the signal processing step, the digital signal is processed, and feature extraction is performed on the preprocessed digital signal to obtain the feature index of human behavior.

Preferably, the signal processing step includes a baseband signal preprocessing step, a feature extraction step and a classification step;

The baseband signal preprocessing step includes an image acquisition step: domain analysis is performed on the baseband signal to obtain a range-time domain image and a micro-Doppler image of the baseband signal;

The feature extraction step: perform feature extraction on the human body in the distance-time domain image and the micro-Doppler image to obtain the feature index of human behavior;

The classifying step: classify the human behavior according to the characteristic index of the human behavior.

Preferably, the characteristic index of the human behavior includes energy gradient, loss duration, maximum displacement, stationary duration and average zero-crossing rate.

Preferably, the baseband signal preprocessing step further includes a clutter suppression step: suppressing the static clutter in the range-time domain image by using a variable-range static clutter suppression method;

In the feature extraction step, feature extraction is performed on the human body in the micro-Doppler image and the range-time domain image after suppressing static clutter to obtain the feature index of human behavior.

Preferably, the baseband signal preprocessing step further includes a ridge line correction step: extracting the ridge line of the human body in the distance-time domain image after the static clutter suppression, and correcting the ridge line of the human body according to the energy threshold, and correcting the ridge line of the human body The posterior ridges are put into the distance-time domain image;

In the feature extraction step, feature extraction is performed on the human body in the range-time domain image and the micro-Doppler image after the ridge line is corrected to obtain the feature index of human behavior.

A radar-based human behavior recognition system provided according to the present invention includes a radar and a signal processing module;

The radar radiates the measured human body through the radar beam, and obtains the baseband signal of the radar;

The signal processing module preprocesses the baseband signal, performs feature extraction on the preprocessed baseband signal, obtains the characteristic index of human behavior, and classifies the human behavior according to the characteristic index of the human behavior.

Preferably, the radar includes a transceiver antenna, a mixer, a power divider and a signal generator;

the signal generator generates a radar input signal;

The power divider divides the radar input signal into a first radar input signal and a second radar input signal;

The transceiver antenna transmits the first radar input signal, the first radar input signal encounters the reflection of the human body, and receives the reflected first radar input signal;

The mixer mixes the second radar input signal and the reflected first radar input signal to obtain the baseband signal.

Preferably, the radar further includes an analog-to-digital converter to convert the baseband signal into a digital signal;

The signal processing module is a digital signal processing module; the digital signal processing module preprocesses the digital signal, and performs feature extraction according to the preprocessed digital signal, so as to obtain the characteristic index of human behavior. Classification of human behavior.

Compared with the prior art, the present invention has the following beneficial effects:

1. The present invention is suitable for all-day, non-contact, and no risk of privacy leakage for human behavior recognition;

2. The present invention can identify: walking-falling, standing-falling, normal walking, standing-waving, standing-sitting and walking-sitting, a total of 6 common human behaviors, which solve the problems of falling and sitting, etc. Difficulty distinguishing and misidentifying similar actions;

3. The present invention can realize the effect of high-precision recognition with a small sample size based on feature driving; use a small amount of sample data as a training set to train the model, obtain a high-efficiency and high-precision training model, and realize high-accuracy behavior recognition.

Description of drawings

Other features, objects and advantages of the present invention will become more apparent by reading the detailed description of non-limiting embodiments with reference to the following drawings:

Figure 1 is a system structure diagram;

Figure 2 is a time-distance image;

Figure 3 is a micro-Doppler map;

Fig. 4 is the schematic diagram before static clutter suppression;

Figure 5 is a schematic diagram after static clutter suppression;

Fig. 6 is the schematic diagram of the energy on the ridge line of the target object;

Fig. 7 is the schematic diagram before ridge line correction;

Fig. 8 is the schematic diagram after ridge line correction;

Figure 9 is a decision tree model diagram.

Detailed ways

The present invention will be described in detail below with reference to specific embodiments. The following examples will help those skilled in the art to further understand the present invention, but do not limit the present invention in any form. It should be noted that, for those skilled in the art, several changes and improvements can be made without departing from the inventive concept. These all belong to the protection scope of the present invention.

Embodiment 1 of the present invention discloses an indoor human behavior recognition method based on millimeter wave radar. As shown in FIG. 1 , the method includes the following steps: a signal acquisition step: the radar beam radiates the detected human body, and the baseband signal of the radar is acquired.

The signal acquisition step includes the following steps:

Signal generation steps: generate radar input signal;

Splitting step: dividing the radar input signal into a first radar input signal and a second radar input signal.

Transceiving step: transmitting the first radar input signal, the first radar input signal encounters the reflection of the human body, and receives the reflected first radar input signal.

The frequency mixing step: mixing the reflected first radar input signal and the second radar input signal to obtain a baseband signal.

The analog-to-digital conversion step: converting the baseband signal into a digital signal.

Signal processing step: preprocessing the baseband signal, extracting the features of the preprocessed baseband signal, obtaining the characteristic index of human behavior, and classifying the human behavior according to the characteristic index of human behavior. That is, the method based on multi-feature fusion is used to classify human behavior.

That is, the digital signal is preprocessed, and the feature extraction is performed according to the preprocessed digital signal, and the feature extraction is performed on the preprocessed digital signal to obtain the feature index of human behavior. Characteristic indicators of human behavior include energy gradient, loss duration, maximum displacement, stationary duration, and average zero-crossing rate.

The signal processing step includes a baseband signal preprocessing step, a feature extraction step and a classification step.

The baseband signal preprocessing step includes an image acquisition step, a clutter suppression step and a ridge line correction step. Image acquisition step: perform domain analysis on the baseband signal, and obtain the range-time domain image and micro-Doppler image of the baseband signal. The step of clutter suppression: the static clutter in the range time domain image is suppressed by the variable range static clutter suppression method.

Ridge line correction step: extract the ridge line of the human body in the distance-time domain image after static clutter suppression, and correct the ridge line of the human body according to the energy threshold, and put the corrected ridge line into the distance-time domain image.

Feature extraction step: perform feature extraction on the human body in the micro-Doppler image and the distance-time domain image after correction of the ridge line to obtain the feature index of human behavior.

Classification step: classify human behavior according to the characteristic index of human behavior.

Embodiment 1 of the present invention also discloses an indoor human behavior recognition system based on a millimeter wave radar, as shown in FIG. 1 , including a radar and a signal processing module. The radar radiates the measured human body through the radar beam, and obtains the baseband signal of the radar.

Radar includes transceiver antennas, mixers, power dividers, signal generators, filters, signal amplifiers, and analog-to-digital converters (AD converters). Signal amplifiers include power amplifiers and low noise amplifiers. The transceiver antenna includes a transmitting antenna and a receiving antenna. The English abbreviation of AD converter is ADC, the full English name is analog to digital converter, and the Chinese translation is analog to digital converter.

The signal generator generates the radar input signal. The power divider divides the radar input signal into a first radar input signal and a second radar input signal. Power divider: Divide the energy of one radar input signal into two channels, which are respectively used for the transmitting antenna and the mixer.

The power amplifier is used for amplifying the first radar input signal.

The transceiver antenna transmits the amplified first radar input signal, the first radar input signal encounters the reflection of the human body, and receives the reflected first radar input signal. Transceiver antennas are used to transmit and receive radar signals. The transmitting antenna is used for transmitting the amplified first radar input signal; the receiving antenna is used for receiving the reflected first radar input signal when the first radar input signal encounters the reflection of the human body.

A low-noise amplifier is used to amplify the reflected radar input signal.

The mixer receives the second radar input signal, and mixes the amplified reflected first radar input signal and the second radar input signal to obtain a baseband signal. Mixer: Mix the transmitted signal (the second radar input signal) and the echo signal (the reflected first radar input signal) to obtain a baseband signal, which is used for subsequent data processing.

The filters include low-pass filters; low-pass filters are used to filter the baseband signal.

An analog-to-digital converter converts the filtered baseband signal to a digital signal.

The signal processing steps of the signal processing module: preprocess the baseband signal, extract the features of the preprocessed baseband signal, obtain the characteristic index of human behavior, and classify the human behavior according to the characteristic index of human behavior.

The signal processing module is a digital signal processing module; the digital signal processing module preprocesses the digital signal, and performs feature extraction according to the preprocessed digital signal to obtain the feature index of human behavior, and analyzes the human behavior according to the feature index of human behavior. sort. Digital signal processing module: According to the baseband signal of the radar, the signal is processed to extract features and classify them.

The invention provides an all-day, non-contact, and no privacy leakage risk based indoor human behavior identification method and system based on millimeter-wave radar. The indoor human behavior recognition system based on millimeter wave radar is shown in Figure 1 below. The method and system for indoor human behavior recognition based on millimeter wave radar includes an indoor human behavior recognition method based on millimeter wave radar and an indoor human behavior recognition system based on millimeter wave radar.

The second embodiment of the present invention also provides an indoor human behavior recognition method based on a millimeter wave radar, including: data preprocessing, feature extraction, and behavior classification.

The specific method flow includes: Step 1: First, use the radar to obtain the baseband signal of human behavior, perform domain analysis on it, and obtain the distance-time domain information (range-time domain image) and micro-Doppler information (micro-Doppler information) of the baseband signal. Doppler image).

Specifically, the baseband signal S _B (t) is obtained after the radar signal is mixed, which can be expressed as:

表示初始相位；exp表示指数函数；π表示圆周率；f_b＝2BR_T/cT，f_b表示差拍频率，为混频器所混合两个信号频率的差值；B表示带宽；R_T表示目标距离雷达的距离；c表示电磁波的传播速度，T表示扫频周期的时长。因此我们对基带信号的每个扫频周期进行快速傅里叶变换便可得到该时刻的目标距离像，对所有扫频周期进行短时傅里叶变换便可获得距离-时间图像。如图2所示为来回走动的距离-时间像，Range表示距离；time表示时间。对每个时刻的距离像之和进行短时傅里叶变换便可得到微多普勒图像。如图3所示为站着摆手的微多普勒图像。Frequency表示频率，Sec英文全称为second，中文译文为秒。Among them, t represents time;

represents the initial phase; exp represents the exponential function; π represents the pi; f _b = 2BR _T /cT, f _b represents the beat frequency, which is the difference between the frequencies of the two signals mixed by the mixer; B represents the bandwidth; R _T represents the target The distance from the radar; c represents the propagation speed of the electromagnetic wave, and T represents the duration of the frequency sweep cycle. Therefore, we can obtain the target range image at this moment by performing fast Fourier transform on each frequency sweep period of the baseband signal, and obtaining the range-time image by performing short-time Fourier transform on all frequency sweep periods. Figure 2 shows the distance-time image of walking back and forth, where Range represents distance; time represents time. The micro-Doppler image can be obtained by performing short-time Fourier transform on the sum of the distance images at each moment. Figure 3 shows a micro-Doppler image of a standing and waving hand. Frequency indicates frequency, Sec is called second in English, and the Chinese translation is second.

Step 2: According to the characteristics of the experiment, propose and use the static clutter elimination method of variable distance to suppress and eliminate the static clutter within the field of view. The field of view is the detection area that the radar can reach. Static clutter, such as a person walking around the house, will produce a curve in the distance-time image, and if there is a large stationary metal reflective object in the house, a straight line will appear in the distance-time image.

Specifically, the static clutter will affect the recognition of the target during the experiment, so the static clutter must be suppressed. In this patent, we propose a variable-range static clutter suppression method. Its method can be expressed as:

R _b (i)=R _b (i)-R _b (i+n(i)) (0.2)

Among them, R _b (i) represents the range image of the i-th frequency sweep cycle, R _b (i) represents the range image, that is, a column of data on the range-time image, and i represents a column of data. n(i) represents the distance between the reference range image and the current range image, n(i) changes with the change of i, and its value can be determined according to the following formula:

Here th is a fixed value, we take th=0.5/T, n represents the time variable, and th is the upper limit of n. The range-time images before and after variable-range static clutter suppression are shown in Figures 4 and 5.

Step 3: Extract the target (human body) ridge line in the distance-time domain image, and correct the target ridge line according to the energy threshold. The energy threshold depends on the environment. Generally, the energy threshold is set according to the energy on the ridge line of the target object arranged from high to low, and the energy is selected at a position of 1:9 on the area enclosed by the rearranged ridge line. As shown in Figure 6, the Chinese translation of normalized amplitude is the normalized amplitude.

Specifically, after static clutter suppression, we can clearly see the distance change of the target, and then we extract the target trajectory (that is, the ridge line), and correct the ridge line according to the energy threshold, and the energy on the ridge line The distance less than the energy threshold is corrected to the position at the previous moment, and the ridge lines before and after the correction are shown in Figures 7 and 8.

Step 4: perform feature extraction on the target, and the extracted features include: energy gradient, loss duration, maximum displacement, stationary duration, and average zero-crossing rate to obtain the characteristic index of the radar data.

Specifically, the range-time image and micro-Doppler image of the target are analyzed, and five features in total, including energy gradient, loss duration, maximum displacement, stationary duration, and average zero-crossing rate, are extracted respectively. The energy gradient and loss duration are mainly used to distinguish between falls and non-falls, the maximum displacement is mainly used to distinguish whether the activity has a displacement or not, the stationary time is mainly used to distinguish whether the activity includes a stationary state, and the average zero-crossing rate is mainly used to distinguish stations. There are two behaviors of standing-sitting and standing-waving. The energy gradient, loss duration, maximum displacement, stationary duration are extracted from the distance-time image. The mean zero-crossing rate was extracted from the dimensional Doppler images. Distinguishing falls and non-falls, distinguishing whether the activity has displacement, distinguishing whether the activity contains a stationary state, distinguishing standing-sitting and standing-waving are all distinguished using the thresholds obtained by SVM support vector machine training. The full name of SVM in English is Support Vector Machine, and the Chinese translation is Support Vector Machine.

Step 5: Put the characteristic index of the radar signal into the designed decision tree, and classify the radar signal according to the size of each characteristic index.

The specific implementation method is as follows: Step 5.1: Collect 6 kinds of behavior data of 6 testers, repeat the test 6 times for each behavior of each person, and obtain a total of 216 sets of experimental data.

Step 5.2: Perform the above-mentioned steps 1-4 for the above 216 groups of experiments, and obtain the characteristic indicators of each group of data to form a characteristic data set.

Step 5.3: Design a decision tree structure according to the characteristics of each feature. The decision tree structure is shown in Figure 9.

Step 5.4: Use SVM and feature dataset to train each layer of the decision tree to obtain a decision tree model for classification.

Step 5.5: Put the characteristic index of the radar signal into the decision tree model, classify the radar signal, and obtain the category of the behavior.

Those skilled in the art know that, in addition to implementing the system provided by the present invention and its various devices, modules, and units in the form of pure computer-readable program codes, the system provided by the present invention and its various devices can be implemented by logically programming the method steps. , modules and units realize the same function in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers and embedded microcontrollers. Therefore, the system provided by the present invention and its various devices, modules and units can be regarded as a kind of hardware components, and the devices, modules and units included in it for realizing various functions can also be regarded as hardware components. The device, module and unit for realizing various functions can also be regarded as both a software module for realizing the method and a structure in a hardware component.

The specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the above-mentioned specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essential content of the present invention. The embodiments of the present application and features in the embodiments may be arbitrarily combined with each other without conflict.

Claims (10)

1. A human behavior recognition method based on radar is characterized by comprising the following steps:

a signal acquisition step: the radar wave beam radiates a detected human body to obtain a baseband signal of the radar;

a signal processing step: preprocessing the baseband signals, extracting the characteristics of the preprocessed baseband signals to obtain characteristic indexes of human behaviors, and classifying the human behaviors according to the characteristic indexes of the human behaviors.

2. The radar-based human behavior recognition method according to claim 1, wherein the signal acquisition step comprises the steps of:

a signal generating step: generating a radar input signal;

a shunting step: dividing the radar input signal into a first radar input signal and a second radar input signal;

a receiving and transmitting step: transmitting a first radar input signal, wherein the first radar input signal meets human body reflection and receives the reflected first radar input signal;

a frequency mixing step: and mixing the second radar input signal and the reflected first radar input signal to obtain the baseband signal.

3. The radar-based human behavior recognition method of claim 1, further comprising an analog-to-digital conversion step of: converting the baseband signal into a digital signal;

in the signal processing step, the digital signals are preprocessed, and feature extraction is carried out on the preprocessed digital signals to obtain feature indexes of human behaviors.

4. The radar-based human behavior recognition method according to claim 1, wherein the signal processing step includes a baseband signal preprocessing step, a feature extraction step, and a classification step;

the baseband signal preprocessing step includes an image acquisition step: carrying out domain analysis on the baseband signals to obtain distance time domain images and micro Doppler images of the baseband signals;

the characteristic extraction step comprises the following steps: extracting the characteristics of the human body in the distance time domain image and the micro Doppler image to obtain characteristic indexes of human body behaviors;

the classification step comprises: and classifying the human body behaviors according to the characteristic indexes of the human body behaviors.

5. The radar-based human behavior recognition method according to claim 4, wherein the characteristic indicators of the human behavior include an energy gradient, a loss duration, a maximum displacement, a rest duration, and an average zero crossing rate.

6. The radar-based human behavior recognition method of claim 4, wherein the baseband signal preprocessing step further comprises a clutter suppression step: static clutter in the distance time domain image is suppressed by a variable-pitch static clutter suppression method;

in the characteristic extraction step, the characteristic extraction is carried out on the human body in the micro Doppler image and the distance time domain image after static clutter suppression, and the characteristic index of the human body behavior is obtained.

7. The radar-based human behavior recognition method of claim 6, wherein the baseband signal preprocessing step further comprises a ridge line modification step: extracting the ridge line of the human body in the distance time domain image after the static clutter suppression, correcting the ridge line of the human body according to an energy threshold value, and putting the corrected ridge line into the distance time domain image;

in the characteristic extraction step, the characteristic extraction is carried out on the human body in the micro Doppler image and the distance time domain image after the ridge line is corrected, and the characteristic index of the human body behavior is obtained.

8. A radar-based human behavior recognition system, which is characterized in that the radar-based human behavior recognition method of any one of claims 1 to 7 is applied, and comprises a radar and a signal processing module;

the radar radiates a detected human body through radar beams to obtain a baseband signal of the radar;

the signal processing module is used for preprocessing the baseband signals, extracting the characteristics of the preprocessed baseband signals to obtain the characteristic indexes of the human behaviors, and classifying the human behaviors according to the characteristic indexes of the human behaviors.

9. The radar-based human behavior recognition system of claim 8, wherein the radar includes a transceiving antenna, a mixer, a power divider, and a signal generator;

the signal generator generates a radar input signal;

the power divider divides the radar input signal into a first radar input signal and a second radar input signal;

the receiving and transmitting antenna transmits a first radar input signal, the first radar input signal meets the reflection of a human body, and the reflected first radar input signal is received;

and the mixer mixes the second radar input signal and the reflected first radar input signal to obtain the baseband signal.

10. The radar-based human behavior recognition system of claim 8, wherein the radar comprises an analog-to-digital converter: converting the baseband signal into a digital signal;

the signal processing module is a digital signal processing module; the digital signal processing module is used for preprocessing the digital signals, extracting features according to the preprocessed digital signals to obtain feature indexes of human behaviors, and classifying the human behaviors according to the feature indexes of the human behaviors.

Images