CN108932479A - A kind of human body anomaly detection method - Google Patents

Abstract

The invention relates to a method for detecting abnormal human behaviors, which collects video sequences of four behaviors of people walking, running, punching, and falling to the ground under a single fixed background, uses the Kalman filter algorithm to extract moving human targets, and extracts moving human motions. The target image stores the detected moving human target image, constructs a training data set, extracts the Hu invariant moment, image entropy and aspect ratio features of the image to establish a Bayesian classifier, and realizes four kinds of human abnormal behaviors. Classification recognition, recognition rate and real-time performance are better. The combination of Bayesian classifier and convolutional neural network method is used to determine the final abnormal behavior category, which can improve the recognition accuracy and reduce misjudgment and missed detection. The invention can realize the detection and classification recognition of the four behaviors of walking, running, punching, and falling to the ground under a fixed background. This method is not limited to these four behaviors, and can also be used for crowd gathering, fighting, etc. Judgment of other abnormal behaviors.

Description

A method for detecting abnormal human behavior

technical field

The invention relates to a human body recognition technology, in particular to a human body abnormal behavior detection method based on a Bayesian classifier and a convolutional neural network.

Background technique

Abnormal behavior analysis technology has a wide range of applications in public security, smart home and other fields. Monitor the fall and coma of the elderly living alone, and monitor abnormal situations such as falling to the ground, fighting, abnormal gathering of crowds, and riots in crowded places.

The current security method is to install cameras in public areas, but there are two main problems in the traditional monitoring mode: first, a large number of monitors are required to view the video around the clock, which can easily cause sensory fatigue, resulting in missed and wrong detections; second, Multiple cameras shot 24 hours a day without interruption, generating a large amount of data, which made it difficult to find later. Therefore, computer-aided detection is needed.

In real scenes, there are often some uncontrollable factors, such as lighting changes, shadows, occlusions, viewing angle conversions, etc., which make the existing abnormal behavior detection methods very limited in practical applications. Therefore, it is necessary to improve the robustness of abnormal behavior analysis. Sexuality is a problem that needs to be solved urgently.

The patent with the application number 201610790306.0 proposes a real-time abnormal human behavior recognition method based on multi-scale convolutional neural network, which adds three-dimensional convolution, three-dimensional downsampling, and three-dimensional pyramid structure to the convolutional neural network for feature extraction. and trained on a specific video set.

The patent with the application number 201610403457.6 proposes an ATM human abnormal behavior detection method based on the linear difference judgment optimization HMM algorithm, which can optimize the imbalance between the feature vectors by standardizing the feature vectors according to the linear difference judgment results between the feature vectors defect.

The patent with application number 201610104559.8 proposes a human abnormal behavior detection method based on motion detection. It adopts the background subtraction method and motion history image for motion detection, and the context information matching method for human body matching. The system judges the geometric features to judge the abnormal behavior of the human body and determine the direction of intrusion.

The patent application number 201710544547.1 proposes a human behavior recognition system based on deep learning, which involves the field of captcha. The neural network model used includes embedding layer, LSTM, fully connected layer and softmax layer, which has high resolution ability.

The patent with the application number 201710496942.7 proposes a method for identifying abnormal human behavior based on a monitoring system. The Mean-shift target tracking algorithm is used to track the denoised foreground image, and the background difference method is used to extract moving targets. A standard behavior database is established to conduct Abnormal judgment.

The patent application number 201710441026.3 proposes a crowd abnormal behavior detection and analysis system based on a deep convolutional neural network. The human body object is extracted through the deep convolutional neural network, and then the optical flow method is used to judge the human body movement state, and clustering and Crowd modeling, and finally identify and judge various crowd behavior abnormalities according to different combinations of crowd density, motion vector value, and duration quantitative index data.

Compared with the above invention patents, the present invention adopts the human body abnormal behavior detection method combining Bayesian classifier and convolutional neural network, which can effectively improve the recognition accuracy, and can detect walking, running, waving of people in a fixed single background. The four behaviors of punching and falling to the ground are detected.

Contents of the invention

Aiming at the problem of human body recognition application, the present invention proposes a human body abnormal behavior detection method, which can detect the four behaviors of walking, running, punching and falling to the ground under a fixed single background.

The technical solution of the present invention is: a method for detecting abnormal human behavior, specifically comprising the following steps:

1) Collect video sequences of the four behaviors of people walking, running, punching, and falling to the ground under a fixed single background, use the Kalman filter algorithm to extract moving human targets, extract moving human moving target images, and convert the detected moving human The target image is stored and the training data set is constructed;

2) Utilize the image in the training data set to train the Bayesian classifier and the convolutional neural network respectively, and obtain the trained Bayesian classifier and the convolutional neural network;

3) For the video image sequence collected in real time on site, Kalman filtering is performed on each frame of image to extract the moving human body target, and then the extracted moving human body image is input into the trained Bayesian classifier and the trained convolution Neural network, for each extracted picture, the Bayesian classifier abnormal behavior classification and recognition and the convolutional neural network abnormal behavior classification and recognition are respectively obtained to obtain test results; when abnormal behavior is detected, the two test results are analyzed Comprehensive judgment, when the two classification recognition results are consistent, the recognition result is directly output; when one classification recognition detects an abnormality, and the other classification recognition detects no abnormality, an early warning of "possibly abnormal" is given, and the next detection is continued. Frame image; when the two types of classification and recognition detect abnormalities, but the detection categories are not the same, an early warning of "abnormality exists, the category is uncertain" is given, and the next frame of image detection is continued.

The establishment process of the Bayesian classifier in the step 2) is as follows: first, for the images in the training data set, according to the four abnormal behavior categories of walking, running, falling to the ground and punching, extract the aspect ratio and image entropy of the image respectively According to the three features of Hu invariant moments and four abnormal behavior image feature values, a Bayesian classifier is established, and the Bayesian formula is used to realize the classification and recognition of abnormal behaviors.

The beneficial effect of the present invention is that: the method for detecting abnormal human behavior of the present invention extracts the Hu invariant moment, image entropy and aspect ratio features of the image to establish a Bayesian classifier to realize the classification and identification of four abnormal human behaviors, and identify The rate and real-time performance are better. The combination of Bayesian classifier and convolutional neural network method is used to determine the final abnormal behavior category, which can improve the recognition accuracy and reduce misjudgment and missed detection. The invention can realize the detection and classification recognition of the four behaviors of walking, running, punching, and falling to the ground under a fixed background. This method is not limited to these four behaviors, and can also be used for crowd gathering, fighting, etc. Judgment of other abnormal behaviors.

Description of drawings

Fig. 1 is the flow chart of Kalman filtering algorithm extracting moving human target among the present invention;

Fig. 2a is the detection result of human walking in the present invention;

Fig. 2b is the detection result of human running in the present invention;

Fig. 2c is the detection result of punching by the human body in the present invention;

Fig. 2d is the detection result of the human body falling to the ground in the present invention;

Fig. 3 is a flowchart of establishing Bayesian classifier among the present invention;

Fig. 4 is a flow chart of convolutional neural network training in the present invention;

Fig. 5 is a flow chart of detection, classification and recognition of abnormal behavior in the present invention.

Detailed ways

The specific implementation mode of the present invention is as follows: first collect the video sequence of these four kinds of behaviors of people walking, running, throwing a fist, and falling to the ground under a fixed single background, use the Kalman filter algorithm to extract the moving human body target, and extract the moving human body moving target image After that, the judgment of the abnormal human behavior category is carried out. The abnormal behavior category judgment part is divided into two parts: training and testing. The training part includes building a training data set, building a Bayesian classifier using the images in the training data set, and training a convolutional neural network. The training part is done offline before the system is used. After the training is completed, it enters the testing stage. For the video image sequence collected in real time, Kalman filtering is performed on each frame of image to extract the moving target, and then the extracted moving human body images are classified and identified by the Bayesian classifier for abnormal behavior. Convolutional neural network abnormal behavior classification and recognition. Then, the classification and recognition results of the two kinds of network abnormal behaviors are comprehensively judged, and the final judgment result of the abnormal behavior of the moving human body is given.

As shown in Figure 1, it is a flow chart of the Kalman filter algorithm to extract moving human targets, including the following steps:

1. Video image acquisition: through a fixed camera to collect videos of four behaviors of people walking, running, punching, and falling to the ground under a fixed and simple background, and input the video image sequence to the PC;

2. Read in a frame of image, perform background estimation on the read image, and generate an initial background image;

3. Read in the next frame image, and use the Kalman filter algorithm (see the invention patent 201010177661.3 for Kalman filter algorithm details), according to the above initial background image and the current frame image, to obtain the foreground moving target area in the current frame image;

4. Calculate the connected domain of the foreground moving target area, filter out the interference area whose area is smaller than the set threshold, and use the area larger than the threshold as the moving human body target area;

5. Use the smallest rectangular frame to mark the moving human body target area, and detect abnormal behavior in the image of this area.

Figure 2a, Figure 2b, Figure 2c, and Figure 2d show the extraction results of pedestrians walking, running, punching, and falling down, respectively. The first picture in the picture is the original picture, the second picture is the moving area selected by the rectangle box, and the third picture is the extracted moving target image.

After extracting the image of the moving human target, the abnormal behavior category of the human body is determined. Abnormal behavior category determination is divided into two parts: training and testing. Before the abnormal behavior detection system is used, the Bayesian classifier and the convolutional neural network are trained based on the training data set. These two processes can be completed offline respectively.

As shown in Figure 3, it is a flow chart of establishing the Bayesian classifier for abnormal behavior detection. The Bayesian classifier has the following advantages: the algorithm logic is simple and easy to implement; the time and space overhead for algorithm implementation is small, and it has high computing speed and recognition accuracy when training large amounts of data; the algorithm performance is stable. Using image aspect ratio, image entropy and Hu invariant moment features to distinguish various abnormal behavior images, it has the advantages of accurate recognition, simple calculation and good real-time performance.

The establishment process of the Bayesian classifier is as follows: firstly, for the images in the training data set, according to the abnormal behavior categories such as walking, running, falling to the ground, and punching, the three aspects of image aspect ratio, image entropy, and Hu invariant moments are respectively extracted. Features, according to the conditional probability distribution of four abnormal behavior image feature values, a Bayesian classifier is established, and the Bayesian formula is used to realize the classification and identification of abnormal behaviors.

As shown in Figure 3, the establishment of a Bayesian classifier for abnormal behavior detection specifically includes the following steps:

1. Construct the training data set. The video used to construct the training data set can be the video collected in real time on site, and it is required that the video should contain human actions for abnormal behavior detection and classification, such as falling to the ground, running, punching, etc. The present invention adopts the public KTH action recognition data set, which includes 600 video clips, each video clip contains about 120 frames, and the image size of each frame is 160×120 pixels. The image background in this dataset video is fixed, and each frame contains a human target. The present invention constructs a training data set by using video clips of four abnormal behaviors of the human body in the data set: walking, punching, running and falling to the ground.

The steps of constructing the training data set are as follows: For the video image sequence, the Kalman filter moving target detection is performed frame by frame, and the moving human target is extracted. The Kalman filtering method is the same as before, and the processing flow is shown in Figure 1. The detected images of moving human targets are stored to construct the training data set of the present invention. Since the video contains abnormal behaviors such as walking, punching, running, and falling to the ground, the constructed training data set contains images of the above four abnormal behaviors.

2. Feature value extraction

The aspect ratio, image entropy and Hu invariant moments of the four abnormal behavior images in the data set were extracted respectively. The specific steps are as follows:

(1) Extraction of image aspect ratio

As shown in Figures 2a, 2b, 2c, and 2d, the segmented target is marked with the smallest rectangular frame, and the aspect ratio of the smallest rectangular frame is calculated as the feature value of the aspect ratio of the target image.

(2) Extraction of image entropy

Select the neighborhood gray value of the image as the spatial feature quantity of the gray distribution, and form a feature pair with the pixel gray value of the image, denoted as (i, j), where i represents the gray value of the pixel (0<= i<=255), j represents the average gray value of the image neighborhood (0<=j<=255), and f(i, j) is the frequency of occurrence of the feature pair (i, j). The joint probability P _ij is:

P _ij =f(i,j)/MN

Indicates the proportion of the feature pair (i, j) in the image, where MN is the total number of pixels in the image. The two-dimensional entropy of the image is:

(3) Extraction of image Hu invariant moments

The present invention uses the Hu5 invariant moment of the image as the feature value. The two-dimensional (p+q) moment of the image G(x, y) is defined as:

Among them, G(x, y) represents the gray value of the image in the space (x, y), and x, y represents the coordinates of the image pixel.

The corresponding center distance is defined as:

in,

The normalized (p+q) order central moment is defined as:

The Hu5 invariant moment φ5 of an image is defined as:

φ ₅ ＝(η ₃₀ -3η ₁ 2)(η ₃₀ +η ₁₂ )[(η ₃₀ +η ₁₂ ) ² -3(η ₂₁ +η ₀₃ ) ² ]+(3η ₂₁ -η ₀₃ )(η ₂₁ + η ₀₃ )[3(η ₃₀ +η ₁₂ ) ² -(η ₂₁ +η ₀₃ ) ² ]

The Hu5 invariant moment eigenvalue of the image can be calculated by the above formula.

3. Establish Bayesian classifier based on eigenvalues

The specific steps are as follows: (see the invention patent 201510981972.8 for the establishment method of the Bayesian classifier):

(1) For the three eigenvalues of image aspect ratio, image entropy and Hu invariant moments extracted above, calculate the mean and variance of the eigenvalues of the four abnormal behavior images respectively, and obtain the conditional probability distribution of the eigenvalues. See Table 1 for the image feature extraction results in the data set used in the present invention. Different abnormal behaviors have different aspect ratio value ranges, among which the value range of the aspect ratio of falling to the ground is 0.7-1.3; the value range of the aspect ratio of normal walking is 0.3-0.5; the value range of the aspect ratio of running is 0.28-0.45 ; The value range of the aspect ratio of punching is 0.4~0.55. Calculate the mean c and variance σ ² , and obtain the Gaussian probability distributions corresponding to the three eigenvalues of the four behaviors of walking, running, punching, and falling to the ground, which are the class conditional probability density functions:

Where x is the eigenvalue, c is the mean value of the eigenvalue, and ^σ2 is the variance of the eigenvalue.

For example, according to the data in Table 1, the function to obtain the probability distribution of Hu5 invariant moment eigenvalues of images of pedestrians walking normally is:

Table 1

(2) Establish a naive Bayesian classifier

Let x={a ₁ , a ₂ , a ₃ } be an item to be classified, and each a is a feature attribute of x, where a ₁ represents the aspect ratio of the image, a ₂ represents the image entropy, and a ₃ represents the image Hu invariant moment.

The set of abnormal behavior categories is C={y ₁ , y ₂ , y ₃ , y ₄ }. Among them, y ₁ means walking, y ₂ means running, y ₃ means punching, and y ₄ means falling to the ground.

Calculate the posterior probability according to the Bayesian formula The Bayesian formula is:

Among them, i takes 1, 2, 3, 4; P(X|y ₁ ), P(X|y ₂ ), P(X|y ₃ ), P(X|y ₄ ) represent walking, running and punching respectively , Class-conditional probability densities of the four behaviors of falling to the ground.

In the formula, P _(x) is the prior probability of each category and is a constant. Comparing Posterior Probabilities for Each Category the size of the comparison

Since each feature attribute is conditionally independent, there are:

if Then x∈y _k , k=1, 2, 3, 4.

Bayesian classifier can be transformed into:

For a test sample feature value X, calculate which behavior corresponds to the highest probability, that is, which type of behavior it belongs to.

For abnormal behavior detection and recognition, there will be missed detection and misjudgment if one classifier is used. Therefore, the present invention uses a method of combining two classification and discrimination methods to reduce abnormal behavior detection and classification errors. The second discriminative method employed is a convolutional neural network. The use of convolutional neural network for abnormal behavior detection has the advantage of simplicity and ease of operation due to the reduction of feature extraction links.

As shown in Table 2, it is the parameters for building a convolutional neural network for abnormal behavior detection. The invention adopts the VGG-16 convolutional neural network model to realize abnormal behavior detection. The configuration of the computer environment used by the present invention to establish the convolutional neural network is: the computer used is a PC with 2.5GHZ*4 CPU, 8GB memory, and windows10 64-bit operating system. First install the anaconda environment, then Pip install tensorflow, and finally install PyCharm, that is, install the required packages to run the convolutional neural network.

Table 2

The specific method of setting up the convolutional neural network is as follows: as shown in Table 2, the present invention selects 9 layers of convolutional neural network structures, which are respectively convolutional layer-pooling layer-convolutional layer-pooling layer-convolutional layer-pool layer-convolution layer-fully connected layer-output layer. The input image is set to a size of 120*120, and Layer0 is a convolutional layer, which is composed of 50 Feature Map feature maps, and the neurons in each Feature Map are connected to a 5*5 neighborhood. The Feature Map size is set to 116*116 to prevent the input connection from running outside the boundary.

Layer1 is downsampling. There are 50 Feature Maps of 58*58. The neighborhood connection is 2*2. The size of each FeatureMap is 1/4 of Layer0 (1/2 for each row and column).

Layer2 is also a convolutional layer with a convolution kernel of 5*5. Each feature map in this layer is connected to all 50 or other feature maps in the previous layer, representing different combinations of Feature Maps.

Layer3 is a downsampling layer consisting of 100 feature maps of 32*32 size.

Layer4 is a convolutional layer, which convolves the previous layer through a 3*3 convolutional kernel to obtain 30*30 neurons.

Layer5 is a downsampling layer consisting of 200 Feature Maps of size 15*15. Select 2*2 for the neighborhood connection.

Layer6 is a convolutional layer with 300 Feature Maps.

Layer7 is 100 units and computes the dot product between the input vector and the weight vector, plus a bias.

Layer8 is the output layer. The output layer is composed of Euclidean Radial Basis Function (Euclidean Radial BasisFunction) units, one unit for each type, each with 100 inputs, and finally an output. For the input sample data, softmax is used to classify, and the probability that the sample image belongs to each category of abnormal behavior is output. Whichever probability is the largest, the image is judged as the category with the highest probability.

After the convolutional neural network is established, it is necessary to use the images in the training data set to train the network, and then it can correctly classify and identify a newly input test image.

As shown in Figure 4, it is the training flow chart of abnormal behavior detection and classification convolutional neural network. All the images in the training data set constructed by Kalman filtering are normalized to the appropriate size of the convolutional neural network, and then input to the convolutional neural network established according to the above parameters to train the network.

The training process is the process of allowing the network to automatically find the image representation characteristics, the key is to use forward propagation and back propagation. The forward propagation performs feature representation on the input image, and then classifies it. Backpropagation updates the network weights and bias values according to the method of error reduction. Specific steps are as follows:

1. First select a part of the training data from the training data set;

2 Obtain the predicted value of the network through forward propagation. The input sample information is calculated layer by layer, and the output result is converted into the number of classes expected by the network. The parameters of the training model enable the network model to better represent the features. The calculation process performed by the network is actually the point multiplication operation of the input information and the small weight matrix of each layer, plus the number of bias item parameters, which is The final output.

3. Through backpropagation, update the network parameter variables. The error of the input layer of the network is backpropagated to the previous layer in turn, and the weight of the convolution layer and the gradient of the deviation are calculated according to the backpropagation of the error, and the model parameters are updated. The weight matrix is adjusted by the direction propagation of the minimization error method.

Initialize the convolutional layer and pooling layer of the convolutional neural network model, and obtain the loss error by forward propagating the expected parameter values of the training model, and cache the result in the network, and then pass the error to the In the previous pooling layer and convolutional layer, the sum of the gradients of the training model is calculated, and the weights and biases are updated. When the sample reaches the number of training times or reaches the training accuracy or the number of training times, the training process ends, otherwise continue the above process.

After the training phase, enter the testing phase. The training phase is generally done offline before the abnormal behavior detection system is used. Detecting and judging abnormal behaviors on site belongs to the testing phase.

As shown in Figure 5, after the Bayesian classifier is established and the convolutional neural network is trained, abnormal behaviors can be detected and classified for the video image sequences collected in real time on site. Specific steps are as follows:

1. Read in a frame of video image.

2. Kalman filter moving target extraction. Carry out Kalman filter moving target extraction on each frame of image (the method is the same as before). If no moving target is detected, continue to read the next frame of image. If a moving target is detected in the image, the extracted moving human body image is subjected to Bayesian classifier and convolutional neural network abnormal behavior category discrimination respectively.

3. Bayesian classifier discrimination. For the extracted moving human body image, first extract the aspect ratio, image entropy and Hu invariant moment features of the image respectively, and then input these feature values into the established Bayesian classifier above, and perform anomaly analysis according to the Bayesian formula Behavior category determination.

4. Convolutional neural network category determination. Normalize the extracted moving human body image to the appropriate size of the convolutional neural network, and then input it to the convolutional neural network, output the probability that the image belongs to the four abnormal behavior categories, and use the highest probability as the abnormal behavior of the image category. For example, if a certain frame of image is input to the network, the output result is: prediction result: running 0.9546, falling to the ground 0.0030, walking 0.0113, punching 0.03097, then the judgment result of the image is running.

5. Comprehensive judgment. The two discriminant results of Bayesian classifier and convolutional neural network are combined to give the final discriminant result.

The situation table of the comprehensive judgment is shown in Table 3:

(1) When no abnormality is detected by the two types of classification and identification, continue to detect the next frame.

(2) When one type of classification recognition detects an abnormality and the other type of classification recognition detects no abnormality, an early warning of "possible abnormality" is given, and the next frame of image detection is continued.

(3) When anomalies are detected in both classifications and recognitions, but the detection categories are different, an early warning of "abnormality exists, category is uncertain" is given. Continue to detect the next frame of image.

(4) When both classifications and identifications detect anomalies, and the categories of the detected anomalies are the same, for example, when both are "fallen down", an early warning of "there is an anomaly and the category is fallen down" is given. Continue to detect the next frame of image.

table 3

(Note: A and B in the table are any of the above abnormal behaviors)

Two classifiers are used for comprehensive judgment. The recognition accuracy rate of each classifier is difficult to achieve 100% accuracy, and there are cases of misjudgment and missed detection. The comprehensive judgment of the two classifiers increases the accuracy of recognition, can avoid missed detection, and do it early An early warning of possible unusual behavior. When one classifier makes a wrong judgment on the abnormal behavior category, the other classifier can avoid individual recognition errors. Only when the two classifiers have the same discrimination results, can the judgment of the abnormal behavior category be given, which increases the credibility. . Since the camera video capture is generally about 25 frames per second, the next frame is continuously recognized, which does not affect the detection of abnormal behavior in real time.

Claims (2)

1. A method for detecting abnormal behaviors of a human body is characterized by comprising the following steps:

1) collecting video sequences of four behaviors of walking, running, punching and falling of a person under a fixed single background, extracting a moving human body target by using a Kalman filtering algorithm, extracting a moving human body moving target image, storing the detected moving human body target image, and constructing a training data set;

2) respectively training a Bayes classifier and a convolutional neural network by using images in the training data set to obtain a trained Bayes classifier and a convolutional neural network;

3) performing Kalman filtering on a video image sequence acquired in real time on site, extracting a moving human body target from each frame of image, then respectively inputting the extracted moving human body image into a trained Bayes classifier and a trained convolutional neural network, and respectively performing Bayes classifier abnormal behavior classification identification and convolutional neural network abnormal behavior classification identification on each extracted image to respectively obtain test results; when abnormal behaviors are detected, comprehensively judging the two test results, and directly outputting the identification result when the two classification identification results are consistent; when one classification identification detects that there is abnormality and the other classification identification detects that there is no abnormality, giving an early warning of possible abnormality and continuously detecting the next frame of image; and when the two kinds of classification identification are detected to have abnormity but the detection types are different, giving an early warning of abnormity and uncertain type, and continuously detecting the next frame of image.

2. The method for detecting abnormal human behavior according to claim 1, wherein the Bayesian classifier in the step 2) is established as follows: firstly, extracting three characteristics of an aspect ratio, an image entropy and a Hu invariant moment of an image according to four abnormal behavior categories of walking, running, falling and punching on an image in a training data set, establishing a Bayes classifier according to the conditional probability distribution of the image characteristic values of the four abnormal behaviors, and realizing classification and identification of the abnormal behaviors by using a Bayes formula.