CN108537286A - A kind of accurate recognition methods of complex target based on key area detection - Google Patents

Abstract

The present invention relates to a method for accurately identifying complex targets based on key region detection, including: using cross-training methods to perform fusion training on the entire neural network, using convolutional neural networks to extract target features, and using detection sub-networks with anchor boxes as references Detect the key areas of complex targets, use regional standard pooling to pool the key areas into fixed-size feature maps, use the classification sub-network to classify the key areas, and fuse the classification results of each key area to achieve accurate recognition of the target. The entire network includes a key area detection sub-network and a key area classification sub-network. The detection sub-network detects key areas with a degree of discrimination for complex targets, and then the classification sub-network classifies the key areas. target identification. These two sub-networks share the features extracted by the VGG convolutional neural network, so that the recognition of complex targets can be achieved quickly and accurately.

Description

A method for accurate recognition of complex targets based on key region detection

technical field

The invention relates to image processing technology, in particular to a method for accurately recognizing complex targets based on key region detection.

Background technique

The classification and recognition of complex objects is an important and basic task in the field of computer vision. Most of the parts of different types of complex targets are the same or similar, but the differences are often reflected in some key local areas. Therefore, there are a lot of interference and redundant information in the images of complex targets. However, some existing classification and recognition methods for complex targets have the problem of low accuracy because they cannot remove the interference and redundant information in complex target images. In order to achieve accurate classification and recognition of complex targets, it is of great significance to study a method for precise recognition of complex targets based on key region detection.

Contents of the invention

In view of this, the main purpose of the present invention is to provide an accurate recognition method for complex targets based on key area detection with high recognition accuracy, which ensures rapid recognition while greatly improving detection accuracy.

In order to achieve the above purpose, the technical solution proposed by the present invention is: a method for accurately identifying complex targets based on key area detection, and the implementation steps are as follows:

Step 1. Read the images of complex targets in the training samples in the database, the coordinate labels of the key areas of complex targets, and the classification labels of complex targets, and use the cross-training method to perform fusion training on the complex target accurate recognition network.

In step 2, the complex target picture to be recognized is used as the input of the complex target accurate recognition network trained in step 1, and features are extracted through the VGG convolutional neural network to obtain the feature map of the complex target picture to be recognized.

Step 3, input the feature map obtained in step 2 into the key region detection subnetwork, slide the feature map with a 3×3 subnetwork, and use the anchor box as a reference to detect the key region of the complex target image, and give The prediction box of the key area and the possibility of whether it is a key area P _is , P _not ;

Step 4: Use non-maximum suppression to filter the detected areas with a high degree of overlap. When the ratio of the area of the intersection of different prediction boxes to the area of the union exceeds the specified threshold IOU_threshold, only the possibility of the key area is retained P _is the largest prediction box, while filtering the other boxes;

Step 5, setting the threshold P_threshold of the possibility P _is of the key area, and mapping the area where the possibility P is of the key area _is greater than the set threshold P_threshold to the feature map extracted by the VGG network;

Step 6, perform regional standard pooling on the region mapped to the feature map obtained in step 5, and pool the detected regions of different sizes into a fixed-size feature map;

Step 7. Use the fixed-size feature map obtained in step 6 as the input of the classification sub-network, use the classification sub-network to classify it accurately, use the softmax function to normalize the classification results, and obtain the probability of classifying the key regions;

In step 8, for a complex target corresponding to the same picture, the mean value of the corresponding probabilities of the classifications of each key area obtained in step 7 is fused to obtain an accurate recognition result of the complex target type.

In the step 1, the whole network cross-training process is as follows:

Step 11, using the ImageNet database picture as the training sample and the weight of the VGG network trained for the classification task as the initial weight, and fine-tuning on this basis;

Step 12, read the complex target picture and the coordinate label of the key area corresponding to the complex target picture, and train the key area detection sub-network, the training loss function is loss=L _P +L _reg , where L _P is the key area detector Whether the network output is the probability of the key area P _is , P _not and the cross entropy of the true value of the label, L _reg is the square of the coordinate offset of the detection area output by the key area detection sub-network and the actual key area coordinate offset in the label and;

Step 13, read the complex target picture and the classification label corresponding to the complex target picture, and train the classification sub-network, the training loss function is the cross entropy between the network output classification result and the actual label result;

Step 14 Repeat step 12 and step 13 several times to perform cross-training on the key region detection sub-network and classification sub-network until the network is stable.

In the step 3, the method of key region detection is as follows:

Step 31, using a sliding window with a size of 3×3, sliding on the feature map obtained in step 2, and obtaining a 512-dimensional vector at each position;

Step 32, set 9 anchor boxes at the position of each sliding window as a reference, the aspect ratio of the anchor box is set to three ratios according to 1:2, 1:1, and 2:1, and the area size is set to 128 ² , 256 ² , 512 ² pixels in three sizes, the center point of the anchor box is the center of the sliding window;

Step 33, output the 512-dimensional vector obtained by each sliding window position above through the fully connected network to output nine 6-dimensional vectors; each vector represents the center point coordinates, length and width of the detection area relative to a reference anchor box Whether the offset d _x , d _y , d _l , d _w is the possibility of the key region P _is , P _not , where: d _x =(xx _a )/l _a , d _y =(yy _a )/w _a , d _l =log(l/l _a ), d _w =log(w/w _a ), x,y,l,w represent the coordinates, length and width of the center point of the detected area, x _a ,y _a ,l _a , w _a represent the coordinates, length and width of the center point of the reference anchor area, P _is , P _not use the softmax function for normalization;

Step 34 Calculate the detection area according to the offsets d _x , d _y , d _l , d _w obtained from the network regression, and the center point coordinates, length and width x _a , y _a , l _a , w _a of the anchor box The actual center point coordinates, length and width x, y, l, w.

In the step 6, the method of regional standard pooling is as follows:

Step 61, express the size of the area to be pooled as m×n, divide the area to be pooled into 7×7 small grids with a size of about m/7×n/7, when m/7 or n/ 7 If it cannot be rounded to an integer, it will be rounded to an approximate integer;

Step 62, in each small grid divided in step 61, use the method of maximum pooling to pool the features in the small grid into 1×1 dimension, so that the feature regions of different sizes are pooled into 7× 7-dimensional fixed-size feature maps.

To sum up, the method for accurate identification of complex targets based on key area detection described in the present invention includes: using cross-training methods to perform fusion training on the entire neural network, using convolutional neural networks to extract target features, and using detectors The network uses the anchor box as a reference to detect the key areas of complex targets, uses regional standard pooling to pool the key areas into feature maps of fixed size, uses the classification sub-network to classify the key areas, and fuses the classification results of each key area to achieve Accurate identification of the target. The entire network includes a key area detection sub-network and a key area classification sub-network. The detection sub-network detects key areas with a degree of discrimination for complex targets, and then the classification sub-network classifies the key areas. target identification. These two sub-networks share the features extracted by the VGG convolutional neural network, so that the recognition of complex targets can be achieved quickly and accurately.

The advantage of the present invention compared with prior art is:

(1) Accuracy: Many different complex targets are often similar in most places, but their differences are often reflected in local key areas. The traditional object recognition method takes the whole picture as the input of the classification network, and the whole picture contains a lot of redundant information and interference information, which limits the accuracy of target recognition. In this method, the detection sub-network is used to first detect the key area, and then the classification sub-network is used to identify the key area, and the identification results of each part of the key area are fused to achieve the effect of accurate target identification.

(2) Rapidity: The present invention uses a deep neural network to extract the features of the original image, and the detection sub-network and the classification sub-network share the features extracted by the same neural network. During the training process, the whole network is trained by cross-training method. During the test, the detection sub-network and the classification sub-network share the features extracted by the same neural network, which greatly reduces the amount of network parameters and calculations, and can achieve fast target recognition results.

Description of drawings

Fig. 1 is a schematic diagram of the implementation flow of the present invention.

Detailed ways

In order to make the purpose, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments.

A method for accurately identifying complex targets based on key region detection according to the present invention includes: using cross-training methods to perform fusion training on the entire neural network, using convolutional neural networks to extract target features, and using detection sub-networks to anchor boxes As a reference to detect key areas of complex targets, use regional standard pooling to pool key areas into fixed-size feature maps, use classification subnetworks to classify key areas, and fuse classification results of each key area to achieve accurate identification of targets . The entire network includes a key area detection sub-network and a key area classification sub-network. The detection sub-network detects key areas with a degree of discrimination for complex targets, and then the classification sub-network classifies the key areas. target identification. These two sub-networks share the features extracted by the VGG convolutional neural network, so that the recognition of complex targets can be achieved quickly and accurately.

As shown in Figure 1, the present invention specifically realizes the following steps:

Step 1, read the complex target picture in the training sample in the database, the coordinate label of the key area corresponding to the complex target picture, and the classification label corresponding to the complex target picture, and use the cross-training method to perform fusion training on the complex target accurate recognition network;

Step 2, using the complex target picture to be recognized as the input of the complex target accurate recognition network after step 1 training, extracting features through the VGG convolutional neural network, and obtaining the feature map of the complex target picture to be recognized;

Step 3, input the feature map obtained in step 2 into the key region detection subnetwork, slide the feature map with a 3×3 subnetwork, and use the anchor box as a reference to detect the key region of the complex target image, and give The prediction box of the key area and the possibility of whether it is a key area P _is , P _not ;

Step 4: Use non-maximum suppression to filter the detected areas with a high degree of overlap. When the ratio of the area of the intersection of different prediction boxes to the area of the union exceeds the specified threshold IOU_threshold, only the possibility of the key area is retained P _is the largest prediction box, while filtering the other boxes;

Step 5, setting the threshold P_threshold of the possibility P _is of the key area, and mapping the area where the possibility P is of the key area _is greater than the set threshold P_threshold to the feature map extracted by the VGG network;

Step 6, perform regional standard pooling on the region mapped to the feature map obtained in step 5, and pool the detected regions of different sizes into a fixed-size feature map;

Step 7. Use the fixed-size feature map obtained in step 6 as the input of the classification sub-network, use the classification sub-network to classify it accurately, use the softmax function to normalize the classification results, and obtain the probability of classifying the key regions;

In step 8, for a complex target corresponding to the same picture, the mean value of the corresponding probabilities of the classifications of each key area obtained in step 7 is fused to obtain an accurate recognition result of the complex target type.

In the step 1, the whole network cross-training process is as follows:

Step 11, using the ImageNet database picture as the training sample and the weight of the VGG network trained for the classification task as the initial weight, and fine-tuning on this basis;

Step 12, read the complex target picture and the coordinate label of the key area corresponding to the complex target picture, and train the key area detection sub-network, the training loss function is loss=L _P +L _reg , where L _P is the key area detector Whether the network output is the probability of the key area P _is , P _not and the cross entropy of the true value of the label, L _reg is the square of the coordinate offset of the detection area output by the key area detection sub-network and the actual key area coordinate offset in the label and;

Step 13, read the complex target picture and the classification label corresponding to the complex target picture, and train the classification sub-network, the training loss function is the cross entropy between the network output classification result and the actual label result;

Step 14 Repeat step 12 and step 13 several times to perform cross-training on the key region detection sub-network and classification sub-network until the network is stable.

In the step 3, the method of key region detection is as follows:

Step 31, using a sliding window with a size of 3×3, sliding on the feature map obtained in step 2, and obtaining a 512-dimensional vector at each position;

Step 32, set 9 anchor boxes at the position of each sliding window as a reference, the aspect ratio of the anchor box is set to three ratios according to 1:2, 1:1, and 2:1, and the area size is set to 128 ² , 256 ² , 512 ² pixels in three sizes, the center point of the anchor box is the center of the sliding window;

Step 33, output the 512-dimensional vector obtained by each sliding window position above through the fully connected network to output nine 6-dimensional vectors; each vector represents the center point coordinates, length and width of the detection area relative to a reference anchor box Whether the offset d _x , d _y , d _l , d _w is the possibility of the key region P _is , P _not , where: d _x =(xx _a )/l _a , d _y =(yy _a )/w _a , d _l =log(l/l _a ), d _w =log(w/w _a ), x,y,l,w represent the coordinates, length and width of the center point of the detected area, x _a ,y _a ,l _a , w _a represent the coordinates, length and width of the center point of the reference anchor area, P _is , P _not use the softmax function for normalization;

Step 34 Calculate the detection area according to the offsets d _x , d _y , d _l , d _w obtained from the network regression, and the center point coordinates, length and width x _a , y _a , l _a , w _a of the anchor box The actual center point coordinates, length and width x, y, l, w.

In the step 6, the process of regional standard pooling is as follows:

Step 61, express the size of the area to be pooled as m×n, divide the area to be pooled into 7×7 small grids with a size of about m/7×n/7, when m/7 or n/ 7 If it cannot be rounded to an integer, it will be rounded to an approximate integer;

Step 62, in each small grid divided in step 61, use the method of maximum pooling to pool the features in the small grid into 1×1 dimension, so that the feature regions of different sizes are pooled into 7× 7-dimensional fixed-size feature maps.

To sum up, the above are only preferred embodiments of the present invention, and are not intended to limit the protection scope of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims (4)

1. a kind of accurate recognition methods of complex target based on key area detection, which is characterized in that include the following steps：

Step 1, the seat of the complex target picture in reading database in training sample, the corresponding key area of complex target picture Label and the corresponding tag along sort of complex target picture are marked, net is precisely identified to complex target using the method for cross-training Network carries out Fusion training；

Step 2, the defeated of network is precisely identified using complex target picture to be identified as the complex target of step 1 after training Enter, extracts feature by VGG convolutional neural networks, obtain the characteristic pattern of complex target picture to be identified；

Step 3, the characteristic pattern that step 2 obtains is input in key area detection sub-network network, is existed with the sub-network of 3 × 3 sizes It is slided on characteristic pattern, as reference with anchor box, the key area of detection of complex Target Photo provides the pre- of key area Survey box and whether be key area possibility P_is,P_not；

Step 4, the higher region of the degree of overlapping detected is filtered using non-maximum suppression, when different prediction box intersections When the ratio of area and union part area is more than defined threshold value IOU_threshold, then it is key area only to retain Possibility P_isMaximum prediction box, and other boxes are filtered；

Step 5, setting is key area possibility P_isThreshold value P_threshold, will be key area possibility P_isMore than setting Determine on the characteristic pattern that the area maps of threshold value P_threshold are extracted to VGG networks；

Step 6, the region being mapped on characteristic pattern that step 5 obtains is subjected to regional standard pond, the difference detected is big Small pool area turns to the characteristic pattern of fixed size；

Step 7, the fixed size for obtaining step 6 characteristic pattern as classification sub-network input, use classification sub-network pair It is accurately classified, and is normalized using softmax function pair classification results, obtains the probability classified to key area；

Step 8, for the corresponding complex target of same pictures, the classification for each key area that step 7 is obtained Corresponding probability takes mean value to be merged, and obtains the accurate recognition result of complex target type.

2. a kind of accurate recognition methods of complex target based on key area detection according to claim 1, feature exist In：In the step 1, cross-training process is as follows：

Step 11, the weights of the VGG networks using ImageNet databases picture as training sample, for classification task training are used As initial weight, it is finely adjusted on this basis；

Step 12, the coordinate label for reading complex target picture and the corresponding key area of complex target picture, to key area Detection sub-network network is trained, and trained loss function is loss=L_P+L_reg, wherein L_PIt is exported for key area detection sub-network network Whether be key area probability P_is,P_notWith the cross entropy of label actual value, L_regIt is exported for key area detection sub-network network Detection zone coordinate shift amount and label in practical key area coordinate shift amount quadratic sum；

Step 13, complex target picture and the corresponding tag along sort of complex target picture are read, classification sub-network is trained, Cross entropy of the trained loss function between network output category result and physical tags result；

Step 14 repeats step 12 and step 13 several times, and intersection instruction is carried out to key area detection sub-network network and classification sub-network Practice, until network stabilization.

3. a kind of accurate recognition methods of complex target based on key area detection according to claim 1, feature exist In：The step 3 specifically includes：

Step 31, the sliding window for a use of size being 3 × 3, is slided on the characteristic pattern that step 2 obtains, each Position obtains the vector of one 512 dimension；

Step 32,9 anchor boxes are set as reference in the position of each sliding window, anchor box length-width ratio presses 1:2、1:1、 2:1 is set as three kinds of ratios, and size is set as 128²、256²、512²The central point of three kinds of sizes of pixel, anchor box is institute At the center of sliding window；

Step 33,512 dimensional vectors above-mentioned each sliding window position obtained by fully-connected network export 96 dimension to Amount；Each vector indicates to refer to anchor box, center point coordinate, length and the wide offset d of detection zone relative to one_x,d_y, d_l,d_wWhether it is key area possibility P_is,P_not, wherein：d_x=(x-x_a)/l_a, d_y=(y-y_a)/w_a, d_l=log (l/ l_a), d_w=log (w/w_a), x, y, l, w indicate the regional center point coordinates, length and the width that detect, x_a,y_a,l_a,w_aIndicate reference Anchoring area domain center point coordinate, length and width, P_is,P_notIt is normalized using softmax functions；

The offset d that step 34 is obtained according to net regression_x,d_y,d_l,d_w, center point coordinate, length and width x with anchor box_a,y_a, l_a,w_a, calculate the actual center point coordinate of detection zone, length and width x, y, l, w.

4. a kind of accurate recognition methods of complex target based on key area detection according to claim 1, feature exist In：In the step 6, the process in regional standard pond is as follows：

Step 61, waiting for that the area size in pond is expressed as m × n, the region division in pond will be waited for into 7 × 7, size is about m/ The sub-box of 7 × n/7, when m/7 or n/7 can not rounding when, then according to the approximate rounding that rounds up；

Step 62, in each sub-box that step 61 divides, using the method in maximum pond, by the feature in sub-box Pond turns to 1 × 1 dimension, in this way, different size of characteristic area pond to be turned to the characteristic pattern of 7 × 7 dimension fixed sizes.