CN113536896A - Small target detection method, device and storage medium based on improved Faser RCNN - Google Patents

Abstract

The invention relates to a small target detection method based on the improved Faser RCNN, which is realized by executing an improved Faser RCNN algorithm instruction by a processor, including: receiving a scene picture containing a small target and extracting a first feature map F; according to the first feature Figure F ₁ obtains a prediction anchor point frame a(x, y, w, h); according to the first feature map F ₁ , the prediction anchor point frame a(x, y, w, h) A second feature map F ₂ with the same size as the feature map F ₁ ; the detection result of the scene picture is obtained according to the second feature map F ₂ and the prediction anchor frame a(x, y, w, h). The present invention modifies the frame based on the Faser RCNN algorithm, and replaces the RPN network under it with an adaptive anchor frame network, so that the generated anchor frame can be more matched with targets of different scales, thereby avoiding the unreasonable size of the anchor frame. Resulting in missed detection, improve detection accuracy.

Description

Small target detection method, device and storage medium based on improved Faser RCNN

technical field

The invention relates to the field of target recognition and recognition, in particular to a small target detection method based on improved Faser RCNN. At the same time, the invention also relates to a small target detection device and a storage medium based on the improved Faser RCNN.

Background technique

Object detection, also called object extraction, is an image segmentation based on the geometric and statistical features of objects. It combines the segmentation and recognition of objects into one, and its accuracy and real-time performance are an important capability of the entire system.

Object detection is a popular direction in computer vision and digital image processing. It is widely used in robot navigation, intelligent video surveillance, industrial inspection, aerospace and many other fields. It is of great practical significance to reduce the consumption of human capital through computer vision. Therefore, object detection has become a research hotspot in theory and application in recent years. In recent years, the vigorous development of machine learning, especially deep learning, makes it possible to achieve low-cost and high-efficiency target detection.

At present, excellent deep learning models can be roughly divided into two categories: the first category belongs to two-stage target detection algorithms (two-stage), such as R-CN, SPP-Net, Fast-RCNN, Faster-RCNN, etc. This kind of algorithm first The target information is extracted from the regional candidate frame (RPN) of the target image, and then the detection network is used to predict the position of the target in the candidate frame and identify the category; the second category belongs to the one-stage target detection algorithm, such as SSD, YOLO, etc., such algorithms do not need to establish an RPN network, but directly perform target prediction and category recognition on the image.

However, in practical applications, the results of target detection often show that the detection effect of small targets is far inferior to that of large and medium targets. This is due to two problems in small target detection: (1) The amount of information is lacking, that is, the target occupies a very small proportion in the image, and the amount of information that can be reflected by the pixels in the corresponding area is very limited; (2) The amount of data is scarce, that is, the data set contains small The image of the target is few, which leads to the imbalance of the categories of the entire training set, resulting in the detection accuracy of small target objects is much lower than that of medium and large objects. At present, for the problem of poor small target detection effect, there are the following methods:

①Amplify the image data and enlarge the image to make the small target bigger. However, this method is simple and rude, the operation is complicated, the calculation amount is too large, and the practical significance is not strong.

②Using the GAN model to enlarge the small target and then detect it. This method is consistent with the idea of image data amplification, but it also has the disadvantage of complicated operation.

③ Modify the parameters of the model training, such as setting the parameter stride to 1, but this method has a general effect.

CN 111985540 A discloses a method for improving the detection rate of small targets based on over-sampling faster-RCNN, which relates to the field of target recognition, and includes step 1: acquiring a target picture data set, and dividing the data set into a training set and a test set; step 2: Obtain a subset of the training set as an oversampling set according to the training set in step 1; Step 3: Build a faster-RCNN model; Step 4: Use the training set and oversampling set to train the faster-RCNN model; Step 5: Use the test set to test the trained faster-RCNN model. If the test result is lower than the average accuracy AP threshold, modify the parameters, and perform step 4 again to test the training result until the test result reaches the AP threshold; Step 6: Input the image to be detected, and use the trained faster-RCNN model for small target detection. The invention improves the detection rate of the small target by over-sampling the small target.

CN 111986160 A discloses a method for improving small target detection effect based on faster-RCNN, which belongs to the field of target detection. The invention includes: acquiring a data set, and dividing it into a training set and a test set according to a corresponding proportion; constructing a faster-RCNN model; using the training set to train the model, during training, if in the nth iteration, the loss of the small target If the value reaches the preset condition, in the n+1th iteration, multiple pictures are reduced, then spliced into the original picture size, and then trained; after the training, the model is tested with the test set, and the AP is obtained. If it is less than the set threshold, modify the corresponding parameters and retrain until the small target AP value of the model reaches the set threshold; use the trained model to detect small targets. The invention can make the distribution of the small targets more uniform, thereby improving the adequacy of the training of the small targets, thereby improving the detection accuracy of the small targets.

CN 111898668 A discloses a small target object detection method based on deep learning, which can overcome the problems of insufficient detection efficiency and low accuracy in the existing small target object detection methods. First, based on the COCO dataset, extract images that do not contain small target objects, adjust the size of the images, and then stitch them together. Combine the stitched images and the images containing small target objects in the COCO dataset to form a new dataset, and use a 4:1 ratio. The data set is divided into training set and test set proportionally; then, the basic feature extraction network of Faster-RCNN is modified to perform feature fusion; then, the fusion features of each level after fusion are selected by the RPN network for candidate regions; The improved network is trained to obtain a training model; finally, the test set is input into the trained model for target detection.

CN 111368769 A Provides a ship multi-target detection method based on an improved anchor frame generation model, comprising: acquiring a SAR ship image; constructing a low-complexity network structure, and placing the image into the low-complexity network to generate a feature map space; The clustering method based on shape similarity is used to generate the initial anchor frame; based on the generated initial anchor frame, the sliding window mechanism is used to generate a new candidate frame in the low-complexity feature space, and the candidate frame is subjected to regression training. for ship multi-target detection. The invention solves the problems of low algorithm efficiency and low detection quality caused by complex network and poor candidate frame quality, and has better accuracy. Since a low-complexity network architecture is used for detection, from the perspective of statistical analysis, the greater the amount of data collected, that is, the more the number of detections, the better the detection effect.

The target detection methods of the above patent applications belong to the two-stage target detection algorithm based on Faster-RCNN, which respectively oversample small targets, improve the adequacy of small target training, improve the model or model training process, and adopt the sliding window mechanism. Generate new candidate boxes in the low-complexity feature space to improve the detection accuracy of small objects. The main steps of Faster-RCNN are as follows: first, the convolutional layer is used to extract the feature maps of the input image; secondly, the RPN network is used to generate proposals; thirdly, the Roi Pooling layer is used to extract feature maps and proposals. Regional feature maps (proposal feature maps); finally, the classification layer calculates the proposal category according to the proposal feature maps, and performs bounding box regression again to obtain the final precise position of the detection frame. However, if the setting size of proposals is unreasonable, it is easy to cause the problem of missed detection, especially when the scales of the two targets are quite different, it is easy to cause the problem of missed detection of small-scale targets, thereby reducing the detection accuracy.

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide a small target detection method based on the improved Faser RCNN, which can generate an anchor point frame more matching the target size, reduce the missed detection rate of small-scale targets, and improve the detection accuracy.

The technical solution provided by the present invention is a small target detection method based on the improved Faser RCNN, which is realized by the processor executing an improved Faser RCNN algorithm instruction, including: receiving a scene picture containing a small target; using the FaserRCNN to its first convolution The module extracts the first feature map F ₁ of the scene picture; the second convolution module of the Faser RCNN is used to obtain the prediction anchor frame center position a (x, y) and the prediction anchor according to the first feature map F ₁ point frame size a(w,h) and obtain the predicted anchor point frame a(x,y, w,h); the third convolution module of the Faser RCNN is used to obtain the first feature map and the first feature map according to the first feature map F ₁ and the prediction anchor frame a(x, y, w, h). The second feature map F ₂ with the same size as F ₁ ; the fourth convolution module of the Faser RCNN is used to obtain the second feature map F ₂ and the predicted anchor point frame a(x, y, w, h) The detection result of the scene picture.

Further, the main part of the first convolution module adopts the convolution structure of ResNet.

Further, the main part of the first convolution module adopts the convolution structure of ResNet50.

Further, the convolutional structure of the ResNet50 includes a multi-layer Deform ResNet50 residual block structure, and the second convolutional layer of the Deform ResNet50 residual block structure is replaced by a depthwise separable convolutional layer.

Further, the first convolution module includes a channel attention mechanism module, and the channel attention mechanism module is configured to obtain the feature weight S _c according to the scene picture; the second convolution module is configured to The first feature map F ₁ and the feature weight S _c obtain the central position a(x, y) of the predicted anchor frame and obtain the predicted anchor according to the central position a(x, y) of the predicted anchor frame Point box a(x,y,w,h).

Further, the backbone part of the second convolution module adopts an adaptive anchor frame network structure.

Further, the adaptive anchor frame network obtains a score feature map F _P according to the first feature map F ₁ , and then obtains the predicted anchor frame center position a(x, y according to the score feature map _FP ) ); and the predicted anchor frame a(x, y, w, h) is obtained according to the predicted anchor frame center position a(x, y) and the predicted anchor frame size a(w, h).

Further, the adaptive anchor point frame network includes an adaptive adjustment module, and the adaptive adjustment module is configured to predict the anchor point frame a(x, y, w, h) and the first feature map according to the F ₁ obtains an adaptive prediction anchor frame a'(x, y, w, h); the third convolution module is configured to, according to the first feature map F ₁ , the adaptive prediction anchor frame a '(x, y, w, h) to obtain a second feature map F _{2 with the same size as the first feature map F 1 .}

Meanwhile, the present invention also provides a small target detection device based on the improved Faser RCNN, including:

processor; and

a memory for storing the processor-executable instructions;

Wherein, the processor is configured to execute the program instructions of the algorithm structure, so as to realize the above-mentioned small target detection method based on the improved Faser RCNN.

In addition, the present invention also provides a computer storage medium based on the small target detection method based on the improved Faser RCNN, when the instructions in the computer storage medium are executed by the processor of the small target detection device based on the improved Faser RCNN, so that the The small target detection device based on the improved Faser RCNN can perform the above-mentioned small target detection method based on the improved Faser RCNN.

The present invention mainly includes four parts: the first part is the process of extracting the features of the scene picture; the second part is to obtain the shape of the predicted anchor point frame according to the center point of the predicted anchor point frame and finally generate the predicted anchor point frame A(x , y, w, h); the third part is the process of generating a feature map of the same size as the anchor box A (x, y, w, h); the fourth part is the process of obtaining the scene picture detection result.

Beneficial effects of the present invention:

1. The present invention is modified based on the framework of the Faser RCNN algorithm, and the RPN network under the framework of the Faser RCNN algorithm is replaced with an adaptive anchor point frame network, so that the generated anchor point frame can be more matched with targets of different scales, and then avoid the anchor point. The missed detection phenomenon caused by the unreasonable size of the frame setting will ultimately achieve the purpose of improving the detection accuracy.

2. The present invention adopts an adaptive anchor box network. The network is mainly composed of two branches: shape prediction and position prediction. The two branches select a position with a predicted probability higher than a certain threshold and the most probable shape of each selected position. Anchor box. Through the above improvement method, an anchor frame with a reasonable size can be obtained, which avoids the situation of missed detection, improves the detection ability of the network to the target, and effectively improves the detection performance. When applied to insulator detection, it is known by comparing the present invention with the existing insulator detection algorithm that the detection accuracy of the present invention is greatly improved.

3. When applied to insulator detection, by comparing the present invention with the existing insulator detection algorithm, it is known that the present invention uses depthwise separable convolution instead of conventional convolution to reduce the amount of network parameters and greatly improve the detection speed.

4. The present invention improves the residual network in the feature extraction network, and adds a channel attention mechanism structure, so that each channel in the feature map is connected to each other and strengthens the important feature information in each channel, which is more helpful for subsequent scenarios on the one hand. On the other hand, the detection process of small objects in pictures can improve the accuracy of detection results.

Description of drawings

1 is a schematic structural diagram of the DeformResNet50 residual block in the small target detection method based on improved Faser RCNN in Embodiment 1 of the present invention;

2 is a schematic diagram of the principle of depth separable convolution in the small target detection method based on improved Faser RCNN in Embodiment 1 of the present invention;

3 is a schematic diagram of an adaptive anchor frame network structure in a small target detection method based on improved Faser RCNN in Embodiment 1 of the present invention;

Fig. 4 is the network training flow chart of improving FaserRCNN in the small target detection method based on improving Faser RCNN in the embodiment of the present invention 1;

5 is a network frame diagram of the improved FaserRCNN in the small target detection method based on the improved Faser RCNN in Embodiment 1 of the present invention;

Fig. 6 is the detection flow chart of the insulator defect in the embodiment 1 of the present invention;

Fig. 7 is the defect detection result diagram of the normal insulator in the embodiment 2 of the present invention;

FIG. 8 is a graph showing the defect detection result of the defective insulator in Example 2 of the present invention.

Detailed ways

Insulators are an important part of overhead transmission lines, which are used to support and fix busbars and live conductors, and to provide sufficient distance and insulation between live conductors or between conductors and the ground. Due to the long-term exposure of overhead transmission lines to the natural environment and the influence of natural or human factors, there are problems such as line aging and damage. If these problems are not regularly checked and repaired, major safety accidents may occur. Those skilled in the art know that insulator defects include erosion, cracking, breaking, core rod exposure, etc., but the dimensions of insulator defects and insulators are quite different. Therefore, in insulator defect detection, the missed detection rate is relatively high.

The technical solutions provided by the present invention and the corresponding technical effects will be described in detail below with reference to the accompanying drawings and taking the insulator detection as an example.

Example 1

This embodiment provides a small target detection method based on the improved Faser RCNN, which is a method of extracting features from the scene picture, generating an anchor frame based on the center position of the anchor frame, and finally obtaining a detection result. For the convenience of understanding, this embodiment describes the specific process through the following steps 100 to 600. In actual implementation, these steps do not represent the order of time. On the premise of realizing the preparation conditions for the implementation of each step, the implementation of changing the order is also the same. different embodiments of the invention.

Step 100: Preprocess the original transmission line image sample picture used for learning, that is, the scene picture.

In this embodiment, in the preprocessing, each sample picture is adjusted to the same size, and data enhancement is performed. Exemplarily, in this embodiment, each sample picture is adjusted to a size of 900×600 using bilinear interpolation, and the data enhancement method used during training enhances the collected sample pictures with data such as rotation, cropping, and contrast increase. The method expands the data set, and the specific parameters of the data enhancement method are shown in Table 1. This step obtains a scene image dataset containing small objects. Specifically, in this embodiment, after the sample picture is expanded with three RGB channel components, one scene picture is represented by one tensor of 900×600×3.

Table 1 Data enhancement methods

Use LabelImg software to manually label the obtained data set, set the labels of insulator and insulator defects as Insulator and defect respectively, and make them into VOC format for detection network training. 80% of the scene image data set was randomly selected as the training set for optimizing the network model, and 20% of the scene image data set was used as the test set for evaluating the model effect.

Step 200, constructing an improved Faser RCNN consisting of a first convolution module, a second convolution module, a third convolution module, a fourth convolution module, and a channel attention mechanism module (as shown in Figure 5). In this embodiment, more specific structural settings and functions of the Faser RCNN used in the present invention can be obtained from the following description of its working principle. The neural network construction in this embodiment includes four main parts: the first part is the convolution layer, the second part is the adaptive anchor box network, the third part is the ROI Pooling layer, and the fourth part is the classification and regression layer. details as follows

The first part: convolution layers (conv layers), used to extract the features of the picture, the input is the tensor (224*224*3) obtained after preprocessing, and the output is the extracted features, referred to as the first feature map F for short ₁ . In one of the specific implementations of this part, we use the convolutional structure of ResNet to perform feature extraction on the scene picture to obtain the first feature map F ₁ . Preferably, the backbone part of the first convolution module of the first neural network adopts the convolution structure of ResNet50, that is, the tensor (224*224*3) is input into the feature extraction layer of ResNet50. The Resnet 50 network is composed of several residual blocks, and the structure diagram is shown in Table 2.

Table 2 Resnet 50 network structure

First, the input image is subjected to a 64-dimensional 7x7 convolution with a stride of 2, then it is downsampled by max pooling with a kernel size of 3x3 and a stride of 2, and finally a series of residuals After the block, do global average pooling and 1000-dimensional fully connected layer, and output to the softmax classifier for classification processing. The backbone network structure of Resnet 50 has not been improved and will not be repeated here. After feature extraction, a first feature map F ₁ is obtained.

As a preference of this embodiment, the residual block of the Resnet 50 network is improved, thereby forming the Deform_Resnet 50 backbone feature extraction network. The convolutional structure of the ResNet50 includes a multi-layer DeformResNet50 residual block structure, each residual block structure includes three convolutional layers (1x1+3x3+1x1), and the middle 3x3 convolutional layer is first in a dimension reduction 1x1 convolution. The calculation is reduced under the layer, and then restored under another 1x1 convolutional layer, which both maintains the accuracy and reduces the amount of calculation.

Exemplarily, one of the specific implementations of this embodiment is: the second convolution layer (ie, the 3x3 convolution layer) of the Deform ResNet50 residual block structure is replaced by a depthwise separable convolution layer. For the Deform_Resnet 50 residual The way the blocks are constructed is shown in Figure 1. Depthwise separable convolutions include channel-wise convolution (Depthwise Convolution) and pointwise convolution (Pointwise Convolution). First, the corresponding C feature maps are generated from the input C channel image, and then the generated feature maps are recombined to form a new feature map through point-by-point convolution. Among them, the size of the convolution kernel of point-by-point convolution is 1x1xM, and M is the number of channels in the previous layer. The schematic diagram of depthwise separable convolution is shown in Figure 2. In the case of the same input, the number of parameters of the depthwise separable convolution is greatly reduced, which greatly reduces the computational complexity.

Exemplarily, another specific implementation of this embodiment is: the first convolution module of the first neural network includes a channel attention mechanism module. Specifically in this embodiment, the channel attention mechanism module is added to the branch part of the residual block (as shown in the right branch of Figure 1), that is, the tensor (112*112*3) is input to the feature extraction layer of ResNet50 , that is, the channel attention mechanism module is called. Input an image of size W×H×C into the detection network, where W, H, and C are the width, height, and number of channels of the image, respectively. First, perform global average pooling on the C channels of the image. To reduce the dimension, convert it into a 1×1×C vector, use a fully connected layer (FC) to reduce the channel dimension of the vector, then activate the function through the ReLU layer, and then restore the original through a FC layer. dimension, and finally use the Sigmoid function to activate the feature weights. Then multiply the input image with this feature weight:

F _scale (u _c ,s _c )=u _c ×s _c

Among them: uc is the input feature map, and s _c is the weight obtained by the excitation operation _.

Exemplarily, the entire channel attention mechanism module can be expressed by the following formula:

δ为激活函数，Q₁、Q₂表示为两个全连接层的权重值。in,

δ is the activation function, and Q ₁ and Q ₂ are the weight values of the two fully connected layers.

Part II: Adaptive Anchor Box Network.

Based on the probability formula P(x, y, w, h|F)=P(x, y|F)*P(w, h|x, y, F), it can be known that when the center point position (x, y) does not affect At the same time, the probabilities of the corresponding prediction boxes are different, denoted as P(x, y|F). When the position of the center point (x, y) is determined, the probability of occurrence of prediction boxes of different sizes is different, which is recorded as P(w,h|x,y,F). Therefore, it is explained that the influence of a group of anchor boxes is affected by the location and shape size factors. Therefore, the adaptive anchor box network of this embodiment has two branches: a position prediction module and a shape prediction module.

Position prediction module: First, the first feature map F ₁ obtained through the first convolution module of the first neural network. The first feature map F ₁ can be represented by four parameters (x, y, w, h), where ( x, y) represents the coordinates of the center point of the prediction anchor box, and (w, h) represents the width and height of the shape of the prediction anchor box. Then the first feature map F ₁ is subjected to a 1x1 convolution operation and activated by the activation function Sigmoid to generate a corresponding score feature map F _p with the same scale, where each point represents each pixel point A score for the presence of an object. The score is then compared to a score threshold τ, which is experimentally taken as 0.6. If the feature value score in F _p is greater than the threshold τ, the corresponding pixel of this point is represented as the center point of the target, and marked as the center position a(x, y) of the anchor point frame.

Shape prediction module: First, the center position a(x,y) of the anchor frame a(x,y,w,h) is determined by position prediction, and then the anchor point needs to be determined by using the nearest real frame B' corresponding to the center position The shape of box a is a(w,h). Since it is very difficult to obtain the value of a(w,h) by regression operation, the method of sampling approximation to w and h is adopted. In the present invention, w:h has three values of 0.5, 1.0 and 2.0. Through the maximum intersection ratio vIOU between anchor box a and a real box B, B'(x', y', w', h') with the largest intersection ratio with a, and a(w at this time) are obtained. ,h), that is, the result outputted by the shape prediction branch is a(w,h). Due to the large and unstable numerical range, the width and height of the prediction frame need to be adjusted by the formulas: W=μ·S· ^edW , H=μ·S· ^edH , where S is the step size and μ is Experience factor (usually 8). Adjust the range of parameters to be learned from [1,1000] to [-1,1] to simplify network training. According to the center position a(x, y) of the anchor frame output by the shape prediction branch and the anchor frame size a(w, h) output by the position prediction branch, the anchor frame a(x, y, w) is finally obtained , h).

As a more preferred implementation of this embodiment, the adaptive anchor box network includes an adaptive adjustment module, and the adaptive adjustment module is configured to adjust according to the anchor box a(x, y, w, h) and the first feature map F ₁ to obtain an adaptive anchor frame a'(x, y, w, h), and the specific operation formula of the adaptive feature is as follows:

f _i '=D(f _i , Wi , H _{i )}

Among them, fi is the eigenvalue of the generated _i -th anchor point frame mapped on the input _first feature map F1, and D( ) is composed of a 3x3 deformable convolution, that is, the Offset described in Figure 3 field. f _i ' is the adjusted feature value, that is, the feature and width and height of the i-th position are processed. The adaptive anchor frame a'(x,y,w,h) is obtained according to f _i '. The network structure diagram of the adaptive anchor box is shown in Figure 3, the input is the first feature map F ₁ , and the output is the predicted anchor box a(x, y, w, h). In FIG. 3 , W×H×1 and W×H×2 are the size representations of the first feature map F ₁ , wherein W and H respectively correspond to the width and height of the first feature map F, and 1 and 2 represent the number of channels. As shown in Figure 3, the first feature map F ₁ is first predicted by location positioning to obtain the anchor frame center position a(x, y), and secondly, through shape prediction, the predicted anchor frame a(x, y, w, h is obtained) ); finally, the adaptive anchor frame a'(x, y, w, h) is obtained after the adaptive adjustment of the features.

The third part: ROI Pooling, this layer collects the input first feature map F ₁ and the prediction anchor frame a (x, y, w, h), after synthesizing these information, the size of the first feature map F ₁ is obtained The same second feature map F ₂ , the second feature map F ₂ is a feature map of a fixed size, and then sent to the subsequent fully connected layer. Preferably, this layer collects the input first feature map F ₁ and the adaptive anchor point frame a' (x, y, w, h), and after synthesizing these information, obtains the same size as the first feature map F ₁ The second feature map F ₂ ,

The fourth part: Classification and regression, the input of this layer is the second feature map F ₂ , and the output is the bounding box of the target and the confidence of the defect category. Using Softmax Loss (detection classification probability) and Smooth L1Loss (detection bounding box regression) to jointly train the classification probability and bounding box regression to obtain the bounding box and defect category confidence of the target (as shown in Figure 6).

Step 300, configure the loss function and train the convolutional neural network.

In this embodiment, the entire training process of the neural network is end-to-end training, and the loss function mainly includes the anchor point location loss function L _loc , the anchor point shape prediction loss function L _shape , the target classification loss function L _cls , and the regression loss function L _reg . :

L _loss =λ ₁ L _loc +λ ₂ L _shape +L _cls +L _reg

where λ ₁ =1 and λ ₂ =0.1

First, for the localization loss function L _loc , the loss function is used to control the corresponding number of anchor boxes so as to place more anchor boxes in the center of the target and place less anchor boxes in the non-center coordinates to prevent the generation of The positive and negative samples in the anchor box are unbalanced. The determination of the position is the central position of the reserved anchor box. That is, when the center point is a negative sample, that is, a simple classification, the prediction score y will be close to 1, so the weight corresponding to the formula will become smaller, so the number of anchor boxes placed on the center point will be reduce. Due to the huge number of generated anchor boxes, the anchor boxes containing negative samples occupy most of them. Therefore, in order to balance the positive and negative samples contained in the anchors, Focal Loss is used to train the position branch. The formula for Focal loss is as follows:

Among them, y represents the predicted score of the anchor box location center as a sample, y' represents the actual label value of the location center in the pre-label, positive samples are set to 1, negative samples are set to 0. The value of α controls the weight of the positive and negative anchor boxes, generally α∈(0,0.5), which is set to 0.25 in this paper. γ is the concentration parameter. When γ=0, the loss function is the traditional cross-entropy loss function, so generally γ≥0, the value of γ in this paper is 2. (1-y') ^γ is the modulation coefficient, which is used to control the weight of easy-to-classify and hard-to-classify samples.

忽略区域表示为

对于其他边界区域定义为负样本区域。其中，

用来调节生成锚点框的数量，一般为

由此确定其锚点框的中心位置。For location prediction, each box is first divided into three types of regions: central region; ignore region; negative sample region. The three specific regions are defined according to the location information corresponding to the real frame mapped to the corresponding feature map, namely (x ₀ , y ₀ , w ₀ , h ₀ ). So the corresponding central region is expressed as

The ignore region is represented as

For other boundary regions, it is defined as a negative sample region. in,

Used to adjust the number of generated anchor boxes, generally

This determines the center position of its anchor box.

Second, for shape prediction loss function

First, the center position (x, y) of the anchor frame a (x, y, w, h) is determined through position prediction, and then the shape of the anchor frame a needs to be determined by using the nearest real frame B' corresponding to the center position. a(w, h). Since it is very difficult to obtain the value of a(w,h) by regression operation, the method of sampling approximation to w and h is adopted. In the present invention, w:h has three values of 0.5, 1.0 and 2.0. Through the maximum intersection ratio vIOU between anchor box a and a real box B, B'(x', y', w', h') with the largest intersection ratio with a, and a(w at this time) are obtained. , h). The loss function for shape prediction is:

Among them, L ₁ is a smoothing function:

Third, for the classification loss function

The loss function of the classification part uses the binary cross entropy function, and the formula is as follows:

为背景真实的概率，

的规定如下：

Among them: p _i is the probability that the anchor box (anchor) is predicted to be the target,

is the true probability of the background,

The provisions are as follows:

Fourth, for the regression loss function L _reg

The regression loss function _Lreg is described as follows:

是与positiveanchor对应的groundtruth的4个坐标参数。R为smoothL1函数：Wherein, t _i ={t _x , _ty , t _w , _th } represents four parameters of the anchor point frame, which are the center position coordinates, width and height of the anchor point frame, respectively.

are the 4 coordinate parameters of the groundtruth corresponding to the positiveanchor. R is the smoothL1 function:

In this embodiment, the entire training process of the neural network is end-to-end training. The first feature map F 1 obtained by the first convolution module of the first neural network is the first feature map F ₁ obtained by the first convolution module of the second neural network. The prediction result of the center position of the anchor point box, followed by the prediction result of the center size of the anchor point box, and finally the prediction result of the anchor point box, which converts the scene picture into the bounding box of the target and the confidence level of the defect category.

Step 400, train the model.

In the training process (as shown in Figure 4), gradient descent is used to optimize the backpropagation stage. The training batch size is set to 16, the momentum value is set to 0.9, and the weight decay is exponentially decayed. The rate is set to 0.004, the parameter num_classes is set to 3 (representing insulators, insulator defects and background), and the warmup training strategy is adopted. The epoch is set to 40000, the model is saved every 3000 epochs and the last model is saved, and the model with the lowest loss is finally selected for detection.

Step 500, model application.

After the above training process, multiple models can be obtained, and the optimal model (with the smallest loss function value) can be selected for application. At this time, the image data processing does not require data enhancement here, and only needs to adjust the image to 900*600 size, and normalization can be used as the input of the model. The parameters of the entire network model are fixed, as long as the image data is input and propagated forward. The first feature map F, the prediction anchor frame a(x, y, w, h), and the second feature map F ₂ are obtained in sequence, and the detection result can be directly obtained through the entire model. When a large number of original transmission line images need to be tested, all images can be integrated into one data file. For example, when using a data table to store the RGB values of each image, an lmdb format file can be used, which is convenient for reading all images at one time.

In this embodiment, based on the ResNet50 with channel attention mechanism in the improved Faser RCNN network architecture as the feature extraction network, the feature extraction of the scene picture is performed; based on the adaptive anchor box network in the improved Faser RCNN network architecture as the prediction anchor The point box network is used to predict and generate target anchor points; finally, based on the improved ROI Pooling layer and classification or regression layer in the Faser RCNN network architecture, the final output category confidence and category bounding box.

Step 500, model validation

In order to verify the effectiveness of this embodiment, the Aut-Faster RCNN detection method and the Fast RCNN, FasterRCNN, SSD, YOLO v3, and YOLO v4 methods are trained on the insulator data set, and their running speed and mAP are compared. The results are shown in Table 3. .

Table 3 Comparison results of six detection methods on the insulator dataset

According to the experimental analysis results, it can be concluded that the average precision (mAP) of the Aut-Faster RCNN algorithm in this embodiment is 93.67%, which is higher than the current mainstream detection network.

Embodiments of the present invention further include a computer-readable storage medium, where the computer-readable storage medium stores program instructions for implementing the method of the present invention and/or model parameters obtained by training the method of the present invention.

Embodiments of the present invention further include a small target detection device based on the improved Faser RCNN, which includes a memory and a processor, the memory stores model parameters obtained by training in the method embodiment of the present invention, and the processor reads and implements the improvement of the present invention The program instructions of the algorithm structure described by Faser RCNN, and the above-mentioned small target detection is realized according to the model parameters.

Example 2

Two scene pictures containing insulators are selected as detection objects, and the detection method provided in Embodiment 1 of the present invention is used for detection. The detection process is as follows.

Example 2.1

(1) Adjust a scene image containing insulators to 900*600 size, and then input the tensor to 900*600*3 into the small target detection device based on improved Faser RCNN in Example 1.

(2) The first feature map F ₁ ∈ R ^37×50×512 obtained by the tensor (900*600*3) through the convolutional layer.

(3) The first feature map F ₁ ∈ R ^37×50×512 obtained by the adaptive anchor frame network is the adaptive anchor frame a' (-212, -419, 183, 359).

(4) The first feature map F ₁ ∈ R ^{37 × 50 × 512} and a' (-212, -419, 183, 359) obtained through the ROI Pooling layer is the second feature map F ₂ ∈ R ^{7 × 7 × 512} .

(5) The second feature map F ₂ ∈ R ^{7 × 7 × 512} and a' (-212, -419, 183, 359) obtained by the classification and regression layer classification and class confidence, the results are shown in Figure 7 .

Example 2.2

(1) Adjust another scene picture containing defective insulators to 900*600 size, and then input the tensor to 900*600*3 into the small target detection device based on improved Faser RCNN in Example 1.

(2) The first feature map F ₁ ∈ R ^37×50×512 obtained by the tensor (900*600*3) through the convolutional layer.

(3) The first feature map F ₁ ∈ R ^37×50×512 obtained by the adaptive anchor frame network is the adaptive anchor frame a' (-415, -425, 286, 431) and the adaptive anchor frame a' (-490, -652, 83, 135).

(4) The first feature map F ₁ ∈ R ^37×50×512 and a' (-415, -425, 286, 431), a' (-490, -652, 83, 135) obtained by the ROI Pooling layer is the second feature map F ₂ ∈ R ^7×7×512 .

(5) The second feature map F ₂ ∈ R ^7×7×512 and a' (-415, -425, 286, 431), a' (-490, -652, 83, 135) are obtained by classification and regression layer The classification category and category confidence of , the results are shown in Figure 8.

It should be noted that, because in the adaptive generation process of the anchor point box, not a single anchor point box is generated, but many anchor point boxes are generated in the corresponding ROI area, and finally each anchor point box is classified. Operation is used to distinguish whether the insulator is normal or damaged.

FIG. 7 is a diagram of the defect detection result of a normal insulator, and FIG. 8 is a diagram of the defect detection result of a defective insulator. It can be seen from the figure that only one bounding box of the insulator target is included in FIG. 7 , and the confidence level of the insulator category is shown above the bounding box of the insulator target. Figure 8 contains two bounding boxes of targets, one is the bounding box of the insulator target, the confidence of the insulator category is shown above the bounding box of the insulator target, and the other is the bounding box of the defective insulator target, which is the bounding box of the defective insulator target. Defective insulator class confidence is shown above the bounding box of the target.

It should be noted that, in order to better distinguish the insulator target and the defective insulator target in this embodiment, when the patent application of the present invention is submitted, FIG. 5 , FIG. 7 and FIG. 8 are simultaneously submitted in the form of other certification documents. Among them, Figure 5 corresponds to the certification material 1 in other certification documents, except that the color is different, all other factors are the same; Figure 7 corresponds to the certification material 2 in other certification documents, except for the color difference, all other factors are the same; Figure 8 corresponds to the proof material 3 in the other proof documents, except for the color difference, all other factors are the same.

The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present invention are all within the protection scope of the present invention. .

Claims (10)

1. a small target detection method based on improving Faser RCNN, implements an improved Faser RCNN algorithm instruction by processor and realizes, it is characterized in that, comprise: receive the scene picture that comprises small target; Use its first convolution of described Faser RCNN The module extracts the first feature map F of the scene picture; the second convolution module of the Faser RCNN is used to obtain the prediction anchor frame center position a (x, y) and the prediction anchor point according to the first feature map F ₁ frame size a(w,h) and obtain the prediction anchor frame a(x,y,w according to the prediction anchor frame center position a(x,y) and the prediction anchor frame size a(w,h) , h); use the Faser RCNN its third convolution module to obtain and the first feature map F according to the first feature map F ₁ , the prediction anchor frame a (x, y, w, h) _1. The second feature map F ₂ of the same size; the fourth convolution module of the Faser RCNN is used to obtain the obtained data according to the second feature map F ₂ and the prediction anchor frame a(x, y, w, h). The detection result of the scene picture. 2. The small target detection method based on improved Faser RCNN according to claim 1, wherein the main part of the first convolution module adopts the convolution structure of ResNet. 3. The small target detection method based on improved Faser RCNN according to claim 2, is characterized in that, the main part of described first convolution module adopts the convolution structure of ResNet50. 4. the small target detection method based on improved Faser RCNN according to claim 3, is characterized in that, the convolution structure of described ResNet50 comprises multi-layer Deform ResNet50 residual block structure, described Deform ResNet50 residual block structure its No. 1 The second convolutional layer is replaced by a depthwise separable convolutional layer. 5. The small target detection method based on improved Faser RCNN according to claim 1, wherein the first convolution module comprises a channel attention mechanism module, and the channel attention mechanism module is configured to The scene picture obtains a feature weight S _c ; the second convolution module is configured to obtain the prediction anchor frame center position a(x, y) according to the first feature map F ₁ and the feature weight S _c and The predicted anchor frame a(x, y, w, h) is obtained according to the central position a(x, y) of the predicted anchor frame. 6. The small target detection method based on improved Faser RCNN according to any one of claims 1-5, wherein the backbone part of the second convolution module adopts an adaptive anchor frame network structure. 7. The small target detection method based on improved Faser RCNN according to claim 6, wherein the adaptive anchor frame network obtains the score feature map F _P according to the first feature map F ₁ , and then according to the The score feature map _FP obtains the prediction anchor frame center position a(x, y); and according to the prediction anchor frame center position a(x, y) and the prediction anchor frame size a(w, h) Obtain the predicted anchor box a(x, y, w, h). 8. The small target detection method based on improved Faser RCNN according to claim 7, wherein the adaptive anchor frame network comprises an adaptive adjustment module, and the adaptive adjustment module is configured to predict according to the prediction The anchor frame a(x, y, w, h) and the first feature map F ₁ obtain the adaptive prediction anchor frame a'(x, y, w, h); the third convolution module is configured To obtain a second feature map F ₂ with the same size as the first feature map F ₁ according to the first feature map F ₁ and the adaptive prediction anchor frame a'(x, y, w, h). 9. a small target detection device based on improved Faser RCNN, is characterized in that, comprises: processor; and a memory for storing the processor-executable instructions; Wherein, the processor is configured to execute the program instructions of the algorithm structure, so as to realize the small target detection method according to any one of claims 1 to 8. 10. A computer storage medium based on the small target detection method of the improved Faser RCNN, characterized in that: when the instructions in the computer storage medium are executed by the processor of the small target detection device based on the improved Faser RCNN, the The small target detection device based on the improved Faser RCNN can perform the small target detection method according to any one of claims 1 to 8.

Images