CN117746252A - A landslide detection method based on improved lightweight YOLOv7 - Google Patents

Abstract

The invention belongs to the technical field of computer image processing, and relates to a landslide detection method based on improved lightweight YOLOv7, which comprises the steps of firstly, collecting mountain satellite images, preprocessing the mountain satellite images by using a super-resolution algorithm GAN, then splicing, rotating and corroding the images, and marking a real frame and a category of the mountain satellite images; secondly, replacing the original Yolov7 features with a lightweight network MobileNet v3 to extract a backbone network, adding a small target detection layer into the model, adding a HAT attention mechanism, simulating different weather conditions, and repeatedly training to obtain an improved lightweight Yolov7 model, and detecting mountain satellite images to obtain a landslide detection result; the invention improves the resolution of the image; the problem of uneven distribution of positive and negative samples is solved; the system has high-efficiency small target detection capability and can be better adapted to complex and changeable weather conditions.

Description

A landslide detection method based on improved lightweight YOLOv7

Technical field

The invention belongs to the technical field of computer image processing. Specifically, it is an improved lightweight YOLOv7 landslide detection method.

Background technique

Landslides are a serious natural disaster that are usually triggered by factors such as rainfall, geological structures, and geological conditions. Such disasters may result in property losses, casualties, and damage to the ecological environment. Therefore, timely detection and monitoring of landslides is crucial to mitigate potential risks. Traditional landslide detection methods usually rely on manual inspections or fixed monitoring equipment. These methods have certain limitations, such as dependence on manpower, time-consuming, and high cost. Traditional landslide monitoring methods mainly rely on manual inspections and the use of various monitoring equipment, such as inclinometers, displacement sensors, depth sounders, etc. Although these methods can provide useful data to a certain extent, they have the following limitations: ① Reliance on manual inspection: Manual inspection requires a lot of time and human resources, which is not only inefficient, but also difficult to carry out in bad weather or at night. ②Limited monitoring range: Traditional monitoring equipment is usually distributed in limited locations, making it difficult to cover the vast landslide potential area. ③High cost: Installing, maintaining and operating monitoring equipment requires expensive capital investment, which is a serious limiting factor for many regions.

Existing landslide detection methods face some challenges: ① Complex terrain and background: Landslides usually occur in changeable and complex terrain, and terrain and background interference are serious, increasing the difficulty of detection. ②Small target detection: The initial signs of landslides are usually relatively small, such as small movements of earth and rock materials or the formation of cracks, so the model needs to have efficient small target detection capabilities. ③Data imbalance: Since landslides are a rare event, the imbalance of positive and negative samples may lead to unstable model training and difficulty in obtaining high accuracy. ④The data set is blurry: Due to the high risk of landslide events, the quality of currently public videos and satellite images is not high, and the environment is relatively single and not rich enough.

In recent years, a large number of target detection algorithms have been proposed, such as the typical traditional detection algorithm SIFT (Scale-Invariant Feature Transform): an algorithm used to detect local feature points in images, which can be used for target detection and matching. With the rise of deep convolutional neural network (CNN), the development of computer vision and deep learning technology has provided new possibilities for landslide detection. Target detection algorithms based on deep learning mainly have two general directions. Faster R-CNN represented by two stages, and YOLO and SSD represented by one stage. Among them, Faster R-CNN is a classic target detection algorithm that improves detection speed by introducing a Region Proposal Network (RPN). Although it performs well in terms of accuracy, there may be some challenges in terms of real-time performance and lightweight. SSD is a single-stage target detection algorithm that detects multi-scale targets at a faster speed. RetinaNet is a target detection algorithm using Focal Loss, focusing on solving the problem of positive and negative sample imbalance. Although it performs well in some specific scenarios, it may require more computing resources.

Therefore, a lightweight and efficient landslide detection method with small target detection capabilities is needed.

Contents of the invention

In view of the problems existing in the above-mentioned prior art, the present invention provides a landslide detection method based on the improved lightweight YOLOv7. The lightweight network MobileNetv3 is used to replace the original YOLOv7 feature extraction backbone network. After that, a small target detection layer is added to the model, and a HAT attention mechanism is added to improve the problem of low resolution of the data set.

In order to achieve the above objectives, the present invention adopts the following technical solution: a landslide detection method based on improved lightweight YOLOv7. The specific steps of the method are as follows:

Step 1, collect satellite images of the mountain;

Step 2: Use the super-resolution algorithm GAN to preprocess the mountain satellite image. At the same time, in order to solve the problem of uneven distribution of positive and negative samples, image enhancement operations such as splicing, rotation, and erosion are then used to increase the number of negative samples. The generalization ability of the model is enhanced, and the processed mountain satellite images are divided into training sets, verification sets, and test sets, and the real frames and categories of the mountain satellite images are marked;

Step 3: Build and train an improved lightweight YOLOv7 model. The improved lightweight YOLOv7 model includes the backbone network, neck network and detection head network; the backbone network includes the MobileNet1 network, MobileNet2 network, MobileNet3 network and MP-ELAN downsampling. network;

Step 4: Input the mountain satellite image processed in step 2 into the backbone network of the improved lightweight YOLOv7 model, and extract the characteristic information of the MobileNet1 network, MobileNet2 network, MobileNet3 network and MP-ELAN downsampling network respectively, and obtain four types of Feature maps of different sizes;

Step 5: Input the four feature maps of different sizes obtained in step 4 into the neck network. The neck network uses the PAFPN feature pyramid to improve features through top-down, bottom-up feature extraction and FPA enhanced feature extraction network. After fusion, four enhanced feature maps of different sizes are obtained respectively. At the same time, in order to further improve the network performance, the HAT attention mechanism is introduced into the neck network to enhance the network's attention and learning of key features, so as to capture mountain satellites more effectively. Important information in the image;

Step 6. The detection head network part includes the detection heads of structural reparameterized convolution RepConv and IDetect of four different target sizes; input the four enhanced feature maps of different sizes obtained in step 5 into the detection head network respectively, and obtain Predict feature maps of four different sizes, and calculate a priori boxes. After calculation, three a priori boxes are generated for the predicted feature maps of each size, and compared with the real box, select the intersection with the real box with the largest IoU Make predictions based on the a priori frame, simulate different weather conditions for repeated training, and complete the training of the improved lightweight YOLOv7 network;

The IoU calculation formula is as follows:

Among them, Intersection_Area represents the area where two bounding boxes intersect; Union_Area represents the area of the union of two bounding boxes;

Step 7: Input the test set into the trained and improved lightweight YOLOv7 model, and pass through the backbone network, neck network and detection head network in sequence to obtain prediction feature maps of four different sizes, and predict the output through model inference The final target frame and target label are used to obtain the landslide detection results.

Further, the super-resolution algorithm GAN described in step 2 is used to preprocess the mountain satellite image, specifically: perform GAN super-resolution processing on the collected mountain satellite image to improve the resolution, and then perform geometric transformation on the image, Geometric transformations include random expansion, random cropping, random splicing, and scaling to a fixed ratio.

Further, in step 2, the ratio of training set, verification set, and test set is 8:1:1.

Further, in step 2, use the LableImg tool to label the real frame and category of the mountain satellite image.

Furthermore, the HAT attention mechanism is divided into two parts: the local window self-attention mechanism and the fusion channel attention mechanism;

Local window self-attention mechanism: normalize the input features, and then divide the input features into local windows through the window self-attention mechanism to focus on local associated information;

Fusion channel attention mechanism: introduce global information, use global information to weight features, and activate more pixels through the channel attention mechanism to obtain more feature information.

Furthermore, in step 5, the PAFPN feature pyramid includes the SPPCSPC module, the ELAN-H module, UP upsampling, MP downsampling and convolutional layers, in which the HAT attention mechanism is introduced at the tail of the fusion of the SPPCSPC module and the ELAN-H module.

Furthermore, the 640*640 image is obtained after processing in step 2, and the 32 times downsampling feature map is output after passing through the backbone network, and then the number of channels is changed from 1024 to 512 through the SPPCSPC module, in a top-down manner with MobileNet1 and MobileNet3 The layer results are fused to obtain a 160*160 size feature map; then the feature fusion is performed with the ELAN-H results and SPPCSPC processing results in a bottom-up manner to obtain a 20*20 size feature map and a 40*40 size feature map; then The MobileNet2 layer information is fused with the ELAN-H output feature map to obtain an 80*80 size feature map.

Furthermore, during the training process, the matching degree and loss value are calculated. The matching degree is mainly compared with the IoU of the prediction box and the real label boundary box, and the matching degree of the prediction box is calculated according to the judgment standard; then, all prediction boxes are given Positive and negative sample labels, and determine the corresponding real object labels to facilitate subsequent loss calculations. After calculating the loss value, use the loss value to backpropagate the gradient to update the weights and biases of the improved lightweight YOLOv7 network. Then the minimized loss function is obtained; the calculation formula of the minimized loss function is as follows:

FocalLoss(p _t )＝-α _t (1-p _t ) ^γ log(p _t )

Among them, p _t represents the proportion of difficult samples, (1-p _t ) ^γ is called the modulation coefficient, α _t defaults to 0.25, and γ defaults to 1.5.

Furthermore, in order to make the prediction box more accurate, an improved non-maximum suppression algorithm is introduced. First, B = {b ₁ ,b ₂ ,…,b _N }, S = {s ₁ ,s ₂ ,…,s _N }, N _t are input; where B is the input prediction box list, S is the confidence score of each prediction box, and N _t is the threshold setting for non-maximum suppression; the formula for calculating the prediction box score is as follows:

Among them, M and σ are constants; s _i is the confidence score of the i-th prediction box; b _i is the i-th prediction box, i∈[1, N];

By calculating the confidence score of the prediction box, the score is continuously updated to improve the accuracy of the prediction box.

The present invention has the following technical effects: (1) The present invention proposes a landslide detection method based on the improved lightweight YOLOv7, using the super-resolution algorithm GAN to preprocess the image, greatly improving the resolution of the image; Use image operations such as splicing, rotation, and corrosion to solve the problem of uneven distribution of positive and negative samples; use the lightweight MobileNetV3 backbone network to replace the YOLOv7 backbone network and add a small target detection layer, and introduce the HAT attention mechanism into the network to achieve Lightweight, and further solve the problem of low target resolution, improve network performance, and enhance the network's attention and learning of key features; in addition, in order to enhance the robustness of the model, image enhancement methods are used to simulate different weather conditions .

(2) The present invention overcomes some limitations in traditional landslide detection methods by constructing and training an improved lightweight YOLOv7 network, and at the same time improves the performance of the existing YOLO model in landslide detection; ① Weather adaptability: The model has been optimized to better adapt to complex and changing weather conditions. ②Small target detection: The improved model has efficient small target detection capabilities and can identify early signs of landslides. ③Data balancing strategy: The data balancing strategy is introduced to solve the problem of imbalance between positive and negative samples and improve the stability and accuracy of the model.

Description of drawings

Figure 1 is a schematic structural diagram of the improved lightweight YOLOv7 of the present invention;

Figure 2 is a schematic structural diagram of the ELAN-H network;

Figure 3 is a schematic structural diagram of the SPPCSPC module.

Detailed ways

The content and drawings of the present invention will be described in detail below. This embodiment is implemented based on the technical solution of the present invention and involves detailed implementation plans and operating processes. However, the protection scope of the present invention is not limited to the following specific implementations. For example, the terminology used in the present invention is for the purpose of describing particular embodiments only and is not intended to limit the invention.

A landslide detection method based on improved lightweight YOLOv7. The specific steps of the method are as follows:

Step 1, collect satellite images of the mountain;

Step 2, use the super-resolution algorithm GAN to preprocess the mountain satellite image to improve the resolution, and then perform geometric transformation on the image. The geometric transformation includes random expansion, random cropping, random splicing and scaling to a fixed ratio. At the same time, in order to To solve the problem of uneven distribution of positive and negative samples, image enhancement operations such as splicing, rotation, and erosion are then used to increase the number of negative samples and enhance the generalization ability of the model. The processed mountain satellite images are then divided into training sets. , verification set and test set, the ratio is 8:1:1, and use the LableImg tool to mark the real frame and category of the mountain satellite image;

Step 3: Build and train an improved lightweight YOLOv7 model. The improved lightweight YOLOv7 model includes the backbone network, neck network and detection head network; the backbone network includes the MobileNet1 network, MobileNet2 network, MobileNet3 network and MP-ELAN downsampling. network;

Step 4: Input the mountain satellite image processed in step 2 into the backbone network of the improved lightweight YOLOv7 model, and extract the characteristic information of the MobileNet1 network, MobileNet2 network, MobileNet3 network and MP-ELAN downsampling network respectively, and obtain four types of Feature maps of different sizes;

Step 5: Input the four feature maps of different sizes obtained in step 4 into the neck network. The neck network uses the PAFPN feature pyramid to improve features through top-down, bottom-up feature extraction and FPA enhanced feature extraction network. After fusion, four enhanced feature maps of different sizes are obtained respectively. At the same time, in order to further improve the network performance, the HAT attention mechanism is introduced into the neck network to enhance the network's attention and learning of key features, so as to capture mountain satellites more effectively. Important information in the image; the PAFPN feature pyramid includes the SPPCSPC module, ELAN-H module, UP upsampling, MP downsampling and convolutional layers, in which the HAT attention mechanism is introduced at the end of the fusion of the SPPCSPC module and ELAN-H module; in the SPPCSPC module Maxpool_9, Maxpool_7, Maxpool_5, and Maxpool_3 respectively correspond to four different scales of maximum pooling and have four receptive fields to distinguish large targets from small targets;

Step 6. The detection head network part includes the detection heads of structural reparameterized convolution RepConv and IDetect of four different target sizes; input the four enhanced feature maps of different sizes obtained in step 5 into the detection head network respectively, and obtain Predict feature maps of four different sizes, and calculate a priori boxes. After calculation, three a priori boxes are generated for the predicted feature maps of each size, and compared with the real box, select the intersection with the real box with the largest IoU Use the a priori frame to predict, simulate different weather conditions (rainy, foggy, snowy weather conditions) for repeated training, and complete the training of the improved lightweight YOLOv7 network;

The IoU calculation formula is as follows:

Among them, Intersection_Area represents the area where two bounding boxes intersect; Union_Area represents the area of the union of two bounding boxes;

During the training process, the matching degree and loss value are calculated. The matching degree is mainly compared with the IoU of the prediction box and the real label boundary box, and the matching degree of the prediction box is calculated according to the judgment standard; then, all prediction boxes are assigned positive and negative Sample labels, and determine the corresponding real object labels to facilitate subsequent loss calculations. After calculating the loss value, use the loss value to backpropagate the gradient to update the weights and biases of the improved lightweight YOLOv7 network, and then obtain the minimum minimize the loss function; the calculation formula for minimizing the loss function is as follows:

FocalLoss(p _t )＝-α _t (1-p _t ) ^γ log(p _t )

Among them, p _t represents the proportion of difficult samples, (1-p _t ) ^γ is called the modulation coefficient, α _t defaults to 0.25, and γ defaults to 1.5;

Step 7: Input the test set into the trained and improved lightweight YOLOv7 model, and pass through the backbone network, neck network and detection head network in sequence to obtain prediction feature maps of four different sizes, and predict the output through model inference The final target frame and target label are used to obtain the landslide detection results.

The HAT attention mechanism is divided into two parts: the local window self-attention mechanism and the fusion channel attention mechanism;

Local window self-attention mechanism: normalize the input features, and then divide the input features into local windows through the window self-attention mechanism to focus on local associated information;

Fusion channel attention mechanism: introduce global information, use global information to weight features, and activate more pixels through the channel attention mechanism to obtain more feature information.

After processing in step 2, a 640*640 image is obtained. After passing through the backbone network, a 32 times downsampled feature map is output. Then the number of channels is changed from 1024 to 512 through the SPPCSPC module, and the results of the MobileNet1 and MobileNet3 layers are processed in a top-down manner. Fusion to obtain a 160*160 size feature map; then perform feature fusion with the ELAN-H results and SPPCSPC processing results in a bottom-up manner to obtain a 20*20 size feature map and a 40*40 size feature map; then the MobileNet2 layer The information is fused with the ELAN-H output feature map to obtain an 80*80 size feature map.

Furthermore, in order to make the prediction box more accurate, an improved non-maximum suppression algorithm is introduced. First, input B={b ₁ , b ₂ ,..., b _N }, S={s ₁ , s ₂ ,..., s _N }, N _t ; where B is the input prediction box list, S is the confidence score of each prediction box, and N _t is the threshold setting of non-maximum suppression; the formula for calculating the prediction box score is as follows:

Among them, M and σ are constants; s _i is the confidence score of the i-th prediction box; b _i is the i-th prediction box, i∈[1, N];

By calculating the confidence score of the prediction box, the score is continuously updated to improve the accuracy of the prediction box.

Comparative experiments were conducted on YOLOv7, YOLOv7_MobileNet after replacing the backbone network, YOLOv7_MobileNet_addlayer with the addition of a small target detection layer, and YOLOv7_MobileNet_addlayer_HAT with the addition of the HAT attention mechanism from the three indicators of parameter quantity, precision, and accuracy, as shown in the following table.

Table 1 Comparison of results

Model Parameter quantity Accuracy Accuracy YOLOv7 72M 71.36% 72.78% YOLOv7_MobileNet 16.7M 70.64% 73.12% YOLOv7_MobileNet_addlayer 17.3M 73.56% 76.74% YOLOv7_MobileNet_addlayer_HAT 17.3M 75.23% 79.51%

The results show that the YOLOv7_MobileNet_addlayer_HAT model has high precision and accuracy when the number of model parameters is small, and the effect is more significant.

The above description of the disclosed embodiments enables those skilled in the art to utilize the present invention. At the same time, the above embodiments are only to illustrate the technical ideas of the present invention and cannot be used to limit the protection scope of the present invention. Any changes made based on the technical solutions based on the technical ideas proposed by the present invention will fall within the protection scope of the present invention. within.

Claims (9)

1. The landslide detection method based on the improved lightweight YOLOv7 is characterized by comprising the following specific steps of:

step 1, acquiring mountain satellite images;

step 2, preprocessing a mountain satellite image by using a super-resolution algorithm GAN, then performing splicing, rotation and corrosion operations on the image, dividing the processed mountain satellite image into a training set, a verification set and a test set, and marking a real frame and a category of the mountain satellite image;

step 3, constructing and training an improved lightweight YOLOv7 model, wherein the improved lightweight YOLOv7 model comprises a backbone network, a neck network and a detection head network; the backbone network comprises a MobileNet1 network, a MobileNet2 network, a MobileNet3 network and an MP-ELAN downsampling network;

step 4, inputting the mountain satellite image processed in the step 2 into a backbone network of an improved lightweight YOLOv7 model, and respectively extracting characteristic information of a MobileNet1 network, a MobileNet2 network, a MobileNet3 network and an MP-ELAN downsampling network to obtain four characteristic diagrams with different sizes;

step 5, inputting the four feature graphs with different sizes obtained in the step 4 into a neck network, wherein the neck network adopts a PAFPN feature pyramid, and four reinforcement feature graphs with different sizes are respectively obtained through feature extraction from top to bottom and feature fusion of an FPA reinforcement feature extraction network, and a HAT attention mechanism is introduced into the neck network;

step 6, the detection head network part comprises detection heads of structural re-parameterized convolution RepConv and four different target sizes; inputting the four reinforcement feature images with different sizes obtained in the step 5 into a detection head network respectively to obtain four prediction feature images with different sizes, calculating prior frames, generating three prior frames for the prediction feature images with each size after calculation, comparing the three prior frames with a real frame, selecting the prior frame with the largest intersection ratio IoU with the real frame for prediction, simulating different weather conditions for repeated training, and completing the training of an improved lightweight YOLOv7 network;

IoU the formula is as follows:

wherein intersectionarea represents the Area where two bounding boxes intersect; union_area represents the Area of the Union of two bounding boxes;

and 7, inputting the test set into a trained improved lightweight YOLOv7 model, sequentially passing through a backbone network, a neck network and a detection head network to respectively obtain four prediction feature diagrams with different sizes, and predicting and outputting a final target frame and a final target label through model reasoning to obtain landslide detection results.

2. The improved lightweight YOLOv 7-based landslide detection method of claim 1, wherein the preprocessing of mountain satellite images using super resolution algorithm GAN in step 2 is specifically as follows: GAN super-division processing is carried out on the acquired mountain satellite images, and then geometric transformation is carried out on the images, wherein the geometric transformation comprises random expansion, random cutting, random splicing and scaling to a fixed proportion.

3. The improved lightweight YOLOv 7-based landslide detection method of claim 1 wherein in step 2 the ratio of training set, validation set, test set is 8:1:1.

4. The improved lightweight YOLOv 7-based landslide detection method of claim 1 wherein step 2 uses LableImg tool to label the landscapes with the true boxes and categories to which they belong.

5. The improved lightweight YOLOv 7-based landslide detection method of claim 1 wherein HAT attention mechanism is split into two parts: a local window self-attention mechanism and a fusion channel attention mechanism;

local window self-attention mechanism: carrying out normalization processing on the input features, and dividing the input features into local windows through a window self-attention mechanism so as to focus local associated information;

fusion channel attention mechanism: introducing global information, weighting the features by using the global information, and activating more pixels through a channel attention mechanism to acquire more feature information.

6. The improved lightweight YOLOv 7-based landslide detection method of claim 1 wherein the PAFPN feature pyramid of step 5 comprises an SPPCSPC module, an ELAN-H module, UP-sampling, MP down-sampling, and a convolution layer, wherein the tail of the SPPCSPC module, ELAN-H module fusion introduces HAT attention mechanisms.

7. The improved lightweight YOLOv 7-based landslide detection method of claim 6, wherein the image 640 x 640 is obtained after processing in step 2, a 32-time downsampling feature map is output after passing through a backbone network, then the number of channels is changed from 1024 to 512 through an SPPCSPC module, and the channel number is fused with the results of the MobileNet1 and the MobileNet3 layers in a top-down manner to obtain a 160 x 160 size feature map; then carrying out feature fusion on the ELAN-H result and the SPPCSPC processing result in a bottom-up mode to obtain a feature map 20 x 20 size and a feature map 40 x 40 size; and then fusing the MobileNet2 layer information with the ELAN-H output characteristic diagram to obtain an 80 x 80 size characteristic diagram.

8. The improved lightweight YOLOv 7-based landslide detection method of claim 1, wherein in the training process, matching degree and loss value are calculated, the matching degree is calculated by comparing a predicted frame with IoU of a real label bounding box, and the matching degree of the predicted frame is calculated according to a judgment standard; then, positive and negative sample labels are given to all the prediction frames, corresponding real object labels are determined so as to facilitate the calculation of subsequent loss, after a loss value is calculated, a back propagation gradient is carried out by using the loss value so as to update the weight and deviation of an improved lightweight YOLOv7 network, and then a minimized loss function is obtained; the minimization loss function calculation formula is as follows:

FocalLoss(p _t )＝-α _t (1-p _t ) ^γ log(p _t )

wherein p is _t Represents the specific gravity of a difficult sample, (1-p) _t ) ^γ Called modulation factor, alpha _t Default to 0.25 and gamma default to 1.5.

9. The method for detecting landslide of improved lightweight YOLOv7 of claim 1 wherein to make the prediction block more accurate, an improved non-maximum suppression algorithm is introduced by first inputting b= { B ₁ ，b ₂ ，...，b _N }，S＝{s ₁ ，s ₂ ，...，s _N }，N _t The method comprises the steps of carrying out a first treatment on the surface of the Where B is a list of input prediction boxes, S is a confidence score for each prediction box, N _t A threshold setting that is non-maximum suppression; the formula for calculating the prediction frame score is as follows:

wherein M and σ are constants; s is(s) _i Is the confidence score of the ith prediction box; b _i Is the i-th prediction frame, i is E [1, N]；

And the confidence score of the prediction frame is calculated, so that the score is continuously updated, and the accuracy of the prediction frame is further improved.