WO2018214195A1

WO2018214195A1 - Remote sensing imaging bridge detection method based on convolutional neural network

Info

Publication number: WO2018214195A1
Application number: PCT/CN2017/089134
Authority: WO
Inventors: 刘兵; 周勇; 郑成浩; 王重秋
Original assignee: 中国矿业大学
Priority date: 2017-05-25
Filing date: 2017-06-20
Publication date: 2018-11-29
Also published as: CN107066995A

Abstract

Disclosed is a remote sensing imaging bridge detection method based on a convolutional neural network. With respect to remote sensing imaging having larger data quantities and larger image sizes, using conventional methods to detect locations of bridges therein is inefficient and time-consuming. The method first establishes a convolutional neural network model and captures from a remote sensing image a bridge image having a size w*h to serve as a training sample; each parameter of the convolutional neural network model is initialized; and the training sample is input into the model for training. During a detection process, a remote sensing image to be detected is scanned by using a window having a size w*h according to a step size of 1, so as to obtain a candidate window and to mark location information accordingly, and a bridge location in the remote sensing image to be detected is output after the candidate window is placed into the model, thus realizing detection. The method does not require performance of feature extraction on bridge images in advance, simplifying detection steps, and significantly accelerating processing of remote sensing images while also maintaining a high detection rate.

Description

Remote sensing image bridge detection method based on convolutional neural network

Technical field

The invention is suitable for the field of image recognition, and is mainly for detecting bridges in remote sensing images, and is a bridge detection method for remote sensing images based on convolutional neural networks.

Background technique

Remote sensing image processing includes acquisition, denoising, enhancement, restoration, compression, segmentation, representation and description, target detection, and the like of remote sensing images. Among them, target detection, as an important part of remote sensing image processing, has important significance in both military and civilian fields. In the military field, it is necessary to conduct military reconnaissance of the enemy and monitor the enemy. Target recognition of remote sensing images obtained from satellite, aerospace or aerospace vehicles can be used to understand the terrain, equipment, and troop movements of the captured area. Early remote sensing image target detection was performed manually. However, since the amount of remotely received image data is usually large, if manual interpretation is used, it is necessary to repeat the work, which is time consuming and laborious, and has poor real-time performance. In modern high-tech warfare, the battlefield situation is changing rapidly. If the image processing speed is too slow, it will not be able to obtain key information in time, resulting in delays in the aircraft and causing significant losses to the party. Therefore, the use of fast automatic identification technology for automatic target detection of remote sensing images is very important for modern warfare. In addition to the important value of the military, remote sensing image target detection has been widely used in other fields such as urban planning, establishment and update of geographic databases, and disaster assessment of natural disasters. With the concept of global positioning system, geographic information system, digital earth system and so on, it is increasingly necessary to accurately detect and locate targets in remote sensing images. In addition, remote sensing image target detection has become an urgent need in accurately mapping two-dimensional or three-dimensional maps of cities, detection of damage caused by natural disasters, and detection of changes in targets.

At present, the detection of bridge targets for remote sensing images mainly uses the saliency method to extract candidate regions and extract features, and uses the classifier to judge the features to obtain the detection results. The remote sensing image of the patent number CN200810232213.1 is used to train and model the waters by using the water features to simulate the waters of the remote sensing image. The bridge detection is performed for the segmented results. The bridge design needs to be different for different bridges. The template then extracts the features and finally completes the bridge detection.

Based on the above research status, the main problems of remote sensing images are as follows: First, after pre-processing, the sample images are often extracted from the shape, aspect ratio or area of the artificial image. To ensure that effective or important features are extracted, the influence of human experience is too great, and the actual application effect is not good. Second, In order not to lose the detailed features of the image, sometimes the process of artificially preset feature extraction is ignored, and all the pixels in the image are directly used as features, and then these features are used as the basic information for classifier training and classification, which is too cumbersome to bring A large amount of redundant information is generated, which reduces the detection efficiency.

Summary of the invention

OBJECT OF THE INVENTION: The object of the present invention is to utilize the advantages of convolutional neural networks in image processing, and to propose a method for detecting bridge images in remote sensing images by using a convolutional neural network. The method overcomes the shortcomings of the traditional method, and automatically mines the features in the image through the convolutional neural network, and finally realizes the detection of the bridge image.

Technical solutions:

A method for detecting remote sensing image bridge based on convolutional neural network, comprising the steps of:

S1: training sample collection and preprocessing;

S1-1: Selecting a remote sensing image including a bridge area, and manually capturing a bridge image of size w*h on the remote sensing image;

S1-2: In the region where the bridge is not included in the remote sensing image, the image with the size of w*h is intercepted and trained as a negative sample of the detector;

S1-3: selecting the positive and negative samples obtained in steps S1-1 and S1-2, and performing horizontal flipping, scale transformation, translation transformation, rotation transformation and the positive and negative sample images under the premise of maintaining the size of the image w*h. Whitening operation;

S2: establishing a convolutional neural network training model to obtain a detector;

S2-1: Establish a convolutional neural network model and initialize various parameters in the convolutional neural network model;

S2-2: putting the positive and negative samples obtained in steps S1-1 and S1-2 into the convolutional neural network model obtained in S2-1, and performing iterative training;

S3: Pretreatment of the test sample:

Select the remote sensing image to be detected, and scan from the upper left corner of the remote sensing image through the w*h size window. The horizontal scanning step is w/2. When scanning to the far right end of the image to be detected, according to the vertical scanning step length h/2 Move one line downward, then scan from the leftmost side according to the step size of horizontal w/2, scan the complete remote sensing picture in turn; record the position coordinates of the upper left corner of the candidate window obtained by each step scanning, as the position information of the candidate picture;

S4: detecting the sample input detector to obtain a result;

S4-1: the candidate window obtained in step S3 is used as the input of the detector trained in step S2, and all the candidate windows are detected, and the candidate images determined by the detector to be included in the bridge are recorded, and the candidate windows are saved;

S4-2: extracting the position information included in the saved candidate window, and then marking the image area represented by the candidate window according to the position information of the candidate window on the picture to be detected, and finally completing the detection of the bridge position in the remote sensing image. .

In the step S1-1, when the bridge picture is intercepted, it is necessary to select a picture with obvious bridge characteristics, and also to intercept the bridge picture including the bridge, but the feature is not obvious, occluded or blurred.

The convolutional neural network model established in the step S2-1 includes an input layer, a convolution layer, a pooling layer, a convolution layer, a pooling layer, a fully connected layer, and an output layer;

1). The input layer takes the positive and negative samples as input and inputs them into the convolutional neural network model;

2). Feature extraction first stage: convolutional layer convolution kernel size is 5*5, input 3 channels, output 64 channels, moving step size is 1; pooling layer is performed by maximum pooling, window size 3*3, the step size is 2, and then the obtained feature map is normalized;

3). Enter the second stage of feature extraction: the convolutional layer of the convolutional layer is still 5*5, input 64 channels, output 64 channels, step size is 1, and then normalize the convolved feature map For pooling, the pooling method still adopts the maximum pooling, the window size is 3*3, and the step size is 2.

4). Finally, the pooled result is placed in the full connection layer, and finally output.

The weight update in the convolutional neural network model established in the step S2-1 is performed by the BP back propagation method; the gradient descent method is selected in the method of updating the weight in each layer; and the learning rate setting of the gradient descent method is set. Between 0.003-0.004.

The final output of the convolutional neural network model established in the step S2-1 adopts Softmax as the second classifier, and the Softmax regression is divided into two steps: the first step is to obtain evidence that a given picture belongs to a specific digital class, and the picture is The pixel value is weighted and summed; if the pixel has strong evidence that the picture does not belong to the class, then the corresponding weight is negative, and if the pixel has favorable evidence to support the picture belongs to this class, then the right The value is a positive number; that is:

Evidence _i represents evidence that a given picture belongs to class i; where w _i represents the weight, b _i represents the offset of the class i, and j represents the pixel index of the given picture x for pixel summation; then the Softmax function can be used These evidences are converted into probabilities y:

y=softmax(evidence)

Among them, Softmax is an excitation function, therefore, given a picture, its fit for each number is converted into a probability value by the Softmax function; the Softmax function is defined as:

Softmax(x)=normalize(exp(x))

Expand the sub-form on the right side of the equation to get:

After obtaining the result of a probability distribution by using the Softmax classifier, the result is compared with the final label, and a threshold T is determined by comparison, which indicates that when the probability value in the Softmax training result is greater than T, then the input is determined. The picture contains a bridge; if the probability value in the training result is less than T, the decision input picture does not contain a bridge.

In the iterative training process in step S2-2, a loop training strategy is adopted; each time a certain number of pictures are randomly selected from all sample pictures for training, the selected batch_size size is 128, and then the same number of other samples are randomly selected. Training is carried out to gradually update the weights in the convolutional neural network model during the continuous cycle.

Advantageous Effects: The present invention eliminates the need for feature extraction of bridge pictures in advance, simplifies the detection steps, and greatly speeds up the detection speed of remote sensing images while maintaining a high detection rate.

DRAWINGS

Figure 1 is a flow chart of the method of the present invention.

Figure 2 is a block diagram of a convolutional neural network.

detailed description

The present invention will be further described below in conjunction with the accompanying drawings.

The invention relates to a remote sensing image bridge detection method based on a convolutional neural network, which mainly comprises a training phase and a detection phase, and the training phase mainly comprises the following steps:

S1: training sample collection and preprocessing;

S2: Establish a convolutional neural network training model to obtain a detector.

The detection phase mainly includes the following steps:

S3: preprocessing of the test sample;

S4: Detecting the sample input detector to obtain a result.

Further, the step S1 described includes the following sub-steps:

S1-1: First select a part of the remote sensing image, and manually intercept the bridge image with the size of w*h on the remote sensing image. When intercepting the bridge image, it is necessary to select the bridge image with obvious features. At the same time, some bridges should also be intercepted. However, the feature is not obvious, the occluded or relatively blurred bridge picture, this can ensure that after training the positive sample, the detector also has a certain detection ability for the bridge picture with insignificant features.

S1-2: In the area where the bridge is not included in the remote sensing image, pictures of size w*h are also intercepted, and these pictures are trained as negative samples of the detector.

S1-3: selecting the positive and negative samples obtained in step S1-1, S1-2, and performing horizontal flipping, scale transformation, translation transformation, rotation transformation and positive and negative sample images under the premise of maintaining the size of the image w*h. The whitening operation further increases the number of training samples and also makes the characteristics of the training pictures more.

Further, the step S2 described includes the following sub-steps:

S2-1; Firstly, a convolutional neural network model should be established. The structure of the whole model is input layer, convolution layer, pooling layer, layer, convolution layer, normalization layer, pooling layer, fully connected layer, Output layer.

2). In the first stage of feature extraction, the convolution kernel of the convolutional layer is 5*5, the input is 3 channels, the output is 64 channels, the moving step is 1, and the pooling layer is performed by the maximum pooling method. 3*3, the step size is 2, and then the obtained feature map is normalized;

3). In the second stage of feature extraction, the convolution kernel of the convolutional layer is still 5*5, input 64 channels, output 64 channels, step size is 1, and then normalize the convolved feature map. For pooling, the pooling method still adopts the maximum pooling, the window size is 3*3, and the step size is 2.

4). Finally, the pooled result is placed in the full connection layer, and finally output. As shown in Figure 2.

S2-2: After designing the model structure of the convolutional neural network, it is necessary to initialize the parameters in the network model. When the data is initialized, the randomness is as high as possible, so that the convergence speed will be faster and not easy to fall into during training. Locally optimal results.

S2-3: The weight update in the convolutional neural network model is performed by the BP back propagation method. The BP back propagation method compares the result of the forward propagation calculation with the target result, and obtains the difference between the two results. The value, the total error, is stepped forward based on the total error, updating the weight of each layer. The method of updating the weights in each layer selects the gradient descent method, and uses the gradient descent method to calculate the optimal weight under the current error. After obtaining the optimal weight, the weight of the previous layer is updated in turn. The Learning Rate of the gradient descent method is set between 0.003-0.004.

S2-4: The final output uses Softmax as the second classifier. Softmax regression is divided into two steps: the first step is to get To the evidence that a given picture belongs to a particular class of numbers, we weight-sum the picture pixel values. If the pixel has strong evidence that the image does not belong to the class, then the corresponding weight is negative, and if the pixel has favorable evidence to support the image belongs to this class, then the weight is positive. which is:

Where w _i represents the weight, b _i represents the offset of the class i, and j represents the pixel index of the given picture x for pixel summation. Then use the Softmax function to convert this evidence into a probability y:

y=softmax(evidence)

Here Softmax can be seen as an activation function, so given a picture, its fit for each number can be converted to a probability value by the Softmax function. The Softmax function can be defined as:

Softmax(x)=normalize(exp(x))

Expand the sub-form on the right side of the equation to get:

The Softmax classifier takes the output value of the fully connected layer as the input value, and evaluates the input value as a power exponent, and then normalizes these result values. This power operation indicates that the larger evidence corresponds to the multiplier weight value in the larger hypothesis model.

Conversely, having less evidence means having a smaller multiplier coefficient in the hypothetical model. Suppose the weights in the model cannot be zero or negative. Softmax then regularizes these weight values so that their sum equals one to construct a valid probability distribution.

S2-5: After obtaining the result of a probability distribution by using the Softmax classifier, the results are compared with the final label, and a threshold T is determined by comparison, which indicates that the probability value in the Softmax training result is greater than T Then, it will be determined that the input picture contains a bridge. If the probability value in the training result is less than T, the decision input picture does not include a bridge.

S2-6: Using the positive and negative samples obtained in steps S1-1 and S1-2, the samples are placed into the convolutional neural network model after initializing the parameters of each layer, and iterative training is performed. During the training process, the strategy of putting all the training samples into the model at one time is not taken. This will make the model input too large, the calculation will be slow, and the general equipment will not support it.

Therefore, the training process will adopt a loop training strategy. Each time, a certain number of pictures are randomly selected from all sample pictures for training. The size of the batch_size selected here is 128, and then the same number of other samples are randomly selected for training. During the loop (the number of loops is set to 10000), the weights in the convolutional neural network model are gradually updated, which not only improves the training speed and efficiency, but also improves the accuracy.

Further, the step S3 includes the following sub-steps:

S3-1: Firstly, the remote sensing image to be detected is selected. For a general remote sensing image, the size of the remote sensing image is very large, so in the present invention, one eighth or one tenth of the remote sensing image to be detected is intercepted. Each time a part of it is detected and then other parts are detected, the detector will be less burdensome to detect, easier to calculate, and more efficient.

S3-2: Select the remote sensing image to be detected, and start scanning from the upper left corner of the remote sensing image through the w*h size window. The horizontal scanning step is w/2. When scanning to the rightmost end of the image to be detected, follow the vertical scanning step. The long h/2 moves down one line, and then scans from the leftmost side according to the step size of the horizontal w/2, and sequentially scans the complete remote sensing picture.

S3-3: Each candidate scan obtains a candidate window. When scanning, the position coordinates of the upper left corner of each candidate window are recorded as position information of the candidate image. Therefore, each candidate window contains information that should be the image area image represented by the candidate window, the upper left coordinate (x, y), and the width and height (w, h) of the candidate image, ie (image, x, y, w, h) .

Further, the step S4 includes the following sub-steps:

S4-1: The candidate window obtained in step S3 is used as an input of the detector trained in step S2, and all the candidate windows are detected, and the candidate pictures judged to include the bridge by the detector are recorded, and the candidate windows are saved.

Since the CPU calculation speed is relatively slow compared to the GPU, the GPU is used for training and calculation at the end, which greatly improves the training speed and greatly improves the detection efficiency.

The above description is only a preferred embodiment of the present invention, and it should be noted that those skilled in the art can also make several improvements and retouchings without departing from the principles of the present invention. It should be considered as the scope of protection of the present invention.

Claims

A method for detecting remote sensing image bridge based on convolutional neural network, which comprises the following steps:

S1: training sample collection and preprocessing;

S1-1: Selecting a remote sensing image including a bridge area, and manually capturing a bridge image of size w*h on the remote sensing image;

S1-2: In the region where the bridge is not included in the remote sensing image, the image with the size of w*h is intercepted and trained as a negative sample of the detector;

S1-3: selecting the positive and negative samples obtained in steps S1-1 and S1-2, and performing horizontal flipping, scale transformation, translation transformation, rotation transformation and the positive and negative sample images under the premise of maintaining the size of the image w*h. Whitening operation;

S2: establishing a convolutional neural network training model to obtain a detector;

S2-1: Establish a convolutional neural network model and initialize various parameters in the convolutional neural network model;

S2-2: putting the positive and negative samples obtained in steps S1-1 and S1-2 into the convolutional neural network model obtained in S2-1, and performing iterative training;

S3: Pretreatment of the test sample:

Select the remote sensing image to be detected, and scan from the upper left corner of the remote sensing image through the w*h size window. The horizontal scanning step is w/2. When scanning to the far right end of the image to be detected, according to the vertical scanning step length h/2 Move one line downward, then scan from the leftmost side according to the step size of horizontal w/2, scan the complete remote sensing picture in turn; record the position coordinates of the upper left corner of the candidate window obtained by each step scanning, as the position information of the candidate picture;

S4: detecting the sample input detector to obtain a result;

S4-1: the candidate window obtained in step S3 is used as the input of the detector trained in step S2, and all the candidate windows are detected, and the candidate images determined by the detector to be included in the bridge are recorded, and the candidate windows are saved;

S4-2: extracting the position information included in the saved candidate window, and then marking the image area represented by the candidate window according to the position information of the candidate window on the picture to be detected, and finally completing the detection of the bridge position in the remote sensing image. .
The remote sensing image bridge detecting method according to claim 1, wherein in the step S1-1, when the bridge picture is intercepted, it is necessary to select a picture with obvious bridge characteristics, and also to intercept the included bridge, but the feature is not obvious. A picture of a bridge that is obscured or obscured.
The remote sensing image bridge detecting method according to claim 1, wherein said step S2-1 is constructed The vertical convolutional neural network model includes an input layer, a convolution layer, a pooling layer, a convolution layer, a pooling layer, a fully connected layer, and an output layer;

1). The input layer takes the positive and negative samples as input and inputs them into the convolutional neural network model;

2). Feature extraction first stage: convolutional layer convolution kernel size is 5*5, input 3 channels, output 64 channels, moving step size is 1; pooling layer is performed by maximum pooling, window size 3*3, the step size is 2, and then the obtained feature map is normalized;

3). Enter the second stage of feature extraction: the convolutional layer of the convolutional layer is still 5*5, input 64 channels, output 64 channels, step size is 1, and then normalize the convolved feature map For pooling, the pooling method still adopts the maximum pooling, the window size is 3*3, and the step size is 2.

4). Finally, the pooled result is placed in the full connection layer, and finally output.
The remote sensing image bridge detection method according to claim 1, wherein the weight update in the convolutional neural network model established in step S2-1 is performed by BP back propagation method; The method uses a gradient descent method; the learning rate of the learning rate of the gradient descent method is set between 0.003-0.004.
The remote sensing image bridge detecting method according to claim 1, wherein the final output of the convolutional neural network model established in the step S2-1 adopts Softmax as the second classifier, and the Softmax regression is divided into two steps: the first step is Obtain evidence that a given picture belongs to a particular number class, and weight the image pixel values; if the pixel has strong evidence that the picture does not belong to the class, then the corresponding weight is negative, instead If the pixel has favorable evidence to support that the picture belongs to this class, then the weight is a positive number;

Evidence i represents evidence that a given picture belongs to class i; where w i represents the weight, b i represents the offset of the class i, and j represents the pixel index of the given picture x for pixel summation; then the Softmax function can be used These evidences are converted into probabilities y:

y=softmax(evidence)

Among them, Softmax is an excitation function, therefore, given a picture, its fit for each number is converted into a probability value by the Softmax function; the Softmax function is defined as:

Softmax(x)=normalize(exp(x))

Expand the sub-form on the right side of the equation to get:

After obtaining the result of a probability distribution by using the Softmax classifier, the result is compared with the final label, and a threshold T is determined by comparison, which indicates that when the probability value in the Softmax training result is greater than T, then the input is determined. The picture contains a bridge; if the probability value in the training result is less than T, the decision input picture does not contain a bridge.
The remote sensing image bridge detection method according to claim 1, wherein in the iterative training process in step S2-2, a loop training strategy is adopted; each time a certain number of pictures are randomly selected from all sample pictures. Training, the selected batch_size is 128, and then randomly select the same number of other samples for training, and gradually update the weights in the convolutional neural network model during the continuous loop.