CN107066995A - A kind of remote sensing images Bridges Detection based on convolutional neural networks - Google Patents

Abstract

The invention discloses a method for detecting bridges in remote sensing images based on a convolutional neural network. For remote sensing images with large data volumes and large image sizes, the detection of bridge positions using traditional methods has low efficiency and takes a long time. The present invention first establishes a convolutional neural network model, intercepts a bridge image with a size of w*h from a remote sensing image as a training sample, initializes each parameter in the convolutional neural network model, and inputs the training sample into the model for training. In the detection process, the remote sensing image to be detected is scanned with a window of w*h size according to the step length l, and the candidate window is obtained and the position information is marked. Finally, the candidate window is put into the model and the position of the bridge in the remote sensing image to be detected is output. Implement detection. The invention does not need to perform feature extraction of bridge pictures in advance, simplifies the detection steps, and greatly accelerates the detection speed of remote sensing images while maintaining a high detection rate.

Description

A method of bridge detection in remote sensing images based on convolutional neural network

technical field

The invention is applicable to the field of image recognition, and is mainly aimed at detecting bridges in remote sensing images, and is a method for detecting bridges in remote sensing images based on a convolutional neural network.

Background technique

Remote sensing image processing includes remote sensing image acquisition, denoising, enhancement, restoration, compression, segmentation, representation and description, target detection, etc. Among them, target detection, as an important part of remote sensing image processing, is of great significance in both military and civilian fields. In the military field, it is necessary to conduct military reconnaissance on the enemy and monitor one's own side. By performing target recognition on remote sensing images obtained by satellites, aviation or aerospace vehicles, it is possible to understand information such as terrain, equipment, and troop mobilization in the photographed area. The early target detection of remote sensing images was carried out manually, but due to the large amount of remote sensing image data usually obtained, if manual interpretation is used, repeated work is required, time-consuming and laborious, and the real-time performance is poor. In modern high-tech warfare, the battlefield situation changes rapidly. If the image processing speed is too slow, it will not be able to obtain key information in time, which will cause delays in fighter planes and cause heavy losses to one's own side. Therefore, it is very important for modern warfare to use fast automatic recognition technology for automatic target detection in remote sensing images. In addition to the important military value, remote sensing image target detection is also widely used in civil fields such as urban planning, establishment and update of geographic databases, and disaster assessment of natural disasters. As concepts such as global positioning system, geographic information system, and digital earth system have been put forward one after another, it is increasingly necessary to accurately detect and locate targets in remote sensing images. In addition, object detection in remote sensing images has become an urgent need in the precise drawing of two-dimensional or three-dimensional maps of cities, the detection of damage caused by natural disasters, and the detection of object changes.

At present, bridge target detection for remote sensing images mainly uses the saliency method to extract candidate regions and extract features, and uses classifiers to judge features to obtain detection results. The remote sensing image bridge target detection with the patent number CN200810232213.1 is trained and modeled by water features, so as to segment the remote sensing image water area, and perform bridge detection based on the segmented results. The bridge detection process needs to be designed for different bridges. template, then extract features and finally complete bridge detection.

Based on the above research status, the target of remote sensing image mainly has the following two problems: first, after preprocessing, the sample image is often extracted with artificially preset specific features such as the shape, aspect ratio or area of the connected region, which cannot To ensure that effective or important features are extracted, the influence of human experience is too great, and the actual application effect is not good; second, in order not to lose the detailed features of the image, sometimes the artificial preset feature extraction process is ignored, and all pixels in the image are directly As features, these features are used as the basic information of classifier training and classification, which is too cumbersome and will bring a lot of redundant information, which will reduce the detection efficiency.

Contents of the invention

Purpose of the invention: The purpose of the present invention is to utilize the advantages of convolutional neural networks in image processing, and propose a method for detecting bridge images in remote sensing images using convolutional neural networks. This method overcomes the shortcomings of low efficiency of the traditional method, and automatically mines the features in the image through the convolutional neural network, and finally realizes the detection of the bridge picture.

Technical solutions:

A method for detecting bridges in remote sensing images based on convolutional neural networks, comprising steps:

S1: training sample collection and preprocessing;

S1-1: Select the remote sensing image containing the bridge area, and manually intercept the bridge image with the size of w*h on the remote sensing image;

S1-2: In the region that does not contain bridges on the remote sensing image, intercept a picture with a size of w*h, and use it as a negative sample for the detector for training;

S1-3: Select the positive and negative samples obtained in steps S1-1 and S1-2, and under the premise of maintaining the size of the picture w*h, horizontally flip the positive and negative sample pictures, scale transformation, translation transformation, rotation transformation and Whitening operation;

S2: Establish a convolutional neural network training model to obtain a detector;

S2-1: Establish a convolutional neural network model, and initialize each parameter in the convolutional neural network model;

S2-2: put the positive and negative samples obtained in steps S1-1 and S1-2 into the convolutional neural network model obtained in S2-1, and perform iterative training;

S3: Preprocessing of detection samples:

Select the remote sensing image to be detected, start scanning from the upper left corner of the remote sensing image through the w*h size window, the horizontal scanning step is w/2, when scanning to the rightmost end of the image to be detected, follow the vertical scanning step h/2 Move down one line, and then scan from the far left according to the horizontal w/2 step length, and scan the complete remote sensing pictures in turn; record the position coordinates of the upper left corner of the candidate window obtained by each step of scanning, as the position information of the candidate picture;

S4: Input the detection sample into the detector to obtain the result;

S4-1: Use the candidate window obtained in step S3 as the input of the detector obtained in step S2 training, detect all candidate windows, record the candidate pictures judged to contain bridges by the detector, and save these candidate windows;

S4-2: Extract the position information contained in the saved candidate window, and then mark the image area represented by the candidate window on the picture to be detected according to the position information of the candidate window, and finally complete the detection of the position of the bridge in the remote sensing image .

In the step S1-1, when intercepting bridge pictures, it is necessary to select pictures with obvious bridge features, and at the same time to intercept bridge pictures containing bridges but not obvious features, being blocked or relatively blurred.

The convolutional neural network model established in step S2-1 includes an input layer, a convolutional layer, a pooling layer, a convolutional layer, a pooling layer, a fully connected layer, and an output layer;

1). The input layer takes positive and negative samples as input and inputs them into the convolutional neural network model;

2). The first stage of feature extraction: the convolution kernel size of the convolution layer is 5*5, the input is 3 channels, the output is 64 channels, and the moving step is 1; the pooling layer adopts the method of maximum pooling, and the window size is is 3*3, the step size is 2, and then normalize the obtained feature map;

3). Enter the second stage of feature extraction: the convolution kernel size of the convolution layer is still 5*5, input 64 channels, output 64 channels, step size is 1, and then normalize the feature map after convolution Perform pooling, the pooling method still adopts the maximum pooling, the window size is 3*3, and the step size is 2;

4). Finally, put the pooling result into the fully connected layer, and finally output it.

The weight update in the convolutional neural network model established by the step S2-1 is carried out using the BP backpropagation method; the method for updating the weight at each layer is the gradient descent method; the Learning Rate of the gradient descent method is set Between 0.003-0.004.

The final output of the convolutional neural network model established by the step S2-1 adopts Softmax as a binary classifier, and the Softmax regression is divided into two steps: the first step is to obtain evidence that a given picture belongs to a certain digital category, and the picture is The pixel values are weighted and summed; if this pixel has strong evidence that this picture does not belong to this category, then the corresponding weight is negative. On the contrary, if this pixel has favorable evidence to support that this picture belongs to this category, then the weight Values are positive numbers; that is:

Evidence _i represents the evidence that the given picture belongs to class i; where w _i represents the weight, b _i represents the bias of the digital i class, and j represents the pixel index of the given picture x for pixel summation; then use the Softmax function to put These evidences translate into probabilities y:

y=softmax(evidence)

Among them, Softmax is an activation function. Therefore, given a picture, its matching degree for each number is converted into a probability value by the Softmax function; the Softmax function is defined as:

softmax(x)=normalize(exp(x))

Expanding the subexpressions on the right-hand side of the equation, we get:

After using the Softmax classifier to obtain the result of a probability distribution, compare the result with the final label, and determine a threshold T through the comparison, which means that when the probability value in the Softmax training result is greater than T, then determine the input The picture contains a bridge; if the probability value in the training result is less than T, it is determined that the input picture does not contain a bridge.

In the iterative training process in the step S2-2, the strategy of cyclic training is adopted; a certain number of pictures are randomly selected from all sample pictures for training each time, the selected batch_size is 128, and then other samples of the same number are randomly selected Perform training, and gradually update the weights in the convolutional neural network model in a continuous loop process.

Beneficial effects: the present invention does not need to perform feature extraction of bridge pictures in advance, simplifies the detection steps, and greatly speeds up the detection speed of remote sensing images while maintaining a high detection rate.

Description of drawings

Fig. 1 is a flow chart of the method of the present invention.

Figure 2 is a structural diagram of a convolutional neural network.

detailed description

The present invention will be further described below in conjunction with the accompanying drawings.

The present invention is a kind of remote sensing image bridge detection method based on convolution neural network, mainly includes training stage and detection stage, and described training stage mainly includes the following steps:

S1: training sample collection and preprocessing;

S2: Establish a convolutional neural network training model to obtain a detector.

The detection phase mainly includes the following steps:

S3: preprocessing of the detection sample;

S4: The detection sample is input into the detector to obtain the result.

Further, the step S1 includes the following sub-steps:

S1-1: First select a part of the remote sensing image, and manually capture a bridge picture with a size of w*h on the remote sensing image. When intercepting the bridge picture, it is necessary to select a picture with obvious bridge features. At the same time, some bridges should also be intercepted. , but the features are not obvious, occluded or relatively blurred bridge pictures, which can ensure that after training positive samples, the detector also has a certain detection ability for bridge pictures with inconspicuous features.

S1-2: In the area that does not contain bridges on the remote sensing image, images of size w*h are also intercepted, and these images are used as negative samples of the detector for training.

S1-3: Select the positive and negative samples obtained in steps S1-1 and S1-2, and under the premise of maintaining the size of the picture w*h, horizontally flip the positive and negative sample pictures, scale transformation, translation transformation, rotation transformation and Whitening operation, which further increases the number of training samples, and also makes the features of the training pictures more.

Further, the step S2 includes the following sub-steps:

S2-1; First, a convolutional neural network model should be established. The structure of the entire model is an input layer, a convolutional layer, a pooling layer, a normalization layer, a convolutional layer, a normalization layer, a pooling layer, and a fully connected layer. output layer.

1). The input layer takes positive and negative samples as input and inputs them into the convolutional neural network model;

2). In the first stage of feature extraction, the convolution kernel size of the convolution layer is 5*5, the input is 3 channels, the output is 64 channels, and the moving step is 1. The pooling layer adopts the method of maximum pooling, and the window size is is 3*3, the step size is 2, and then normalize the obtained feature map;

3). Enter the second stage of feature extraction, the convolution kernel size of the convolution layer is still 5*5, input 64 channels, output 64 channels, step size is 1, and then normalize the feature map after convolution Perform pooling, the pooling method still adopts the maximum pooling, the window size is 3*3, and the step size is 2;

4). Finally, put the pooling result into the fully connected layer, and finally output it. As shown in Figure 2.

S2-2: After designing the model structure of the convolutional neural network, it is necessary to initialize each parameter in the network model. The randomness of the data initialization is as high as possible, so that the convergence speed during training will be faster and it is not easy to fall into local optimal results.

S2-3: The weight update in the convolutional neural network model is carried out by the BP backpropagation method. The BP backpropagation method compares the result calculated by the forward propagation with the target result and obtains the difference between the two results. Value, that is, the total error, according to the total error step by step, update the weight of each layer. The method of updating weights in each layer uses the gradient descent method, and uses the gradient descent method to calculate the optimal weight under the current error, and then updates the weights of the previous layer in turn after obtaining the optimal weight. The Learning Rate learning rate of the gradient descent method is set between 0.003-0.004.

S2-4: The final output uses Softmax as the binary classifier, and Softmax regression is divided into two steps: the first step is to obtain evidence (evidence) that a given picture belongs to a specific digital class, we weight and sum the pixel values of the picture . If the pixel has strong evidence that the image does not belong to the class, then the corresponding weight is negative, and on the contrary, if the pixel has favorable evidence that the image belongs to the class, the weight is positive. which is:

where w _i represents the weight, b _i represents the bias of the number i class, and j represents the pixel index of the given image x for pixel summation. Then use the Softmax function to convert these evidences into probability y:

y=softmax(evidence)

The Softmax here can be regarded as an activation function. Therefore, given a picture, its matching degree for each number can be converted into a probability value by the Softmax function. Softmax function can be defined as:

softmax(x)=normalize(exp(x))

Expanding the sub-expression on the right side of the equation, we can get:

The Softmax classifier takes the output value of the fully connected layer as the input value, and evaluates the input value as a power index, and then regularizes these result values. This power operation means that greater evidence corresponds to a greater multiplier weight value in the hypothetical model.

Conversely, having less evidence means having smaller multiplier coefficients in the hypothesis model. Assume that the weights in the model cannot be 0 or negative. Softmax then regularizes these weight values so that their sum equals 1, thereby constructing an efficient probability distribution.

S2-5: After using the Softmax classifier to obtain a probability distribution result, compare these results with the final label, and determine a threshold T by comparison, which means that when the probability value in the Softmax training result is greater than T , then it will be determined that the input picture contains bridges. If the probability value in the training result is less than T, it will be determined that the input picture does not contain bridges.

S2-6: Utilize the positive and negative samples obtained in steps S1-1 and S1-2, put these samples into the convolutional neural network model after initializing the parameters of each layer, and perform iterative training. During the training process, the strategy of putting all the training samples into the model at one time will not be adopted, which will make the model input too large, the calculation will be slower, and general equipment will not support it.

Therefore, a loop training strategy will be adopted in the training process, and a certain number of pictures are randomly selected from all sample pictures for training each time. The batch size selected here is 128, and then the same number of other samples are randomly selected for training. In the continuous loop process (the number of loops is set to 10000), the weights in the convolutional neural network model are gradually updated. This will not only improve the training speed and efficiency, but also increase the accuracy.

Further, the step S3 includes the following sub-steps:

S3-1: first select the remote sensing picture to be detected. For general remote sensing images, the size of the remote sensing image is very large, so in the present invention, one-eighth or one-tenth of a remote sensing picture to be detected will be intercepted , to detect a part of it each time and then detect other parts, so that the burden of the detector will be relatively small, easier to calculate, and more efficient.

S3-2: Select the remote sensing picture to be detected, start scanning from the upper left corner of the remote sensing picture through the w*h size window, the horizontal scanning step is w/2, when scanning to the rightmost end of the picture to be detected, follow the vertical scanning step Long h/2 moves down one line, and then scans from the far left according to the horizontal step of w/2, and scans the complete remote sensing pictures in turn.

S3-3: Each scan step obtains a candidate window, and during scanning, record the position coordinates of the upper left corner of each candidate window as the position information of the candidate picture. Therefore, the information contained in each candidate window should be the image area represented by the candidate window, the coordinates of the upper left corner (x, y), the width and height of the candidate picture (w, h), that is (image, x, y, w, h) .

Further, the step S4 includes the following sub-steps:

S4-1: Use the candidate windows obtained in step S3 as the input of the detector obtained in step S2 training, detect all candidate windows, record the candidate pictures judged to include bridges by the detector, and save these candidate windows.

S4-2: Extract the position information contained in the saved candidate window, then mark the image area represented by the candidate window on the picture to be detected according to the position information of the candidate window, and finally complete the detection of the position of the bridge in the remote sensing image .

Since the calculation speed of using CPU is relatively slow compared with that of GPU, GPU is used for training and calculation at the end, which greatly improves the training speed and detection efficiency.

The above is only a preferred embodiment of the present invention, it should be pointed out that for those of ordinary skill in the art, without departing from the principle of the present invention, some improvements and modifications can also be made, and these improvements and modifications are also possible. It should be regarded as the protection scope of the present invention.

Claims (6)

1. a kind of remote sensing images Bridges Detection based on convolutional neural networks, it is characterised in that：Including step：

S1：Training sample is gathered and pretreatment；

S1-1：The remote sensing images for including bridge area are chosen, manual interception size size is the bridge of w*h sizes on remote sensing images Beam picture；

S1-2：The region of bridge is not included on remote sensing images, interception size size is w*h picture, is used as the negative of detector Sample is trained；

S1-3：The positive negative sample obtained in selecting step S1-1, S1-2, on the premise of picture w*h sizes are kept, is aligned Negative sample picture carries out flip horizontal, change of scale, translation transformation, rotation transformation and whitening operation；

S2：Convolutional neural networks training pattern is set up, detector is obtained；

S2-1：Convolutional neural networks model is set up, and the parameters in convolutional neural networks model are initialized；

S2-2：The positive negative sample that step S1-1, S1-2 is obtained is put into the convolutional neural networks model that S2-1 is obtained, and is iterated Training；

S3：Detect the pretreatment of sample：

Remote sensing image to be detected is chosen, is scanned by w*h size windows since the upper left corner of remote sensing image, transversal scanning step A length of w/2, when the low order end of scanning to picture to be detected, a line is moved down according to longitudinal scanning step-length h/2, then from most left While starting the step scan according to horizontal w/2, a completely remote sensing image is scanned successively；Record the candidate that each step scanning is all obtained The position coordinates in the window upper left corner, is used as the positional information of candidate's picture；

S4：Detection sample input detector obtains result；

S4-1：The candidate window that step S3 is obtained trains the input of obtained detector as step S2, to all candidates Window is detected, records the candidate's picture for being judged as including bridge by detector, and preserve these candidate windows；

S4-2：The positional information that the candidate window of preservation is included is extracted, then according to candidate on picture to be detected The positional information of window marks the image-region representated by candidate window, is finally completed the inspection to remote sensing images Bridge position Survey work.

2. remote sensing images Bridges Detection according to claim 1, it is characterised in that：The step S1-1 is in interception bridge When beam picture, the obvious picture of bridge feature should be chosen, while also to intercept comprising bridge, but feature is not obvious, It is blocked or more fuzzy bridge picture.

3. remote sensing images Bridges Detection according to claim 1, it is characterised in that：The volume that the step S2-1 is set up Product neural network model includes input layer, convolutional layer, pond layer, convolutional layer, pond layer, full articulamentum and output layer；

1) input layers are, as input, to be input to positive negative sample in convolutional neural networks model；

2) the feature extractions first stage：The convolution kernel size of convolutional layer is 5*5, inputs 3 passages, exports 64 passages, mobile step A length of 1；Pond layer is carried out by the way of maximum pond, and window size is 3*3, and step-length is 2, then enters obtained characteristic pattern Row normalization；

3) enters feature extraction second stage：The convolution kernel size of convolutional layer remains 5*5, inputs 64 passages, and output 64 is led to Road, step-length is 1, then pond will be carried out after the characteristic pattern normalization operation after convolution, pond mode, which remains unchanged, takes maximum pond Change, window size is 3*3, and step-length is 2；

4) pond result is finally put into full articulamentum by, is finally exported.

4. remote sensing images Bridges Detection according to claim 1, it is characterised in that：The volume that the step S2-1 is set up Right value update in product neural network model is carried out using BP back propagations；Under every layer of method selection gradient for updating weights Drop method；The Learning Rate learning rates of the gradient descent method are arranged between 0.003-0.004.

5. remote sensing images Bridges Detection according to claim 1, it is characterised in that：The volume that the step S2-1 is set up The last output of product neural network model is using Softmax as two graders, and Softmax is returned in two steps：The first step in order to The evidence that a given picture belongs to some optional network specific digit class is obtained, summation is weighted to picture pixels value；If this picture There is element very strong evidence to illustrate that this pictures is not belonging to such, then corresponding weights are negative, if this opposite pixel Possessing favourable evidence supports this pictures to belong to this class, then weights are positive numbers；I.e.：

<mrow> <msub> <mi>evidence</mi> <mi>i</mi> </msub> <mo>=</mo> <munder> <mi>&amp;Sigma;</mi> <mi>j</mi> </munder> <msub> <mi>w</mi> <mrow> <mi>i</mi> <mo>,</mo> <mi>j</mi> </mrow> </msub> <msub> <mi>x</mi> <mi>j</mi> </msub> <mo>+</mo> <msub> <mi>b</mi> <mi>i</mi> </msub> </mrow>

evidence_iRepresent that given picture belongs to the evidence of i classes；Wherein w_iRepresent weight, b_iRepresent the amount of bias of numeral i classes, j The pixel index for representing given picture x is summed for pixel；Then these evidences can be converted into probability with Softmax functions y：

Y=softmax (evidence)

Wherein, Softmax is an excitation function, therefore, gives a pictures, it is for each digital goodness of fit quilt Softmax functions are converted into a probable value；Softmax functions are defined as：

Softmax (x)=normalize (exp (x))

Deploy the minor on the right of equation, obtain：

<mrow> <mi>s</mi> <mi>o</mi> <mi>f</mi> <mi>t</mi> <mi>max</mi> <msub> <mrow> <mo>(</mo> <mi>x</mi> <mo>)</mo> </mrow> <mi>i</mi> </msub> <mo>=</mo> <mfrac> <mrow> <mi>exp</mi> <mrow> <mo>(</mo> <msub> <mi>x</mi> <mi>i</mi> </msub> <mo>)</mo> </mrow> </mrow> <mrow> <msub> <mi>&amp;Sigma;</mi> <mi>j</mi> </msub> <mi>exp</mi> <mrow> <mo>(</mo> <msub> <mi>x</mi> <mi>j</mi> </msub> <mo>)</mo> </mrow> </mrow> </mfrac> <mo>;</mo> </mrow>

After the result of a probability distribution is obtained using Softmax graders, result is compared with final label, and A threshold value T is determined by comparing, the threshold value is represented when the probable value in Softmax training results is more than T, then judged defeated Enter and bridge is included in picture；If the probable value in training result is less than T, that judges not including bridge in input picture.

6. remote sensing images Bridges Detection according to claim 1, it is characterised in that：Iteration in the step S2-2 In training process, the strategy of circuit training is taken；A number of picture is randomly selected from all samples pictures every time to carry out Training, the batch_size sizes of selection are 128, then randomly select other same amount of samples and are trained, continuous In ground cyclic process, the weights in convolutional neural networks model are gradually updated.