CN107301420A - A kind of thermal infrared imagery object detection method based on significance analysis - Google Patents

Abstract

The invention discloses a thermal infrared image target detection method based on saliency analysis, comprising: S1 constructing a global saliency map of a thermal infrared image; S2 constructing a local saliency map of a thermal infrared image; S3 constructing a fusion saliency map of a thermal infrared image; For infrared image target detection, threshold segmentation is performed on the fused saliency map to obtain a binary result map of the marked target position. The method of the invention makes up for the deficiencies of morphological filtering, median filtering and two-dimensional least square method, and can be used for image enhancement of thermal infrared images, target detection and the like. The invention is based on the gray scale feature and direction feature of the single-band image, is not limited to the number of channels of the thermal infrared imager, and is applicable to single-band and multi-band thermal infrared images at the same time; the detection accuracy is high, no auxiliary information and manual intervention are required, and the calculation speed is fast , which can be automated.

Description

A thermal infrared image target detection method based on saliency analysis

technical field

The invention is based on the field of remote sensing image technology processing, and in particular relates to a thermal infrared image target detection method based on saliency analysis.

Background technique

Thermal infrared imaging technology obtains the temperature information of the object by receiving the radiation information of the object in the thermal infrared band, so that it has the ability to detect thermal anomalies. Its good working performance at night and in bad weather conditions makes thermal infrared target detection technology widely used in target recognition and tracking, alarm systems, and forest fire monitoring. Thermal infrared images have low signal-to-noise ratio and lack texture information, so background suppression or target modeling methods are generally used for target detection.

Traditional thermal infrared image target detection methods include morphological filtering, median filtering, two-dimensional least squares method, etc. When the signal-to-noise ratio of the thermal infrared image is low, the target structure information is weak, and the noise interference is serious, the morphological filter can easily reduce the signal-to-noise ratio of the original image, or even lose the target. However, the window size of the median filter is difficult to determine, and it will damage the important details in the image while filtering out the noise, and the preservation of the target edge is poor. When the two-dimensional least squares method is applied to thermal infrared image target detection, when the filter is located in the target area with a small correlation range, it is prone to non-convergence, which leads to inaccurate estimation of target pixels. Traditional thermal infrared image target detection algorithms are based on specific targets and rely on the prior knowledge of the target to set the filter window, which lacks versatility.

Contents of the invention

The object of the present invention is to provide a thermal infrared image target detection method based on saliency analysis for thermal infrared remote sensing images.

In order to achieve the above object, a thermal infrared image target detection method based on saliency analysis of the present invention comprises the following steps:

Step 1, read the thermal infrared image and construct the global saliency map of the thermal infrared image;

Step 2, read the thermal infrared image, construct the local saliency map of the thermal infrared image, this step further includes the following sub-steps;

Step 2.1, extracting the grayscale feature map of the thermal infrared image;

Step 2.2, extracting the thermal infrared image direction feature map;

Step 2.3, fusing the grayscale feature map and the direction feature map to construct a local saliency map of the thermal infrared image, and normalize the local saliency map;

Step 3, normalize the global saliency map in step 1, combine the normalized global saliency map and local saliency map to construct a thermal infrared image fusion saliency map, the specific implementation method is as follows,

The calculation formula of the fused saliency map is S=S _G οS _L , where ο can be any one of [+, *, max], that is, for the normalized global saliency map S _G and the normalized local saliency map S _L Perform any one of addition, multiplication, and maximum value operations, and perform normalization operations on the resulting fused saliency map.

Step 4, thermal infrared image target detection, performing threshold segmentation on the fused saliency map to obtain a binary result map of the marked target position.

Further, the implementation of step 1 is as follows,

Step 1.1, read the thermal infrared image, and perform Gaussian filter preprocessing on the thermal infrared image to remove the influence of noise on target detection;

Step 1.2, calculate the global saliency map, the formula is in In order to read the average gray value of the thermal infrared image, I(x, y) is the gray value of the thermal infrared image at the pixel (x, y), and || || represents the calculation of the Euclidean formula between two gray values distance.

Further, the implementation of step 2.1 is as follows,

In step 2.1.1, the thermal infrared image is smoothed and filtered by Gaussian functions of different scales to obtain a grayscale image pyramid from scale 0 to scale 8: define the Gaussian convolution kernel, and the Gaussian convolution kernel formula is σ standard deviation, μ mean value μ=0, scale factor x=[1,2,...8], the thermal infrared image and the Gaussian convolution kernel are sequentially convolved, downsampled step by step, and the scale from 0 to 8-scale grayscale image pyramid;

In step 2.1.2, use the central and surrounding difference operation to calculate the grayscale feature map corresponding to the thermal infrared image, and the central and surrounding difference operation formula is expressed as Among them, I(c) represents the central scale image of the gray-scale feature pyramid image, and I(s) represents the surrounding scale image of the gray-scale feature pyramid image, Operate for the difference around the center.

Further, the implementation of step 2.2 is as follows,

Step 2.2.1, define the two-dimensional Gabor function, the formula is Where (x, y) represents the pixel coordinates, σ _x and σ _y represent the standard deviation of the Gaussian function in the x and y directions, and the four directions of 0°, 45°, 90° and 135° are selected for filtering, and the obtained Gabor filter template , the formula is h(x,y)=g(x',y')cos(2πω _f x'), where (x',y')=(x cos(θ _f )+y sin(θ _f )- x sin(θ _f )+y cosn(θ _f )), ω _f is the modulation frequency, θ _f ∈ {0°,45°,90°,135°}, the filter template is convolved with the thermal infrared image, Obtain four direction features of 0°, 45°, 90° and 135°;

Step 2.2.2, repeat step 2.1.1 for the four thermal infrared direction features of 0°, 45°, 90° and 135° to establish the directions from scale 0 to scale 8 corresponding to 0°, 45°, 90° and 135° feature pyramid;

Step 2.2.3, for each direction feature pyramid in the above four thermal infrared directions, select the central scale image I(c) and the surrounding scale image I(s) to perform central and surrounding difference calculations to obtain 0°, 45°, 90° °, 135°, a total of four groups of sub-directional feature maps;

In step 2.2.4, the four groups of sub-direction feature maps are linearly fused to obtain the final direction feature map.

Further, the implementation of step 2.3 is as follows:

Step 2.3.1, perform normalization operation on the grayscale feature map, the specific method is to separately count the maximum value max and the minimum value min of the pixel grayscale in the grayscale feature map, and reset the new pixel value of the pixel x in the grayscale feature map for

Step 2.3.2, repeat step 2.3.1 to normalize the direction feature map;

Step 2.3.3, calculate the local saliency map, add the normalized grayscale feature map and direction feature map pixel by pixel, resample to the same size as the original thermal infrared image, and obtain the local saliency map after normalization .

Further, the threshold segmentation in step 4 is calculated by the formula T=m+kσ, where m is the mean value of the fused saliency map, σ is the variance of the fused saliency map, and k is the empirical threshold.

Further, the value of the empirical threshold k is 5-10.

Further, the binary result map of the marked target position obtained in step 4 is realized in the following manner, a binary result map of the same size as the original thermal infrared image is established, and the pixels at the coordinates (x, y) of the fusion saliency map are compared one by one. The size of the grayscale value and the threshold value T, if it is greater than the threshold value T, then assign the corresponding value of 1 to the pixel at (x, y) in the binary result map.

The present invention has following advantage and beneficial effect:

(1) High detection accuracy and less misjudgment; it is also applicable to thermal infrared images collected by different imaging platforms.

(2) The detection is based on the grayscale and directional features of single-band images, and is not limited by the number of channels of the thermal infrared imager, and is applicable to single-band and multi-band thermal infrared images at the same time.

(3) No auxiliary information and manual intervention are needed, the calculation speed is fast, and it can be processed automatically.

(4) Strong scalability. When conditions permit, the method of the present invention can be used as an initial criterion, and a compound decision-making method can be used to obtain more accurate thermal infrared target detection results.

Description of drawings

Fig. 1 is a flowchart of an embodiment of the present invention;

Fig. 2 is a thermal infrared target detection result diagram according to an embodiment of the present invention, (a) is a thermal infrared image containing a vehicle engine target, (b) is a fusion saliency map corresponding to the image, and (c) is a binary value obtained by segmenting and fusion saliency map Result map, white indicates target.

Fig. 3 is a thermal infrared target detection result diagram according to an embodiment of the present invention, (a) is a thermal infrared image containing a pedestrian target, (b) is a fusion saliency map corresponding to the image, and (c) is a binary result obtained by segmentation and fusion saliency map Figure, white indicates the target.

detailed description

In order to better understand the technical solutions of the present invention, the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments.

This specific implementation mode is implemented by Matlab language programming under the Matlab b2011 environment, and the whole process can realize automatic processing. The specific implementation of this step is as follows:

Step 1, read the thermal infrared image, construct the global saliency map of the thermal infrared image, this step further includes:

Step 1.1, read the thermal infrared image, and perform Gaussian filter preprocessing on the thermal infrared image to remove the influence of noise on target detection.

Step 1.2, calculate the global saliency map, the formula in In order to read the average gray value of the thermal infrared image, I(x, y) is the gray value of the thermal infrared image at the pixel (x, y), and || || represents the calculation of the Euclidean formula between two gray values distance.

Step 2, constructing the local saliency map of the thermal infrared image, the specific implementation of this step is as follows:

Step 2.1, extracting the grayscale feature map of the thermal infrared image, which further includes:

In step 2.1.1, the grayscale feature pyramid image of the thermal infrared image is established. The thermal infrared image is smoothed and filtered by Gaussian functions of different scales to obtain grayscale image pyramids from scale 0 to scale 8. Define the Gaussian convolution kernel, the Gaussian convolution kernel formula is: The parameters are specifically set as standard deviation σ=2, mean value μ=0, scale factor x=[1,2,...8]. The thermal infrared image and the Gaussian convolution kernel are sequentially convolved, down-sampled step by step, and a gray-scale image pyramid from scale 0 (original image) to scale 8 is established;

In step 2.1.2, the grayscale feature map corresponding to the thermal infrared image is calculated by using the difference operation around the center. The operating formula of the difference around the center is expressed as Among them, I(c) represents the central scale image of the gray-scale feature pyramid image, and I(s) represents the surrounding scale image of the gray-scale feature pyramid image, Operate for the difference around the center. The specific operation method is to up-sample the image I(s) to the same resolution as I(c), then perform pixel-by-pixel subtraction to obtain the grayscale feature map, where I(c) can choose a 0-2 scale image, and I( s) 3-5 scale images can be selected. .

Step 2.2, extracting the thermal infrared image direction feature map, which further includes:

Step 2.2.1, thermal infrared image Gabor direction feature extraction, define two-dimensional Gabor function, the formula is: Where (x, y) represents the pixel coordinates, σ _x and σ _y represent the standard deviation of the Gaussian function in the x and y directions, and four directions of 0°, 45°, 90° and 135° are selected for filtering, and the obtained Gabor filter template , the formula is h(x,y)=g(x',y')cos(2πω _f x'), where (x',y')=(x cos(θ _f )+y sin(θ _f )- x sin(θ _f )+y cosn(θ _f )), ω _f is the modulation frequency, θ _f ∈ {0°,45°,90°,135°}, the filter template is convolved with the thermal infrared image, Obtain four direction features of 0°, 45°, 90° and 135°;

Step 2.2.2, establish the direction feature pyramid, similar to step 2.1.1, for the four thermal infrared direction features of 0°, 45°, 90° and 135°, establish 0°, 45°, 90° and 135° in sequence Corresponding 0-scale (original image) to 8-scale directional feature pyramid;

In step 2.2.3, use the central and surrounding difference operation to calculate the direction feature map. Similar to step 2.1.2, select the central scale image I(c) and the surrounding scale image I(s) in each direction feature pyramid to perform the central and surrounding difference After calculation, a total of four groups of sub-direction feature maps of 0°, 45°, 90°, and 135° are obtained.

In step 2.2.4, the four groups of sub-directional feature maps are linearly fused to obtain the final directional feature map.

Step 2.3, fusing the grayscale feature map and the direction feature map to construct a local saliency map, further including:

In step 2.3.1, the normalization operation is performed on the grayscale feature map and the direction feature map respectively. The specific method is to count the maximum pixel gray value max and the minimum value min in the feature map, and reset the new pixel value of the pixel in the feature map where x is the original pixel value;

Step 2.3.2, calculate the local saliency map, add the normalized grayscale feature map and direction feature map pixel by pixel, resample to the same size as the original thermal infrared, and obtain the local saliency map after normalization.

Step 3, building a thermal infrared image fusion saliency map, this step further includes:

Step 3.1, normalize the global saliency map obtained in step 1;

Step 3.2, construct the fusion saliency map S, the formula is S=S _G οS _L , where ο can be any one of [+, *, max], that is, for the normalized global saliency map S _G and the normalized local The saliency map S _L performs any one of addition, multiplication, and maximum value operations, and performs normalization operations on the resulting fused saliency map. In this embodiment, taking the addition operation as an example, the normalized global saliency map S _G and the normalized local saliency map S _L are added pixel by pixel, and the normalization operation is performed to obtain the fused saliency map.

Step 4, thermal infrared image target detection, performing threshold segmentation on the fused saliency map to obtain a binary result map of the marked target position, this step further includes:

Step 4.1, calculate the segmentation threshold, the formula is T=m+kσ, where m is the mean value of the fused saliency map, σ is the variance of the fused saliency map, and k is the empirical threshold, generally selected from 5 to 10;

Step 4.2: Create a binary result map of the same size as the original thermal infrared image, compare the gray value of the pixel at the coordinates (x, y) of the fusion saliency map with the size of the threshold T one by one, if it is greater than the threshold T, the binary result The corresponding value of the pixel at (x, y) in the picture is 1.

The specific embodiments described herein are merely illustrative of the spirit of the invention. Those skilled in the art to which the present invention belongs may make various modifications, additions or similar substitutions to the described specific embodiments without departing from the spirit of the present invention or exceeding the definition defined in the appended claims. scope.

Claims (8)

1. a kind of thermal infrared imagery object detection method based on significance analysis, it is characterised in that comprise the following steps：

Step 1, thermal infrared imagery is read, the global notable figure of thermal infrared imagery is built；

Step 2, thermal infrared imagery is read, the local notable figure of thermal infrared imagery is built, this step further comprises following sub-step；

Step 2.1, thermal infrared imagery gray feature figure is extracted；

Step 2.2, thermal infrared imagery direction character figure is extracted；

Step 2.3, fusion gray feature figure and direction character figure, build the local notable figure of thermal infrared imagery, and to local notable Figure is normalized；

Step 3, the global notable figure in step 1 is normalized, with reference to the global notable figure and part after normalization Notable figure, builds thermal infrared imagery fusion notable figure, and specific implementation is as follows,

Fusion notable figure calculation formula beWhereinCan be any one in [+, *, max], i.e., to normalizing Global notable figure S_GWith the local notable figure S of normalization_LAny one operation in maximum is added, is multiplied and is taken, and to obtaining To fusion notable figure operation is normalized.

Step 4, thermal infrared imagery target acquisition, the two-value knot that row threshold division obtains marking target location is entered to fusion notable figure Fruit is schemed.

2. a kind of thermal infrared imagery object detection method based on significance analysis as claimed in claim 1, it is characterised in that： The implementation of the step 1 is as follows,

Step 1.1, thermal infrared imagery is read, and gaussian filtering pretreatment is carried out to thermal infrared imagery, noise is removed and target is visited The influence of survey；

Step 1.2, global notable figure is calculated, formula isWhereinGray scale for reading thermal infrared imagery is averaged Value, I (x, y) is gray value of the thermal infrared imagery at pixel (x, y) place, | | | | it is European between representative two gray values of calculating Distance.

3. a kind of thermal infrared imagery object detection method based on significance analysis as claimed in claim 2, it is characterised in that： The implementation of the step 2.1 is as follows,

Step 2.1.1, carries out smothing filtering to thermal infrared imagery by different scale Gaussian function and obtains from 0 yardstick to 8 yardsticks Grayscale image pyramid：Gaussian convolution core is defined, Gaussian convolution core formula isσ standards Difference, μ mean μ=0, scale factor x=[1,2 ... 8], thermal infrared imagery and Gaussian convolution core are subjected to convolution operation successively, It is down-sampled step by step, set up the grayscale image pyramid from 0 yardstick to 8 yardsticks；

Step 2.1.2, poor operation around the corresponding gray feature figure of thermal infrared imagery, center is calculated using poor operation around center Formula is expressed asWherein I (c) represents the center yardstick image of gray feature pyramid image, I (s) tables Show surrounding's yardstick image of gray feature pyramid image,For poor operation around center.

4. a kind of thermal infrared imagery object detection method based on significance analysis as claimed in claim 3, it is characterised in that： The implementation of the step 2.2 is as follows,

Step 2.2.1, defines two-dimensional Gabor function, and formula isWherein (x, y) is represented Pixel coordinate, σ_xAnd σ_yStandard deviation of the Gaussian function in x and y directions is represented, 0 °, 45 °, 90 ° and 135 ° four direction is chosen and carries out Filtering, obtained Gabor Filtering Templates, formula is h (x, y)=g (x', y') cos (2 π ω_fX'), wherein (x', y')= (xcos(θ_f)+ysin(θ_f)-xsin(θ_f)+ycosn(θ_f)), ω_fFor modulating frequency, θ_f∈ { 0 °, 45 °, 90 °, 135 ° }, will be filtered Ripple template carries out convolution operation with thermal infrared imagery, obtains 0 °, 45 °, 90 ° and 135 ° four direction feature；

Step 2.2.2, repeat step 2.1.1 set up 0 ° successively to 0 °, 45 °, 90 ° and 135 ° four thermal infrared direction characters, Direction character pyramid of 45 °, 90 ° and 135 ° corresponding 0 yardsticks (raw video) to 8 yardsticks；

Step 2.2.3, to each direction character pyramid in aforementioned four thermal infrared direction, selection center yardstick image I (c) Around yardstick image I (s) carries out central surrounding difference operation, obtains 0 °, 45 °, 90 °, 135 ° have four groups of sub- direction character figures altogether；

Step 2.2.4, the sub- direction character figure of four groups of linear fusion, obtains final direction character figure.

5. a kind of thermal infrared imagery object detection method based on significance analysis as claimed in claim 4, it is characterised in that： The implementation of the step 2.3 is as follows：

Step 2.3.1, gray feature figure is normalized operation, and concrete mode is pixel ash in statistics gray feature figure respectively Maximum max, minimum value min are spent, the new pixel value for reseting the pixel x of gray feature figure is

Step 2.3.2, repeat step 2.3.1 are to direction characteristic pattern normalization operation；

Step 2.3.3, calculates local notable figure, the gray feature figure after normalization is added pixel-by-pixel with direction character figure, weight New sampling to the same size of original thermal infrared imagery, and local notable figure is obtained after normalized.

6. a kind of thermal infrared imagery object detection method based on significance analysis as claimed in claim 1, it is characterised in that： Threshold segmentation is calculated by formula T=m+k σ and obtained in the step 4, and wherein m is the average of fusion notable figure, and σ merges notable Figure variance, k is empirical value.

7. a kind of thermal infrared imagery object detection method based on significance analysis as claimed in claim 6, it is characterised in that： Empirical value k value is 5~10.

8. a kind of thermal infrared imagery object detection method based on significance analysis as claimed in claim 7, it is characterised in that： Obtain marking the binary result image of target location to be accomplished by the following way in the step 4, set up and original thermal infrared imagery An equal amount of binary result image, compares the size of the gray value and threshold value T of fusion notable figure coordinate (x, y) place pixel one by one, If more than threshold value T, pixel correspondence in binary result image (x, y) place is entered as into 1.