CN114092404A - Infrared target detection method and computer readable storage medium - Google Patents

Abstract

The invention provides an infrared target detection method and a computer storage medium, which specifically include: inputting continuous video frame data; correcting the lens shake and background change of the continuous video frame data through an image registration method based on a foreground mask algorithm; using time, Spatial constraints are used to associate adjacent frame targets in continuous video frame data; combined with the KCF algorithm, the associated targets are further tracked to obtain the tracking results; according to the target association times and tracking results, it is finally determined whether the target is a moving target. According to the infrared target detection method provided by the present invention, shaking and background changes are corrected through feature point matching based on KLT, and adjacent frame targets are constrainedly associated with features such as time and space, and combined with the KCF-based kernel correlation filter tracking method to perform multiple Dimensional target association, joint detection of moving targets, improves the detection rate of infrared weak and small targets under the condition of airborne forward-looking and downward-looking, and reduces the false alarm rate.

Description

A kind of infrared target detection method and computer readable storage medium

technical field

The present invention relates to the technical field of target detection, in particular to an infrared target detection method and a computer-readable storage medium.

Background technique

The detection and tracking of airborne forward-looking infrared weak and small targets is an extremely difficult problem. In the airborne environment, the infrared imaging system shakes severely, the background changes, the resolution of the infrared image is low, and the target to be detected is small and atmospheric disturbance. Influence and other factors make object detection very difficult.

Infrared weak and small target detection plays an important application in infrared imaging technology, which can be used for long-distance target detection, identification and early warning. There are currently many algorithms for the detection of infrared weak and small targets, including methods based on the human visual system, methods based on pipeline filtering, methods based on morphological filtering, and methods based on deep learning.

The detection and tracking performance of airborne forward-looking infrared weak and small targets largely depends on the correction of lens shake, the suppression of the ground background, and the temporal and spatial constraints of the target on the relevance of adjacent frame targets.

SUMMARY OF THE INVENTION

The technical problem to be solved by the present invention is how to improve the detection rate of infrared weak and small targets, and to provide an infrared target detection method and a computer-readable storage medium.

According to the infrared target detection method provided by the present invention, the method includes:

Input continuous video frame data;

Correct the lens shake and background change of the continuous positive video frame data by the image registration method based on the foreground mask algorithm;

Use time and space constraints to associate adjacent frame targets in the continuous video frame data;

Combined with the KCF algorithm, further track the associated target to obtain the tracking result;

It is finally determined whether the target is a moving target according to the association times of the target and the tracking result.

According to the infrared target detection method provided by the present invention, shaking and background changes are corrected through feature point matching based on KLT, and adjacent frame targets are constrainedly associated with features such as time and space, and combined with the KCF-based kernel correlation filter tracking method to perform multiple Dimensional target association, joint detection of moving targets, improves the detection rate of infrared weak and small targets under the condition of airborne forward-looking and downward-looking, and reduces the false alarm rate.

In some embodiments of the present invention, lens shake and background variation are corrected by an image registration method based on a foreground mask algorithm, which specifically includes:

For the three frames of images continuously input in the continuous video frame data, with the last frame of image as a reference, a candidate region is given;

Through the feature point matching algorithm based on the KLT corner detection method, the offsets of the candidate regions of the other two frames of images are calculated, and the image offsets of the three frames of images are corrected.

According to some embodiments of the present invention, the method further comprises:

The foreground mask data is obtained by the three-frame difference method for the three-frame images after the corrected image offset;

The foreground image is finally obtained by post-processing the foreground mask image.

In some embodiments of the present invention, the post-processing includes normalization, erosion, dilation, and binarization.

According to some embodiments of the present invention, the use of time and space constraints to associate adjacent frame objects in the video frame data is specifically:

According to the foreground image, segment the target, extract location information and image information;

For the targets of adjacent frames, the similarity is calculated according to the position information for target association.

In some embodiments of the present invention, for the targets of adjacent frames, the similarity is calculated according to the position information, and the target association is specifically:

If the Euclidean distance between the targets of adjacent frames is less than a certain threshold, it is a potential target. The smaller the distance, the higher the similarity. When the similarity meets the preset condition, the targets of adjacent frames are associated.

According to some embodiments of the present invention, the combination is based on the KCF algorithm to further correlate and confirm the moving target, specifically:

Set the target whose number of associations is greater than the preset value as a very suspicious target;

A regressor is trained according to the KCF algorithm, and the regressor is used to track extremely suspicious targets.

In some embodiments of the present invention, using the regressor to track extremely suspicious targets is specifically:

According to the target data of the latest frame of the extremely suspicious target and the target data of the previous frame of the latest frame, the response value is obtained respectively through the regressor. When the distance and response value between the latest frame and the previous frame respectively meet the preset conditions, the extremely suspicious target The latest frame of and the target of the previous frame of the latest frame are associated with the same target.

According to some embodiments of the present invention, the infrared target detection method further includes: collecting positive and negative samples by using a circulant matrix in the area around the target, and training a regressor by using ridge regression.

The present invention also provides a computer-readable storage medium, the computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement some aspects of the present invention. The method steps described in the examples.

Description of drawings

1 is a flowchart of an infrared target detection method according to an embodiment of the present invention;

2 is a flowchart of an infrared target detection method according to a specific embodiment of the present invention;

FIG. 3 is a schematic diagram of the effect of an infrared target detection method according to a specific embodiment of the present invention.

Detailed ways

In order to further illustrate the technical means and effects adopted by the present invention to achieve the predetermined purpose, the present invention will be described in detail below with reference to the accompanying drawings and preferred embodiments.

The description of the method flow in the specification of the present invention and the steps of the flowchart in the accompanying drawings of the present invention are not necessarily strictly executed according to the step numbers, and the execution order of the method steps can be changed. Also, certain steps may be omitted, multiple steps may be combined to be performed in one step, and/or one step may be decomposed into multiple steps to be performed.

According to the infrared target detection method provided by the present invention, as shown in FIG. 1 , the method includes:

S100: Input continuous video frame data.

S200: Correct lens shake and background change of continuous video frame data through an image registration method based on a foreground mask algorithm.

S300: Use time and space constraints to associate adjacent frame targets in the continuous video frame data.

S400: Combine the KCF algorithm to further track the associated target to obtain a tracking result.

S500: Finally determine whether the target is a moving target according to the association times of the target and the tracking result.

According to the infrared target detection method provided by the present invention, shaking and background changes are corrected through feature point matching based on KLT, and adjacent frame targets are constrainedly associated with features such as time and space, and combined with the KCF-based kernel correlation filter tracking method to perform multiple Dimensional target association, joint detection of moving targets, improves the detection rate of infrared weak and small targets under the condition of airborne forward-looking and downward-looking, and reduces the false alarm rate.

In some embodiments of the present invention, S200 : correcting lens shake and background change through an image registration method based on a foreground mask algorithm, which specifically includes: for three frames of images continuously input in the continuous video frame data, use the last frame of image For reference, candidate regions are given. Through the feature point matching algorithm based on the KLT corner detection method, the offsets of the candidate regions of the other two frames of images are calculated, and the image offsets of the three frames of images are corrected.

Among them, the KLT corner detection is specifically, if in the image I and the image J, two points match, then take the two points as the center and W as the window, there is a very small gray squared difference ε, which is defined as follows:

偏移量

将公式(1)变为where the coordinates

Offset

Change formula (1) into

处进行泰勒展开，截断高阶保留线性项，In order to get the minimum matching and make the value of ε the smallest, put the above formula at the point

Taylor expansion is performed at , truncating higher-order retained linear terms,

利用上式，分别令

和

带入上式：令

知

带入上式得到：in

Using the above formula, let

and

Bring into the above formula: let

Know

Bringing in the above formula gets:

Therefore there is,

Bringing (2) into (3) approximately has:

可得：make

Available:

equivalent to

根据公式(5)求出最佳偏移位置

in

Calculate the optimal offset position according to formula (5)

According to some embodiments of the present invention, the method further includes:

The foreground mask data is obtained by the three-frame difference method for the three-frame images after the corrected image offset;

The foreground image is finally obtained by post-processing the foreground mask image.

In some embodiments of the present invention, post-processing includes normalization, erosion, dilation, binarization.

According to some embodiments of the present invention, the temporal and spatial constraints are used to associate adjacent frame targets in the video frame data, specifically: according to the foreground image, segment the target, extract position information and image information, and for the adjacent frame targets, Calculate similarity based on location information for target association.

In some embodiments of the present invention, for the targets of adjacent frames, the similarity is calculated according to the position information to perform target association. Specifically, the Euclidean distance of the adjacent frame targets is less than a specific threshold as a potential target, and the smaller the distance, the more similar The higher the degree, when the similarity satisfies the preset condition, the targets of adjacent frames are associated.

Among them, associating the targets of adjacent frames specifically includes: 1. Since the target area is not very large, for the foreground mask obtained by the foreground mask detection algorithm, the large-area target is filtered first. 2. Do target segmentation, collect location information and image information for each target, including location, size, and image template. 3. The Euclidean distance of adjacent frame targets is less than a certain threshold as a potential target, and the smaller the distance, the higher the similarity. 4. As the number of associations increases, the possibility of the target being a moving target becomes more and more likely. For example, suspicious targets with an association number greater than 5 can be set as extremely suspicious targets. 5. Use tracking algorithm to track extremely suspicious targets.

According to some embodiments of the present invention, combined with the KCF-based algorithm, the moving target is further correlated and confirmed, specifically: setting the target whose correlation times are greater than the preset value as an extremely suspicious target, training a regressor according to the KCF algorithm, and using the regressor to Very suspicious targets to be tracked.

Specifically, as a discriminative tracking method, KCF generally trains a target detector during the tracking process, uses the target detector to detect whether the predicted position of the next frame is a target, and then uses the new detection results to update The training set in turn updates the object detector. Use the circulant matrix of the surrounding area to collect positive and negative samples, use ridge regression to train the object detector, and successfully use the circulant matrix to be diagonalizable in the Fourier space to convert the matrix operation into a vector Hadamad product, that is, the point of the element Multiplication, which greatly reduces the amount of operation and improves the operation speed.

Through the search area Z, through the result of the objective function f(z)=w ^T z, determine the position of the target and minimize the objective function:

是第i个训练样本，yi是对应区域中心点的像素值，对公式(6)w求导等于0可以得到：in,

is the i-th training sample, yi is the pixel value of the center point of the corresponding area, and the derivative of formula (6) w equals to 0 can be obtained:

w=(X ^H X+λI) ^-1 X ^H y (7)

where matrix X is a cyclic matrix formed by target vectors x=[x ₁ , x ₂ , ..., x _n ], ie:

Circular matrices can be diagonalized in Fourier space using discrete Fourier matrices, ie:

表示对向量x进行傅里叶变换，即

其中，

Represents the Fourier transform of the vector x, that is

in,

Bring (8) into (7), simplify:

表示两个向量的元素间点乘。引入核函数后，目标函数为：in,

Represents the element-wise dot product of two vectors. After introducing the kernel function, the objective function is:

where k represents the kernel function, which is defined and operated as follows

Finally solved:

Generally, k takes the Gaussian kernel function:

Through formula (9), a regressor can be trained to search for the position of the target. The position regression formula is as follows:

最大值的位置即为目标的移动位置。Generally, during training, X takes the area where the target is located. When the position returns, the response value

The position of the maximum value is the moving position of the target.

In some embodiments of the present invention, using the regressor to track extremely suspicious targets is specifically:

According to the target data of the latest frame of the extremely suspicious target and the target data of the previous frame of the latest frame, the response value is obtained respectively through the regressor. When the distance and response value between the latest frame and the previous frame respectively meet the preset conditions, the extremely suspicious target The latest frame of and the target of the previous frame of the latest frame are associated with the same target.

会呈现良好的高斯分布。为此，可以根据公式(10)的结果响应值

来判断两个目标是否相似，取响应值

标准差阈值σ＝0.05，当σ≥0.05时，视为两目标相似。另外由于时间空间约束性，目标帧之间位移不会太大，取目标距离阈值d＝15，当距离d>15时且σ>0.05时，两目标可关联为同一目标。When the regressor obtained by the KCF algorithm tracks the target, it can be known from formula (10) that when the target x in the previous frame is similar to the target z in the next frame, the response value

will present a nice Gaussian distribution. To this end, the response value can be based on the result of formula (10)

To judge whether the two targets are similar, take the response value

The standard deviation threshold σ=0.05, when σ≥0.05, the two targets are regarded as similar. In addition, due to the constraints of time and space, the displacement between target frames will not be too large. Take the target distance threshold d=15. When the distance d>15 and σ>0.05, the two targets can be associated with the same target.

According to some embodiments of the present invention, the infrared target detection method further includes: collecting positive and negative samples by using a circulant matrix in the area around the target, and training a regressor by using ridge regression. Among them, when training the regressor, the target area is generally selected as a positive sample, and the surrounding area of the target is a negative sample. Of course, the area closer to the target is more likely to be a positive sample.

The present invention also provides a computer-readable storage medium, where one or more programs are stored in the computer-readable storage medium, and the one or more programs can be executed by one or more processors to implement the methods according to some embodiments of the present invention The difference lies in that, in terms of engineering implementation, this embodiment can be implemented by means of software plus a necessary general hardware platform, of course, it can also be implemented by hardware, but the former is a better implementation in many cases.

Based on this understanding, the method of the present invention can be embodied in the form of a computer software product, which is stored in a computer-readable storage medium (eg, ROM/RAM, magnetic disk, optical disk), and includes several instructions for making A device (which may be a device with functions of data operation and graphics processing, such as a computer) executes the method of the embodiment of the present invention.

The infrared target detection method according to the present invention will be described in detail below with a specific embodiment with reference to the accompanying drawings. It should be understood that the following description is only an exemplary description, rather than a specific limitation of the present invention.

The invention discloses an infrared weak and small target detection method based on feature point matching and kernel correlation filtering. Image correction is performed by feature point matching based on KLT, and target correlation is performed by kernel correlation filtering algorithm based on KCF, so as to solve the problem of airborne downward movement. The detection and tracking of weak and small targets, under complex ground conditions, detect moving targets and obtain good detection and tracking results. Similar feature point matching methods include, but are not limited to, gray-scale image-based feature point matching, contour-based feature point matching. Curve feature point matching, binarization-based feature point matching, and phase-based feature point matching. Kernel correlation filtering methods include but are not limited to CSK-based kernel correlation filtering, STC-based kernel correlation filtering, and KCF-based kernel correlation filtering;

The algorithm proposed in this paper first uses the Harris-based corner detection method to perform multi-point correction, and then uses the KLT optical flow method to determine the background image offset between frames. At the same time, the multi-scale median filtering operation is performed on the current frame image and the previous two frame images respectively, and then the frame sequence with offset compensation is performed three-frame differential pairing in the offset compensation by using frame skipping or fixed frame interval method. Do poorly after that. After inputting three pictures to obtain two differential images, perform AND operation on the two differential images, and finally combine the Ostu algorithm and morphological filtering to binarize them to obtain the target foreground.

The target has target scale invariance in adjacent frames or adjacent frames, and adopts the frame skipping mechanism of the previous section, and the displacement of the adjacent frame target is near the detection area of the current frame target. Based on the above premise, the target correlation algorithm based on kernel correlation filtering can effectively ensure the robustness of target detection. In signal processing, convolution is often used to realize the Fourier transform of discrete signals. The kernel correlation filtering algorithm is combined with ridge regression, and the cyclic shift matrix diagonalization is used to convert the image from the spatial domain to the frequency domain for operation. On the premise, the computational complexity is reduced.

Please refer to the accompanying drawings, which is a flowchart of an infrared weak and small target detection method based on feature point matching and nuclear correlation filtering disclosed in an example of the present invention, and an infrared weak and small target detection method based on feature point matching and nuclear correlation filtering disclosed in an example of the present invention, As shown in Figure 2 and Figure 3, the specific implementation steps are:

Step S1: Extract three frames of adjacent image data from the input video sequence.

Step S2: Perform feature point detection on three frames of images (as shown in a in Figure 3).

Step S3: Based on the background correction algorithm of KLT, the offset of the feature points is calculated, and the image is corrected.

Step S4: three-frame difference to obtain a foreground mask image (as shown in b in Figure 3).

Step S5: Binarize the foreground mask and perform target segmentation and blind flash element elimination (as shown in c in Figure 3).

Step S6: Associate the current frame data with the historical frame data to obtain new target information (as shown in d in FIG. 3 ).

In addition, after the above steps and combined with the KCF tracking algorithm, the detection result shown in e in Figure 3 is obtained.

The invention discloses an infrared weak and small target detection method based on feature point matching and kernel correlation filtering. Through the KLT-based feature point matching and KCF-based target association algorithm, the detection and tracking problems of airborne downward moving weak and small targets are solved. Under complex ground conditions, moving targets are detected and good detection and tracking effects are obtained. The main feature is to propose a target detection process that improves the performance of the algorithm. From the subjective visual effects and objective numerical evaluation, we can see that we The proposed method has improved the detection performance of infrared weak and small targets, especially the suppression of false alarms caused by complex background clutter.

The above description of the disclosed embodiments enables any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Through the description of the specific embodiments, it should be possible to have a more in-depth and specific understanding of the technical means and effects adopted by the present invention to achieve the predetermined purpose. However, the accompanying drawings are only for reference and description, not for the present invention. limit.

Claims (10)

1. an infrared target detection method, is characterized in that, described method comprises: Input continuous video frame data; Correct the lens shake and background change of the continuous video frame data through the image registration method based on the foreground mask algorithm; Use time and space constraints to associate adjacent frame targets in the continuous video frame data; Combined with the KCF algorithm, further correlate and confirm the target to obtain the tracking result; It is finally determined whether the target is a moving target according to the association times of the target and the tracking result. 2. infrared target detection method according to claim 1, is characterized in that, by the image registration method based on foreground mask algorithm correction lens shake and background change, specifically comprises: For the three frames of images continuously input in the continuous video frame data, with the last frame of image as a reference, a candidate region is given; Through the feature point matching algorithm based on the KLT corner detection method, the offsets of the candidate regions of the other two frames of images are calculated, and the image offsets of the three frames of images are corrected. 3. The infrared target detection method according to claim 2, wherein the method further comprises: Obtaining foreground mask data through a three-frame difference method for the three frames of images after the corrected image offset; The foreground image is finally obtained by post-processing the foreground mask image. 4. The infrared target detection method according to claim 3, wherein the post-processing includes normalization, corrosion, expansion, and binarization. 5. infrared target detection method according to claim 3, is characterized in that, described utilizes the adjacent frame target in the constrained association video frame data on time and space, specifically: According to the foreground image, segment the target, and extract position information and image information; For the targets of adjacent frames, the similarity is calculated according to the position information for target association. 6. infrared target detection method according to claim 5, is characterized in that, to the target of adjacent frame, calculate similarity according to position information, to carry out target association specifically: If the Euclidean distance of the target of adjacent frames is less than a certain threshold, it is a potential target, and the potential targets of adjacent frames are associated. 7. infrared target detection method according to claim 1, is characterized in that, described in conjunction with KCF algorithm, further correlates and confirms moving target, is specially: Set the target whose number of associations is greater than the preset value as a very suspicious target; A regressor is trained according to the KCF algorithm, and the regressor is used to track the extremely suspicious target. 8. infrared target detection method according to claim 7, is characterized in that, using regressor to track extremely suspicious target is specifically: According to the target data of the latest frame of the extremely suspicious target and the target data of the previous frame of the latest frame, the response value is obtained respectively through the regressor. When the distance and the response value between the latest frame and the previous frame respectively satisfy the preset conditions, the The latest frame of the suspicious target and the target of the previous frame of the latest frame are associated with the same target. 9 . The infrared target detection method according to claim 7 , wherein the infrared target detection method further comprises: collecting positive and negative samples using a cyclic matrix in the area around the target, and using ridge regression to train a regressor. 10 . 10. A computer-readable storage medium, characterized in that, the computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to realize the claim The method steps of any one of claims 1 to 9 are required.

Images