CN101666682B - Neural network non-uniformity correction method based on scene statistics - Google Patents

Abstract

The present invention claims protection for a neural network non-uniformity correction method based on scene statistics, and belongs to the field of infrared focal plane detection. In view of the deficiency that the traditional neural network correction method is difficult to eliminate low-frequency spatial noise, a neural network non-uniformity correction method based on scene statistics is proposed. The method includes initializing the image correlation matrix and parameters; performing blind pixel detection and compensation according to the gray value of the image pixel; using the scene statistics method to perform non-uniformity correction on the image bias; using the neural network correction method, judging the regional attributes of the pixel according to the standard deviation threshold of the correction error, and performing gain nonlinear correction on the image after scene statistics correction and without low-frequency spatial noise. The present invention has a good correction effect on the desired signal of the image, suppresses target degradation and ghosting, and the change of the background image has almost no effect on the correction effect. The method can be widely used in image detection and processing.

Description

Non-uniformity Correction Method of Neural Network Based on Scene Statistics

technical field

The invention relates to the technical field of image detection and processing, in particular to an image correction method in the infrared focal plane detection technology.

Background technique

Infrared focal plane array imaging system has become the development trend of infrared imaging technology due to its advantages of high sensitivity, small size, compact structure, long range, good anti-interference, strong ability to penetrate smoke, and can work all-weather and all-day. , and the staring infrared focal plane array has become the mainstream detection device for the development of future infrared thermal imaging systems. However, due to the limitations of materials and technology levels, there is generally non-uniformity among the response characteristics of each detection unit of the infrared focal plane array (IRFPA), which will lead to a significant decrease in the performance of the infrared imaging system such as temperature resolution, so that it is difficult to To meet the requirements of engineering applications, the infrared focal plane arrays used in engineering almost without exception adopt non-uniformity correction technology.

At present, there have been a variety of non-uniformity correction methods for infrared focal plane arrays at home and abroad, but they can be roughly divided into two categories: one is calibration-based correction methods, mainly including two-point temperature calibration (TPC) and multi-point temperature calibration (TPC). Point Temperature Calibration (ETPC). This type of correction method has the advantages of simple and flexible algorithm, fast operation speed, and easy hardware implementation, etc., and is the main method used in engineering applications at present. However, due to the influence of the infrared focal plane array's working time and environmental changes, its response parameters will drift slowly, which will affect the calibration accuracy. Therefore, calibration methods usually require periodic calibration and calibration. In this way, not only the real-time imaging process needs to be interrupted, but also the operation is complicated. The other is the scene-based correction method, which mainly includes constant statistical average method (CSC), temporal high-pass filter method (THPFC) and artificial neural network method (ANNC). This type of method can effectively eliminate the drift of the response parameters of the infrared focal plane array with the change of working time and environment, without calibration, and only needs to realize the adaptive correction of IRFPA non-uniformity according to the scene information. Among the scene-based correction methods, the neural network correction method is the most representative.

Although the traditional neural network correction method (Scribner) does not need to calibrate the infrared focal plane array in theory, it does not have high requirements for the linearity and stability of the detector parameters. However, the traditional neural network correction method also has obvious deficiencies, especially for low-frequency spatial noise. The reason is that the average value of the four neighborhoods is simply used as the expected output, and too few pixels are involved in the calculation, especially the current pixel value is not considered, resulting in a large deviation between the expected value and the actual value. When the signal changes slowly in the same area in the scene, it is reasonable to calculate the expected value of the four-neighborhood mean; when the signal changes drastically in space, and the window spans multiple different areas, it is more difficult to use the four-neighborhood mean as the expected value. big error. When the target moves, this error will not accumulate, and the effect on the correction effect will not be obvious; target degradation. And when the target changes from static to sudden movement, it will leave a reverse image correction ghost in the original position.

Zhang Tianxu, Shi Yan et al. analyzed the spatial frequency characteristics of infrared focal plane array non-uniformity noise and pointed out that Low frequency noise is its main component. Aiming at the shortcomings of the traditional spatial adaptive correction method for removing low-frequency spatial noise, a method combining one-point correction and spatial adaptive correction is proposed. The one-point correction in this method is obtained by time-averaging the background of continuous finite frames without targets. If the image background remains unchanged during the working of the camera, the background image obtained by time-averaging the background of the continuous finite frame without the target is close to the actual background, and a better correction effect can be obtained at this time. However, when the background image changes in real time, there is a large error between the background image obtained by this method and the actual background, so the preprocessing correction result will inevitably affect the final correction result.

Due to the existence of blind elements of infrared focal plane array, the infrared image is blurred. At present, blind elements are mainly detected through signal processing technology, and effective compensation is performed through neighboring pixels to improve the imaging quality of infrared focal plane array. The processing of blind elements includes two aspects of blind element detection and compensation. The traditional blind pixel processing method is difficult to realize the online detection and compensation of blind pixels. In Dai Shaosheng and Zhang Tianqi's "A New Algorithm for Processing Blind Elements of Infrared Focal Plane Arrays", a new algorithm for real-time detection and compensation of IRFPA blind elements is proposed. The algorithm is simple to implement and highly versatile. It can detect and compensate randomly generated blind pixels in real time, but it only involves the detection and compensation of blind pixels, and does not involve non-uniformity correction and gain nonlinearity correction for image signal bias. .

The spatial noise suffered by the infrared focal plane array has the following two properties:

Nature 1: Spatial noise caused by non-uniformity mainly appears as low-frequency components.

Nature 2: Spatial noise caused by gain non-uniformity alone is mainly presented as high-frequency components. However, the premise of the traditional neural network correction method is that the spatial frequency of the spatial noise caused by non-uniformity is mainly high frequency or white noise. Based on this premise, the traditional neural network correction method uses the mean value of the four neighborhoods of the pixel as the correction expected value of the pixel output to update the correction coefficient, so that it presents a spatial low-pass frequency characteristic. Therefore, when the spatial noise of the actual infrared focal plane array is dominated by low frequencies, the traditional neural network correction method is powerless.

For example, the algorithm that combines one-point correction and spatial adaptive correction (onepoint_nn_nuc), the change of the background image will affect the correction effect. Difference.

If we use some kind of preprocessing correction in advance to eliminate the noise of the spatial low frequency part, leaving only the spatial high frequency noise caused by gain non-uniformity, and then use the neural network method for subsequent correction, a better correction effect can be obtained . It is based on the above reasons that the present invention proposes a neural network non-uniformity correction method based on scene statistics, that is, firstly eliminates the spatial low-frequency noise caused by IRFPA bias non-uniformity through scene statistics, and then uses the neural network method for subsequent corrections .

Contents of the invention

Aiming at the change of response characteristics and stability of infrared focal plane array caused by complex and changeable environmental conditions, and the difficulty of eliminating low-frequency spatial noise by traditional neural network correction methods, the present invention proposes a method based on scene statistics and neural network non-uniformity correction method. The method includes performing blind element detection and compensation, using a scene statistics method to correct non-uniformity of image bias, and using a neural network correction method to perform gain nonlinearity correction on an image after scene statistics correction and without low-frequency spatial noise.

Blind pixel detection and compensation. Before correcting the non-uniformity of the infrared focal plane array, it is first necessary to perform blind element detection and compensation. Take the time-domain mean value of consecutive k-frame image sequences as the response rate B _ij of each pixel in the current frame, that is $B_{\mathrm{ij}}=\frac{\underset{t=\mathrm{no}-(k-1)}{\overset{\mathrm{no}}{\mathrm{\Σ}}}{x}_{i,j}\left(t\right)}{k};$ To judge whether the pixel B _ij is a valid pixel, first query the maximum and minimum pixel gray value B _max , B _min in a window whose center size is (2h+1) _× (2h+1) with the pixel B ij as the center; then Remove B _max and B _min , and calculate the average gray value B of the remaining pixels in the window, that is $\stackrel{\‾}{B}=\frac{\underset{i=h}{\overset{i+h}{\mathrm{\Σ}}}\underset{j=h}{\overset{j+h}{\mathrm{\Σ}}}{B}_{\mathrm{ij}}-B_{\max }-B_{\min }}{(2h+1)\×(2h+1)-2};$ Perform blind element judgment on each pixel, when the condition is satisfied $\mathrm{\Δ}=\frac{B_{\max }-\stackrel{\‾}{B}}{\stackrel{\‾}{B}}\&Greater\; Equal;9$ or $\mathrm{\Δ}=\frac{\stackrel{\‾}{B}-B_{\min }}{B_{\min }}\&Greater\; Equal;9$ When , it can be judged as a blind pixel, otherwise it is a valid pixel. If the pixel B _ij is a blind element, set the corresponding position in the blind element matrix (i.e. row i, column j) to 1; replace the blind element position with the blind element pixel 4 neighborhood mean value for compensation in the original image of the current frame , get the image with blind elements removed.

The image bias is corrected for non-uniformity using scene statistics. Calculate the original image input mean value E[X _{i, j} ] and the corrected output mean value E[Y _{i, j} ] according to the time domain mean value of the original image, and obtain the gray value Z _{i, j} of each pixel that does not contain low-frequency spatial noise ( n).

Using the time-domain average value of the original image from the first frame to the nth frame as the input of the infrared focal plane array, E[X _{i, j} ] can be calculated recursively: that is $\mathrm{E.}\left[x_{i,j}\right]=\stackrel{\‾}{x_{i,j}}=\frac{x_{i,j,\mathrm{no}}+(\mathrm{no}-1)\·x_{i,j,\mathrm{no}-1}}{\mathrm{no}}.$ The spatial average of X _{i, j} can be used as the average output E[Y _{i, j} ] of the pixels in the nth frame of the infrared focal plane array: $\mathrm{E.}\left[Y_{i,j}\right]=\stackrel{\‾}{x}=\frac{1}{m\×N}\×\underset{i=1}{\overset{m}{\mathrm{\Σ}}}\underset{j=1}{\overset{N}{\mathrm{\Σ}}}\stackrel{\‾}{x_{i,j}}.$

The original image X _{i, j} (n) of the current frame subtracts the average input X _i, j of the (i, j)th pixel of the infrared focal plane array in time, and the gray value of each pixel without low-frequency spatial noise can be obtained Z _{i, j} (n).

Z _i,j (n)=X _i,j (n)-X _i,j

The neural network correction method is used to correct the scene statistics, and the image without low-frequency spatial noise is corrected for gain nonlinearity. If an adaptive weighted average filter is used, the expected signal output is determined. According to the standard deviation threshold of the correction error to judge the regional attribute of the pixel, a larger weight is assigned to the pixels in the same region, and a smaller weight is assigned to the pixels in different regions. The weighted average filter weight W _{p, q} (n) Determine the expected value F _i,j (n) of the output.

Among them, the weight is determined by the following formula:

${\mathrm{<span\; class="notranslate">\; W</span>}}_{\mathrm{<span\; class="notranslate">\; pq</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{<span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; Z</span>}}_{\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; q</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; Z</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; |</span><span\; class="notranslate">\; \&Center\; Dot;</span>\mathrm{<span\; class="notranslate">\; \&eta;</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; \&CenterDot;</span><span\; class="notranslate">\; (</span><span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; Z</span>}}_{\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; q</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; Z</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; |</span><span\; class="notranslate">\; <</span>\mathrm{<span\; class="notranslate">\; the\; th</span>}<span\; class="notranslate">\; )</span>$

Among them, η is the weight coefficient.

The output of the weighted filter is the expected value F _{i, j} (n), and the expected value F _{i, j} (n) is calculated according to the following formula:

${\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\underset{\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; h</span>}}{\overset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; h</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; q</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; h</span>}}{\overset{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; h</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; W</span>}}_{\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; q</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span>{\mathrm{<span\; class="notranslate">\; Z</span>}}_{\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; q</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span>}{\underset{\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; h</span>}}{\overset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; h</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; q</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; h</span>}}{\overset{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; h</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; W</span>}}_{\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; q</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span>}$

Compared with the traditional neural network method and the method of combining one-point correction and neural network, the present invention has stronger non-uniformity correction capability, thereby being able to obtain ideal image correction effect. In the real-time correction of continuous scene changes, it can not only eliminate low-frequency spatial noise, but also eliminate target degradation and ghost images, and improve the calculation speed of the algorithm.

Description of drawings

Fig. 1 is a schematic flow chart of the correction method of the present invention.

Figure 2 is a schematic diagram of blind element detection. Among them, Fig. 2(a) is a schematic diagram of a k-frame motion scene; Fig. 2(b) is a 3×3 blind pixel detection window.

Detailed ways

The following first analyzes the implementation process of the algorithm in principle:

A linear model is used to correct the infrared focal plane array, and the correction model is:

Y _{i, j} (n) = G _{i, j} (n) X _{i, j} (n) + O _{i, j} (n) (1)

where Y _i,j represents the corrected output, Xi _,j represents the original image input, G _i,j represents the corrected gain, and O _i,j represents the corrected image bias.

Taking expectation on both sides of the above formula can get an expression of the following form:

E[Y _{i, j} ]=G _i,j (n)·E[X _i,j ]+O _i,j (n) (2)

Where E[] is the expected operator, E[Y _{i, j} ] is the mean value of the corrected output, E[X _{i, j} ] is the mean value of the original image input,

After subtracting formula (1) and formula (2), the following formula is obtained:

Y _i,j (n)-E[Y _i,j ]=G _i,j (n)·(X _i,j (n)-E[X _i,j ]) (3)

Comparing formula (3) with formula (1), it can be seen that E[X _{i, j} ] term is added in formula (3), and offset term O _{i, j} is missing. The calculation and estimation of the offset O _{i, j} in the original algorithm is transformed into the calculation of the input signal mean value E[X _{i, j} ] and the corrected output mean value E[Y _{i, j} ]. If z _i,j (n)=Y _i,j (n)-E[Y _i,j ], Zi _,j (n)=X _i,j (n)-E[X _i,j ] then Available:

z _{i, j} (n) = G _{i, j} (n) Z _{i, j} (n) (4)

Equation (4) no longer contains low-frequency spatial noise, and then the neural network method can be used to correct the high-frequency spatial noise caused by gain non-uniformity. The corrected output is:

Y _i,j (n)=G _i,j (n)·(X _i,j (n)-E[X _i,j ])+E[Y _i,j ] (5)

The calculation of the input signal mean value E[X _i,j ] and the corrected output mean value E[Y _i,j ] is further discussed below.

Due to the time correlation of the image sequence, the average gray value of the original image of the first n frames is used as the time-domain average input value of the infrared focal plane array. Use Xi _{, j, k} to represent the (i, j) pixel original value of the Kth frame image, and use Xi _{, j} to represent the time average input of the (i, j) pixel of the infrared focal plane array, then: $\stackrel{\&OverBar;}{x_{i,j}}=\frac{\underset{k=1}{\overset{\mathrm{no}}{\mathrm{\&Sigma;}}}{x}_{i,j,k}}{\mathrm{no}}.$ Substituting X _{i, j} for E[X _{i, j} ], the following formula is obtained:

$\mathrm{<span\; class="notranslate">\; E.</span>}<span\; class="notranslate">\; [</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; =</span>\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}}}<span\; class="notranslate">\; =</span>\frac{\underset{\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; no</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; k</span>}}}{\mathrm{<span\; class="notranslate">\; no</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 6</span>}<span\; class="notranslate">\; )</span>$

Since the spatial correlation of Xi _{, j} of the original image will continue to increase as the number of accumulated frames increases, the spatial mean X of _{Xi, j} can be used to replace the corrected average output of the (i, j)th pixel E[Y _{i, j} ]: namely

$\mathrm{<span\; class="notranslate">\; E.</span>}<span\; class="notranslate">\; [</span>{\mathrm{<span\; class="notranslate">\; Y</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; =</span>\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; N</span>}}<span\; class="notranslate">\; \&times;</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; R</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; L</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 7</span>}<span\; class="notranslate">\; )</span>$

M and N are the number of rows and columns of the focal plane array.

Substitute (6) and (7) into (5) to get the final corrected output:

Y _i,j (n)=G _i,j (n)·(X _i,j (n)-X _i,j )+X (8)

Among them, Xi _{, j} and X are the calculation results of formula (6) and formula (7) respectively.

The following describes the implementation of the present invention in detail with reference to the accompanying drawings and examples. Figure 1 is a schematic flow chart of the correction method of the present invention, which specifically includes the following steps: initialization, blind element detection and compensation, scene statistics, and neural network correction.

(1) Initialization process

Initialize first, initialize the image two-dimensional matrix and parameters. Initialize the gain correction coefficient G of each pixel as a matrix of all 1s, the storage matrix of blind elements is a matrix of all 0s, the mean value matrix of the time domain input is a matrix of all 0s, the mean value of the spatial domain output is 0, the sequence number of the original image to be corrected is frame=1, and the error threshold is th=0.2, set the original image frame number k used for blind element detection calculation as a natural number.

(2) Blind element detection and compensation

Calculate the response rate of the pixel according to the gray value of the image pixel, take the response rate as the center, query the mean value of the gray value of the pixel in the determined window, find out the maximum and minimum pixel gray value B _max , B _min ; remove B _max , B _min , calculate the average gray value B of the remaining pixels in the window, thereby determine the position of the blind element, and set the pixel at the corresponding position of the blind element matrix to 1; replace the detected blind element position with the mean value of the 4 neighborhoods of the blind element pixel Compensation to obtain an image with blind pixels removed.

The invention adopts a scene-based real-time blind element detection algorithm to detect random blind elements generated during the working process of the infrared focal plane array in real time. Using the current frame and the previous k-1 consecutive frames of images, the gray value of each pixel is averaged in time domain to calculate its response rate, and recorded as B _ij . Then B _ij can be expressed as: $B_{\mathrm{ij}}=\frac{\underset{t=\mathrm{no}-\mathrm{<figure-callout\; id="1"\; label="k\; +"\; filenames="H2009101045511E00012.png"\; state="\{\{state\}\}">k</figure-callout>}+1}{\overset{\mathrm{no}}{\mathrm{\&Sigma;}}}{x}_{i,j}\left(t\right)}{k}$

Figure 2 is a schematic diagram of blind element detection. In the figure, the original image pixel gray value of 10 consecutive frames (from frame n-9 to frame n) including the current frame is used for time-domain average as the response rate of the corresponding pixel. In order to improve the operation efficiency, we use an iterative method to calculate B _ij :

SumV _i,j (n)=SumV _i,j (n-1)+X _i,j (n)-X _i,j (n-9)

${\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; ij</span>}}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; Sum</span>}{\mathrm{<span\; class="notranslate">\; V</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span>}{\mathrm{<span\; class="notranslate">\; 10</span>}}$

Then use the scene-based real-time blind element detection algorithm to perform blind element detection on the obtained response rate, and set the element at the corresponding position of the blind element matrix to 1.

The blind element detection process is as follows:

(a) Take the response rate B _ij as the center, query the average pixel gray value in the (2h+1)×(2h+1) window, and find out the maximum and minimum pixel gray value B _max , B _min .

(b) Remove B _max and B _min in the window, and calculate the average value B of the gray scale of the remaining pixels in the window, that is $\stackrel{\&OverBar;}{B}=\frac{\underset{i=h}{\overset{i+h}{\mathrm{\&Sigma;}}}\underset{j=h}{\overset{j+h}{\mathrm{\&Sigma;}}}{B}_{\mathrm{ij}}-B_{\max }-B_{\min }}{(2h+1)\&times;(2h+1)-2}$

(c) compare the percentage of B _max , B _min and B difference, that is, $\mathrm{\&Delta;}=\frac{B_{\max }-\stackrel{\&OverBar;}{B}}{\stackrel{\&OverBar;}{B}}$ or $\mathrm{\&Delta;}=\frac{\stackrel{\&OverBar;}{B}-B_{\min }}{B_{\min }}.$ According to the national standard GB/T1744421998 "Technical Standards for Infrared Focal Plane Acceptance Test", when Δ≥9, the pixel is considered to be a blind element, and the position of the blind element is recorded, and the element at the corresponding position in the blind element matrix is set to 1.

(d) Blind element compensation. The blind pixel position detected in the original image of the current frame is replaced by the blind pixel pixel 4-neighborhood mean value for compensation, and the image with the blind pixel removed is obtained.

${\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}}}{\mathrm{<span\; class="notranslate">\; 4</span>}}$

(2) Scene statistics steps

Using the scene statistics method to correct the non-uniformity of the image bias, the purpose is to calculate the original image input mean value E[X _{i, j} ] and the corrected output mean value E[Y _{i, j} ] according to the time domain mean value of the original image, and obtain The gray value Z _i,j (n) of each pixel that does not contain low-frequency spatial noise.

Due to the temporal correlation of the image sequence, the time domain mean value of the original image gray value of the previous n frames is used as the average input of the infrared focal plane array (IRFPA). The time-average input of the (i, j)th pixel of the infrared focal plane array is given by formula (6).

In order to improve computing efficiency and occupy less storage space, the present invention uses a recursive method to calculate the original image input mean E[X _{i, j} ], namely:

$\mathrm{<span\; class="notranslate">\; E.</span>}<span\; class="notranslate">\; [</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; =</span>\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}}}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; no</span>}}<span\; class="notranslate">\; +</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&Center\; Dot;</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}}{\mathrm{<span\; class="notranslate">\; no</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 9</span>}<span\; class="notranslate">\; )</span>$

The average output of the infrared focal plane array pixels, that is, the corrected output mean E[Y _{i, j} ], can be calculated from the spatial correlation of the image:

$\mathrm{<span\; class="notranslate">\; E.</span>}<span\; class="notranslate">\; [</span>{\mathrm{<span\; class="notranslate">\; Y</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; =</span>\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; N</span>}}<span\; class="notranslate">\; \&times;</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; N</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 10</span>}<span\; class="notranslate">\; )</span>$

The original image X _{i, j} (n) of the current frame subtracts the average input X _{i, j} (original image input mean value E[X _{i, j} ]) of the pixel (i, j) of the infrared focal plane array in time, then The gray value Z _i,j (n) of each pixel without low-frequency spatial noise can be obtained.

Z _i,j (n)=X _i,j (n)-X _i,j

(4) Neural network correction

The neural network correction method is used to correct the gain nonlinearity of the scene statistically corrected image without low-frequency spatial noise.

The traditional neural network correction algorithm proposed by DAScribner et al. simply uses the 4-neighborhood mean of the current pixel to calculate its expected value F _{i, j} (n). This method is reasonable to calculate the expected value of the current pixel inside the same area of the scene, but at the edge of the area, using this method to calculate the expected value will have obvious shortcomings. At the same time, when using the four-neighborhood mean to calculate the expected value of the current pixel, due to the small number of pixels involved in the calculation, the IRFPA with large spatial low-frequency noise cannot obtain the closest to the real expected value. For this reason, the present invention adopts the self-adaptive weighted average filter, uses more pixels to participate in the average operation, judges the area of the pixel according to the standard deviation threshold value of the correction error, for the same area as the central pixel (i-th row j column pixel) Pixels are assigned larger weights, and pixels in different regions are assigned smaller weights.

For the pixel pq within the (2h+1) filter window centered on the ijth pixel, its weight is determined by the following formula:

${\mathrm{<span\; class="notranslate">\; W</span>}}_{\mathrm{<span\; class="notranslate">\; pq</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{<span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; Z</span>}}_{\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; q</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; Z</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; |</span><span\; class="notranslate">\; \&Center\; Dot;</span>\mathrm{<span\; class="notranslate">\; \&eta;</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; \&Center\; Dot;</span><span\; class="notranslate">\; (</span>{<span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; Z</span>}}_{\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; q</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; Z</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; |</span><span\; class="notranslate">\; <</span>\mathrm{<span\; class="notranslate">\; the\; th</span>}<span\; class="notranslate">\; )</span>$

η is the weight coefficient.

The output of the weighted filter is the expected value F _i,j (n):

${\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\underset{\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; h</span>}}{\overset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; h</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; q</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; h</span>}}{\overset{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; h</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; W</span>}}_{\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; q</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span>{\mathrm{<span\; class="notranslate">\; Z</span>}}_{\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; q</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span>}{\underset{\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; h</span>}}{\overset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; h</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; q</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; h</span>}}{\overset{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; h</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; W</span>}}_{\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; q</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span>}$

After obtaining the expected value F _{i, j} (n), use the following formula to perform iterative correction frame by frame:

Y _i,j (n)=G _i,j (n) Z _i,j (n)+X

G _i,j (n)=G _i,j (n-1)-2λZ _i,j (n)·(Z _i,j (n)-F _i,j (n))

Among them, Y _{i, j} (n) is the correction output of the nth frame, G _{i, j} (n) is the gain correction coefficient of the nth frame, and λ is the iteration step size.

For the pixel pq in the 3×3 filter window centered on pixel i, j, its 4 neighborhood weights (W1 represents the weight of pixel i-1, j, called the weight; W2 represents the pixel i+1, The weight of j is called the lower weight; W3 represents the weight of pixel i, j-1, which is called the left weight; W4 represents the weight of pixel i, j+1, which is called the right weight) is determined by the following formula:

$\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="W"\; filenames="H2009101045511E00012.png"\; state="\{\{state\}\}">W</figure-callout></span>}\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{<span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; Z</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; Z</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; |</span><span\; class="notranslate">\; \&CenterDot;</span>\mathrm{<span\; class="notranslate">\; \&eta;</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; \&CenterDot;</span><span\; class="notranslate">\; (</span><span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; Z</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; Z</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; |</span><span\; class="notranslate">\; <</span>\mathrm{<span\; class="notranslate">\; the\; th</span>}<span\; class="notranslate">\; )</span>$

$\mathrm{<span\; class="notranslate">\; <figure-callout\; id="2"\; label="W"\; filenames="H2009101045511E00012.png"\; state="\{\{state\}\}">W</figure-callout></span>}\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{<span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; Z</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; Z</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; |</span><span\; class="notranslate">\; \&CenterDot;</span>\mathrm{<span\; class="notranslate">\; \&eta;</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; \&CenterDot;</span><span\; class="notranslate">\; (</span><span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; Z</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; Z</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; |</span><span\; class="notranslate">\; <</span>\mathrm{<span\; class="notranslate">\; the\; th</span>}<span\; class="notranslate">\; )</span>$

$\mathrm{<span\; class="notranslate">\; W</span>}\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{<span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; Z</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; Z</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; |</span><span\; class="notranslate">\; \&CenterDot;</span>\mathrm{<span\; class="notranslate">\; \&eta;</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; \&CenterDot;</span><span\; class="notranslate">\; (</span><span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; Z</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; Z</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; |</span><span\; class="notranslate">\; <</span>\mathrm{<span\; class="notranslate">\; the\; th</span>}<span\; class="notranslate">\; )</span>$

$\mathrm{<span\; class="notranslate">\; W</span>}\mathrm{<span\; class="notranslate">\; 4</span>}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{<span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; Z</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; Z</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; |</span><span\; class="notranslate">\; \&Center\; Dot;</span>\mathrm{<span\; class="notranslate">\; \&eta;</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; \&Center\; Dot;</span><span\; class="notranslate">\; (</span><span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; Z</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; Z</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; |</span><span\; class="notranslate">\; <</span>\mathrm{<span\; class="notranslate">\; the\; th</span>}<span\; class="notranslate">\; )</span>$

η is the weight coefficient.

The output of the weighted filter is the expected value F _i,j (n):

${\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="W"\; filenames="H2009101045511E00012.png"\; state="\{\{state\}\}">W</figure-callout></span>}\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; \&Center\; Dot;</span>{\mathrm{<span\; class="notranslate">\; Z</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; <figure-callout\; id="2"\; label="W"\; filenames="H2009101045511E00012.png"\; state="\{\{state\}\}">W</figure-callout></span>}\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; \&Center\; Dot;</span>{\mathrm{<span\; class="notranslate">\; Z</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; W</span>}\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; \&Center\; Dot;</span>{\mathrm{<span\; class="notranslate">\; Z</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; W</span>}\mathrm{<span\; class="notranslate">\; 4</span>}<span\; class="notranslate">\; \&Center\; Dot;</span>{\mathrm{<span\; class="notranslate">\; Z</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; Z</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span>}{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="W"\; filenames="H2009101045511E00012.png"\; state="\{\{state\}\}">W</figure-callout></span>}\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; <figure-callout\; id="2"\; label="W"\; filenames="H2009101045511E00012.png"\; state="\{\{state\}\}">W</figure-callout></span>}\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; W</span>}\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; W</span>}\mathrm{<span\; class="notranslate">\; 4</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}}$

After obtaining the expected value F _{i, j} (n), use the following formula to perform iterative correction frame by frame:

Y _i,j (n)=G _i,j (n) Z _i,j (n)+X

G _i,j (n)=G _i,j (n-1)-2λZ _i,j (n)·(Z _i,j (n)-F _i,j (n))

Among them, Y _{i, j} (n) is the correction output of the nth frame, G _{i, j} (n) is the gain correction coefficient of the nth frame, and λ is the iteration step size.

The traditional neural network correction method uses the original image to calculate the expected signal. In the neural network correction of the present invention, the corrected output image is used to calculate the expected signal, and the current pixel is added for the expected estimation, and the expected value is calculated by adaptive weighted filtering method. Therefore, the present invention can effectively overcome defects such as target degeneration and ghost images existing in traditional neural network correction. Both the method based on the combination of scene statistics and neural network correction proposed by the present invention and the method of combining one-point correction and neural network can eliminate spatial low-frequency noise well. However, when the image background is constantly changing, the method of combining one-point correction with neural network will make the correction effect worse because of the difference between the initial sampling background image and the moving background image. When this error increases, the correction error will also increase. , and even the target will be submerged in the background. The correction method of the present invention is based on the combination of scene statistics and neural network, uses the scene correlation to continuously carry out statistics, and updates the background image in real time. In the real-time correction of continuously changing scenes, the method of the invention has a good correction effect, suppresses target degradation and ghost images, and the change of the background image has almost no influence on the correction effect.

Claims (4)

1. A neural network non-uniformity correction method based on scene statistics is characterized in that it specifically includes the following steps: initializing image correlation matrix and parameters; carrying out blind element detection and compensation according to image pixel gray values; Calculate the original image input mean value E[X _{i, j} ] by the domain mean value, calculate the corrected output mean value E[Y _{i, j} ] of the image pixel by the spatial correlation of the image, use the original image X _{i, j} (n ) minus the original image input mean value E[X _{i, j} ] to obtain the image pixel gray value Z _{i, j} (n) that does not contain low-frequency spatial noise, and perform non-uniformity correction on the image bias; use the neural network correction method , according to the standard deviation threshold of the correction error, the regional attribute of the image pixel is judged, and the gain nonlinear correction is performed on the image after the scene statistical correction and does not contain low-frequency spatial noise. The filter assigns larger weights to pixels in the same area, and assigns smaller weights to pixels in different areas. The weighted average filter weight W _{p, q} (n) determines the output image expected value F _{i, j} (n). 2. The neural network non-uniformity correction method according to claim 1, wherein the blind element detection and compensation specifically include: calculating the pixel response rate according to the image pixel gray value; centering on the response rate, searching for the maximum and the smallest image pixel gray value B _max , B _min ; remove B _max , B _min , and then calculate the average gray value of the remaining image pixels in the window

Blind element determination is performed on each pixel to determine the position of the blind element, and the image pixel at the corresponding position in the blind element matrix is set to 1; finally, the detected blind element position is replaced by the mean value of the blind element pixel 4 neighbors for compensation , get the image with blind elements removed. 3. The neural network non-uniformity correction method according to claim 2, characterized in that, the specific method for determining the blind cell position is: when the condition is satisfied

or

, the position is a blind pixel, otherwise it is a valid pixel. 4. The neural network non-uniformity correction method according to claim 2, characterized in that, the specific step of calculating the pixel response rate is: taking the time-domain mean value of the pixel gray value of the continuous k-frame image sequence as the current frame each Pixel responsivity B _ij .

Images