CN107895356A - A kind of near-infrared image Enhancement Method based on steerable pyramid - Google Patents

Abstract

The invention relates to a near-infrared image enhancement method based on a steerable pyramid. The original image undergoes multi-scale decomposition of the steerable pyramid to obtain high-frequency coefficients and low-frequency coefficients at multiple scales, and then separately processes the high-frequency coefficients and low-frequency coefficients to achieve noise removal and contrast improvement of the original image. The invention performs multi-scale decomposition and reconstruction on the original image, and obtains high-frequency coefficients and low-frequency coefficients in multiple directions for each scale in the decomposition process. The high-frequency coefficients use the threshold method to remove image noise, and the low-frequency coefficients use fuzzy set nonlinear transformation. Improve the image contrast, and then reconstruct the image through the inverse transformation of the steerable pyramid, and finally achieve near-infrared image enhancement.

Description

A Near Infrared Image Enhancement Method Based on Steerable Pyramid

technical field

The invention relates to a near-infrared image enhancement method based on a steerable pyramid, which belongs to the technical field of image processing.

Background technique

Near-infrared imaging mainly uses the near-infrared ambient light imaging reflected by the target to be measured. Compared with visible light imaging, it has better atmospheric penetration performance and human skin penetration. Therefore, near-infrared imaging is widely used in military, medical and many industrial productions. have a wide range of applications. However, limited by the production process, it is difficult for the devices and methods in the prior art to improve the quality of near-infrared images from the hardware level, and practical applications have higher and higher requirements for the quality of near-infrared images, which cannot meet the application requirements of the prior art ; Image processing through image enhancement algorithms to improve image quality has become a mainstream research direction.

The enhancement of near-infrared images mainly solves the problems of enhancing image contrast, highlighting image details, and eliminating noise. Traditional image enhancement algorithms are divided into spatial domain enhancement and frequency domain enhancement. The spatial domain enhancement directly processes the pixel gray value, the main methods are gray scale stretching, histogram equalization, unsharp mask, etc.; the frequency domain enhancement first transforms the image into the frequency domain, and then uses the frequency domain filter to process the frequency domain image Implement enhancements. Pure spatial domain enhancement or frequency domain enhancement cannot meet the requirements of existing systems to eliminate noise and enhance details. In recent years, in response to actual engineering needs, many researchers have proposed new infrared image enhancement algorithms, but these methods often rely on application scenarios and have targeted functions. For example, Yun Haijiao et al. proposed a combination of histogram equalization and fuzzy set theory Infrared image enhancement (Yun Haijiao, Wu Zhiyong, Wang Guanguan, etc. Infrared image enhancement combined with histogram equalization and fuzzy set theory [J]. Journal of Computer-Aided Design and Graphics, 2015,27(8):1498-1505.) Primarily improves image contrast.

Contents of the invention

Aiming at the deficiencies of the prior art, the present invention provides a near-infrared image enhancement method based on a steerable pyramid.

Summary of the invention:

The invention proposes a near-infrared image enhancement method based on a steerable pyramid. The original image undergoes multi-scale decomposition of the steerable pyramid to obtain high-frequency coefficients and low-frequency coefficients at multiple scales, and then separately processes the high-frequency coefficients and low-frequency coefficients to achieve noise removal and contrast improvement of the original image.

Specifically, the main technical content of the present invention includes multi-scale decomposition and reconstruction of the original image, obtaining high-frequency coefficients and low-frequency coefficients in multiple directions for each scale in the decomposition process, and using the threshold method to remove the high-frequency coefficients from the image. Noise and low-frequency coefficients use fuzzy set nonlinear transformation to improve image contrast, and then perform image reconstruction through steerable pyramid inverse transformation, and finally achieve near-infrared image enhancement.

Technical scheme of the present invention is:

A near-infrared image enhancement method based on a steerable pyramid, comprising the following steps:

1) Decompose the input image into a steerable pyramid to obtain low-frequency coefficients at the nth scale and high-frequency coefficients in K directions; n=1 at the first steerable pyramid decomposition;

2) The high-frequency coefficients in the K directions obtained in step 1) are respectively used to remove image noise by the threshold method; that is, the threshold is set in the frequency domain to isolate the noise information of the image; wherein, the processing result of the threshold method is mainly composed of the threshold value and The threshold function is determined by two factors;

In the high-frequency coefficients, the image signal coefficient value is generally larger, and the noise coefficient value is smaller. Therefore, by selecting an appropriate threshold, the signal coefficient and the noise coefficient can be separated through the threshold function.

3) The low-frequency image obtained in step 1) is down-sampled by 2 to obtain image I1, and the resolution of image I1 is reduced to 1/4 of the original image; n=n+1, repeat steps 1), 2) N-1 Times; Wherein, image I1 is the input image of the next layer of steerablepyramid decomposition process; Setting variable m=N;

4) The low-frequency coefficients of the mth scale are subjected to fuzzy set nonlinear transformation; that is, the pixel values of the low-frequency image are transformed into fuzzy functions in the spatial domain; the low-pass coefficients retain the general features of the original image, and the present invention processes each scale through fuzzy set theory Low-pass factor to enhance image contrast.

5) Perform steerable pyramid inverse transformation on the processed high-frequency coefficients and low-frequency coefficients on the mth scale, and reconstruct the enhanced image on the mth scale;

6) The enhanced image obtained in step 5) is subjected to 2 up-sampling to obtain a low-frequency image on the m-1th scale; m=m-1;

7) Repeat steps 4), 5), and 6) until an image with the same resolution as the original image is obtained;

8) Perform an unsharp mask on the image obtained in step 7), enhance the image details, and obtain an enhanced image.

Preferably according to the present invention, in the step 1), the value of K includes 2, 3 or 4; in the step 3), the value of N includes 2, 3 or 4.

Preferably according to the present invention, the threshold in the step 2) Wherein, σ is the mean square error of the noise signal, and n is the signal length of the steerable pyramid; the mean square error σ of the noise signal is estimated by the median value of the high-frequency coefficient, that is, σ=median(I), wherein, I is the image pixel value, and median(I ) represents the median value of the selected image pixel value; the selected threshold is the minimum value among multiple high-frequency coefficient thresholds at the smallest scale; because there are high-frequency information in multiple directions in each scale, there are also multiple thresholds obtained through calculation, The present invention selects the smallest one.

The threshold function used in the threshold method in the step 2) is a hard threshold function, and remains unchanged when the signal value is greater than the threshold, and is set to zero when it is less than the threshold;

Preferably according to the present invention, in said step 4), the specific steps of fuzzy set nonlinear transformation are as follows:

4.1) Transform the image to the fuzzy domain through a linear membership function; linear membership function:

Among them, x _max and x _min are the maximum value and minimum value of the original image pixel respectively, and x _ij is the pixel value of each point; the linear membership function normalizes the pixel value of the digital image to between 0 and 1;

4.2) Calculate fuzzy contrast; fuzzy contrast calculation formula:

in is the mean value of μ _ij after the original image is transformed by the linear membership function;

4.3) Non-linear transformation of fuzzy contrast; select transformation function ψ(x) to stretch image gray scale, Fc'=ψ(Fc); requirements for function ψ(x): ψ(0)=0, ψ(1)= 1, ψ(x)≥x x∈(0,1);

4.4) inverse transformation; after the transformation function in step 4.3), the value of fuzzy contrast F _c is stretched and compressed accordingly; the image pixel value after the fuzzy contrast transformation is obtained by following formula inverse transformation; the function of two-step inverse transformation as follows:

x _ij '=μ _ij '(x _max -x _min )+x _min .

Further preferably, in step 4.3), ψ(x)=4x-6x ² +4x ³ -x ⁴ .

Preferably according to the present invention, the up-sampling process in step 6) is realized by bilinear interpolation.

Preferably according to the present invention, in said step 8), the mathematical expression of the unsharp mask algorithm:

v=u+γ(u-w),

Among them, v is the enhanced image, u is the input image, w is the result of linear low-pass filtering; γ is the weighting coefficient, γ>0.

Further preferably, the low-pass filter is an L1_2 filter in a steerable pyramid at the first scale.

The beneficial effects of the present invention are:

1. The near-infrared image enhancement method of the present invention uses the reversibility of the steerable pyramid decomposition model to decompose the original image into multiple resolutions for processing, and simultaneously separates high-frequency and low-frequency information on each resolution, respectively Deal with image noise and contrast issues; use frequency-domain threshold segmentation for high-frequency images at each resolution, and use space-domain fuzzy set nonlinear transformation for low-frequency images; high-frequency coefficients of images retain image detail information and noise information, The low-frequency coefficients retain the general features of the original image; by processing the high-frequency coefficients of the image, the noise of the image can be filtered out, and the processing of the low-frequency coefficients of the image can improve the contrast of the image; the combination of the two processes can effectively achieve image removal. noise and increased contrast requirements;

2. The near-infrared image enhancement method of the present invention utilizes the multi-directionality of the steerable pyramid decomposition model to better preserve the texture features of the image and effectively reduce the loss of detail in the process of removing noise from the image.

Description of drawings

Figure 1 is a block diagram of steerable pyramid decomposition and reconstruction;

Fig. 2 is a schematic diagram of low-pass filter L1_1;

Fig. 3 is a schematic diagram of high-pass filter H1_1;

Fig. 4 is a schematic diagram of low-pass filter L1_2;

Fig. 5 is a schematic diagram of a directional bandpass filter B1_1;

Fig. 6 is the schematic diagram of directional bandpass filter B1_2;

Fig. 7 is the schematic diagram of directional bandpass filter B1_3;

Fig. 8 is the original image in the experiment;

Fig. 9 is the image processed by the present invention;

Fig. 10 is the image after histogram equalization processing;

Figure 11 is the result after gray scale stretching processing;

Fig. 12 is the result after processing the references in the background technology.

Detailed ways

The present invention will be further described below in conjunction with the embodiments and the accompanying drawings, but is not limited thereto.

Example 1

As shown in Figure 1.

A near-infrared image enhancement method based on a steerable pyramid, the image shown in Figure 8 is processed, the steps are as follows:

1) Decompose the input image into a steerable pyramid to obtain low-frequency coefficients at the nth scale and high-frequency coefficients in three directions; n=1 during the first steerable pyramid decomposition; filters L1_1 and H1_1 used in the steerable pyramid decomposition , L1_2, B1_1, B1_2, B1_3 are shown in Fig. 2-Fig. 7 respectively.

2) The high-frequency coefficients in the three directions obtained in step 1) are respectively used to remove image noise by the threshold method; that is, the threshold is set in the frequency domain to isolate the noise information of the image;

The threshold Wherein, σ is the mean square error of the noise signal, and n is the signal length of the steerable pyramid; the mean square error σ of the noise signal is estimated by the median value of the high-frequency coefficient, that is, σ=median(I), wherein, I is the image pixel value, and median(I ) represents the median value of the selected image pixel value; the selected threshold is the minimum value among multiple high-frequency coefficient thresholds at the smallest scale; because there are high-frequency information in multiple directions in each scale, there are also multiple thresholds obtained through calculation, The present invention selects the smallest one.

The threshold function used by the threshold method is a hard threshold function, which remains unchanged when the signal value is greater than the threshold, and is set to zero when it is less than the threshold;

Among them, the processing result of the threshold method is mainly determined by two factors: the threshold size and the threshold function;

In the high-frequency coefficients, the image signal coefficient value is generally larger, and the noise coefficient value is smaller. Therefore, by selecting an appropriate threshold, the signal coefficient and the noise coefficient can be separated through the threshold function.

3) The low-frequency image obtained in step 1) is down-sampled by 2 to obtain image I1, and the resolution of image I1 is reduced to 1/4 of the original image; n=n+1, repeat steps 1), 2) 2 times; Among them, image I1 is the input image of the next layer of steerable pyramid decomposition process; set variable m=3;

4) The low-frequency coefficients of the mth scale are subjected to fuzzy set nonlinear transformation; that is, the pixel values of the low-frequency image are transformed into fuzzy functions in the spatial domain; the low-pass coefficients retain the general features of the original image, and the present invention processes each scale through fuzzy set theory Low-pass factor to enhance image contrast.

The specific steps of fuzzy set nonlinear transformation are as follows:

4.1) Transform the image to the fuzzy domain through a linear membership function; linear membership function:

Among them, x _max and x _min are the maximum value and minimum value of the original image pixel respectively, and x _ij is the pixel value of each point; the linear membership function normalizes the pixel value of the digital image to between 0 and 1;

4.2) Calculate fuzzy contrast; fuzzy contrast calculation formula:

in is the mean value of μ _ij after the original image is transformed by the linear membership function;

4.3) Non-linear transformation of fuzzy contrast; select transformation function ψ(x) to stretch image gray scale, Fc'=ψ(Fc); requirements for function ψ(x): ψ(0)=0, ψ(1)= 1, ψ(x)≥xx∈(0,1); ψ(x)=4x-6x ² +4x ³ -x ⁴ .

4.4) inverse transformation; after the transformation function in step 4.3), the value of fuzzy contrast F _c is stretched and compressed accordingly; the image pixel value after the fuzzy contrast transformation is obtained by following formula inverse transformation; the function of two-step inverse transformation as follows:

x _ij '=μ _ij '(x _max -x _min )+x _min .

5) Perform steerable pyramid inverse transformation on the processed high-frequency coefficients and low-frequency coefficients on the mth scale to reconstruct the enhanced image on the mth scale; the inverse transformation is shown in the right half of Figure 1.

6) The enhanced image obtained in step 5) is subjected to 2 up-sampling to obtain a low-frequency image on the m-1th scale; m=m-1; the up-sampling process is realized by bilinear interpolation.

7) Repeat steps 4), 5), and 6) until an image with the same resolution as the original image is obtained;

8) Perform an unsharp mask on the image obtained in step 7), enhance the image details, and obtain an enhanced image.

In said step 8), the mathematical expression of the unsharp mask algorithm:

v=u+γ(u-w),

Among them, v is the enhanced image, u is the input image, w is the result of linear low-pass filtering; γ is the weighting coefficient, γ>0. γ=1. The low-pass filter used in this step is the L1_2 filter in the steerable pyramid at the first scale.

The image obtained after the original image in Fig. 8 is processed by the present invention is shown in Fig. 9 .

Comparative example 1:

Using the method of "Infrared Image Enhancement Combining Histogram Equalization and Fuzzy Set Theory" in the background technology to perform image enhancement processing on Fig. 8; the processing result is shown in Fig. 12 .

Comparative example 2:

The image enhancement processing of FIG. 8 is performed by using the histogram equalization method in the prior art; the processing result is shown in FIG. 10 .

Comparative example 3:

Image enhancement processing is performed on Fig. 8 by using the grayscale stretching method in the prior art; the processing result is shown in Fig. 11 .

Comparing Figures 9 to 12, it can be seen that the quality of the image processed by the present invention is better than that of several other processing methods.

Several objective evaluation index data of embodiment 1 and comparative example processing result are shown in the following table:

Through the comparison of various indicators, it can be seen that the image enhancement in Example 1 has a higher contrast gain and information entropy, and has advantages compared to the traditional image enhancement method and the method in Comparative Example 1. Mean square error and peak signal-to-noise ratio , indicating that the image enhancement method in Example 1 has an obvious effect in the near-infrared image enhancement processing.

Claims (8)

1. a kind of near-infrared image Enhancement Method based on steerable pyramid, it is characterised in that as follows including step：

1) input picture is subjected to steerable pyramid decomposition, obtains the low frequency coefficient under n-th of yardstick and K direction High frequency coefficient；N=1 when first time steerable pyramid is decomposed；

2) threshold method is respectively adopted in the high frequency coefficient in the K direction obtained in step 1) and removes picture noise；I.e. in frequency domain Given threshold, to isolate the noise information of image；

3) the down-sampled of the low-frequency image obtained in step 1) progress 2 is obtained into image I1, image I1 resolution ratio is reduced to original The 1/4 of image；N=n+1, repeat step 1), 2) N-1 times；Wherein, image I1 is next layer of pyramid points of steerable The input picture of solution preocess；Set variable m=N；

4) low frequency coefficient of m-th of yardstick is subjected to fuzzy set nonlinear transformation；Pixel i.e. in spatial domain to low-frequency image Value carries out ambiguity function conversion；

5) high frequency coefficient on m-th of yardstick after processing and low frequency coefficient are subjected to steerable pyramid inverse transformations, weight Structure goes out enhanced image on m-th of yardstick；

6) the enhanced image obtained in step 5) is carried out to 2 up-sampling, obtains the low-frequency image on the m-1 yardstick；m =m-1；

7) repeat step 4), 5), 6), until obtaining the image with the equal resolution ratio of original image；

8) image obtained in step 7) is subjected to unsharp mask, strengthens image detail, obtain enhanced image.

2. the near-infrared image Enhancement Method according to claim 1 based on steerable pyramid, its feature exist In in the step 1), K value includes 2,3 or 4；In the step 3), N value includes 2,3 or 4.

3. the near-infrared image Enhancement Method according to claim 1 based on steerable pyramid, its feature exist In the threshold value in the step 2)Wherein, σ is noise signal mean square deviation, and n is steerable pyramid Signal length；Noise signal meansquaredeviationσ is by the mediant estimation of high frequency coefficient, i.e. σ=median (I), wherein, I is image Pixel value, median (I) represent to choose the intermediate value of image pixel value；

The threshold function table that threshold method uses in the step 2) is hard threshold function, and signal value keeps constant when being more than threshold value, small Zero setting when threshold value；

<mrow> <mi>y</mi> <mo>=</mo> <mfenced open = "{" close = ""> <mtable> <mtr> <mtd> <mi>x</mi> </mtd> <mtd> <mrow> <mo>|</mo> <mi>x</mi> <mo>|</mo> <mo>&amp;GreaterEqual;</mo> <mi>&amp;lambda;</mi> </mrow> </mtd> </mtr> <mtr> <mtd> <mn>0</mn> </mtd> <mtd> <mrow> <mo>|</mo> <mi>x</mi> <mo>|</mo> <mo>&lt;</mo> <mi>&amp;lambda;</mi> </mrow> </mtd> </mtr> </mtable> </mfenced> <mo>.</mo> </mrow>

4. the near-infrared image Enhancement Method according to claim 1 based on steerable pyramid, its feature exist In in the step 4), fuzzy set nonlinear transformation comprises the following steps that：

4.1) image is transformed to by fuzzy field by linear membership function；Linear membership function：

<mrow> <msub> <mi>&amp;mu;</mi> <mrow> <mi>i</mi> <mi>j</mi> </mrow> </msub> <mo>=</mo> <mfrac> <mrow> <msub> <mi>x</mi> <mrow> <mi>i</mi> <mi>j</mi> </mrow> </msub> <mo>-</mo> <msub> <mi>x</mi> <mi>min</mi> </msub> </mrow> <mrow> <msub> <mi>x</mi> <mi>max</mi> </msub> <mo>-</mo> <msub> <mi>x</mi> <mi>min</mi> </msub> </mrow> </mfrac> <mo>,</mo> </mrow>

Wherein, x_maxAnd x_minRespectively original image pixel maximum and minimum value, x_ijFor each point pixel value；Linear membership function will The pixel value of digital picture is normalized between 0 and 1；

4.2) fuzzy contrast is calculated；Fuzzy contrast calculation formula：

<mrow> <msub> <mi>F</mi> <mi>c</mi> </msub> <mo>=</mo> <mo>|</mo> <mfrac> <mrow> <msub> <mi>&amp;mu;</mi> <mrow> <mi>i</mi> <mi>j</mi> </mrow> </msub> <mo>-</mo> <msub> <mover> <mi>&amp;mu;</mi> <mo>&amp;OverBar;</mo> </mover> <mrow> <mi>i</mi> <mi>j</mi> </mrow> </msub> </mrow> <mrow> <msub> <mi>&amp;mu;</mi> <mrow> <mi>i</mi> <mi>j</mi> </mrow> </msub> <mo>+</mo> <msub> <mover> <mi>&amp;mu;</mi> <mo>&amp;OverBar;</mo> </mover> <mrow> <mi>i</mi> <mi>j</mi> </mrow> </msub> </mrow> </mfrac> <mo>|</mo> <mo>,</mo> </mrow>

WhereinFor μ after the linear membership function conversion of original image_ijAverage；

4.3) fuzzy contrast nonlinear transformation；Choose transforming function transformation function ψ (x) stretching gradation of images, Fc'=ψ (Fc)；To function ψ (x) requirement：ψ (0)=0, ψ (1)=1, ψ (x) >=x x ∈ (0,1)；

4.4) inverse transformation；After the transforming function transformation function processing in step 4.3), fuzzy contrast F_cValue by phase strain stretch and pressure Contracting；Image pixel value after fuzzy contrast conversion is obtained by following formula inverse transformation；The function of two step inverse transformations is as follows：

<mrow> <msup> <msub> <mi>&amp;mu;</mi> <mrow> <mi>i</mi> <mi>j</mi> </mrow> </msub> <mo>&amp;prime;</mo> </msup> <mo>=</mo> <mfenced open = "{" close = ""> <mtable> <mtr> <mtd> <mfrac> <mrow> <msub> <mover> <mi>u</mi> <mo>&amp;OverBar;</mo> </mover> <mrow> <mi>i</mi> <mi>j</mi> </mrow> </msub> <mrow> <mo>(</mo> <mn>1</mn> <mo>-</mo> <msup> <msub> <mi>F</mi> <mi>c</mi> </msub> <mo>&amp;prime;</mo> </msup> <mo>)</mo> </mrow> </mrow> <mrow> <mn>1</mn> <mo>+</mo> <msup> <msub> <mi>F</mi> <mi>c</mi> </msub> <mo>&amp;prime;</mo> </msup> </mrow> </mfrac> </mtd> <mtd> <mrow> <msub> <mi>u</mi> <mrow> <mi>i</mi> <mi>j</mi> </mrow> </msub> <mo>&amp;le;</mo> <msub> <mover> <mi>u</mi> <mo>&amp;OverBar;</mo> </mover> <mrow> <mi>i</mi> <mi>j</mi> </mrow> </msub> </mrow> </mtd> </mtr> <mtr> <mtd> <mrow> <mn>1</mn> <mo>-</mo> <mfrac> <mrow> <mo>(</mo> <mn>1</mn> <mo>-</mo> <msub> <mover> <mi>u</mi> <mo>&amp;OverBar;</mo> </mover> <mrow> <mi>i</mi> <mi>j</mi> </mrow> </msub> <mo>)</mo> <mo>(</mo> <mn>1</mn> <mo>-</mo> <msup> <msub> <mi>F</mi> <mi>c</mi> </msub> <mo>&amp;prime;</mo> </msup> <mo>)</mo> </mrow> <mrow> <mn>1</mn> <mo>+</mo> <msup> <msub> <mi>F</mi> <mi>c</mi> </msub> <mo>&amp;prime;</mo> </msup> </mrow> </mfrac> </mrow> </mtd> <mtd> <mrow> <msub> <mi>u</mi> <mrow> <mi>i</mi> <mi>j</mi> </mrow> </msub> <mo>&gt;</mo> <msub> <mover> <mi>u</mi> <mo>&amp;OverBar;</mo> </mover> <mrow> <mi>i</mi> <mi>j</mi> </mrow> </msub> </mrow> </mtd> </mtr> </mtable> </mfenced> <mo>,</mo> </mrow>

x_ij'=μ_ij′(x_max-x_min)+x_min。

5. the near-infrared image Enhancement Method according to claim 4 based on steerable pyramid, its feature exist In ψ (x)=4x-6x in the step 4.3)²+4x³-x⁴。

6. the near-infrared image Enhancement Method according to claim 1 based on steerable pyramid, its feature exist In the upsampling process in the step 6) is realized by bilinear interpolation method.

7. the near-infrared image Enhancement Method according to claim 1 based on steerable pyramid, its feature exist In, in the step 8), unsharp mask algorithm mathematics expression formula：

V=u+ γ (u-w),

Wherein, v is enhanced image, and u is input picture, and w is the result after linear low-pass ripple；γ is weight coefficient, γ ＞ 0.

8. the near-infrared image Enhancement Method according to claim 7 based on steerable pyramid, its feature exist In the low pass filter is the L1_2 wave filters in steerable pyramid under first yardstick.