CN114187196B - Self-adaptive multi-integration time infrared image sequence optimizing method - Google Patents
Self-adaptive multi-integration time infrared image sequence optimizing method Download PDFInfo
- Publication number
- CN114187196B CN114187196B CN202111439718.7A CN202111439718A CN114187196B CN 114187196 B CN114187196 B CN 114187196B CN 202111439718 A CN202111439718 A CN 202111439718A CN 114187196 B CN114187196 B CN 114187196B
- Authority
- CN
- China
- Prior art keywords
- image
- images
- gray
- optimal
- sequence
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 43
- 230000005855 radiation Effects 0.000 claims abstract description 48
- 238000011156 evaluation Methods 0.000 claims abstract description 29
- 230000011218 segmentation Effects 0.000 claims abstract description 23
- 230000010354 integration Effects 0.000 claims abstract description 20
- 230000006870 function Effects 0.000 claims abstract description 18
- 238000012216 screening Methods 0.000 claims abstract description 6
- 238000013441 quality evaluation Methods 0.000 claims description 22
- 238000004364 calculation method Methods 0.000 claims description 8
- 230000004044 response Effects 0.000 claims description 7
- 230000003044 adaptive effect Effects 0.000 claims description 5
- 238000005259 measurement Methods 0.000 claims description 3
- NAWXUBYGYWOOIX-SFHVURJKSA-N (2s)-2-[[4-[2-(2,4-diaminoquinazolin-6-yl)ethyl]benzoyl]amino]-4-methylidenepentanedioic acid Chemical compound C1=CC2=NC(N)=NC(N)=C2C=C1CCC1=CC=C(C(=O)N[C@@H](CC(=C)C(O)=O)C(O)=O)C=C1 NAWXUBYGYWOOIX-SFHVURJKSA-N 0.000 claims description 2
- 239000011159 matrix material Substances 0.000 claims description 2
- 238000003384 imaging method Methods 0.000 abstract description 18
- 238000003331 infrared imaging Methods 0.000 abstract description 9
- 230000004927 fusion Effects 0.000 description 20
- 238000003709 image segmentation Methods 0.000 description 8
- 230000000694 effects Effects 0.000 description 7
- 238000009826 distribution Methods 0.000 description 5
- 238000010187 selection method Methods 0.000 description 4
- 238000001931 thermography Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 238000005057 refrigeration Methods 0.000 description 3
- 230000000007 visual effect Effects 0.000 description 3
- 230000005587 bubbling Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 230000016776 visual perception Effects 0.000 description 2
- 201000004569 Blindness Diseases 0.000 description 1
- 230000002411 adverse Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 125000004122 cyclic group Chemical group 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000004445 quantitative analysis Methods 0.000 description 1
- 238000012163 sequencing technique Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/90—Dynamic range modification of images or parts thereof
- G06T5/94—Dynamic range modification of images or parts thereof based on local image properties, e.g. for local contrast enhancement
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/40—Image enhancement or restoration using histogram techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/136—Segmentation; Edge detection involving thresholding
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/70—Circuitry for compensating brightness variation in the scene
- H04N23/741—Circuitry for compensating brightness variation in the scene by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10048—Infrared image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30168—Image quality inspection
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Radiation Pyrometers (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a self-adaptive multi-integration time infrared image sequence optimizing method, and belongs to the technical field of high-dynamic infrared imaging. The implementation method of the invention comprises the following steps: dividing the sequence image based on the region growing points, respectively searching the optimal functions of the normal temperature target and the strong radiation target for the divided image, respectively obtaining local optimal images from the input sequence image according to the optimal functions of the normal temperature target and the strong radiation target, reserving image details as far as possible, enabling the normal temperature target to be imaged clearly by adopting a high integration time exposure mode, and enabling the strong radiation target to be imaged clearly by adopting a low integration time exposure mode; screening a global optimal image from the input sequence images based on gray information evaluation indexes, and eliminating discontinuity of the local optimal image selected based on the region growing point segmentation method; and obtaining an infrared high dynamic range HDR image by fusing the local optimal image and the global optimal image, and performing enhanced display to improve the imaging quality in a high dynamic scene.
Description
Technical Field
The invention relates to a self-adaptive multi-integration time infrared image sequence optimizing method, and belongs to the technical field of high-dynamic infrared imaging.
Background
In an actual Infrared application scene, when a strong radiation target such as sun, flame or interference bomb exists together with a normal temperature target, the radiation brightness span is large, and for a refrigeration Infrared Focal plane (Infrared Focal PLANE ARRAY, IRFPA) thermal imaging system of the current 14-bit A/D, even if the nonlinear S effect of the response of a detector is not considered, the equivalent dynamic range is only about 84dB, which is far smaller than the natural scene radiation difference of the radiation interference. That is, no matter how the imaging parameters of the system are adjusted, underexposure or overexposure phenomenon can be caused, and all details of the scene can not be captured at one time, which can cause adverse effects on various target detection and identification tasks, and a High dynamic range (High-DYNAMIC RANGE, HDR) imaging method is required to adapt to the effective imaging of the full-radiation scene.
The current methods for HDR infrared thermal imaging include: 1) Based on pixel a/D technology. The method designs pixel-level AD on each pixel respectively to realize HDR imaging with high bit width and low noise, is an advanced IRFPA design method, but the detector has complex process, large development difficulty and higher cost at the present stage, and because the NETD is lower, the dynamic range of a scene obtained by adopting 18-bit AD is difficult to meet the requirement of an actual HDR scene. 2) Variable integration time imaging techniques based on superframes. Based on the prior IRFPA, the method adopts periodic cyclic variable integration time ultrahigh frame frequency imaging, expands the imaging into HDR thermal imaging by fusing Low dynamic range (Low-DYNAMIC RANGE, LDR) images with different integration time, completely adopts a digital program control mode, does not need to increase a mechanical structure, has higher imaging frame frequency, generally adopts a larger relative aperture for infrared signals with lower visible light to pursue detection of long-distance and weak and small targets, and adopts a variable integration time method for scenes with strong radiation interference to realize the HDR imaging.
Unlike a general visible light image, an infrared thermal image is generally low in contrast, large in noise, unclear in detail, and concentrated in gray distribution. The variable integration time method has the following problems: 1) In the prior study, the fusion image selection of the HDR infrared thermal image has a certain blindness, the response value change and the exposure degree are generally judged by combining the LDR images obtained by the acquisition software according to priori knowledge, and two to three LDR images are selected in the image sequence for fusion, or the acquired image sequence is fused by a certain weight calculation method. But inaccuracy misjudgment of manual selection, gray level non-uniformity caused by noise points and blind points can influence the selection deviation of LDR images from 'best'. 2) Due to the limitation of real-time performance, when the imaging device shoots a scene, parameters with moderate image exposure are difficult to adjust, and many students expect to adjust exposure parameters of a next frame image according to current scene frame information. Some researches try to compare the full-pixel gray average value of the current image with a preset optimal gray average value, and the method has the advantages of simple processing flow, capability of optimizing the image quality to a certain extent, neglecting local gray difference and poor overall effect. The self-adaptive exposure method based on the gray histogram (fixed block theory, fuzzy logic calculation weight theory, pixel saturation threshold theory, scene area segmentation theory and the like) developed later and the research of the method for adjusting the integration time by utilizing other image information (such as Sobel operator to extract image evaluation indexes of edge information, information entropy, gradient difference and the like) also provide an integration time self-adaptive adjustment way for evaluating the front-end LDR output image sequence, and the method has the problems of weak self-adaptability or single evaluation index and non-ideal imaging.
In view of the foregoing, it is desirable to develop a method for adaptively selecting a multi-integral infrared image sequence to improve the quality of a high dynamic fusion image. How to define the image selection standard, solve the exposure quality problem caused by strong radiation of a high dynamic scene, and verify the validity of the fused image effect is a key problem worth solving.
Disclosure of Invention
Aiming at the problem that the dynamic range in a high dynamic scene cannot be effectively and comprehensively captured, the invention aims to provide a self-adaptive multi-integration time infrared image sequence preferred method, aiming at the response requirement of the high dynamic range in the high dynamic scene, inputting a sequence image, dividing the sequence image based on region growing points, respectively searching the divided images for optimal functions of a normal temperature target and a strong radiation target, respectively obtaining local optimal images from the input sequence image according to the optimal functions of the normal temperature target and the strong radiation target, wherein the local optimal images comprise a normal temperature target local optimal image (high exposure image) and a strong radiation target local optimal image (low exposure image) so as to keep image details as far as possible, and enabling the normal temperature target to be clearly imaged by adopting a high integration time exposure mode, so that the normal temperature target local optimal image is called a high exposure image, and enabling the strong radiation target to be clearly imaged by adopting a low integration time exposure mode, and the strong radiation target local optimal image is called a low exposure image; in addition, comprehensively screening global optimal images from the input sequence images based on gray information evaluation indexes, and eliminating discontinuity of local optimal images selected based on a region growing point segmentation method; and the local optimal image and the global optimal image are fused to obtain an infrared high dynamic range HDR image, and enhanced display is performed, namely, under the condition that the dynamic range of an infrared imaging device is limited, the dynamic range of infrared imaging is effectively enlarged, and the imaging quality under a high dynamic scene is improved.
The aim of the invention is achieved by the following technical scheme.
The invention discloses a self-adaptive multi-integration time infrared image sequence optimizing method, which comprises the following steps:
and acquiring infrared sequence images under different integration time aiming at the same high dynamic response scene.
Inputting the acquired sequence images to a region growing point segmentation module, segmenting the sequence images based on a region growing point method, respectively searching the segmented images for optimal functions of a normal temperature target and a strong radiation target, respectively obtaining local optimal images from the input sequence images according to the optimal functions of the normal temperature target and the strong radiation target, wherein the local optimal images comprise a normal temperature target local optimal image (high exposure image) and a strong radiation target local optimal image (low exposure image), so that image details are kept as far as possible, the normal temperature target can be imaged clearly by adopting a high integration time exposure mode, the normal temperature target local optimal image is called a high exposure image, the strong radiation target can be imaged clearly by adopting a low integration time exposure mode, and the strong radiation target local optimal image is called a low exposure image. The parameters used for segmentation in the segmentation sequence images based on the region growing points are changed, the gray scale relation between the normal temperature region and the strong radiation region is calculated through the gray scale straight graph, and the gray scale relation is set as the threshold segmentation value of the current scene.
And inputting the acquired sequence images to a global image selection module based on gray information evaluation indexes, comprehensively and quantitatively screening global optimal images containing more information from the input sequence based on the gray information evaluation indexes, and eliminating the discontinuity of the local optimal images selected based on the region growing point segmentation method.
The infrared high dynamic range HDR image is obtained by fusing the normal temperature target local optimal image, the strong radiation target local optimal image and the global optimal image, the preferred selection of the self-adaptive infrared image sequence is realized, the fusion effect error of the artificial experience selection image is reduced, namely, the dynamic range of infrared imaging is effectively enlarged under the condition that the dynamic range of an infrared imaging device is limited, and the imaging quality under a high dynamic response scene is improved.
Based on the change of the parameters for segmentation in the region growing point segmentation sequence images, the gray scale multiple relation between the normal temperature region and the strong radiation region is calculated through the gray scale histogram, and is set as a segmentation threshold value mu of the current scene. Preferably, the threshold parameter μ for segmentation is obtained according to formula (1) for different high dynamic range infrared scenes:
Wherein M is the number of histogram intervals; n is the pixel number of each interval; x and y are the positions corresponding to the maximum N value in the intervals (1, M/2) and (M/2, M) respectively; the edges are one-dimensional tuples.
The specific implementation method for dividing the input image sequence based on the region growing points comprises the following steps: presetting an initial seed point, calculating the relation between the gray scale of each pixel in the sequence image and the seed point, and dividing the sequence image into a strong radiation area and marking the strong radiation area with 1 if the gray scale of the pixel is larger than the gray scale value of the seed point which is mu times; if the gray level of the pixel is less than mu times of the gray level value of the seed point, the pixel is divided into normal temperature areas and marked with 0, so that the target division based on the gray threshold mu is realized, and the expression is shown in the formula (2).
Searching the optimal functions of the normal temperature target and the strong radiation target for the segmented image respectively, and obtaining a local optimal image from the input sequence image according to the optimal functions of the normal temperature target and the strong radiation target, wherein the specific implementation method comprises the following steps: and (3) searching the optimal functions of the normal temperature target and the strong radiation target for the segmented images respectively, namely respectively acquiring gray level average values of two areas of the multi-integration sequence images, and respectively calculating the image closest to the medium gray level according to the bubbling sequence.
Wherein, I x is the input x-th frame image; A gray average value of the calculated area; w is the bit width of the original data; range is the image areas L range and H range to be calculated.
And obtaining local optimal images, namely a normal temperature target local optimal image (high exposure image) and a strong radiation target local optimal image (low exposure image), from the input sequence images according to an optimal function formula (3) of the normal temperature target and the strong radiation target.
Preferably, the overall optimal image containing more information is quantitatively screened from the input sequence comprehensively based on the gray information evaluation index, and the specific implementation method is as follows:
An index system for evaluating the image quality of an input image sequence is established, wherein the index system comprises an information entropy H (X) image quality evaluation index, a gradient difference S obel image quality evaluation index, a gray average value M eanI image quality evaluation index and a gray mean square error S tdI image quality evaluation index.
And comprehensively quantitatively screening from the input sequence based on the gray information evaluation index.
For information entropy H (X) image quality evaluation index: the image is a matrix of m n pixels, each pixel corresponding to a gray scale level between 0 and 255. The probability of defining each gray level is Pi:
Where α i is the sum of the pixels for each gray level. The information entropy is expressed as:
where w is the system bit width.
For gradient difference S obel image quality evaluation index:
Wherein, I x and I y are images of the original image after convolution with h x,hy, respectively. The Sobel operator h x,hy is:
the image quality evaluation index calculation mode for the gray average value M eanI is as follows:
The image quality evaluation index calculation method for the gray mean square error S tdI is as follows:
the calculation formula of the comprehensive evaluation measurement standard is as follows
Mulrank＝SUM(Hrank,Srank,Mrank,Stdrank) (10)
M ulrank is the comprehensive ranking of the images calculated by the sum of the ranking of all the evaluation indexes; h rank is the ranking of information entropy H; s rank is the ranking of the image gradient difference S obel; m rank is the rank of the image gray-scale mean M eanI; s tdrank is the rank name of the image mean square error S tdI.
According to the formula (10), the image quality evaluation index information entropy H (X), the gradient difference S obel, the gray average value M eanI and the gray average value S tdI are comprehensively ordered, and the image with the highest comprehensive ranking is selected, so that the overall situation that all evaluation indexes are in the relatively optimal state is realized, namely the globally optimal image containing more information is quantitatively screened from the input sequence.
Preferably, the fused infrared high dynamic range HDR image is enhanced and displayed based on a detail enhancement cascade algorithm.
The beneficial effects are that:
1. The invention discloses a self-adaptive multi-integration time infrared image sequence optimizing method, which aims at the high dynamic response requirement in a high dynamic scene, inputs a sequence image, segments the sequence image based on a region growing point, respectively searches the segmented image for the optimal functions of a normal temperature target and a strong radiation target, obtains a local optimal image from the input sequence image according to the optimal functions of the normal temperature target and the strong radiation target, comprises the normal temperature target local optimal image (high exposure image) and the strong radiation target local optimal image (low exposure image), and can realize that the image detail is kept as far as possible. In order to enable the normal-temperature target to be imaged clearly, a high-integration time exposure mode is required, so that a local optimal image of the normal-temperature target is called a high-exposure image; to enable clear imaging of the strongly radiating target, a low integration time exposure mode is required, so the locally optimal image of the strongly radiating target is referred to as a low exposure image.
2. The invention discloses a self-adaptive multi-integration time infrared image sequence optimizing method, which is characterized in that a global image selection module based on gray information evaluation indexes carries out comprehensive sorting on image quality evaluation index information entropy, gradient difference S obel, gray mean value M eanI and gray mean square error S tdI according to established global image comprehensive evaluation measurement standards, and an image with highest comprehensive ranking is selected, so that the infrared multi-integration time image selection is converted from qualitative judgment to quantitative analysis, global images with optimal overall imaging quality can be selected from an infrared image sequence, namely global optimal images containing more information are quantitatively screened from an input sequence, and the overall scene has continuity.
3. The invention discloses a self-adaptive multi-integration time infrared image sequence optimizing method, which is used for adaptively selecting a segmentation threshold according to infrared scenes in different high dynamic ranges.
4. According to the self-adaptive multi-integration time infrared image sequence optimizing method disclosed by the invention, the self-adaptive infrared image sequence optimizing selection is realized by fusing the normal-temperature target local optimal image, the strong-radiation target local optimal image and the global optimal image, so that the fusion effect error of the artificial experience selection image is reduced, an infrared high-dynamic-range HDR image is obtained, namely, under the condition that the dynamic range of an infrared imaging device is limited, the dynamic range of infrared imaging is effectively enlarged, and the imaging quality under a high-dynamic-range scene is improved.
5. According to the self-adaptive multi-integration time infrared image sequence preferred method disclosed by the invention, a quantitative standardized flow can provide a guiding method for variable-integration infrared imaging, so that the self-adaptive infrared image sequence preferred selection is realized, the fusion effect error of the artificial experience selected images is reduced, the dynamic range of the high-dynamic-range infrared image is further expanded, and the imaging quality of the thermal infrared imager is improved.
Drawings
FIG. 1 is a flow chart of an infrared image multi-integration time adaptive selection method based on gray information evaluation and region growing point gray segmentation.
Fig. 2 is a mid-wave infrared thermal imaging system for experiments, fig. 2 (a) is a refrigeration mid-wave infrared thermal imager ImageIR, and fig. 2 (b) is a refrigeration mid-wave thermal imager jace.
Fig. 3 is a set of scene graph gray scale distributions. Fig. 3 (a) shows the ir image gray scale distribution of the scene at 100 clock cycles, fig. 3 (b) shows the ir image gray scale distribution of the scene at 5000 clock cycles, and fig. 3 (c) shows the ir image gray scale distribution of the scene at 30000 clock cycles.
Fig. 4 is a sequence of images based on growing point region segmentation. Fig. 4 (a) shows the scene image segmentation situation in fig. 3 at 100 clock cycles, fig. 4 (b) shows the scene image segmentation situation in fig. 3 at 500 clock cycles, fig. 4 (c) shows the scene image segmentation situation in fig. 3 at 1000 clock cycles, fig. 4 (d) shows the scene image segmentation situation in fig. 3 at 2500 clock cycles, fig. 4 (e) shows the scene image segmentation situation in fig. 3 at 5000 clock cycles, fig. 4 (f) shows the scene image segmentation situation in fig. 3 at 10000 clock cycles, fig. 4 (g) shows the scene image segmentation situation in fig. 3 at 15000 clock cycles, and fig. 4 (h) shows the scene image segmentation situation in fig. 3 at 30000 clock cycles.
Fig. 5 is an LDR image of scene 1 and its HDR fusion image. Fig. 5 (a) is an infrared enhanced image of scene 1 at 50 clock cycles, fig. 5 (b) is an infrared enhanced image of scene 1 at 1050 clock cycles, and fig. 5 (c) is an infrared enhanced image of scene 1 at 1150 clock cycles.
Fig. 6 is an LDR image of scene 2 and its HDR fusion image. Fig. 6 (a) is an infrared enhanced image of scene 2 at 400 clock cycles, fig. 6 (b) is an infrared enhanced image of scene 2 at 900 clock cycles, and fig. 6 (c) is an infrared enhanced image of scene 2 at 1050 clock cycles.
Fig. 7 is an LDR image of scene 3 and its HDR fusion image. Fig. 7 (a) is an infrared enhanced image of scene 3 at 500 clock cycles, fig. 7 (b) is an infrared enhanced image of scene 3 at 1500 clock cycles, and fig. 7 (c) is an infrared enhanced image of scene 3 at 30000 clock cycles.
Fig. 8 is an LDR image of scene 4 and its HDR fusion image. Fig. 8 (a) is an infrared enhanced image of scene 4 at 1500 clock cycles, and fig. 8 (b) is an infrared enhanced image of scene 4 at 16000 clock cycles. Since the normal temperature target local optimum image and the global optimum image screened out by the scene are identical, fig. 8 (c) is fig. 8 (b).
The images (a), (b) and (c) in fig. 5 to 8 are respectively a normal-temperature target local optimum image, a strong-radiation target local optimum image and a global optimum image screened out based on the method of the invention in the current scene.
Fig. 5 to 8 (d) are HDR fusion diagrams obtained by artificial experience, and the selection method is as follows: and judging the exposure condition of the image by calculating the relation between the gray level average value and the medium gray level of the image, and selecting three images close to the medium gray level for fusion.
Fig. 5 to 8 (e) show HDR fusion maps obtained from image evaluation indexes, and the selection method is as follows: and calculating the information entropy, gradient difference, gray average value and gray mean square error of each image in the sequence, and respectively selecting the optimal images corresponding to the four indexes for fusion.
Fig. 5 to 8 (f) are HDR fusion diagrams implemented by the method of the present invention.
Fig. 9 is an index evaluation result of the HDR image after the four sets of scenes shown in fig. 5 to 8 are fused.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be further described in detail below with reference to the accompanying drawings.
As shown in fig. 1, the adaptive multi-integration time infrared image sequence selection method disclosed in this embodiment includes the following detailed steps:
step one: the thermal imager of fig. 2 was used for image acquisition. And adjusting the integration time of the thermal infrared imager, collecting scene image sequences under different integration times, inputting images into software, and initializing parameters. An initial threshold μ is set to 1.5, and one pixel point in the normal temperature region is set as a "seed". Traversing the infrared image sequence, and carrying out the second to fifth steps.
Step two: a threshold μ of the current scene is calculated. The gray scale of the input image I x is divided into equivalent intervals edges, and the number N of pixels with gray scale values falling in each interval is calculated. Taking the medium gray level as a gray level dividing line, calculating the maximum value of the count N forwards, and determining the gray level value corresponding to the maximum value as the gray level value L gray of the normal temperature region of the frame image; and calculating the maximum value of the count N, and determining the gray value corresponding to the maximum value as the gray value H gray of the strong radiation area of the frame image. The threshold μ can be obtained according to formula (1).
Step three: and (5) performing region segmentation. Each pixel value I (I, j) is compared with the mu-times of the gray value I (I 0,j0) of the seed point, and when the pixel value is smaller than the mu-times of the gray value of the seed point, the pixel is classified as a seed area, regarded as a normal temperature point, marked with '0', and if not, the pixel is regarded as a high temperature point, marked with '1'. The image is divided into two major parts, i.e., a normal temperature region L range and a strong radiation region H range according to formula (2), as shown in fig. 4.
Step four: and optimizing the objective function. And acquiring a gray average value of a high-temperature region of the multi-integration sequence image, and calculating an image closest to a medium gray level according to bubbling sequencing to be used as a local optimal image (low exposure image) of a strong radiation target. The normal-temperature target local optimum image (high exposure image) is selected in the same manner.
Step five: and calculating four groups of image evaluation indexes and performing comprehensive evaluation. Ordering the information entropy indexes from big to small; for the image gradient index, sorting from large to small according to the absolute value; for the gray average value index, sorting from small to large according to the difference value with the medium gray level; and for the gray mean square error index, the gray mean square error index is ranked from large to small. And finally, obtaining the comprehensive ranking of all the image evaluation indexes, and obtaining the image with the first comprehensive ranking as a global optimal image.
Step six: and (5) image fusion and enhanced display. And carrying out nonlinear fusion and enhancement processing on the gray level of the strong radiation target local optimal image, the normal temperature target local optimal image and the global optimal image to form an HDR image displayed by 8 bits.
Step seven: the HDR image obtained by the present invention was compared with other methods (d, e diagrams in fig. 5 to 8) related to the present invention, and the image was evaluated by subjective evaluation and objective evaluation to verify the validity of the present invention.
The HDR image achieved in this embodiment, both strongly radiated and ambient temperature regions, can be imaged clearly, without distortion, subjectively by analyzing fig. 5-8.
Because no known HDR infrared fusion image quality evaluation index exists at present, in order to evaluate the effectiveness of the adaptive multi-integration time infrared image sequence preferred method disclosed by the embodiment, the invention also provides an objective evaluation index for evaluating the fused high dynamic range infrared image quality, and the image is objectively evaluated from three aspects of noise level, fidelity and visual perception quality by means of roughness (ρ), fusion visual information fidelity (VIFF) and natural image quality evaluation index (NIQE). Objectively analyzing, the HDR image roughness (rho) obtained by the method is low, namely the noise is low; the visual information fidelity (VIFF) is higher, namely the fusion image can keep the visual information of the source image more accurately; the natural image quality evaluation index (NIQE) is higher, namely the visual perception quality of the image is better, and the natural image can be better matched with a natural scene.
While the foregoing is directed to embodiments of the present invention, other and further details of the invention may be had by the present invention, it should be understood that the foregoing description is merely illustrative of the principles of the present invention, and that various modifications, equivalents, improvements and modifications may be made without departing from the spirit and principles of the present invention.
Claims (2)
1. An adaptive multi-integration time infrared image sequence optimizing method is characterized in that: comprises the following steps of the method,
For the same high dynamic response scene, acquiring infrared sequence images under different integration time;
Inputting the acquired sequence images to a region growing point segmentation module, segmenting the sequence images based on a region growing point method, respectively searching the segmented images for optimal functions of a normal temperature target and a strong radiation target, respectively obtaining local optimal images from the input sequence images according to the optimal functions of the normal temperature target and the strong radiation target, wherein the local optimal images comprise the normal temperature target local optimal images and the strong radiation target local optimal images so as to keep image details as far as possible, and adopting a high integration time exposure mode to enable the normal temperature target to be imaged clearly, so that the normal temperature target local optimal images are called high exposure images, and adopting a low integration time exposure mode to enable the strong radiation target to be imaged clearly, so that the strong radiation target local optimal images are called low exposure images; the parameters used for segmentation in the segmentation sequence images based on the region growing points are changed, and the gray scale relation between the normal temperature region and the strong radiation region is calculated through a gray scale histogram and is set as a threshold segmentation value of the current scene;
Inputting the collected sequence images to a global image selection module based on gray information evaluation indexes, comprehensively quantitatively screening global optimal images containing more information from the input sequence based on the gray information evaluation indexes, and eliminating discontinuity of local optimal images selected based on a region growing point segmentation method;
The high dynamic range HDR infrared thermal image is obtained by fusing the normal temperature target local optimal image, the strong radiation target local optimal image and the global optimal image, so that the preferred selection of the self-adaptive infrared image sequence is realized;
For different high dynamic range infrared scenes, a threshold parameter μ for segmentation is obtained according to equation (1):
Wherein M is the number of histogram intervals; n is the pixel number of each interval; x and y are the positions corresponding to the maximum N value in the intervals (1, M/2) and (M/2, M) respectively; the edges are one-dimensional tuples;
Dividing an input image sequence based on region growing points, wherein an initial seed point is preset, the relation between the gray scale of each pixel in the sequence image and the seed point is calculated, and if the gray scale of the pixel is larger than the gray scale value of the seed point which is mu times, the pixel is divided into a strong radiation region and marked by '1'; if the gray level of the pixel is less than mu times of the gray level value of the seed point, dividing the pixel into normal temperature areas, and marking the normal temperature areas with 0 so as to realize target segmentation based on a gray level threshold mu, wherein the representation is shown in a formula (2);
Searching the optimal functions of the normal temperature target and the strong radiation target for the segmented image respectively, and obtaining a local optimal image from the input sequence image according to the optimal functions of the normal temperature target and the strong radiation target;
Wherein, I x is the input x-th frame image; A gray average value of the calculated area; w is the bit width of the original data; range is the image areas L range and H range to be calculated;
Obtaining a local optimal image from the input sequence image according to an optimal function formula (3) of the normal temperature target and the strong radiation target, namely, a normal temperature target local optimal image and a strong radiation target local optimal image;
The method comprehensively and quantitatively screens the global optimal image containing more information from the input sequence based on the gray information evaluation index, and comprises the following steps of,
Establishing an index system for evaluating the image quality of an input image sequence, wherein the index system comprises an information entropy H (X) image quality evaluation index, a gradient difference S obel image quality evaluation index, a gray average value M eanI image quality evaluation index and a gray mean square error S tdI image quality evaluation index;
comprehensively quantitatively screening from the input sequence based on the gray information evaluation index;
For information entropy H (X) image quality evaluation index: the image is an m x n pixel matrix, and each pixel corresponds to a gray level between 0 and 255; the probability of defining each gray level is Pi:
where α i is the sum of pixels for each gray level; the information entropy is expressed as:
Wherein w is the system bit width;
for gradient difference S obel image quality evaluation index:
Wherein, I x and I y are images after the original images are respectively convolved with h x,hy; the Sobel operator h x,hy is:
the image quality evaluation index calculation mode for the gray average value M eanI is as follows:
The image quality evaluation index calculation method for the gray mean square error S tdI is as follows:
the calculation formula of the comprehensive evaluation measurement standard is as follows
Mulrank＝SUM(Hrank,Srank,Mrank,Stdrank) (10)
M ulrank is the comprehensive ranking of the images calculated by the sum of the ranking of all the evaluation indexes; h rank is the ranking of information entropy H; s rank is the ranking of the image gradient difference S obel; m rank is the rank of the image gray-scale mean M eanI; s tdrank is the rank of image mean square error S tdI;
According to the formula (10), the image quality evaluation index information entropy H (X), the gradient difference S obel, the gray average value M eanI and the gray average value S tdI are comprehensively ordered, and the image with the highest comprehensive ranking is selected, so that the overall situation that all evaluation indexes are in the relatively optimal state is realized, namely the globally optimal image containing more information is quantitatively screened from the input sequence.
2. An adaptive multi-integration time infrared image sequence preferentially method as claimed in claim 1, characterized in that: and carrying out enhanced display on the fused infrared high dynamic range HDR image based on a detail enhancement cascade algorithm.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111439718.7A CN114187196B (en) | 2021-11-30 | 2021-11-30 | Self-adaptive multi-integration time infrared image sequence optimizing method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111439718.7A CN114187196B (en) | 2021-11-30 | 2021-11-30 | Self-adaptive multi-integration time infrared image sequence optimizing method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114187196A CN114187196A (en) | 2022-03-15 |
| CN114187196B true CN114187196B (en) | 2024-06-14 |
Family
ID=80602975
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111439718.7A Active CN114187196B (en) | 2021-11-30 | 2021-11-30 | Self-adaptive multi-integration time infrared image sequence optimizing method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114187196B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116934882B (en) * | 2023-06-15 | 2024-12-17 | 北京理工大学 | Infrared image dynamic range compression display method based on local information quantity statistical histogram |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110035239A (en) * | 2019-05-21 | 2019-07-19 | 北京理工大学 | One kind being based on the more time of integration infrared image fusion methods of gray scale-gradient optimizing |
| CN110889806A (en) * | 2019-11-19 | 2020-03-17 | 常州工学院 | Adaptive gain image enhancement method based on fractional order multi-scale entropy fusion |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP5457652B2 (en) * | 2008-09-01 | 2014-04-02 | キヤノン株式会社 | Image processing apparatus and method |
| US20120288217A1 (en) * | 2010-01-27 | 2012-11-15 | Jiefu Zhai | High dynamic range (hdr) image synthesis with user input |
| CN110910323A (en) * | 2019-11-19 | 2020-03-24 | 常州工学院 | An Underwater Image Enhancement Method Based on Adaptive Fractional Multiscale Entropy Fusion |
| CN113674186B (en) * | 2021-08-02 | 2024-10-22 | 中国科学院长春光学精密机械与物理研究所 | Image synthesis method and device based on self-adaptive adjustment factors |
-
2021
- 2021-11-30 CN CN202111439718.7A patent/CN114187196B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110035239A (en) * | 2019-05-21 | 2019-07-19 | 北京理工大学 | One kind being based on the more time of integration infrared image fusion methods of gray scale-gradient optimizing |
| CN110889806A (en) * | 2019-11-19 | 2020-03-17 | 常州工学院 | Adaptive gain image enhancement method based on fractional order multi-scale entropy fusion |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114187196A (en) | 2022-03-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107292830B (en) | Low-illumination image enhancement and evaluation method | |
| CN108154479A (en) | A kind of method that remote sensing images are carried out with image rectification | |
| CN111656781A (en) | System and method for image signal processor tuning using reference images | |
| CN110009688A (en) | A kind of infrared remote sensing image relative radiometric calibration method, system and remote sensing platform | |
| Venkatesh et al. | Image enhancement and implementation of CLAHE algorithm and bilinear interpolation | |
| CN111630854A (en) | System and method for image signal processor tuning | |
| CN114187196B (en) | Self-adaptive multi-integration time infrared image sequence optimizing method | |
| CN109658405B (en) | Image data quality control method and system in crop live-action observation | |
| Gil et al. | Fieldscale: Locality-aware field-based adaptive rescaling for thermal infrared image | |
| CN112771568A (en) | Infrared image processing method, device, movable platform and computer readable medium | |
| CN106027911B (en) | A kind of in-orbit focus adjustment method of the spaceborne transmission of visible light type camera of earth observation | |
| Kurmi et al. | Pose error reduction for focus enhancement in thermal synthetic aperture visualization | |
| Hasikin et al. | Adaptive fuzzy intensity measure enhancement technique for non-uniform illumination and low-contrast images | |
| Purohit et al. | Enhancing the surveillance detection range of image sensors using HDR techniques | |
| CN113432723B (en) | Image processing method, system and computer system for weakening stray radiation | |
| CN117237779B (en) | Image recognition method and system for visible light image and infrared image combined analysis | |
| Rossi et al. | Dynamic range reduction and contrast adjustment of infrared images in surveillance scenarios | |
| CN108470325B (en) | Space-time three-dimensional noise identification and compensation method for area array staring infrared remote sensing image | |
| Petrova et al. | Contrast enhancing by applying histogram analysis in image processing | |
| EP3054668B1 (en) | Method and device for processing a video stream | |
| van Zwanenberg et al. | A tool for deriving camera spatial frequency response from natural scenes (NS-SFR) | |
| CN112348771B (en) | Imaging consistency evaluation method based on wavelet transformation | |
| Louvat et al. | Advanced software solutions for IR images | |
| CN109615592A (en) | A method of image-region compensation is realized by grey scale curve | |
| CN117495984B (en) | An intelligent calibration method for coaxial hybrid optical zoom pinhole lens |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |