CN118247161B - Infrared and visible light image fusion method under weak light - Google Patents

Abstract

The present invention discloses a method for fusing infrared and visible light images under weak light, comprising the following steps: enhancement of visible light images: decomposing the original visible light image into a base layer and a detail layer through a guided filter; adjusting the brightness of the base layer through a brightness correction function; enhancing the detail layer through a detail adjustment function; integrating the detail layer and the base layer of the visible light image to obtain an enhanced visible light image; target extraction of infrared images: reconstructing an infrared background image through a morphological opening operation; roughly extracting infrared targets through background subtraction, eliminating redundant background information according to the difference between infrared and visible light images, and obtaining infrared targets; image fusion: integrating the infrared targets into the enhanced visible light image using a target compression ratio to obtain a final fused image. The technical solution of the present invention obtains a fused image with clearer local details under weak light conditions, and makes the contrast of the targets in the fused image higher.

Description

A method for fusion of infrared and visible light images under weak light conditions

Technical Field

The present invention relates to the technical field of image processing, and in particular to a method for fusing infrared and visible light images under weak light conditions.

Background technique

The image information provided by a single sensor is limited and cannot fully describe the scene, which may cause decision makers to make wrong judgments. Image fusion technology can integrate the advantages of each sensor to obtain a complex and detailed image, which helps to improve the accuracy of subsequent task decisions. Therefore, fusion technology plays an increasingly important role in computer vision applications.

Among the many fusion source images, the fusion of infrared and visible light images has certain advantages in many aspects. First, the visible light sensor relies on capturing reflected light to form an image. The obtained visible light image is rich in details and has a high spatial resolution, which is suitable for human visual system perception, but is easily disturbed by climatic and environmental factors. The infrared sensor relies on capturing thermal radiation to form an image. The obtained infrared image can better describe the thermal target and can work in all weather conditions at all times, but generally lacks detailed features. Therefore, infrared and visible light images have good complementarity, and the fusion image of the two has good robustness; secondly, infrared and visible light images can almost present the inherent characteristics of all objects; finally, the imaging technology of infrared and visible light images is relatively simple, and the corresponding equipment cost is low. In summary, compared with other fusion schemes, the application range of infrared and visible light image fusion is wider, for example, road monitoring, hidden weapon detection, field reconnaissance, target recognition and target tracking are all its typical applications. However, the existing image fusion methods for low light have the following problems: the local details of the fused image are blurred and the target contrast is low.

The Chinese patent publication number is "CN 115049570A", and the name is "A method for fusion of visible light and infrared images under low illumination". This method first enhances the visible light image through a nonlinear mapping function and denoises it with a guided filter; secondly, the enhanced visible light image and infrared image are decomposed at multiple scales using non-subsampled shearlet; thirdly, the high-frequency sub-band and low-frequency sub-band are fused by constructing Gaussian membership functions through adaptive dual-channel pulse-coupled convolutional neural network and intuitionistic fuzzy sets respectively; finally, the fused image is obtained by applying non-subsampled shearlet inverse transform. The fused image obtained by this method has unclear local details and poor contrast of infrared thermal targets.

In summary, how to obtain a fused image with clearer local details in a low-light environment and make the contrast of the target in the fused image higher is a technical problem that those skilled in the art need to solve urgently.

Summary of the invention

The main purpose of the embodiment of the present invention is to propose a method for fusing infrared and visible light images under low light conditions, aiming to design an image fusion method that can obtain a fused image with clearer local details and make the contrast of the target in the fused image higher in a low light environment.

The technical solution of the present invention to solve the above technical problem is to provide a method for fusing infrared and visible light images under weak light, comprising the following steps:

(1) Enhancement of visible light images:

The original visible light image is decomposed into a base layer and a detail layer by using a guided filter;

Adjust the brightness of the base layer through the brightness correction function;

Enhance the detail layer through the detail adjustment function;

Integrate the detail layer and the base layer of the visible light image to obtain an enhanced visible light image;

(2) Target extraction from infrared images:

Reconstruct the infrared background image through morphological opening operation;

The infrared target is roughly extracted by background subtraction. According to the difference between infrared and visible light images, redundant background information is eliminated to obtain the infrared target.

(3) Image fusion:

The infrared target is integrated into the enhanced visible light image through the target compression ratio to obtain the final fused image.

Furthermore, the step of decomposing the original visible light image into a base layer and a detail layer by using a guided filter comprises:

The base layer is calculated by smoothing the original visible light image through guided filtering. The calculation formula is: ;in, represents guided filtering, represents the base layer, represents a visible light image, Indicates the filter window size, and Represent the width and height of the visible light image, respectively. Indicates the edge preservation degree;

The detail layer is calculated on a logarithmic scale using the following formula: ;in, Represents the detail layer, represents a logarithmic operator with a constant e as the base. Further, the step of adjusting the brightness of the base layer by using the brightness correction function comprises:

The brightness of the base layer is corrected using gamma correction on a logarithmic scale. The correction formula is: ;

in, is the scale factor, parameter Controls the degree of contrast compression, parameters Controls the brightness of the image; when When , the base layer contrast is compressed; By combining the infrared image grayscale value with the Sigmoid function, we can get: ;

in, represents the normalized infrared image;

Will Set as: ;in, Used to find the maximum value of an image, exp is an exponential function with base e.

Furthermore, the step of enhancing the detail layer by using the detail adjustment function includes:

Adjust through the detail adjustment function, the formula is: ;in, represents the normalized local standard deviation of the detail layer; DAF represents the detail adjustment function;

Enhance the detail layer, the formula is: ; Enhance visible light image, the formula is: in, To enhance the detail layer, Denotes enhanced visible light image.

Furthermore, the step of reconstructing the infrared background image through the morphological opening operation includes:

The infrared background is obtained by morphological opening operation, and the expression is: ;in, represents the infrared background image, represents an infrared image, represents the structural element, represents a Gaussian filter, is the filter window size.

Furthermore, the step of roughly extracting the infrared target by background subtraction and removing redundant background information according to the difference between the infrared and visible light images to obtain the infrared target includes:

Subtract the background image from the original infrared image to roughly extract the infrared target. The formula is: ;in, Indicates a rough infrared target; based on the difference between infrared and visible light images, the precise infrared target is expressed as follows: ;

in, For precise infrared targeting , Represents redundant background, parameter Through the nonlinear transformation function, we get: ;in, is the inverse tangent function, is a parameter used to control the overall suppression level to prevent the loss of too many infrared targets. , , , represents the normalized redundant background, Reflects the distribution of redundant background features.

Furthermore, the step of integrating the infrared target into the enhanced visible light image by the target compression ratio to obtain the final fused image includes: compressing the infrared target, the formula is: ;in, represents the final infrared target, Indicates the target compression ratio;

Enhanced Visible Light Image and precise infrared targeting Sum and get the maximum value. When the maximum value is less than or equal to 255, , the infrared target remains unchanged; otherwise, it is calculated by the following formula: ;in, , Control the degree of compression;

The fused image is obtained by the following formula: , For image fusion.

The technical solution of the present invention divides the visible light image into a base layer and a detail layer through guided filtering transformation, and designs corresponding enhancement methods according to the characteristics of different layers. In the base layer, gamma correction on a logarithmic scale is introduced to ensure that the global contrast is improved and hidden details can be displayed; an adaptive detail adjustment function is designed in the detail layer to improve the local contrast and solve the problem of blurred local details. The infrared target features are extracted through morphological opening operation and background subtraction, and the pixel grayscale distribution of the redundant background is used for optimization. The infrared target obtained is injected into the enhanced visible light image through a compression ratio to obtain a fused image. The size of the compression ratio is related to the contrast of the target in the result, thereby solving the problem of poor contrast of the infrared target in the fused image.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying any creative work.

FIG1 is a flowchart of the steps of the method for fusing infrared and visible light images under weak light according to the present invention;

FIG. 2 is a fusion block diagram of the infrared and visible light image fusion method under weak light conditions described in the present invention.

Detailed ways

The following will be combined with the drawings in the accompanying drawings of the present invention specification to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

In addition, in the present invention, descriptions such as "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "several" or "multiple" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

In the present invention, unless otherwise clearly specified and limited, the terms "connection", "fixation", etc. should be understood in a broad sense. For example, "fixation" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly defined. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

In addition, the technical solutions between the various embodiments of the present invention can be combined with each other, but it must be based on the fact that ordinary technicians in the field can implement it. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

The present invention proposes a method for fusing infrared and visible light images under weak light conditions, aiming to design an image fusion method that can obtain a fused image with clearer local details and make the contrast of the target in the fused image higher under weak light conditions.

The infrared and visible light image fusion method under weak light conditions proposed by the present invention will be described in the following specific embodiments:

In the technical solution of this embodiment, as shown in FIG. 1 and FIG. 2 , a method for fusing infrared and visible light images under weak light conditions includes the following steps:

(1) Enhancement of visible light images:

The original visible light image is decomposed into a base layer and a detail layer by using a guided filter;

Adjust the brightness of the base layer through the brightness correction function;

Enhance the detail layer through the detail adjustment function;

Integrate the detail layer and the base layer of the visible light image to obtain an enhanced visible light image;

Specifically, during the enhancement process, a brightness correction function and a detail adjustment function are introduced to avoid manual adjustment of enhancement parameters, thereby obtaining a visible light image with high overall contrast and clear local details.

(2) Target extraction from infrared images:

Reconstruct the infrared background image through morphological opening operation;

The infrared target is roughly extracted by background subtraction. According to the difference between infrared and visible light images, redundant background information is eliminated to obtain the infrared target.

Specifically, in the target extraction process, the morphological opening operation is introduced to efficiently realize the rough extraction of the target. At the same time, the redundant background feature distribution is combined to remove the interference information, so as to obtain a more accurate infrared target.

(3) Image fusion:

The infrared target is integrated into the enhanced visible light image through the target compression ratio to obtain the final fused image.

Specifically, the compression ratio is used to suppress the overexposure phenomenon during fusion, thereby obtaining a fused image with high target contrast and clear local details.

It can be understood that the visible light image is divided into a base layer and a detail layer through guided filtering transformation, and corresponding enhancement methods are designed according to the characteristics of different layers. In the base layer, gamma correction on a logarithmic scale is introduced to ensure that the global contrast is improved and hidden details can be displayed; an adaptive detail adjustment function is designed in the detail layer to improve the local contrast and solve the problem of local detail blur. The infrared target features are extracted through morphological opening operation and background subtraction, and the pixel grayscale distribution of the redundant background is used for optimization. The infrared target obtained is injected into the enhanced visible light image through the compression ratio to obtain a fused image. The size of the compression ratio is related to the contrast of the target in the result, thereby solving the problem of poor contrast of the infrared target in the fused image.

Furthermore, the step of decomposing the original visible light image into a base layer and a detail layer by using a guided filter comprises:

The base layer is calculated by smoothing the original visible light image through guided filtering. The calculation formula is: ;in, represents guided filtering, represents the base layer, represents a visible light image, Indicates the filter window size, and Represent the width and height of the visible light image, respectively. Indicates the edge preservation degree;

The detail layer is calculated on a logarithmic scale using the following formula: ;in, Represents the detail layer, Represents the logarithm operator with a constant base e.

It can be understood that guided filtering has the characteristics of edge preservation, artifact suppression and high efficiency, and can achieve good results in image decomposition. and The contrast between them can be expressed as Inspired by this, the detail layer can be obtained on a logarithmic scale, thereby improving the effect of subsequent local enhancement. Further, the step of adjusting the brightness of the base layer by a brightness correction function includes:

The brightness of the base layer is corrected using gamma correction on a logarithmic scale. The correction formula is: ;in, is the scale factor, parameter Controls the degree of contrast compression, parameters Controls the brightness of the image; when When , the base layer contrast is compressed; By combining the infrared image grayscale value with the Sigmoid function, we can get: ;in, represents the normalized infrared image;

Specifically, through the above operation, high-contrast details are fully compressed and displayed, but some low-contrast details may be invisible, so it is necessary to adjust the parameters to restore them. will make the image brighter, and in order to avoid the output exceeding the maximum grayscale, Set as: :in, Used to find the maximum value of an image, exp is an exponential function with base e.

Understandably, the base layer mainly contains low-frequency information, and brightness correction can improve the global contrast and thus display the details submerged in the highlight and dark areas. , the base layer contrast is compressed. In order to make the compression adaptive, It is obtained by combining the grayscale value of the infrared image with the Sigmoid function. The Sigmoid function can reduce the dark pixels of the infrared image and prevent excessive compression.

Furthermore, the step of enhancing the detail layer by using the detail adjustment function includes:

Affected by factors such as lighting and environment, there are some blurred details in the visible light image, which need to be enhanced to make it clearer. The detail layer is adjusted through the detail adjustment function;

Specifically, the adjustment is performed through the detail adjustment function, and the formula is: ;in, It represents the normalized local standard deviation of the detail layer; DAF represents the detail adjustment function; the local standard deviation reflects the image contrast, the larger its value is, the clearer the image is. The working principle is as follows: When it is close to 1, it means that the detail feature has a higher contrast, and the corresponding Small to avoid artifacts and over-enhancement around strong edges; when When it is close to 0, it indicates a flat area, and the corresponding Small to prevent noise from being amplified; when hour, Reach the maximum value. Enhance the detail layer, the formula is: It can be understood that after being processed by the detail adjustment function, the local contrast of the visible light image is adaptively enhanced and the blurred details become clear.

To enhance the visible light image, the formula is: ;in, To enhance the detail layer, Denotes enhanced visible light image.

From the characteristics of infrared imaging systems, we can see that in low-light environments, the infrared background is usually darker and smoother than the infrared target. Therefore, the infrared target can be obtained by background subtraction: first reconstruct the background in the infrared image, and then subtract the background from the original image to obtain the image of the infrared target.

Furthermore, the step of reconstructing the infrared background image through the morphological opening operation includes:

The infrared background is obtained by morphological opening operation, and the expression is: ;in, represents the infrared background image, represents an infrared image, Represents the structural element. The structural element has a great influence on the result of background reconstruction. The structural element should be large enough not to fit any target. Therefore, is a disk-shaped structural element with a radius of 30. represents a Gaussian filter, is the filter window size, which plays a role in smoothing the infrared background and suppressing artifacts. It can be understood that the morphological opening operation is a process in which the image undergoes corrosion operation and expansion operation in sequence, which has the effect of deleting bright targets on the dark background. Therefore, this method can effectively reconstruct the infrared background. For an infrared image, the expression of the infrared background is obtained through the morphological opening operation.

Furthermore, the step of roughly extracting the infrared target by background subtraction and removing redundant background information according to the difference between the infrared and visible light images to obtain the infrared target includes:

Subtract the background image from the original infrared image to roughly extract the infrared target. The formula is: ;in, Indicates a rough infrared target;

Understandably, since the infrared background is usually complex, the reconstruction effect is not perfect, which will cause the extracted infrared target to be mixed with some redundant background information. To solve this problem, the infrared target needs to be further optimized. Generally speaking, the infrared target has a larger gray value than the visible light image. Therefore, the area with a smaller gray value in the infrared image than the visible light image is likely to belong to the background. The significant difference between infrared and visible light images can effectively estimate the redundant background. However, there are some infrared target features in the background. Directly subtracting with the rough infrared target may result in the loss of more target features, so it is suppressed by multiplying by an appropriate parameter.

According to the difference between infrared and visible light images, the infrared target can be accurately determined using the formula: ;in, For precise infrared targeting , Represents redundant background, parameter Through the nonlinear transformation function, we get: ;in, is the inverse tangent function, is a parameter used to control the overall suppression level to prevent the loss of too many infrared targets. , , , represents the normalized redundant background, Reflects the distribution of redundant background features.

Specifically, when When the value is large, it means that the feature is more likely to belong to an infrared target, and the corresponding parameters is also larger, indicating a greater degree of suppression, avoiding the loss of more infrared target features. Used to globally control the degree of suppression. As the value of increases, the corresponding nonlinear transformation gradually strengthens, indicating that the suppression effect is more obvious. is set to 20. Furthermore, the step of integrating the infrared target into the enhanced visible light image through the target compression ratio to obtain the final fused image includes:

The fused image is obtained by directly injecting the infrared target into the enhanced visible light image. This method is effective and efficient, but sometimes it will cause overexposure, affecting the fusion quality. Therefore, in order to solve the above problem while maintaining the visible light information, the infrared target needs to be further compressed.

To compress the infrared target, the formula is: ;in, represents the final infrared target, represents the target compression ratio; for an image, the exposure level can be measured by the maximum grayscale value; therefore, for enhanced visible light images and precise infrared targeting Sum and get the maximum value. When the maximum value is less than or equal to 255, , the infrared target remains unchanged; otherwise, it is calculated by the following formula: ;in, , Control the degree of compression;

specifically, The larger the value, the smaller the compression degree, and the more prominent the target in the obtained fused image. Set to 2. The fused image is obtained by the following formula: , For image fusion.

The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims (4)

1. A method for fusing infrared and visible light images under weak light, characterized by comprising the following steps: (1) Enhancement of visible light images: The original visible light image is decomposed into a base layer and a detail layer by using a guided filter; Adjust the brightness of the base layer through the brightness correction function; Enhance the detail layer through the detail adjustment function; Integrate the detail layer and the base layer of the visible light image to obtain an enhanced visible light image; (2) Target extraction from infrared images: Reconstruct the infrared background image through morphological opening operation; The infrared target is roughly extracted by background subtraction. According to the difference between infrared and visible light images, redundant background information is eliminated to obtain the infrared target. (3) Image fusion: The infrared target is integrated into the enhanced visible light image through the target compression ratio to obtain the final fused image; The step of integrating the infrared target into the enhanced visible light image by the target compression ratio to obtain the final fused image comprises: To compress the infrared target, the formula is: ; in, represents the final infrared target, Indicates the target compression ratio; Enhanced Visible Light Image and precise infrared targeting Sum and get the maximum value. When the maximum value is less than or equal to 255, , the infrared target remains unchanged; otherwise, it is calculated by the following formula: ; in, , Control the degree of compression; the fused image is obtained by the following formula: ; For image fusion; The step of adjusting the brightness of the base layer by using the brightness correction function comprises: The gamma correction on the logarithmic scale corrects the brightness of the base layer, and the correction formula is: ; in, is the scale factor, parameter Controls the degree of contrast compression, parameters Controls the brightness of the image; when When , the base layer contrast is compressed; By combining the infrared image grayscale value with the Sigmoid function, we can get: ;in, represents the normalized infrared image; Will Set as: ; in, Used to find the maximum value of an image, exp is an exponential function with base e; The steps of enhancing the detail layer by the detail adjustment function include: Adjust through the detail adjustment function, the formula is: ; in, represents the normalized local standard deviation of the detail layer, and DAF represents the detail adjustment function; Enhance the detail layer, the formula is: ; To enhance the visible light image, the formula is: ; in, To enhance the detail layer, Denotes enhanced visible light image. 2. The method for fusion of infrared and visible light images under weak light according to claim 1, characterized in that the step of decomposing the original visible light image into a base layer and a detail layer by means of a guided filter comprises: The base layer is calculated by smoothing the original visible light image through guided filtering. The calculation formula is: ; in, represents guided filtering, represents the base layer, represents a visible light image, Indicates the filter window size, and Represent the width and height of the visible light image, respectively. Indicates the edge preservation degree; The detail layer is calculated on a logarithmic scale using the following formula: ;in, Represents the detail layer, Represents the logarithm operator with a constant base e. 3. The method for fusion of infrared and visible light images under weak light according to claim 1, wherein the step of reconstructing the infrared background image by morphological opening operation comprises: The infrared background is obtained by morphological opening operation, and the expression is: ; in, represents the infrared background image, represents an infrared image, represents the structural element, represents a Gaussian filter, is the filter window size. 4. The method for fusion of infrared and visible light images under weak light according to claim 3 is characterized in that the step of roughly extracting the infrared target by background subtraction and removing redundant background information according to the difference between the infrared and visible light images to obtain the infrared target comprises: Subtract the background image from the original infrared image to roughly extract the infrared target. The formula is: ; in, Indicates a rough infrared target; According to the difference between infrared and visible light images, the infrared target can be accurately determined using the formula: ; in, For precise infrared targeting , Represents redundant background, parameter Through the nonlinear transformation function, we get: ; in, is the inverse tangent function, is a parameter used to control the overall suppression level to prevent the loss of too many infrared targets. , , , represents the normalized redundant background, Reflects the distribution of redundant background features.