CN113269763A - Underwater image definition recovery method based on depth image recovery and brightness estimation - Google Patents

Abstract

The invention provides an underwater image clarity restoration method based on depth map restoration and brightness estimation. The method of the present invention includes the following steps: firstly, performing equalization processing on the underwater image; secondly, using the monocular depth estimation model to estimate the relative depth of the equalized underwater image, and then using the image segmentation strategy to segment the background depth Estimate the wrong part and re-estimate; then, use guided filtering to smooth the depth-re-estimated area; then, convert the relative depth to absolute depth through a depth normalization operation; divide the image pixels into multiple intervals equally by their depth values , search for the potential minimum pixel point of the degraded image in each interval; use the underwater image imaging model, fit the parameters by channel and estimate and remove the backscatter; finally, use the automatic brightness value estimation method to estimate the brightness Use the optimal brightness parameter on scattered images to adjust brightness and remove color casts in underwater images.

Description

Underwater Image Sharpness Restoration Method Based on Depth Map Restoration and Brightness Estimation

technical field

The invention relates to the technical field of image processing, in particular, to an underwater image clarity restoration method based on depth map restoration and brightness estimation.

Background technique

Underwater images play an important role in marine geological exploration, marine ecological protection and marine resource development. However, due to the absorption and scattering of light by water and suspended particles, underwater images and videos generally suffer from low contrast, low definition and low chroma. Severely degraded underwater images have no practical application value. In order to effectively improve the quality of underwater images, underwater image enhancement methods and underwater image restoration methods are two commonly used methods. In recent years, with the rapid development of deep learning methods in the field of image processing, the use of deep learning to restore the clarity of underwater images has also become a hot development trend.

Among them, underwater image enhancement methods mainly use methods such as filter and histogram equalization to restore image color and saturation. Although these methods can effectively improve the visual effect of the image, they ignore the relationship between the degree of degradation and the depth of field, and cannot restore the true color of the scene. The underwater image restoration method mainly uses the underwater imaging model to reverse the degradation process of the underwater image. Such methods usually use prior knowledge to solve the model parameters, but the underwater image restoration method based on prior has the problem that the prior and the target scene do not match. The methods based on deep learning are still immature and there are many problems. On the one hand, deep learning-based methods have fixed parameter estimates after training and lack sufficient flexibility when dealing with complex underwater environments. When the new underwater image type is different from the underwater environment type of the training set, the trained model may not output satisfactory results. On the other hand, the limitations of deep learning itself, such as the need for a large number of parameters to learn complex mapping functions and whether to find a suitable training set, also limit the potential value of deep learning methods in practical applications. The underwater image sharpness restoration method based on depth map restoration and brightness estimation not only considers the influence of imaging range on the degradation of underwater images, but also uses a more accurate imaging model. This ensures that the method can obtain stable and better restoration results when dealing with underwater images of different types of water bodies. In addition, the color correction method used in this method results in a significant improvement in brightness, chroma and contrast of underwater images.

SUMMARY OF THE INVENTION

According to the technical problems raised above, a method for restoring underwater image sharpness based on depth map restoration and brightness estimation is provided. The invention uses the refined depth map and the minimum pixel point, estimates and removes backscattering according to the physical model of underwater imaging, uses information entropy to automatically select the optimal brightness parameter, and adjusts the overall brightness of the image to obtain a brightly colored underwater image .

The technical means adopted in the present invention are as follows:

An underwater image clarity restoration method based on depth map restoration and brightness estimation, comprising the following steps:

Step S01: stretching the contrast of the original RGB image, ensuring that the minimum and maximum pixel values of the original image are 0 and 255 respectively, and using a monocular depth estimation model to estimate the relative depth map of the image after the stretched contrast;

Step S02: use the segmentation method to segment out the background region error estimation part of the contrast-stretched image, re-estimate the depth of this part, and use the guided filtering to smooth the re-estimated region;

Step S03: Selecting the upper and lower limits of the depth according to the actual depth of the scene and performing depth normalization processing to obtain the absolute depth map of the original RGB image;

Step S04: Divide the pixels of the original RGB image into 10 groups according to their depth values from small to large, and sort each group according to the size of the sum of the RGB values of the pixels from small to large, and take the top 200 pixels in the order. point;

的值，其中A_c表示大气光；

表示后向散射系数；J_c表示未经退化的水下图像；β_c ^D表示带宽系数；Step S05: According to the underwater image imaging model, use the minimum pixel point and its depth value to respectively fit

, where _Ac represents atmospheric light;

is the backscattering coefficient; J _c is the undegraded underwater image; β _c ^D is the bandwidth coefficient;

值，估算并去除后向散射；Step S06: Imaging the model and parameters through the underwater image

value, estimate and remove backscatter;

Step S07: on the backscattered image, the pixel values of the three channels are sorted respectively, and the pixel values of the pixel points between the 0.5% and 2% are taken as the brightness parameter according to the interval of 0.15;

Step S08: Each brightness parameter corresponds to an image enhanced by the brightness parameter, and the image with the highest information entropy is selected as the final enhancement result.

Further, the image contrast stretching formula in step S01 is:

Among them, x _min and x _max represent the minimum and maximum pixel points in the original image, respectively, x represents each pixel point of the image, and y represents the contrast-stretched image.

Further, the image segmentation method in step S02 adopts the Mahalanobis distance to measure the similarity between points in the RGB space; for the Mahalanobis distance D(z, m) between any point z in the RGB space and the average color m is given by:

where C represents the covariance matrix of the selected samples.

Further, in step S03, for each relative depth value x in the original depth map, linear conversion is used to convert it into an absolute depth value y, and the specific conversion formula is as follows:

表示原深度图中深度值的最大值和最小值；

表示需要转换的深度值的最小值和最大值。in,

Represents the maximum and minimum values of the depth values in the original depth map;

Indicates the minimum and maximum value of depth values that need to be converted.

Further, the underwater image imaging model in step S05 is:

表示后向散射系数；J_c表示未经退化的水下图像；

表示带宽系数。Among them, A _c represents atmospheric light;

is the backscattering coefficient; J _c is the undegraded underwater image;

Indicates the bandwidth factor.

Further, in step S08, the method of automatically obtaining the optimal enhanced image is as follows:

表示对三个通道的像素值排序后，在每个通道第0.5％～2％间的像素点的像素值，按照0.15为间隔取的值；Dc是去除散射后的图像，W₀＝0.1；

即为最优增强图像，其中，信息熵的计算方式如下：Among them, H(D _ri ) represents the maximum information entropy,

Indicates that after sorting the pixel values of the three channels, the pixel values of the pixels between the 0.5% and 2% of each channel are taken at intervals of 0.15; Dc is the image after the scattering is removed, W ₀ =0.1;

That is, the optimal enhanced image, where the information entropy is calculated as follows:

Among them, _i represents the gray level of the pixel, and pi represents the proportion of the pixel whose gray level is i in the whole image.

Compared with the prior art, the present invention has the following advantages:

1. For the problem of color distortion in the image enhancement method and the large deviation of transmittance estimation in the traditional DCP-based method, the present invention considers the degradation mechanism of the underwater image, and uses a new underwater physical imaging model to estimate the backscattering. The scattering effect is obvious, and the restoration result is close to the real non-degraded underwater scene.

2. The present invention only needs to obtain the depth map of the image, but does not need to estimate the transmittance and background light of the image. Compared with the traditional restoration method, the present invention has lower complexity.

3. The present invention has better practicability and robustness, and can handle various types of underwater images without problems such as over-enhancement and artifacts.

Based on the above reasons, the present invention can be widely promoted in the fields of image processing and the like.

Description of drawings

In order to illustrate the embodiments of the present invention or the technical solutions in the prior art more clearly, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description These are some embodiments of the present invention, and for those of ordinary skill in the art, other drawings can also be obtained from these drawings without any creative effort.

FIG. 1 is a schematic flowchart of the underwater image clarity restoration method according to the present invention.

Fig. 2 is a comparison effect diagram of the present invention and other underwater image restoration methods in a near-coastal scene; wherein, Fig. 2-1 is the original image (marine creatures) collected underwater; Fig. 2-2 is Li et al.UWCNN Figure 2-3 is a processing effect diagram of the Peng et al. GDCP method; Figure 2-4 is a processing effect diagram of the Peng et al. IBLA method; Figure 2-5 is a processing effect diagram of the method according to the present invention.

Fig. 3 is a comparison effect diagram of the present invention and other underwater image methods in turbid water bodies, wherein Fig. 3-1 is the original image (sea turtle) of the underwater collected image; Fig. 3-2 is the processing by Li et al. UWCNN method Effect diagram; Figure 3-3 is a processing effect diagram of Peng et al. GDCP method; Figure 3-4 is a processing effect diagram of Peng et al. IBLA method; Figure 3-5 is a processing effect diagram of the method of the present invention.

Detailed ways

In order to make those skilled in the art better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only Embodiments are part of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

It should be noted that the terms "first", "second" and the like in the description and claims of the present invention and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific sequence or sequence. It is to be understood that the data so used may be interchanged under appropriate circumstances such that the embodiments of the invention described herein can be practiced in sequences other than those illustrated or described herein. Furthermore, the terms "comprising" and "having" and any variations thereof, are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those expressly listed Rather, those steps or units may include other steps or units not expressly listed or inherent to these processes, methods, products or devices.

Example 1

As shown in Figure 1, the present invention provides a method for restoring underwater image clarity based on depth map restoration and brightness estimation, comprising the following steps:

Step S01: Stretch the contrast of the original RGB image to ensure that the minimum and maximum pixel values of the original image are 0 and 255 respectively, and use a monocular depth estimation model to estimate the relative depth map of the stretched contrast image; image contrast stretching The formula is:

Among them, x _min and x _max represent the minimum and maximum pixel points in the original image, respectively, x represents each pixel point of the image, and y represents the contrast-stretched image;

Step S02: use the segmentation method to segment out the background region misestimated part of the image after contrast stretching, re-estimate the depth of this part, and use the guided filtering to smooth the re-estimated region; the image segmentation method adopts the Mahalanobis distance to measure the difference between each point in the RGB space. The similarity between ; for any point z in the RGB space and the average color m, the Mahalanobis distance D(z,m) is given by:

Among them, C represents the covariance matrix of the selected sample; since the background area of the image is the area farthest from the camera in the whole image, we estimate the depth value of the sample points in the result area as the original depth map in an ascending manner. The depth value of the 1st percentile after sorting;

Step S03: According to the actual depth of the scene, select the appropriate approximate depth upper and lower limits (within the range of 0 to 20 meters) to perform depth normalization processing to obtain the absolute depth map of the original RGB image; The relative depth value x is converted to the absolute depth value y by linear conversion. The specific conversion formula is as follows:

表示原深度图中深度值的最大值和最小值；

表示需要转换的深度值的最小值和最大值，此处

表示该图像真实的深度估计范围(单位：米)；in,

Represents the maximum and minimum values of the depth values in the original depth map;

Indicates the minimum and maximum values of depth values that need to be converted, here

Indicates the real depth estimation range of the image (unit: meters);

Step S04: Divide the pixels of the original RGB image into 10 groups according to their depth values from small to large, and sort each group according to the size of the sum of the RGB values of the pixels from small to large, and take the top 200 pixels in the order. point;

的值，其中A_c表示大气光；

表示后向散射系数；J_c表示未经退化的水下图像；β_c ^D表示带宽系数；水下图像成像模型为：Step S05: According to the underwater image imaging model, use the minimum pixel point and its depth value to respectively fit

, where _Ac represents atmospheric light;

represents the backscattering coefficient; J _c represents the undegraded underwater image; β _c ^D represents the bandwidth coefficient; the underwater image imaging model is:

值，估算并去除后向散射；Step S06: Imaging the model and parameters through the underwater image

value, estimate and remove backscatter;

Step S07: on the backscattered image, the pixel values of the three channels are sorted respectively, and the pixel values of the pixel points between the 0.5% and 2% are taken as the brightness parameter according to the interval of 0.15;

Step S08: each brightness parameter corresponds to an image enhanced by the brightness parameter, and the image with the highest information entropy is selected as the final enhancement result; the method of automatically obtaining the optimal enhanced image is as follows:

表示对三个通道的像素值排序后，在每个通道第0.5％～2％间的像素点的像素值，按照0.15为间隔取的值；D_c是去除散射后的图像，W₀＝0.1；

即为最优增强图像。其中，信息熵的计算方式如下：Among them, H(D _ri ) represents the maximum information entropy,

Indicates that after sorting the pixel values of the three channels, the pixel values of the pixels between the 0.5% and 2% of each channel are taken at intervals of 0.15; D _c is the image after de-scattering, W ₀ =0.1 ;

is the optimal enhanced image. Among them, the calculation method of information entropy is as follows:

Among them, _i represents the gray level of the pixel, and pi represents the proportion of the pixel whose gray level is i in the whole image.

In order to verify the effectiveness of the de-scattering of the present invention, underwater images of different scenes are selected as the test set, and the results are compared with Li et al. video enhancement,"Pattern Recognit.98,1-11(2020).UWCNN), Peng et al.GDCP(Y.Peng,K.Cao,and P.C.Cosman,"Generalization of the Dark Channel Prior for Single Image Restoration,"IEEE Trans.Image Process. 27(6), 2856-2868(2018). GDCP), Peng et al. IBLA (Y. Peng and P.C. Cosman, “Underwater Image Restoration Based on Image Blurriness and Light Absorption,” IEEE Trans. Image Process. 26(4), 1579-1594(2017).IBLA) The experimental results of the method were compared qualitatively and quantitatively.

As shown in FIG. 2 , the present invention provides a comparison effect diagram with other underwater image restoration methods in a near-shore scene (sea creature). It can be seen from the comparison that the image contrast in the effect map processed by the method of the present invention is significantly improved, and is better than other methods (Li et al. UWCNN, Peng et al. GDCP, Peng et al. IBLA). Therefore, the method of the present invention can correct the color, enhance the contrast of the image, and improve the visual effect of the image.

As shown in FIG. 3 , the present invention provides a comparison chart of the experimental effects of other methods in turbid water bodies (sea turtles). Through comparative analysis with (Li et al. UWCNN, Peng et al. GDCP, Peng et al. IBLA) methods, the color restoration effect of sea turtles processed in this paper is significantly better than other methods, and the image clarity is higher. Therefore, the method of the present invention can correct the color, enhance the contrast of the image, and improve the visual effect of the image.

In this embodiment, in order to verify the robustness of the present invention, no reference image quality evaluation indicators UIQM and UCIQE are compared and analyzed, and the specific data are shown in Table 1 and Table 2. The larger the no-reference image quality evaluation index is, the better the chroma, saturation and contrast of the image generated by this method, and the better the visual effect can be obtained. The two index data values of the image processed by the method of the present invention are superior to other methods. It is proved that the method of the present invention can effectively improve the color and contrast of the image.

Table 1 Unreferenced image quality evaluation index (UIQM) of the processing results of the method of the present invention and other methods

Raw image UWCNN GDCP IBLA Our 1.1217 1.2694 1.4964 1.4435 1.6065 0.6224 0.5232 0.9900 1.0983 1.4034

Table 2 The unreferenced image quality evaluation index (UCIQE) of the processing results of the method of the present invention and other methods

Raw image UWCNN GDCP IBLA Our 0.4328 0.4572 0.5594 0.5431 0.6577 0.3220 0.3126 0.3730 0.4754 0.6341

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, but not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that it can still be The technical solutions described in the foregoing embodiments are modified, or some or all of the technical features thereof are equivalently replaced; and these modifications or replacements do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present invention.

Claims (6)

1. the underwater image clarity restoration method based on depth map restoration and brightness estimation, is characterized in that, comprises the following steps: Step S01: stretching the contrast of the original RGB image, ensuring that the minimum and maximum pixel values of the original image are 0 and 255 respectively, and using a monocular depth estimation model to estimate the relative depth map of the image after the stretched contrast; Step S02: use the segmentation method to segment out the background region error estimation part of the contrast-stretched image, re-estimate the depth of this part, and use the guided filtering to smooth the re-estimated region; Step S03: Selecting the upper and lower limits of the depth according to the actual depth of the scene and performing depth normalization processing to obtain the absolute depth map of the original RGB image; Step S04: Divide the pixels of the original RGB image into 10 groups according to their depth values from small to large, and sort each group according to the size of the sum of the RGB values of the pixels from small to large, and take the top 200 pixels in the order. point; Step S05: According to the underwater image imaging model, use the minimum pixel point and its depth value to respectively fit A _c ,

J _c ,

, where _Ac represents atmospheric light;

is the backscattering coefficient; J _c is the undegraded underwater image;

represents the bandwidth coefficient; Step S06: through the underwater image imaging model and the parameter A _c ,

J _c ,

value, estimate and remove backscatter; Step S07: on the backscattered image, the pixel values of the three channels are sorted respectively, and the pixel values of the pixel points between the 0.5% and 2% are taken as the brightness parameter according to the interval of 0.15; Step S08: Each brightness parameter corresponds to an image enhanced by the brightness parameter, and the image with the highest information entropy is selected as the final enhancement result. 2. the underwater image clarity restoration method based on depth map restoration and brightness estimation according to claim 1, is characterized in that, the image contrast stretching formula in step S01 is:

Among them, x _min and x _max represent the minimum and maximum pixel points in the original image, respectively, x represents each pixel point of the image, and y represents the contrast-stretched image. 3. the underwater image clarity restoration method based on depth map restoration and brightness estimation according to claim 1, is characterized in that, the image segmentation method in step S02 adopts Mahalanobis distance to measure similarity between each point in RGB space The Mahalanobis distance D(z,m) between any point z in the RGB space and the average color m is given by:

where C represents the covariance matrix of the selected samples. 4. the underwater image clarity restoration method based on depth map restoration and brightness estimation according to claim 1, is characterized in that, in step S03, for each relative depth value x in the original depth map, the mode of linear conversion is used Convert it to an absolute depth value y, the specific conversion formula is as follows:

in,

Represents the maximum and minimum values of the depth values in the original depth map;

Indicates the minimum and maximum value of depth values that need to be converted. 5. the underwater image clarity restoration method based on depth map restoration and brightness estimation according to claim 1, is characterized in that, the underwater image imaging model in step S05 is:

Among them, A _c represents atmospheric light;

is the backscattering coefficient; J _c is the undegraded underwater image;

Indicates the bandwidth factor. 6. the underwater image clarity restoration method based on depth map restoration and brightness estimation according to claim 1, is characterized in that, in step S08, obtains optimally enhanced image mode automatically as follows:

in,

represents the maximum information entropy,

Indicates that after sorting the pixel values of the three channels, the pixel values of the pixels between the 0.5% and 2% of each channel are taken at intervals of 0.15; D _c is the image after de-scattering, W ₀ =0.1 ;

That is, the optimal enhanced image, where the information entropy is calculated as follows:

Among them, _i represents the gray level of the pixel, and pi represents the proportion of the pixel whose gray level is i in the whole image.

Images