CN108564549B - Image defogging method based on multi-scale dense connection network - Google Patents
Image defogging method based on multi-scale dense connection network Download PDFInfo
- Publication number
- CN108564549B CN108564549B CN201810361296.8A CN201810361296A CN108564549B CN 108564549 B CN108564549 B CN 108564549B CN 201810361296 A CN201810361296 A CN 201810361296A CN 108564549 B CN108564549 B CN 108564549B
- Authority
- CN
- China
- Prior art keywords
- image
- convolution
- defogging
- scale
- convolution layer
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 34
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 26
- 230000000694 effects Effects 0.000 claims abstract description 25
- 238000007781 pre-processing Methods 0.000 claims abstract description 12
- 238000004364 calculation method Methods 0.000 claims description 27
- 238000013527 convolutional neural network Methods 0.000 claims description 10
- 230000004913 activation Effects 0.000 claims description 6
- 238000010606 normalization Methods 0.000 claims description 6
- 230000003044 adaptive effect Effects 0.000 claims description 3
- 238000005457 optimization Methods 0.000 claims description 3
- 230000001105 regulatory effect Effects 0.000 claims 1
- 238000013135 deep learning Methods 0.000 abstract description 9
- 238000012549 training Methods 0.000 abstract description 6
- 238000013528 artificial neural network Methods 0.000 abstract description 5
- 230000016776 visual perception Effects 0.000 abstract 1
- 230000005540 biological transmission Effects 0.000 description 8
- 238000010586 diagram Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 238000002474 experimental method Methods 0.000 description 3
- 238000005286 illumination Methods 0.000 description 3
- 238000002203 pretreatment Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 239000003086 colorant Substances 0.000 description 2
- 235000009508 confectionery Nutrition 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 238000003909 pattern recognition Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 238000009825 accumulation Methods 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 239000000428 dust Substances 0.000 description 1
- 238000012417 linear regression Methods 0.000 description 1
- 239000002245 particle Substances 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 230000005855 radiation Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000002834 transmittance Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/73—Deblurring; Sharpening
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/40—Image enhancement or restoration using histogram techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses an image defogging method based on a multi-scale dense connection network, which reconstructs images with different degrees of fog into clearer images and greatly improves the quality and visual perception of the images. The image preprocessing for improving the contrast of the image by a self-adaptive histogram equalization mode is put forward for the first time, so that the defogging effect is obviously improved; the method has the advantages that the characteristics of the mists with different scales can be described by adopting the multi-scale dense connection convolution neural network, the characteristics of the mists are effectively combined, the most effective defogging effect is achieved, a formula based on the Retinex defogging problem is provided, the end-to-end deep learning defogging is more concise and effective, and the method is compared with other defogging algorithms based on the deep learning. The invention not only greatly reduces the number of model parameters, but also can achieve ideal defogging effect under the condition of few training data.
Description
Technical Field
The invention relates to the field of image enhancement, in particular to an image defogging method based on a multi-scale dense connection network.
Background
Fog is a weather phenomenon that results from the accumulation of minute dust and moisture particles under dry conditions. Haze, fog, and the like turbid media absorb atmospheric light and cause scattering of the atmospheric light, which leads to image degradation of outdoor scenes captured in such weather. In general, degraded images lose contrast and color fidelity.
The light passing through a certain scattering medium gradually reduces the light intensity in the original direction, and the reduced light intensity is scattered to other directions due to the law of conservation of energy. Furthermore, the energy lost by scattering depends on its distance from the camera. Based on this physical phenomenon, people often use a physical model based on atmospheric scattering to describe foggy images. The foggy image can be represented as a linear model of:
I(x)＝J(x)t(x)+A(1-t(x))
where I (x) represents the fogged image, J (x) is the original radiation of the object, A is the global atmospheric illumination, and t (x) is called the medium transmittance. However, if only a single image of information is provided, and t (x) and a are solved simultaneously, this is an under-adapted problem.
To solve this problem, most conventional defogging algorithms rely on assumptions and a priori conditions to estimate the transmission map and thus solve for other unknowns. (1) Contrast-based methods: tan et al found that the contrast of haze-free images was lower than that of haze-images, while the change in throw ratio was only related to the depth of the object, so that projection maps were modeled using Markov random fields[1](ii) a (2) Method based on color attenuation prior: zhu et al used a simple linear regression model based on priors to predict scene depth[2]This is done by using physical characteristics that the brightness and color saturation in the haze-free area are very similar, but the brightness and color saturation in the haze-free area are very different. (3) Dark channel based method: he et al defogging a priori using a dark channel. The so-called dark channel is the value at which the light intensity is at its minimum in the vast majority of non-sky local areas. With the aid of the dark channel map, all the required parameter values can be obtained from the foggy image[3]. (4) Method based on global pixels: berman et al are based on a priori knowledge: the number of image pixel points in a clear and clean image is far greater than the number of different colors. Generally, for a normal image, in RGB space, the colors of the pixels of the image may be aggregated into hundreds of small clusters. And the pixel points belonging to the same cluster can be gathered on the straight lines of the RGB space, and the straight lines become fog lines. The method is just utilizingThe fog lines estimate the transmissivity, and then the defogged image is obtained through an atmospheric scattering model[4]. As can be seen from the above method, the general defogging algorithm greatly depends on the accuracy of the transmission map estimation. The estimation of the transmission map needs to be based on various priors and assumptions. Once the actual image does not conform to the prior assumptions, the defogging effect of the image is very poor.
In order to improve the accuracy of transmission map estimation, in recent years, the academia has begun to use deep learning to solve the image defogging problem. Cai et al, the first proposal, which is to use deep learning to learn the mapping relationship between the foggy image and the projection map, and then reconstruct a clearer image by using an atmospheric scattering model[5]. Ling et al and Ren et al improved methods for estimating transmission maps for Cai, and respectively proposed transmission map estimation based on depth CNN[6]And multi-scale based transmission map estimation[7]. However, this approach of separately estimating the transmission map and the global atmospheric illuminance may lead to sub-optimal solution problems. Because the errors generated when the two partial parameters are estimated separately accumulate and then are amplified when the two parameters are optimized in parallel. Therefore, Li et al completely transform the image defogging problem into an end-to-end problem, and directly learn the mapping relation from the foggy image to the fogless image through the neural network[8]。
Although the related researches have achieved a good image defogging effect, there are some problems. When image defogging is performed in a conventional manner, if the generation of a foggy image is inconsistent with the prior conditions or assumptions of the algorithm, the defogging performance of the foggy image is reduced. When the defogging algorithm based on deep learning is adopted, the robustness of the defogging algorithm is limited by the data set, and the defogging algorithm has poor effect when some images are processed.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provides an image defogging method based on a multi-scale dense connection network, which can reconstruct images with fog of different concentrations into clearer images and remarkably improve the image quality and visual effect.
The technical scheme adopted by the invention is as follows:
an image defogging method based on a multi-scale dense connection network comprises the following steps:
step 1, preprocessing a single foggy image I (x), and preprocessing the image I (x) obtained4) As input data for the model;
the step 1 specifically comprises the following steps:
step 1.1, image equalization: a single foggy image I (x) passes through an adaptive histogram equalization method based on contrast limitation[9]Carrying out image preprocessing to obtain an image I (x)1)；
Step 1.2, image normalization: the preprocessed foggy image I (x)1) Is divided by 255 such that each pixel is between 0,1]Get image I (x)2)；
Step 1.3, image negation: normalizing the processed foggy image I (x)2) Is multiplied by-1 such that each pixel is between-1, 0]Get image I (x)3)；
Step 1.4, image normalization: negating the processed foggy image I (x)3) Is added with 1, so that each pixel is between 0,1]Get image I (x)4)；
Step 2, the image I (x) obtained after the pretreatment is carried out4) Respectively executing an initial stage, a multi-scale convolution layer calculation stage, a multi-scale convolution characteristic mixing calculation stage and a variable substituting formula calculation stage to obtain a corresponding defogged image;
the step 2 specifically comprises the following steps:
step 2.1, initial stage: input pre-processed resulting image I (x)4) Convolution operation and activation function operation are carried out to obtain the corresponding layer 1 output result F (I (x)4) The calculation formula is:
F(I(x4))＝max(W*I(x4)+b，0) (1)
wherein W and b are the convolution weight parameter and the bias parameter of the first layer network of the invention respectively;
step 2.2, multi-scale convolutional layer calculation stage: in order to extract the characteristics of fog with different scales, the convolution characteristics extracted in the initial stage are input into three convolution layer groups consisting of a plurality of convolution kernels with three sizes, wherein the sizes of the convolution kernels are 3x3, 5x5 and 7x7 respectively. The result of the initial stage F (I (x)) will be compared4) Each convolution layer group of the multi-scale convolution layer, after which the result is convolved again and then the result of the previous two result stacks is input to the next convolution layer. The calculation formula is as follows:
wherein
And
convolution weight parameters of the 1 st, 2 nd and 3 rd convolution layers of the convolution layer group of i scale (i is 3, 5, 7) respectively,
and
bias parameters for the 1 st, 2 nd and 3 rd convolutional layers of the convolutional layer group of the i-scale (i-3, 5, 7), respectively.
And
the output results of the 1 st, 2 nd and 3 rd convolutional layers of the convolutional layer group of i scale (i is 3, 5, 7), respectively;
step 2.3, multi-scale convolution feature mixing stage: the feature mixing stage of the invention consists of 1 feature stacking operation, 2 activation function operations and 2 convolution operations, and the calculation formula is as follows:
F5(I(x))＝max(W5*F4(I(x))+b5，0) (6)
wherein, W4And W5Convolution weight parameters of the 1 st and 2 nd convolution layers of the multi-scale convolution feature mixing stage, respectively, b4And b5Bias parameters, F, for the 1 st and 2 nd convolutional layers, respectively, of the multi-scale convolutional feature mixing stage5(i (x)) is the output of the convolutional neural network of the present invention, i.e., the intermediate variables learned by the convolutional neural network;
step 3, defogging area calculation stage: the invention reconstructs a clearer image by using the intermediate variable learned by the convolutional neural network.
The traditional defogging algorithm is based on an atmospheric scattering physical model, and the formula is as follows:
I(x)＝J(x)t(x)+A(1-t(x)) (7)
wherein, I (x) is the image with fog, J (x) is the original image without fog, A is the global atmosphere illumination, and t (x) is the projection image.
Since the conventional defogging model needs to calculate more parameters at the same time, the image defogging problem becomes an underdetermined problem. Therefore, on the basis of Galdran et al theory[10]The invention provides a formula for solving the defogging problem from the Retinex perspective. Among them, Galdran et al demonstrate that there is a relationship between Retinex theory and image defogging problem,the relationship is as follows:
Dehazing(I(x))＝1-Retinex(1-I(x)) (8)
wherein Dehazing represents a defogging algorithm, and Retinex represents an algorithm for defogging an image based on the Retinex theory. This formula demonstrates the relationship between these two methods.
The Retinex theory obeys the following physical model:
logR(I(x))＝logI(x)-logL(I(x)) (9)
wherein, L (I (x)) represents the reflection map of the foggy image, and R (I (x)) represents the image enhanced by the Retinex algorithm. According to the above equation, the following calculation formula is derived based on the formula for solving the defogging problem by Retinex:
wherein epsilon is an adjusting factor, epsilon is 0.0001 through experiment selection, and the adjusting factor can avoid the condition that the logarithm value is zero. In conclusion, the invention only needs to optimize F in the defogging formula through the multi-scale dense connection convolutional neural network5(I(x))。
Step 4, comparing the defogged image with the y fog-free image, and calculating the Euclidean distance between the two images;
step 5, continuously updating and optimizing based on the calculated Euclidean distance to obtain optimal convolution weight parameters and bias parameters;
when the reconstructed clearer image is compared with the corresponding clearer image, the preset defogging effect is not obtained, then the backward propagation is continued, the convolution weight parameter and the bias parameter are updated by using the gradient descent optimization algorithm, and then the step 2-5 is executed;
and when the recovered clearer image is compared with the corresponding clearer image and a preset defogging effect is obtained, stopping back propagation, and finally obtaining the convolution weight parameter and the offset parameter obtained in the step 2.
By adopting the technical scheme, the invention has the following four advantages: firstly, the multi-scale dense connection convolution neural network can describe the characteristics of fog with different scales, and effectively combines the characteristics of the fog with the characteristics to achieve the most effective defogging effect; secondly, the invention provides a formula based on Retinex defogging problem, so that the end-to-end deep learning defogging is more concise and effective; thirdly, compared with other defogging algorithms based on deep learning, the method not only greatly reduces the number of model parameters, but also can achieve ideal defogging effect under the condition of few training data; fourthly, the invention firstly provides the image preprocessing for improving the contrast of the image in a self-adaptive histogram equalization mode, and the defogging effect is obviously improved.
Drawings
The invention is described in further detail below with reference to the accompanying drawings and the detailed description;
FIG. 1 is a schematic overall principle diagram of the image defogging method based on the multi-scale dense connection network of the invention;
FIG. 2 is a schematic diagram of the present invention of a multi-scale dense connectivity network;
FIG. 3 is an original fog-free image;
FIG. 4 is a foggy image formed from a fogless image;
FIG. 5 is a diagram illustrating the effect of the AODNet defogging technique on the treatment;
FIG. 6 is a process effect diagram of a dark channel defogging algorithm;
FIG. 7 is a graph of the effect of the process without the pre-treatment step using the present invention;
FIG. 8 is a graph of the effect of processing using the complete steps of the present invention.
Detailed Description
As shown in one of figures 1-8, the invention discloses an image defogging method based on a multi-scale dense connection network, which comprises the following steps:
step 1, training data preparation phase.
The step 1 specifically comprises the following steps:
step 1.1, a training data set is selected. Used by the present invention is CVPR NTIRE2018 Outdoor Dehaze match data, which contains a fog image and fog-free image pair. Wherein, the foggy image is formed by a certain algorithm for the fogless image.
Step 1.2, preprocessing the image database to form a pairing set of the fog-containing image and the high-definition fog-free image. Adaptive histogram equalization method using contrast-based constraints[9]Preprocessing the foggy image I (x) to obtain a foggy image I (x) with adjusted contrast1). From foggy image I (x)1) In the present invention, d is 256, and the sub-image I is capturedcAnd simultaneously intercepting a sub-image J with a corresponding size from the fog-free image J (x)cForming a pairing set comprising N sub-images
i∈{1,2,…,N}。
Step 1.3, image normalization: preprocessing the foggy image
And fog-free image
Is divided by 255 such that each pixel is between 0,1]To (c) to (d);
step 1.4, image negation: the normalized foggy sub-image
Is multiplied by-1 such that each pixel is between-1, 0]To (c) to (d);
step 1.5, image normalization: negating the processed foggy sub-image
Is added with 1, so that each pixel is between 0,1]To (c) to (d);
step 2, utilizing a deep convolution neural network to preprocess the obtained foggy sub-image
As input data of the model, respectively executing an initial stage, a multi-scale convolution layer calculation stage, a multi-scale convolution characteristic mixing calculation stage and a variable substituting formula calculation stage to finally obtain a corresponding defogged image;
the step 2 specifically comprises the following steps:
step 2.1, initial stage: input pre-processed resulting image I (x)4) Convolution operation and activation function operation are carried out to obtain the corresponding layer 1 output result F (I (x)4) The calculation formula is:
F(I(x4))＝max(W*I(x4)+b，0) (1)
wherein W and b are the convolution weight parameter and the bias parameter of the first layer network of the invention respectively;
step 2.2, multi-scale convolutional layer calculation stage: in order to extract the characteristics of fog with different scales, the convolution characteristics extracted in the initial stage are input into three convolution layer groups consisting of a plurality of convolution kernels with three sizes, wherein the sizes of the convolution kernels are 3x3, 5x5 and 7x7 respectively. The result of the initial stage F (I (x)) will be compared4) Each convolution layer group of the multi-scale convolution layer, after which the result is convolved again and then the result of the previous two result stacks is input to the next convolution layer. The calculation formula is as follows:
wherein
And
convolution weight parameters of the 1 st, 2 nd and 3 rd convolution layers of the convolution layer group of i scale (i is 3, 5, 7) respectively,
and
bias parameters for the 1 st, 2 nd and 3 rd convolutional layers of the convolutional layer group of the i-scale (i-3, 5, 7), respectively.
And
the output results of the 1 st, 2 nd and 3 rd convolutional layers of the convolutional layer group of i scale (i is 3, 5, 7), respectively;
step 2.3, multi-scale convolution feature mixing stage: the feature mixing stage of the invention consists of 1 feature stacking operation, 2 activation function operations and 2 convolution operations, and the calculation formula is as follows:
F5(I(x))＝max(W5*F4(I(x))+b5，0) (6)
wherein W4And W5Convolution weight parameters of the 1 st and 2 nd convolution layers of the multi-scale convolution feature mixing stage, respectively, b4And b5Bias parameters, F, for the 1 st and 2 nd convolutional layers, respectively, of the multi-scale convolutional feature mixing stage5(I (x)) is the output of the convolutional neural network of the present invention;
step 3, defogging area calculation stage: the invention learns the intermediate variable through the convolutional neural network and reconstructs a clearer image by utilizing the intermediate variable.
The traditional defogging algorithm is based on an atmospheric scattering physical model, and the formula is as follows:
I(x)＝J(x)t(x)+A(1-t(x)) (7)
wherein J (x) is a foggy image, J (x) is an original fogless image, A is global atmospheric illumination, and t (x) is a projection image.
Since the conventional defogging model needs to calculate more parameters at the same time, the image defogging problem becomes an underdetermined problem. Therefore, on the basis of Galdran et al theory[10]The invention provides a formula for solving the defogging problem from the Retinex perspective. Galdran et al have demonstrated a relationship between Retinex theory and image defogging problems, as follows:
Dehazing(I(x))＝1-Retinex(1-I(x)) (8)
wherein Dehazing represents a defogging algorithm, and Retinex represents an algorithm for defogging an image based on the Retinex theory. This formula demonstrates the relationship between these two methods.
The Retinex theory obeys the following physical model:
logR(I(x))＝logI(x)-logL(I(x)) (9)
wherein, L (I (x)) represents the reflection map of the foggy image, and R (I (x)) represents the image enhanced by the Retinex algorithm. According to the above equation, the following calculation formula is derived based on the formula for solving the defogging problem by Retinex:
wherein epsilon is an adjusting factor, epsilon is 0.0001 through experiment selection, and the adjusting factor can avoid the condition that the logarithm value is zero. In summary, the invention only needs to optimize F in the defogging formula through the multi-scale dense connection convolutional neural network5(I(x))。
Step 4, comparing the defogged image with the y fog-free image, and calculating the Euclidean distance between the two images;
step 5, continuously updating and optimizing based on the calculated Euclidean distance to obtain optimal convolution weight parameters and bias parameters;
when the reconstructed clearer image is compared with the corresponding clearer image, the preset defogging effect is not obtained, then the backward propagation is continued, the convolution weight parameter and the bias parameter are updated by using the gradient descent optimization algorithm, and then the step 2-5 is executed;
and when the recovered clearer image is compared with the corresponding clearer image and a preset defogging effect is obtained, stopping back propagation, and finally obtaining the convolution weight parameter and the offset parameter obtained in the step 2.
To verify the effectiveness of the present invention, experiments were conducted using the game data set of NTIRE2018 Outdoor Dehaze. The data set contains 45 pairs of ultra-high resolution images. The invention divides the training data set into 35 sheets, the verification set into 5 sheets and the test set into 5 sheets. As shown in FIGS. 3-8, the defogging effect obtained by the present invention is comparable to some conventional AODNets[8]And dark channel defogging algorithm[3]A comparison is made.
The invention adopts Peak Signal to Noise Ratio (PSNR) to measure the defogging performance of the image.
Table 1 PSNR averages for the present invention and prior art in NTIRE2018 out door Dehaze dataset
As can be seen from Table 1, the PSNR values of the present invention are respectively improved by 7.29dB and 6.94dB compared with the PSNR value of the classic single-image defogging algorithm in the prior art. In addition, it can also be seen from table 1 that the PSNR mean value of the present invention increased 1.832dB after the histogram equalization method based on the contrast limit was used as the pre-treatment, thus proving that the method obtained better defogging effect after the pre-treatment.
| Defogging algorithm | C2MSNet | CANDY | The invention |
| Number of model parameters | 3584 | 20990 | 2254 |
TABLE 2 comparison of number of model parameters of the present invention with those of the prior art
As can be seen from Table 2, the present invention greatly reduces the model parameters. Compared with C2MSNet, the number of the model parameters of the invention is reduced by 1330, and compared with CANDY, the parameters of the invention are only one tenth of the parameters. Therefore, the invention can achieve ideal defogging effect under the condition of using less parameters.
The image defogging method based on the multi-scale dense connection convolutional neural network mainly embodies in four aspects: firstly, the multi-scale dense connection convolution neural network can describe the characteristics of fog with different scales, and effectively combines the characteristics of the fog with the characteristics to achieve the most effective defogging effect; secondly, the invention provides a formula based on Retinex defogging problem, so that the end-to-end deep learning defogging is more concise and effective; thirdly, compared with other defogging algorithms based on deep learning, the method not only greatly reduces the number of model parameters, but also can achieve ideal defogging effect under the condition of few training data; fourthly, the invention firstly provides the image preprocessing for improving the contrast of the image in a self-adaptive histogram equalization mode, and the defogging effect is obviously improved.
The present invention relates to the following references:
[1]Tan R T.Visibility in bad weather from a single image[C]//Computer Vision and Pattern Recognition,2008.CVPR 2008.IEEE Conference on.IEEE,2008:1-8.
[2]Zhu Q,Mai J,Shao L.A Fast Single Image Haze Removal Algorithm Using Color Attenuation Prior[J].IEEE Trans Image Process,2015,24(11):3522-3533.
[3]He K,Sun J,Tang X.Single Image Haze Removal Using Dark Channel Prior.[J].IEEE Trans Pattern Anal Mach Intell,2011,33(12):2341-2353.
[4]Berman D,Treibitz T,Avidan S.Non-local Image Dehazing[C]//Computer Vision and Pattern Recognition.IEEE,2016:1674-1682.
[5]Cai B,Xu X,Jia K,et al.DehazeNet:An End-to-End System for Single Image Haze Removal[J].IEEE Transactions on Image Processing,2016,25(11):5187-5198.
[6]Ling Z,Fan G,Wang Y,et al.Learning deep transmission network for single image dehazing[C]//IEEE International Conference on Image Processing.IEEE,2016:2296-2300.
[7]Ren W,Liu S,Zhang H,et al.Single Image Dehazing via Multi-scale Convolutional Neural Networks[M]//Computer Vision–ECCV 2016.Springer International Publishing,2016:154-169.
[8]Li B,Peng X,Wang Z,et al.AOD-Net:All-in-One Dehazing Network[C]//IEEE International Conference on Computer Vision.IEEE Computer Society,2017:4780-4788.
[9]Zuiderveld K.Contrast limited adaptive histogram equalization[M]//Graphics gems IV.Academic Press Professional,Inc.1994:474-485.
[10]Galdran A,Alvarezgila A,Bria A,et al.On the Duality Between Retinex and Image Dehazing[J].2017.
Claims (3)
1. an image defogging method based on a multi-scale dense connection network is characterized by comprising the following steps: which comprises the following steps:
step 1, preprocessing a single foggy image I (x), and preprocessing the image I (x) obtained4) As input data for the model;
the step 1 specifically comprises the following steps:
step 1.1, image equalization: carrying out image preprocessing on a single foggy image I (x) by an adaptive histogram equalization method based on contrast limitation to obtain an image I (x)1)；
Step 1.2, image normalization: the preprocessed foggy image I (x)1) Is divided by 255 such that each pixel is between 0,1]Get image I (x)2)；
Step 1.3, image negation: normalizing the processed foggy image I (x)2) Is multiplied by-1 such that each pixel is between-1, 0]Get image I (x)3)；
Step 1.4, image normalization: negating the processed foggy image I (x)3) Is added with 1, so that each pixel is between 0,1]Get image I (x)4)；
Step 2, the image I (x) obtained after the pretreatment is carried out4) Respectively executing an initial stage, a multi-scale convolution layer calculation stage, a multi-scale convolution characteristic mixing calculation stage and a variable substituting formula calculation stage to obtain a corresponding defogged image;
the step 2 specifically comprises the following steps:
step 2.1, initial stage: input pre-processed resulting image I (x)4) Convolution operation and activation function operation are carried out to obtain the corresponding layer 1 output result F (I (x)4) The calculation formula is:
F(I(x4))＝max(W*I(x4)+b，0) (1)
wherein W and b are the convolution weight parameter and the bias parameter of the first layer network respectively;
step 2.2, multi-scale convolutional layer calculation stage: inputting convolution characteristics extracted in an initial stage into a convolution layer group consisting of convolution kernels with three different sizes, wherein the convolution layer group consists of a plurality of i convolution layer groups, namely each i convolution layer group consists of 3 convolution kernels, the convolution kernels in the group have the same size, and the convolution kernels among different convolution layer groups have different sizes; specifically, the result of the initial stage F (I (x)4) Respectively inputting each i convolution layer group, stacking the output result of the 1 st convolution layer and the output result of the 2 nd convolution layer in the same group as the input of the 3 rd convolution layer, and executing convolution operation, wherein the calculation formula is as follows:
wherein
And
the convolution weight parameters of the 1 st, 2 nd and 3 rd convolution layers of each i convolution layer group respectively,
and
bias parameters of the 1 st, 2 nd and 3 rd convolutional layers of each i convolutional layer group respectively;
and
respectively outputting the 1 st convolution layer, the 2 nd convolution layer and the 3 rd convolution layer of each i convolution layer group;
step 2.3, multi-scale convolution feature mixing stage: the feature mixing stage consists of 1 feature stacking operation, 2 activation function operations and 2 convolution operations, and the calculation formula is as follows:
wherein, W4And W5Convolution weight parameters of the 1 st and 2 nd convolution layers of the multi-scale convolution feature mixing stage, respectively, b4And b5Bias parameters, F, for the 1 st and 2 nd convolutional layers, respectively, of the multi-scale convolutional feature mixing stage5(i (x)) is the output of the convolutional neural network;
step 3, defogging area calculation stage: solving and deriving a defogging calculation formula based on Retinex:
wherein epsilon is a regulating factor, I (x) is an image with fog, and D (x) is an image after defogging treatment;
step 4, comparing the defogged image with the non-fog image, and calculating the Euclidean distance between the two images;
step 5, continuously updating and optimizing based on the calculated Euclidean distance to obtain optimal convolution weight parameters and bias parameters;
comparing the defogged image D (X) with the defogged image J (x) without a preset defogging effect, continuing to perform reverse propagation, updating the convolution weight parameter and the bias parameter by using a gradient descent optimization algorithm, and then executing the step 2-5;
and (3) when the defogged image D (X) is compared with the defogged image J (x), the preset defogging effect is obtained, stopping the back propagation, and finally obtaining the convolution weight parameter and the offset parameter obtained in the step (2).
2. The image defogging method based on the multi-scale dense connection network as claimed in claim 1, wherein: the convolution kernel sizes of the three convolution layer groups in step 2.2 are 3x3, 5x5, and 7x7, respectively, i.e., i is 3, 5, and 7.
3. The image defogging method based on the multi-scale dense connection network as claimed in claim 1, wherein: in the step 3, the value of the adjusting factor epsilon is 0.0001.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810361296.8A CN108564549B (en) | 2018-04-20 | 2018-04-20 | Image defogging method based on multi-scale dense connection network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810361296.8A CN108564549B (en) | 2018-04-20 | 2018-04-20 | Image defogging method based on multi-scale dense connection network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108564549A CN108564549A (en) | 2018-09-21 |
| CN108564549B true CN108564549B (en) | 2022-04-05 |
Family
ID=63536010
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810361296.8A Active CN108564549B (en) | 2018-04-20 | 2018-04-20 | Image defogging method based on multi-scale dense connection network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108564549B (en) |
Families Citing this family (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109859120B (en) * | 2019-01-08 | 2021-03-02 | 北京交通大学 | Image dehazing method based on multi-scale residual network |
| CN110189262B (en) * | 2019-04-29 | 2023-03-24 | 复旦大学 | Image defogging method based on neural network and histogram matching |
| CN110097522B (en) * | 2019-05-14 | 2021-03-19 | 燕山大学 | Single outdoor image defogging method based on multi-scale convolution neural network |
| CN110443761B (en) * | 2019-07-16 | 2022-02-15 | 大连理工大学 | Single image rain removing method based on multi-scale aggregation characteristics |
| CN110544213B (en) * | 2019-08-06 | 2023-06-13 | 天津大学 | An image defogging method based on global and local feature fusion |
| CN110569738B (en) * | 2019-08-15 | 2023-06-06 | 杨春立 | Natural scene text detection method, equipment and medium based on densely connected network |
| CN110517203B (en) * | 2019-08-30 | 2023-06-23 | 山东工商学院 | Defogging method based on reference image reconstruction |
| CN110569928B (en) * | 2019-09-23 | 2023-04-07 | 深圳大学 | Micro Doppler radar human body action classification method of convolutional neural network |
| CN111161360B (en) * | 2019-12-17 | 2023-04-28 | 天津大学 | Image defogging method of end-to-end network based on Retinex theory |
| CN112734673B (en) * | 2021-01-13 | 2022-06-21 | 山东大学 | A low-light image enhancement method and system based on multi-expression fusion |
| CN113033215B (en) * | 2021-05-18 | 2021-08-13 | 华南师范大学 | Emotion detection method, device, device and storage medium |
| CN116681626A (en) * | 2023-08-01 | 2023-09-01 | 山东科技大学 | Target image defogging method based on deep learning in haze weather |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102436640A (en) * | 2011-09-21 | 2012-05-02 | 北京航空航天大学 | A Fog Image Clearing Method Based on Multi-scale Retinex Model in HIS Space |
| CN105574827A (en) * | 2015-12-17 | 2016-05-11 | 中国科学院深圳先进技术研究院 | Image defogging method and device |
| CN106960425A (en) * | 2017-04-05 | 2017-07-18 | 上海矽奥微电子有限公司 | Single frames defogging method based on multiple dimensioned filtering of deconvoluting |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10043243B2 (en) * | 2016-01-22 | 2018-08-07 | Siemens Healthcare Gmbh | Deep unfolding algorithm for efficient image denoising under varying noise conditions |
-
2018
- 2018-04-20 CN CN201810361296.8A patent/CN108564549B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102436640A (en) * | 2011-09-21 | 2012-05-02 | 北京航空航天大学 | A Fog Image Clearing Method Based on Multi-scale Retinex Model in HIS Space |
| CN105574827A (en) * | 2015-12-17 | 2016-05-11 | 中国科学院深圳先进技术研究院 | Image defogging method and device |
| CN106960425A (en) * | 2017-04-05 | 2017-07-18 | 上海矽奥微电子有限公司 | Single frames defogging method based on multiple dimensioned filtering of deconvoluting |
Non-Patent Citations (3)
| Title |
|---|
| A mixed-scale dense convolutional neural network for image analysis;Daniel M Pelt等;《Proceedings of the National Academy of Sciences of the United States of America》;20171226;全文 * |
| 一种基于多尺度Retinex算法的图像去雾方法;罗会兰等;《计算机应用与软件》;20130415(第04期);全文 * |
| 一种新的基于图像增强的融合算法;张雷等;《激光与红外》;20130920(第09期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108564549A (en) | 2018-09-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108564549B (en) | Image defogging method based on multi-scale dense connection network | |
| CN106530246B (en) | Image defogging method and system based on dark Yu non local priori | |
| CN106157267B (en) | Image defogging transmissivity optimization method based on dark channel prior | |
| Wang et al. | Dehazing for images with large sky region | |
| CN105976330B (en) | An embedded foggy real-time video stabilization method | |
| CN108549892B (en) | A method for clearing license plate images based on convolutional neural network | |
| CN116823686B (en) | Night infrared and visible light image fusion method based on image enhancement | |
| CN110807742B (en) | A low-light image enhancement method based on integrated network | |
| CN102968772B (en) | A kind of image defogging method capable based on dark channel information | |
| CN104217404B (en) | Haze sky video image clearness processing method and its device | |
| TW202226141A (en) | Image dehazing method and image dehazing apparatus using the same | |
| Tang et al. | Single image dehazing via lightweight multi-scale networks | |
| CN103578083B (en) | Single image defogging method based on associating average drifting | |
| CN107680054A (en) | Multisource image anastomosing method under haze environment | |
| CN108537756A (en) | Single image to the fog method based on image co-registration | |
| CN101783012A (en) | Automatic image defogging method based on dark primary colour | |
| CN109272455A (en) | Image defogging method based on weak supervision to generate confrontation network | |
| CN111861939B (en) | Single image defogging method based on unsupervised learning | |
| CN106097256B (en) | A kind of video image fuzziness detection method based on Image Blind deblurring | |
| CN102306384A (en) | Color constancy processing method based on single image | |
| CN113920018B (en) | An improved MSCNN underwater image dehazing method | |
| CN114881875A (en) | Single image defogging network based on U-Net structure and residual error network and defogging method thereof | |
| CN117593235A (en) | Retinex variational underwater image enhancement method and device based on deep CNN denoising prior | |
| CN113487509B (en) | A remote sensing image fog removal method based on pixel clustering and transmittance fusion | |
| CN113421210A (en) | Surface point cloud reconstruction method based on binocular stereo vision |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |