CN103729828B - video rain removing method - Google Patents

Abstract

The present invention proposes a kind of video rain removing method, that takes into account illumination variation factor, first determine whether whether ambient lighting changes, realize there be rainy the going of illumination variation situation according to illumination variation by changing Kalman gain Kg on this basis, and the extent by the brightness measurements of present frame with the brightness actual value of former frame, judge under illumination, whether pixel is covered by raindrop, and then remove raindrop.Described video rain removing method goes to consider ambient lighting changing factor in the rain at video, it is possible to is effectively taking place video when environment exists illumination variation and removes rain.

Description

How to remove rain from video

technical field

The invention relates to the technical field of computer vision, in particular to a method for removing raindrops from a video that includes illumination variation factors.

Background technique

Rain has a great impact on video image imaging, which will cause blurring of video image imaging and coverage of information. The direct result is that the definition of video images will decrease, and the digital processing of video images will also be affected by this and the performance will decline. Restoring the video image polluted by raindrops is beneficial to the further processing of the video image. The target detection, tracking, recognition or segmentation technology of video images has been widely used in many fields such as modern military, transportation and security monitoring.

Video rain removal technology has made great progress since it was proposed in 2003. Various methods based on different mathematical and physical models have been proposed by scholars, and the effect of raindrop removal has gradually been improved. In the existing rain removal methods, the algorithm based on continuous video frames incorporates the time information of the video, and the effect of removing raindrops in the video is generally better than the rain removal method based on a single frame image, especially for the edge of the background object. The retention effect is more pronounced. Rain removal methods based on pixel brightness skewness (skew), gray tone, K-means clustering, Kalman filter, and fuzzy connectivity are relatively novel and effective. The following takes the Kalman filter as an example to introduce a method for raindrop removal based on multi-frame video images.

The Kalman filter is an optimal autoregressive data processing algorithm. For the discrete control process system represented by the following formula:

X(k)=AX(k-1)+BU(k)+W(k) (1)

System measurements:

Z(k)=HX(k)+V(k) (2)

Among them, X(k) and X(k-1) represent the system state at time k and k-1 respectively, U(k) represents the system control parameters, W(k) and V(k) represent the process and measurement noise, they follow a Gaussian distribution. A and B are system parameters, H is a measurement system parameter, and they are all matrices for a multi-factor system.

For the system that satisfies the above two linear stochastic differential equations, the Kalman filter can be used for optimal estimation. The core of the Kalman filter is the following five equations:

X(k|k-1)=AX(k-1|k-1)+BU(k) (3)

P(k|k-1)=AP(k-1|k-1)A'+Q(4)

X(k|k)=X(k|k-1)+Kg(k)(Z(k)-HX(k|k-1)) (5)

Kg(k)=P(k|k-1)H'/(HP(k|k-1)H'+R) (6)

P(k|k)=(I-Kg(k)H)P(k|k-1) (7)

Among them, P(k|k) corresponds to the variance of X(k|k), Q is the variance of the system process, and Kg(k) is called Kalman Gain. In simple terms, the Kalman filter is to estimate the actual value through the system predicted value and measured value.

When the Kalman filter is used to remove raindrops, set the initial value X(0)=100, A=1, H(0)=1, P(0)=1. Since raindrops can be regarded as noise with a large variance, the initial variance of V(k) is set to 50, and the initial variance of W(k) is set to 5. The size of the variance determines the speed of reaching the estimated value and the effect of noise removal. A small variance means that it is slower to reach the predicted value, but the removal effect is better.

In the prior art, the deraining method based on the luminance information of continuous multi-frames of video does not take into account the factors of environmental light changes, so if there is a change in light in practical applications, it will greatly affect the deraining effect. Taking the Kalman filter as an example, if the light intensity of k increases at a certain moment, the "actual value" calculated according to the above formula (5) pixel brightness will be much smaller than the real actual value.

Contents of the invention

In view of the above problems, the object of the present invention is to provide a video rain removal method that includes illumination variation factors, so as to increase the robustness of the existing video rain removal method.

A video rain removal method is characterized in that, comprising the steps of:

S11. Calculate the difference between the measured brightness value of the current frame and the actual brightness value of the previous frame, as the first element and construct the first matrix;

S12. Calculate the mean value of the sum of other first elements in the surrounding preset area of each first element as the second element and construct a second matrix;

S13. Comparing the absolute value of the second element with the threshold value of the illumination change, if the absolute value of the second element is greater than the threshold value of the illumination change, then there is an illumination change, execute S14, otherwise execute S16;

S14. Calculate the element difference between the first element and the second element, and compare it with a preset threshold value. If the element difference is greater than the preset threshold value, it is judged that the first element corresponds to pixels are covered by raindrops, and execute S16, otherwise execute S15;

S15. Use the Kalman filter to calculate the actual brightness value of the current frame, and change the Kalman gain to Kg=Kg+a*|y|+b, where a and b are used to return the second element to Unified parameters;

S16. Using a Kalman filter to calculate the actual brightness value of the current frame;

S17. Execute the next frame until the end.

In a preferred implementation manner of the present invention, before step S11, further include: S10, read in the video, and obtain the actual brightness value of the previous frame.

In a preferred implementation manner of the present invention, before step S11, further includes: setting an initial value of the Kalman filter.

In a preferred embodiment of the present invention, the preset area is a 21x21 square area.

Compared with the prior art, the video deraining method provided by the present invention takes into account the environmental light change factor in the video deraining, and can effectively perform video deraining when there is a light change in the environment.

The above description is only an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention, it can be implemented according to the contents of the description, and in order to make the above and other purposes, features and advantages of the present invention more obvious and understandable , the following specific examples, and with the accompanying drawings, are described in detail as follows.

Description of drawings

Fig. 1 is a flow chart of a video rain removal method provided by a preferred embodiment of the present invention.

detailed description

The present invention will be described in further detail below in conjunction with the accompanying drawings and specific embodiments.

Referring to Fig. 1, an embodiment of the present invention provides a method for removing rain from a video, which includes the following steps:

S11. Calculate the difference between the brightness measurement value of the current frame and the actual brightness value of the previous frame as the first element and construct a first matrix.

Specifically, the difference between the measured brightness value of the current frame and the actual brightness value calculated in the previous frame, that is, the brightness difference between two frames before and after, is calculated, and stored as the first element in the first matrix x. It can be understood that the first matrix x contains multiple first elements.

In this embodiment, before the step S11, the method further includes: S10, reading in the video, and obtaining the actual brightness value of the previous frame.

Thereafter, a step (not shown in the figure) is included: setting the initial value of the Kalman filter. That is, set the initial value X(0)=100, A=1, H(0)=1, P(0)=1, for details, please refer to the content of the background technology of the present invention, which will not be repeated here.

S12. Calculate the mean value of the sum of other first elements within the preset area around each first element as the second element and construct a second matrix.

In this embodiment, the preset area is a 21x21 square area. Specifically, calculate the mean value of the sum of other first elements in a 21×21 square area around each first element in the first matrix x, and store it as a second element in the second matrix y. Of course, the range of the preset area is not limited to this embodiment, and can also be set according to needs, as long as the square area satisfies: on the one hand, the square area is large enough to exclude that the increase in brightness is caused by a large number of raindrops; on the other hand On the one hand, to avoid misjudgment of illumination changes in other areas due to illumination changes in local areas caused by point light sources when the square area is too large.

It can be understood that the second matrix y also includes multiple second elements.

S13. Comparing the absolute value of the second element with the threshold value of illumination change, if the absolute value of the second element is greater than the threshold value of illumination change, then there is illumination change, and execute S14, otherwise execute S16.

In this embodiment, for each second element in the second matrix y, it is judged whether there is |y|>c, where c represents the threshold of illumination change, and when |y| is large, it is considered that there is illumination change, and execution Step S14, otherwise execute step S16.

S14. Calculate the element difference between the first element and the second element, and compare it with a preset threshold value. If the element difference is greater than the preset threshold value, it is judged that the first element corresponds to pixels are covered by raindrops, and execute S16, otherwise execute S15.

In this embodiment, it is judged whether (x-y)>T, where T is a threshold that can be set manually. Thus, it can be judged whether x is covered by raindrops. If it is true, it means that x is very large. At this time, it can be judged that x is covered by raindrops. Step S16 is executed; otherwise, step S15 is executed.

S15. Use the Kalman filter to calculate the actual brightness value of the current frame, and change the Kalman gain to Kg=Kg+a*|y|+b, where a and b are used to return the second element to Unified parameters.

In this embodiment, the actual brightness value of the current frame is calculated according to formulas (3) to (7) in the background technology of the present invention (not described here), and Kg is changed to Kg=Kg+a*|y| +b, where a and b are the parameters used to normalize y.

It can be understood that when |y|>c, it means that the illumination changes, whether it is illumination enhancement (y>0) or illumination reduction (y<0), the weight Kg of the measured value Z(k) must be increased to adapt to Lighting changes.

S16. Calculate the actual brightness value of the current frame by using a Kalman filter.

In this embodiment, the actual brightness value of the current frame is calculated according to formulas (3)-(7) in the background art of the present invention (not described in detail here), and it is not prepared for the next frame.

S17. Execute the next frame until the end.

It can be understood that the video rain removal method provided by the present invention takes into account the illumination change factor, firstly judges whether the ambient illumination changes, and on this basis, according to the illumination change by changing the Kalman gain Kg to realize the rain removal under the illumination change situation , and according to the difference between the brightness measurement value of the current frame and the actual brightness value of the previous frame, it is judged whether the pixel is covered by raindrops under the light, and then the raindrops are removed.

Compared with the prior art, the video deraining method provided by the present invention takes into account the environmental light change factor in the video deraining method, and can effectively perform video deraining when there is a light change in the environment.

The above description is only an embodiment of the present invention, and does not limit the present invention in any form. Although the present invention has been disclosed as above with the embodiment, it is not intended to limit the present invention. Without departing from the scope of the technical solution of the present invention, when the technical content disclosed above can be used to make some changes or be modified into equivalent embodiments with equivalent changes, but if it does not deviate from the technical solution of the present invention, the technical essence of the present invention can be used for the above Any simple modifications, equivalent changes and modifications made in the embodiments still fall within the scope of the technical solution of the present invention.

Claims (4)

1. a video method for removing rain, is characterized in that, comprises the steps: S11. Calculate the difference between the measured brightness value of the current frame and the actual brightness value of the previous frame, as the first element and construct the first matrix; S12. Calculate the mean value of the sum of other first elements in the surrounding preset area of each first element as the second element and construct a second matrix y; S13. Comparing the absolute value of the second element with the threshold value of the illumination change, if the absolute value of the second element is greater than the threshold value of the illumination change, then there is an illumination change, execute S14, otherwise execute S16; S14. Calculate the element difference between the first element and the second element, and compare it with a preset threshold value. If the element difference is greater than the preset threshold value, it is judged that the first element corresponds to pixels are covered by raindrops, and execute S16, otherwise execute S15; S15. Use the Kalman filter to calculate the actual brightness value of the current frame, and change the Kalman gain to Kg=Kg+a*|y|+b, where a and b are used to return the second element to Unified parameters; S16. Using a Kalman filter to calculate the actual brightness value of the current frame; S17. Execute the next frame until the end. 2. The video rain removal method according to claim 1, characterized in that, before step S11, further comprising: S10, reading in the video, and obtaining the actual brightness value of the previous frame. 3. The method for removing rain from video according to claim 1, further comprising: setting an initial value of a Kalman filter before step S11. 4. The video rain removal method according to claim 1, wherein the preset area is a 21x21 square area.