CN109788297A - A Video Frame Rate Up-Conversion Method Based on Cellular Automata - Google Patents

Abstract

The invention discloses a kind of up-conversion method of video frame rate based on cellular automata, are related to technical field of video coding, are somebody's turn to do the up-conversion method of video frame rate based on cellular automata, comprising: according to interleave f_t+ΔtForward frame f_tWith backward frame f_t+1Treat interleave f_t+ΔtBi-directional motion estimation is carried out, is generated to interleave f_t+ΔtInitial motion vector field V_t+Δt,0；It will be to interleave f_t+ΔtInitial motion vector field V_t+Δt,0Motion vector smoothing algorithm is controlled using cellular automata, is obtained to interleave f_t+ΔtFinal motion vector field V_t+Δt；According to neighboring reference frame f_t、f_t+1And to interleave f_t+ΔtFinal motion vector field V_t+ΔtOverlapped block motion compensation method is carried out, is obtained to interleave f_t+Δt.The present invention is by construction cellular automata, using the evolution rule of design, the problem of improving the ability for screening abnormal motion vector, effectively inhibited smooth motion vector, has been obviously improved the visual quality for above turning video.

Description

A Video Frame Rate Up-Conversion Method Based on Cellular Automata

technical field

The present invention relates to the technical field of video coding, and more particularly to a method for up-conversion of video frame rate based on cellular automata.

Background technique

Frame rate is an important indicator to determine the visual quality of video. Outdated imaging technology and industry standards have caused low frame rates to be popularized in various visual media. The rapid development of information technology has stimulated people's strong demand for picture realism, and high frame rate has become the common pursuit goal of the film, television and video industries. Frame Rate Up-Conversion (FRUC) is a video post-processing technology that converts video from a lower frame rate to a higher frame rate by inserting intermediate frames between adjacent frames. frame rate. For low frame rate video output by ordinary cameras, FRUC can be used to generate high frame rate video to overcome the adverse effects such as motion blur and screen freeze caused by low frame rate. It can be seen that in order to improve the visual quality of video, it is urgent to propose a high-performance FRUC method. FRUC can be divided into two categories: (1) do not consider the motion of the object, which is called a non-motion compensation method; (2) track the motion trajectory of the object, which is called a motion compensation method. There are three common methods for non-motion compensation methods, including inserting black frames, frame averaging, and frame repetition. These three methods are simple and fast to operate, but have a narrow scope of application. Only when the picture in the video is relatively static or contains a very small amount of motion , non-motion compensation methods can be used. However, when the video contains a lot of motion, if such methods are used, problems such as blurred images and inconsistent scene changes will occur. Aiming at the problem that the non-motion compensation methods cannot effectively consider the motion of objects, a motion-compensated FRUC (Motion-Compensated FRUC, MC-FRUC) method is proposed.

MC-FRUC consists of motion estimation (Motion Estimation, ME) and motion-compensated interpolation (Motion-Compensated Interpolation, MCI). The ME outputs the motion vector field of the interpolated frame, and the MCI uses the motion vector field to predict the interpolated pixels. Of the above two components, ME is more important. As long as the output motion vector field is as accurate as possible, the prediction performance of MCI can be greatly improved. In order to avoid pixel overlap and holes in the interpolated frame, bi-directional ME (Bi-directional ME, BME) is often used in the prior art. However, BME assumes that the motion vector of each pixel in the interpolated frame has bidirectional symmetry, and the motion vector does not have the condition of motion symmetry. The occlusion area and texture area, which cause abnormal motion vectors. In order to reduce motion anomalies, Motion Vector Smoothing (MVS) has become an essential part of MC-FRUC and is a key technical step to improve the accuracy of ME. Median filtering is a more mature MVS method, which uses the median vector of 8 adjacent motion vectors as the current motion vector. Compared with mean filtering, it improves the robustness of error correction performance. However, median filtering often causes excessive errors. Smooth, that is, the accuracy cannot be guaranteed in the edge area of the object, and the correct motion vector is often misjudged as abnormal. The method to solve the problem of misjudgment is to add certain prior knowledge in the smoothing process. At present, the existing technology often uses spatial correlation and temporal correlation as prior knowledge. For example, the document "New Frame Rate Up-Conversion Using Bi- directional Motion Estimation (Byung-Tae Choi, Sung-Hee Lee, and Sung-Jea Ko, IEEE Transactions on Consumer Electronics, 2000, vol.46, no.3, pp.603-609.) "Using the space between motion vectors Correlation, the 8 spatial adjacent motion vectors of the current block are used as candidate motion vectors, and the motion vector with higher reliability is selected as the final output. The document "Direction-Select Motion Estimation for Motion-Compensated Frame Rate Up-Conversion (Dong-Gon Yoo, Suk-Ju Kang, and Young Hwan Kim, Journal of Display Technology, 2013, vol.9, no.10, pp.840-850.)" Using the temporal correlation between motion vectors, several temporal The adjacent motion vectors are used as candidate motion vectors, and the final motion vector is output based on the minimum matching error. Although MVS methods based on prior knowledge have better error correction performance, they need to invest more computation. Due to the limited frequency of motion anomalies, the investment of too much computation is often not worth the gain, and even individual correct motion vectors will Misled by prior knowledge, resulting in over-smoothing.

To sum up, to improve the performance of FRUC technology, it is necessary to design a more efficient MVS method, and the MVS method adopted by the existing FRUC technology has the problem of over-smoothing the motion vector field, resulting in an abnormal number of motion vectors in the interpolation frame. increase, which seriously affects the prediction quality of the interpolated frame.

SUMMARY OF THE INVENTION

The embodiment of the present invention provides a video frame rate up-conversion method based on cellular automata, which is used to solve the problem that the motion vector field is easily over-smoothed in the prior art, which causes an increase in the number of abnormal motion vectors in the interpolation frame, which seriously affects the internal The problem of prediction quality of interpolated frames.

Embodiments of the present invention provide a cellular automata-based video frame rate up-conversion method, comprising:

Step a. Carry out bidirectional motion estimation according to the forward frame ft and the backward frame ft ₊₁ of the frame ft _{+Δt to} be inserted, and generate the initial motion vector field of the frame ft _{+Δt to} be inserted. V _t+Δt,0 ; the frame to be inserted f _t+Δt is a frame located between the reference frames f _t and f _t+1 , and located at the position Δt on the right side of the frame number t;

Step b. The initial motion vector field V _t+Δt,0 of the frame to be inserted f _t +Δt is controlled by the cellular automata to control the motion vector smoothing algorithm to obtain the final motion vector field V _t+Δt of the frame to be inserted f _t +Δt ;

Step c: Perform an overlapping block motion compensation method according to the adjacent reference frames f _t , f _t+1 and the final motion vector field V _t+Δt of the frame to be inserted f _t +Δt to obtain the frame to be inserted f _t+Δt .

Preferably, according to the forward frame f _t and the backward frame f _t+1 of the frame to be inserted f _{t+Δt ,} two-way motion estimation is performed on the frame to be inserted f t+Δt, and the initial frame f _t+Δt of the frame to be inserted is generated. Motion vector field V _t+Δt,0 ;:

According to the reference frame f _t , f _t+1 and bidirectional motion estimation, the frame to be inserted f _t+Δt is divided into mutually non-overlapping blocks of size B×B, and the motion vector of each block is calculated one by one according to formula (1) to form the size is the initial motion vector field V _t+Δt,0 of R×C;

Among them, in formula (1), s is the pixel coordinate, B _r,c is the pixel coordinate set of the r-th row and c-th column block in the video frame, v is the candidate motion vector, and S _r,c is the block B _r,c . In the candidate motion vector search area, f _t (s+v) and f _t+1 (sv) represent the luminance values of f _t and f _t+1 at pixel coordinates s+v and sv, respectively.

Preferably, the initial motion vector field V _t+Δt,0 of the frame to be inserted f _{t +Δt} is smoothed by cellular automata control, and the final motion vector field V _t of the frame to be inserted f t+Δt is obtained. _+Δt includes:

Step b1, set the iteration variable i to 0, let

Step b2, one by one The abnormality detection of each motion vector in the formula (2) is performed, and the mark E[r,c] of each motion vector after abnormality detection is generated. The abnormal distribution map E of ;

in,

Among them, in formula (2), E[r,c] is used to mark The motion vector in row r and column c in Whether it is abnormal, 1 means abnormal, 0 means normal, v ₀ means for The median vector of the 8 adjacent motion vectors of , 0 represents the zero vector, median{·} represents the median vector of the calculation input vector set, represents computing v ₀ with The cosine value of the included angle, T stands for transpose operation, & stands for logical AND, | stands for logical OR, all vectors are column vectors;

Step b3, constructing a cellular automaton;

Among them, consider any element in the two-dimensional anomaly distribution map E as a cell, the cell has two states 0 and 1, and the cell in the rth row and the cth column is E[r,c], then its Moore neighborhood is defined as follows:

The VonNeuman neighborhood is defined as follows:

In step b4, each cell state in the cellular automaton evolves according to the following formula (8); wherein, after the evolution of each cell state, the two-dimensional anomaly distribution map E evolves into a new anomaly distribution map

Among them, in formula (8), Λ _k represents the kth element of Λ, k=1,2,...,8, Ω _l represents the lth element of Ω, l=1,2,3,4;

Step b5, according to the abnormal distribution map and formula (9) for to smooth;

Step b6, calculate according to formula (10) and The residual value between;

Among them, in formula (10), ||·|| ₁ is the 1-norm of the input column vector, and is defined as follows:

Step b7, if the value of ε is greater than 3, then let i=i+1, and go to step b2 for the next iteration, otherwise, exit the loop iteration, and let the final motion vector field V _t+ of the frame to be inserted f _{t+Δt Δt} is

In the embodiment of the present invention, a cellular automata-based video frame rate up-conversion method is provided, and the beneficial effects compared with the prior art are as follows:

The present invention obtains the final motion vector field V t+Δt of the frame to be inserted f _{t +Δt} by using the cellular automaton to control the motion vector smoothing algorithm by controlling the initial motion vector field V _t+Δt,0 of the frame to be inserted f t _+Δt ; Carry out the overlapping block motion compensation method according to the final motion vector field V _t+Δt of the adjacent reference frames f _t , f _t+1 and the frame to be inserted f _t +Δt, and obtain the frame to be inserted f _t+Δt , that is, to The initial motion vector field V _t+Δt,0 uses the cellular automata to control the motion vector smoothing algorithm to obtain the final motion vector field V _t+Δ of the frame to be inserted f _t+Δt by constructing a cellular automaton, using the designed The evolution rule improves the ability to identify abnormal motion vectors and effectively suppresses the problem of over-smooth motion vectors. And adopt the method provided by the present invention to test 23 groups of CIF format video sequences, evaluate the interpolated video frame quality by the peak signal-to-noise ratio and structural similarity, and use the motion vector smoothing method based on median filtering, and take the spatiotemporal correlation as the first. Compared with the video frame rate up-conversion technology of the tested motion vector smoothing method, the present invention has a remarkable effect of improving the quality of the interpolated video frame.

Description of drawings

In order to explain the embodiments of the present invention or the technical solutions in the prior art more clearly, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only These are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained according to these drawings without creative efforts.

1 is a diagram illustrating an example of implementation of a cellular automaton-based video frame rate up-conversion method to improve frame rate provided by an embodiment of the present invention;

2 is an implementation flowchart of generating a single video frame by a cellular automaton-based video frame rate up-conversion method provided by an embodiment of the present invention;

3 is a diagram illustrating an implementation example of a bidirectional motion estimation module in a cellular automaton-based video frame rate up-conversion method provided by an embodiment of the present invention;

4 is a flowchart of the implementation of a cellular automaton-controlled motion vector smoothing module in a cellular automaton-based video frame rate up-conversion method provided by an embodiment of the present invention;

5 is a diagram illustrating an example implementation of an overlapping block motion compensation module in a cellular automaton-based video frame rate up-conversion method provided by an embodiment of the present invention;

6 is a subjective quality comparison diagram of the 33rd frame of the foreman test video sequence of the CIF format generated by interpolation of a cellular automaton-based video frame rate up-conversion method and a comparison method according to an embodiment of the present invention;

7 is a comparison diagram of subjective quality of the 57th frame of a mobile test video sequence in CIF format generated by interpolation between a cellular automaton-based video frame rate up-conversion method and a comparison method according to an embodiment of the present invention.

Detailed ways

A specific embodiment of the present invention will be described in detail below with reference to the accompanying drawings, but it should be understood that the protection scope of the present invention is not limited by the specific embodiment.

FIG. 1 is a diagram illustrating an example of implementation of a cellular automaton-based video frame rate up-conversion method to improve the frame rate according to an embodiment of the present invention. As shown in FIG. 1 , the video sequence to be up-converted is split into several video frames, which are regarded as reference frames, and then the adjacent reference frames are input into a cellular automaton-based video frame rate up-conversion method provided by the embodiment of the present invention. , the frame to be inserted is generated at a certain position between adjacent reference frames. As shown in FIG. 2 , a cellular automata-based video frame rate up-conversion method provided by an embodiment of the present invention utilizes two input adjacent reference frames f _t and f _t+1 to perform bidirectional motion estimation, The motion vector smoothing and overlapping block motion compensation controlled by cellular automata are used to generate the frame to be inserted f _t+Δt . FIG. 3 to FIG. 5 are respectively an implementation example of a bidirectional motion estimation module in a cellular automata-based video frame rate up-conversion method provided by an embodiment of the present invention, and an implementation flowchart of a cellular automata-controlled motion vector smoothing module. And an example diagram of the implementation of the overlapping block motion compensation module. 3 to 5, the execution process of the embodiment of the present invention is described as follows:

Step a, set the spatial resolution of each frame in the video sequence to be uploaded to M×N, according to the t-th frame, the t+1-th frame reference frame f _t , f _t+1 , the bidirectional motion estimation will insert the frame f _t to be inserted. _+Δt is divided into non-overlapping blocks of size B×B. As shown in Figure 3, the initial motion vector V _t+Δt,0 [r,c] of the block B _r,c in the rth row and the cth column is calculated as follows :

Among them, in formula (1), s is the pixel coordinate, B _r,c is the pixel coordinate set of the r-th row and c-th column block in the video frame, v is the candidate motion vector, S _r,c is the block B _r,c The candidate motion vector search area of , f _t (s+v) and f _t+1 (sv) represent the luminance values of f _t and f _t+1 at pixel coordinates s+v and sv, respectively. The initial motion vector of each block is calculated block by block according to formula (1) to form an initial motion vector field V _t+Δt,0 of size R×C.

Step b, input the initial motion vector field V _t+Δt,0 of the frame f _t+Δt to be inserted into the motion vector smoothing module controlled by the cellular automaton, as shown in Figure 4, and perform the following steps:

Step b1, set the iteration variable i to 0, let

Step b2, one by one Each motion vector in the following formula (2) implements anomaly detection, After the abnormality detection is performed for each motion vector in the Anomaly distribution map E.

in,

Among them, in formula (2), E[r,c] is used to mark The motion vector in row r and column c in Whether it is abnormal, 1 means abnormal, 0 means normal, v ₀ means for The median vector of the 8 adjacent motion vectors of , 0 represents the zero vector, median{·} represents the median vector of the calculation input vector set, represents computing v ₀ with The cosine of the included angle, T stands for transpose operation, & stands for logical AND, | stands for logical OR, and all vectors are column vectors.

In addition, according to the anomaly detection formula (2), when v ₀ , not all zero vectors and less than but is abnormal, the flag E[r,c] is 1, and when v ₀ , an all-zero vector or greater or equal to but is normal, the flag E[r,c] is 0.

In step b3, a cellular automaton is constructed, and any element in the two-dimensional anomaly distribution map E is regarded as a cell, with two cell states 0 and 1. Let the cell of row r and column c be E[r,c], then its Moore neighborhood is defined as follows:

The VonNeuman neighborhood is defined as follows:

In step b4, each cell state in the cellular automaton evolves according to the following formula (8); wherein, after the evolution of each cell state, the two-dimensional anomaly distribution map E evolves into a new anomaly distribution map

Wherein, in formula (8), Λ _k represents the kth element of Λ, k=1, 2, . . . , 8, and Ω _l represents the lth element of Ω, and l=1, 2, 3, and 4. Through cellular automata, the two-dimensional anomaly distribution map E evolves into a new anomaly distribution map

Step b5, according to the abnormal distribution map Press formula (9) to for smoothing.

Step b6, calculate according to formula (10) and residual value between .

Among them, ||·|| ₁ in formula (10) is the 1-norm of calculating the input column vector, which is defined as follows:

Step b7, if the value of ε is greater than 3, then let i=i+1, and go to step b2 for the next iteration, otherwise, exit the loop iteration, and let the final motion vector field V _t+ of the frame to be inserted f _{t+Δt Δt} is

Step c, according to the final motion vector field V _t+Δt of the frame to be inserted f _t+ Δt, using the reference frames f _t and f _t+1 to perform motion compensation of overlapping blocks to generate the frame to be inserted f _t+Δt . As shown in FIG. 5 , for any block B ₄ in f _t+Δt , it overlaps with the three adjacent blocks B ₁ , B ₂ , and B ₃ in the upper left corner, and B ₁ , B ₂ , B ₃ , and B ₄ each have Motion vectors v ₁ , v ₂ , v ₃ , v ₄ . For the O ₁ area, where the four blocks B ₁ , B ₂ , B ₃ , and B ₄ overlap, the interpolation formula for this area is executed:

For the O ₂ area, where the two blocks B ₃ and B ₄ overlap, the interpolation formula is executed in this area:

For the _O3 region, which is completely contained within the B4 block, the interpolation _formula is performed:

Among them, s in equations (12)-(14) is the pixel coordinate, f _t (s), f _t+1 (s), f _t+Δt (s) represent f _t , f _t+1 , f _{t+ Δt} is the luminance value at pixel coordinate s. The four overlapping areas of B ₄ and the adjacent blocks in the upper right corner, the lower left corner and the lower right corner are calculated according to the formula (12), and the two overlapping areas are calculated according to the formula (13).

Embodiment 2. Description of simulation results

In the second embodiment of the present invention, 23 groups of test video sequences in CIF format are used to evaluate the technology proposed by the present invention. The comparison methods are:

1) Document "New Frame Rate Up-Conversion Using Bi-directional MotionEstimation (Byung-Tae Choi, Sung-Hee Lee, and Sung-Jea Ko, IEEE Transactions on Consumer Electronics, 2000, vol.46, no.3, pp.603 -609.)" proposed a frame rate up-conversion technology using spatial correlation motion vector smoothing, denoted as SC-MVS; 2) Document "Direction-Select Motion Estimation for Motion-Compensated Frame Rate Up-Conversion (Dong-Gon Yoo, Suk -Ju Kang, and Young Hwan Kim, Journal of Display Technology, 2013, vol.9, no.10, pp.840-850.)" proposed a frame rate up-conversion technique using temporal correlation motion vector smoothing, denoted as TC- MVS; 3) The frame rate up-conversion technology based on median filtering motion vector smoothing is adopted, which is denoted as MF-MVS. The method of the present invention and the comparison method are used to interpolate odd-numbered frames from the even-numbered frames in the first 100 frames of the test video sequence, and calculate the peak signal-to-noise ratio (Peak Signal-to-Noise Ratio, PSNR) between the interpolated odd-numbered frames and the original odd-numbered frames, Structural SIMilarity (SSIM) is used as an objective quality evaluation performance indicator for interpolated frames. The hardware platform is a Core i5CPU computer with a main frequency of 3.10GHz and a memory of 4GB, and the software platform is Windows 764-bit operating system and Matlab R2014b simulation experiment software.

Table 1 lists the average PSNR values of interpolated video frames for different frame rate up-conversion techniques. Compared with the TC-MVS and SC-MVS methods, the method proposed in the present invention significantly improves the PSNR value, up to 9.49dB, and the effect of improving the quality of the interpolation frame is remarkable. For video sequences that are static or contain a small amount of motion, such as akiyo, news, and waterfall, the MF-MVS method is relatively better, and the PSNR value is improved by about 0.81dB on average compared with the method of the present invention. However, for video sequences containing a lot of motion, such as bus, city, and mobile, compared with the MF-MVS method, the method of the present invention can provide better interpolation quality, and the highest PSNR value gain is 4.02dB. Table 2 lists the average SSIM values of interpolated video frames for different frame rate up-conversion techniques. It can be seen from Table 2 that the SSIM value of the method of the present invention is significantly higher than that of the TC-MVS and SC-MVS methods, and only the processing of hall and highway video sequences is slightly lower than that of the MF-MVS method. Combining the results in Tables 1 and 2, it can be seen that, compared with the comparison method, the present invention has a significant effect of improving the objective quality of the interpolated video frame.

Table 1 Comparison of average PSNR values of interpolated video frames with different frame rate up-conversion techniques

Table 2 Comparison of average SSIM values of interpolated video frames with different frame rate up-conversion techniques

Further, Figures 6 and 7 respectively show the subjective quality comparison diagrams of the 33rd frame of the foreman test video sequence and the 57th frame of the mobile test video sequence in the CIF format generated by the method of the present invention and the comparison method. It can be seen that there are interpolated block mismatches caused by abnormal motion vectors in the comparison methods. Especially for the TC-MVS and SC-MVS methods, there are obvious block mismatches in the mouth and eyes of the foreman sequence. For mobile sequences, not only the TC-MVS and SC-MVS methods cannot suppress the abnormal motion vector in the digital area of the station calendar, but also the digital area of the station calendar in the interpolation frame of the MF-MVS method is chaotic. However, the interpolation result of the method of the present invention produces a satisfactory subjective visual effect, no block dislocation phenomenon, and at the same time the digital area restoration of the desk calendar is basically accurate. It can be seen from this that the present invention uses cellular automata to control motion vector smoothing, which can effectively suppress over-smoothing and significantly improve the interpolation quality.

Although preferred embodiments of the present invention have been described, additional changes and modifications to these embodiments may occur to those skilled in the art once the basic inventive concepts are known. Therefore, the appended claims are intended to be construed to include the preferred embodiment and all changes and modifications that fall within the scope of the present invention.

It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit and scope of the invention. Thus, provided that these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims (3)

1. A video frame rate up-conversion method based on cellular automata is characterized by comprising the following steps:

step a, according to the frame f to be inserted_t+ΔtForward frame f_tAnd a backward frame f_t+1Frame f to be interpolated_t+ΔtPerforming bidirectional motion estimation to generate a frame f to be interpolated_t+ΔtInitial motion vector field V_t+Δt,0(ii) a The frame f to be inserted_t+ΔtIs located in the reference frame f_t、f_t+1And at a position Δ t to the right of the frame number t;

step b, frame f to be inserted_t+ΔtInitial motion vector field V_t+Δt,0Utilizing cellular automaton to control the motion vector to be smooth, and obtaining a frame f to be interpolated_t+ΔtOf the final motion vector field V_t+Δt；

Step c, according to adjacent reference frame f_t、f_t+1And frame f to be inserted_t+ΔtOf the final motion vector field V_t+ΔtMethod for motion compensation of overlapped block to obtain frame f to be inserted_t+Δt。

2. The cellular automaton-based video frame rate up-conversion method according to claim 1, wherein the frames f to be interpolated are based on_t+ΔtForward frame f_tAnd a backward frame f_t+1Frame f to be interpolated_t+ΔtPerforming bidirectional motion estimation to generate a frame f to be interpolated_t+ΔtInitial motion vector field V_t+Δt,0The method comprises the following steps:

from the reference frame f_t、f_t+1And bidirectional motion estimation to-be-interpolated frame f_t+ΔtDividing the blocks into non-overlapped blocks of B × B size, calculating the motion vector of each block one by one according to formula (1) to form an initial motion vector field V of R × C size_t+Δt,0；

Wherein s in the formula (1) is a pixel coordinate, B_r,cIs the pixel coordinate set of the r-th row and c-th column block in the video frame, v is the candidate motion vector, S_r,cIs a block B_r,cCandidate motion vector search area of f_t(s+v)、f_t+1(s-v) each represents f_t、f_t+1Luminance values at pixel coordinates s + v, s-v.

3. The cellular automaton-based video frame rate up-conversion method according to claim 1, wherein the frame f to be interpolated_t+ΔtInitial motion vector field V_t+Δt,0Performing cellular automaton to control motion vector smoothing, and obtaining a frame f to be interpolated_t+ΔtIs the most important ofFinal motion vector field V_t+ΔtThe method comprises the following steps:

step b1, setting the iteration variable i to 0 and enabling

Step b2, pair one by oneThe abnormality detection is performed for each motion vector according to the formula (2), and the label E [ r, c ] of each motion vector after the abnormality detection is performed]GeneratingThe abnormality distribution map E;

wherein,

wherein, in the formula (2), E [ r, c]For markingMotion vector of middle r row and c columnWhether or not abnormal, 1 is abnormal, 0 is normal, v₀Represents Is composed of0 represents a zero vector, mean {. cndot } represents the median vector of the set of computed input vectors,representative calculation v₀Andthe cosine value of the included angle, T represents the transposition operation,&represents a logical AND, | represents a logical OR, and all vectors are column vectors;

step b3, constructing a cellular automaton;

wherein, regarding any element in the two-dimensional abnormal distribution map E as a cellular cell, the cellular cell has two states 0 and 1, and the cellular cell in the r-th row and c-th column is E [ r, c ], and its Moore neighborhood is defined as follows:

the von neuman neighborhood is defined as follows:

b4, evolving the states of the cells in the cellular automaton according to the following formula (8); wherein the two-dimensional abnormal distribution map E evolves into a new abnormal distribution map after the evolution of each cellular state

Wherein in the formula (8), Λ_kThe kth element representing Λ, k ═ 1,2, …,8, Ω_lThe i-th element representing Ω, i ═ 1,2,3, 4;

step b5, distributing the abnormal mapAnd the pair of formula (9)Smoothing;

step b6, calculating according to formula (10)Andthe residual value between;

wherein, in the formula (10), | · | ceiling₁To calculate the 1 norm of the input column vector, and is defined as follows:

step b7, if the epsilon value is larger than 3, making i equal to i +1, and going to step b2 to perform the next iteration, otherwise, exiting the loop iteration, and making the frame f to be inserted_t+ΔtOf the final motion vector field V_t+ΔtIs composed of