CN115100236A - Video processing method and device - Google Patents

Abstract

The application discloses a video processing method and a video processing device, and belongs to the technical field of video processing. The method comprises the steps of obtaining a first frame and a second frame of a target video, wherein the second frame is a previous frame adjacent to the first frame; determining a first motion vector and a weighting proportion of the first frame according to the second frame; adjusting the first motion vector according to the weighting proportion to obtain a second motion vector of the first frame; aligning the first frame with the second frame based on the second motion vector.

Description

Video processing method and device

technical field

The present application belongs to the technical field of video processing, and in particular relates to a video processing method and a device thereof.

Background technique

Image alignment, also known as frame alignment or image registration, is a technique of twisting and rotating an image to align it with another image. Image alignment is a key technique in many video processing scenarios, such as video noise reduction, motion detection, portrait segmentation, and more.

The optical flow method is usually used for image alignment. Optical flow method is a method of inferring the motion information of objects by detecting the change of the intensity of image pixels over time. When applying the optical flow method for image alignment, it is necessary to meet the conditions that the brightness of the object will not change when the object moves, and that the motion is a small movement. However, in real scenes, these two conditions are difficult to meet, resulting in the use of the optical flow method for image alignment. Alignment is less accurate.

SUMMARY OF THE INVENTION

The purpose of the embodiments of the present application is to provide a video processing method and device thereof, which can improve the accuracy and robustness of image alignment.

In a first aspect, an embodiment of the present application provides a video processing method, the method includes: acquiring a first frame and a second frame of a target video, wherein the second frame is an upper frame adjacent to the first frame a frame; determine the first motion vector of the first frame according to the second frame; adjust the first motion vector according to the weighting ratio to obtain the second motion vector of the first frame; The second motion vector aligns the first frame with the second frame.

In a second aspect, an embodiment of the present application provides a video processing apparatus, the apparatus includes: a first acquiring module, configured to acquire a first frame and a second frame of a target video, wherein the second frame is the same as the The previous frame adjacent to the first frame; the first determining module is used to determine the first motion vector and the weighting ratio of the first frame according to the second frame; the first weighting module is used to determine the weighting ratio according to the first frame Adjusting the first motion vector to obtain a second motion vector of the first frame; a first alignment module for aligning the first frame and the second frame based on the second motion vector .

In a third aspect, an embodiment of the present application provides an electronic device, the electronic device includes a processor and a memory, the memory stores a program or an instruction that can be executed on the processor, and the program or instruction is processed by the processor The video processing method according to the first aspect is implemented when the processor is executed.

In a fourth aspect, an embodiment of the present application provides a readable storage medium, where a program or an instruction is stored on the readable storage medium, and when the program or instruction is executed by a processor, the video processing method according to the first aspect is implemented .

In a fifth aspect, an embodiment of the present application provides a chip, the chip includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is configured to run a program or an instruction, and implement the first aspect the video processing method.

In a sixth aspect, an embodiment of the present application provides a computer program product, where the program product is stored in a storage medium, and the program product is executed by at least one processor to implement the video processing method according to the first aspect.

In the embodiment of the present application, the motion vectors of the two frames before and after are first determined according to the two frames before and after, and the relationship between the two frames is determined from the spatial dimension to ensure the robustness of the frame alignment in space; The weighted ratio of a frame adjusts the motion vector, so that the front and rear frames maintain smoothness in the time dimension, and the robustness of frame alignment in time is improved. Combining the temporal dimension with the spatial dimension can also improve the accuracy of frame alignment. In addition, the technical solution of this embodiment has a simple process and a small data throughput, which is beneficial to reduce the power consumption pressure of the system, and can increase application scenarios.

Description of drawings

1 is a flowchart of a video processing method provided by an embodiment of the present application;

2 is a schematic diagram of a sub-block in a video processing method provided by an embodiment of the present application;

3 is a schematic structural diagram of a video processing apparatus provided by an embodiment of the present application;

4 is one of the schematic structural diagrams of the electronic device provided by the embodiment of the present application;

FIG. 5 is a second schematic structural diagram of an electronic device provided by an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be clearly described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art fall within the protection scope of this application.

The terms "first", "second" and the like in the description and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It is to be understood that the data so used are interchangeable under appropriate circumstances so that the embodiments of the present application can be practiced in sequences other than those illustrated or described herein, and distinguish between "first", "second", etc. The objects are usually of one type, and the number of objects is not limited. For example, the first object may be one or more than one. In addition, "and/or" in the description and claims indicates at least one of the connected objects, and the character "/" generally indicates that the associated objects are in an "or" relationship.

The video processing method, video processing apparatus, and electronic device provided by the embodiments of the present application will be described in detail below with reference to the accompanying drawings through specific embodiments and application scenarios thereof.

The embodiments of the present application first provide a video processing method, and the video processing method can be applied to mobile phones, tablet computers, notebook computers, wearable electronic devices (such as smart watches), augmented reality (AR)/virtual reality (virtual reality) In electronic devices such as reality (VR) devices and in-vehicle devices, the embodiments of the present application do not impose any limitations on this.

FIG. 1 shows a flowchart of a video processing method provided by an embodiment of the present application. As shown in Figure 1, the video processing method includes the following steps:

Step 10: Acquire the first frame and the second frame of the target video, wherein the second frame is the previous frame adjacent to the first frame.

The image sensor can acquire images in a certain period of time, such as acquiring one frame of image every 30 milliseconds, etc. Multiple frames of images can form a video. The first frame may refer to the currently acquired image, or the image to be processed. The second frame refers to the image acquired before the first frame. For example, if the image sensor can generate one frame of image every 30 milliseconds, 30 milliseconds can be regarded as a moment, the first frame can be the image of the second moment, and the second frame can be the image of the first moment.

Step 20: Determine the first motion vector and the weighting ratio of the first frame according to the second frame.

The motion vector refers to the motion trajectory of an image or an area in the image relative to the reference frame, that is, the direction and size of the movement in the spatial perspective. In this embodiment, the first motion vector refers to the motion vector of the first frame relative to the second frame.

Exemplarily, the process of determining the first motion vector is as follows: first, both the first frame and the second frame are divided into multiple sub-blocks, such as M sub-blocks. M is a positive integer greater than 0. The size of the first frame and the second frame is the same. For example, for the first frame and the second frame whose size is W×H, W is the width of the image and H is the height of the image, and the image can be divided into m×n sub-blocks , M=m×n, then the size of each sub-block is W/m×H/n. For example, if the second frame and the first frame are respectively divided into 2×2 sub-blocks, as shown in FIG. 2 , the second frame 21 can be divided into sub-block 1, sub-block 2, sub-block 3 and sub-block 4, The first frame 22 is also divided into four sub-blocks: sub-block 1, sub-block 2, sub-block 3 and sub-block 4.

Then, motion estimation is performed for each pair of sub-blocks in the first frame and the second frame, respectively, and a first local vector of the sub-blocks in the first frame relative to the corresponding sub-blocks in the second frame is determined. The local vector refers to the estimated value of the motion vector of the sub-block, and may include the estimated value of the horizontal dimension and the estimated value of the vertical dimension. Exemplarily, grayscale processing may be performed on the first frame and the second frame first, and the value of each pixel point is converted into a grayscale value, thereby reducing the amount of calculation and improving the calculation efficiency. For each sub-block in the first frame and the second frame, the pixel values of each column in the sub-block, that is, the grayscale values, can be accumulated to obtain a one-dimensional array composed of the accumulated sum of the column pixel values. For example, if each sub-block includes 10 columns, the one-dimensional array formed by the cumulative sum of each column includes 10 elements. This one-dimensional array can be represented as:

For convenience of distinction, the sub-block 1 of the second frame 21 is denoted as pre1, and the sub-block 1 of the first frame 22 is denoted as cur1. In the above formula, Ver _pre1 is the one-dimensional array corresponding to the first sub-block of the second frame, Ver _pre1 (i) represents the i-th element in the one-dimensional array; blockH is the width of each sub-block, that is, the width of the column. The total number; blockW is the height of each sub-block, that is, the total number of rows; pre1(i, j) represents the pixels in the first sub-block of the first frame, that is, the i-th pixel in the i-th row; in this one-dimensional array There are blockW elements in total.

Moreover, the pixel values of each row of the first sub-block of the second frame are accumulated, and a one-dimensional array can also be obtained according to the accumulated sum of the row pixel values. For the convenience of partitioning, the one-dimensional array Ver formed by the above column pixels is called a column array, and the one-dimensional array formed by row pixels is called a row array. The row array of the first subblock of the second frame can be represented as:

Among them, Hor represents an array of columns. Other sub-blocks of the second frame, ie, row arrays and column arrays of pre2, pre3, and pre4, and row arrays and column arrays of each sub-block of the first frame can also be determined according to the above formulas (1) and (2). Through the row array and column array of each sub-block, the characteristics of the horizontal dimension and the vertical dimension of different regions in the first frame or the second frame can be counted. After the above processing, m×n×2×2 one-dimensional arrays can be obtained, that is, 8 arrays in the first frame and 8 arrays in the second frame.

Optionally, the estimated values of the motion vectors in the horizontal dimension and the vertical dimension of the second frame and the first frame are calculated by means of offset subtraction. Dislocation refers to moving the second frame or the first frame by a certain distance and then subtracting it from another frame. The range of dislocation is preset, such as [-5,5], [-9,9], etc., which is not particularly limited in this embodiment. Exemplarily, different misalignment ranges may be set for the horizontal dimension and the vertical dimension, respectively. After the misalignment range is determined, a misalignment value with the smallest accumulated difference value between the second frame and the first frame is searched within the range, and the misalignment value can be determined as the first local vector of the sub-block. For example, when the misalignment value is 0, for the first sub-block pre1 of the second frame and the first sub-block cur1 of the first frame, calculate the result of subtracting the first element of the sub-block pre1 and the sub-block cur1 in turn. , the result of the subtraction of the second element, the result of the subtraction of the third element, etc., and then add the absolute value of each pair of subtraction results to get the sub-block pre1 and sub-block cur1 in the case of a misalignment of 0 in the horizontal dimension Or the cumulative difference value for the vertical dimension. After the difference value corresponding to each misalignment position within the misalignment range is calculated, the misalignment value with the smallest difference value is determined, and this value is the first local vector of the sub-block pre1 and the sub-block cur1. For example, for the first sub-block of the first frame, the calculation process in the horizontal dimension is expressed as follows:

Wherein, difVer _block1 (k) represents the difference value between the first sub-block block1 of the first frame and the first sub-block of the second frame when the dislocation is k; k takes a value within the dislocation range, for example k=- 10, -9, …, 0, 1, …, 9, 10. In this range, the difference value difVer between the corresponding sub-blocks is calculated for each k value in turn, and the k value with the smallest difference difVer value is used as the estimated value of the horizontal dimension of the sub-block, denoted as X _block1 . Similarly, the difference value difHor of the vertical dimension of the first sub-block block1 can be calculated to obtain the estimated value Y _block1 when the difference value difHor is the smallest. The first local vector of the first sub-block block1 is: X _block1 , Y _block1 .

In this embodiment of the present application, each sub-block in the first frame is processed by the above formula (3), and the first local vector of each sub-block can be obtained. Then, the local vector of the first frame is spatially filtered to improve the robustness and smoothness of the spatial dimension.

For the j-th sub-block in the first frame, perform mean filtering on the first local vector of the j-th sub-block according to the adjacent sub-blocks of the j-th sub-block to obtain the second local vector of the j-th sub-block, if starting from 0 count, then 0≤j≤M. j can also start counting from 1. If the adjacent sub-blocks of the j-th sub-block are the j+1-th sub-block and the j-1-th sub-block, then the average value of the first local vectors of the j-1, j, and j+1-th sub-blocks is taken as the j-th sub-block The second local vector of the block. Exemplarily, take the surrounding 8 sub-blocks centered on the j-th sub-block as the adjacent sub-blocks with the j-th sub-block, and calculate the average value of the first local vectors of the 9 sub-blocks together with the j-th sub-block, and set The result is obtained as the second local vector of the jth subblock. The second local vector is the first motion vector of the first frame.

Exemplarily, the first motion vector of the first frame may include a vector diagram composed of the second local vector of each sub-block, and the length and width of the vector diagram are the number of sub-blocks in the horizontal direction and the number in the vertical direction, that is, m. , n. Referring to FIG. 2 , when the image is divided into 2×2 sub-blocks, the vector diagram of the first frame can be represented as MVmap1(x,y), x=1,2; y=1,2. Specifically: MVmap1(1,1)={X _block1 ,Y _block1 }, MVmap1(1,2)={X _block2 ,Y _block2 }, MVmap1(2,1)={X _block3 ,Y _block3 }, MVmap1( 2,2)={X _block4 , Y _block4 }. Alternatively, the second local vector of each sub-block of the first frame may be represented as a one-dimensional vector as the first motion vector. For example, the first element of the one-dimensional vector MVmap1 MVmap1(1)={X _block1 , Y _block1 }, the second element MVmap1(2)={X _block2 , Y _block2 } and so on.

In this embodiment, the local vector of the first frame is first estimated according to the intra-frame information of the first frame, and then the average filtering is performed on the local vector to enhance the smoothness of the local vector and ensure the overall consistency of the local vector.

Step 30: Adjust the first motion vector according to the weighting ratio to obtain the second motion vector of the first frame.

It can be understood that the first motion vector of the first frame can be determined according to the second frame, and the first motion vector of the second frame can also be determined according to the frame before the second frame. That is, the first motion vector of the next frame can be determined according to every two adjacent frames in the video. Furthermore, the weighting ratio of the first frame can be determined according to the first motion vector of the second frame, and then the first motion vector of the first frame is processed according to the weighting ratio. Similarly, the weighting ratio of the second frame can also be determined according to the frame before the second frame.

Exemplarily, the weighting ratio may include two coefficients, which are a first coefficient and a second coefficient, respectively. In terms of the horizontal dimension and the vertical dimension, the weighting ratio may include the first coefficient and the second coefficient of the horizontal dimension and the first coefficient and the second coefficient of the vertical dimension. The weighted ratio of the first frame may be determined according to the weighted ratio of the second frame. The frame at the historical moment is integrated into the motion vector result of the first frame through the weighted ratio, which can improve the robustness of the motion vector estimation result of the first frame in the time domain.

Specifically, the first candidate coefficient of the first frame is first determined based on the first target parameter and the first coefficient in the weighting ratio of the second frame. If the second frame is the start frame in the video, such as counting from 0, that is, the 0th frame, the weighting ratio of the 0th frame is 0, that is, the first coefficient and the second coefficient are both 0. Weights can be introduced by adding a first objective parameter. Specifically, for the horizontal dimension or the vertical dimension, the first coefficient of the second frame is first updated based on the first target parameter, as shown in the following formula:

pre _bx (i)=pre _bx (i)+paramx1 (4)

Wherein, pre _bx (i) is the first coefficient of the horizontal dimension of the ith sub-block of the second frame, i is greater than or equal to 0, and less than or equal to m×n, and paramx1 represents the first target parameter. That is, the updated result is the sum of the first target parameter and the original first coefficient of the second frame. Then, the updated first coefficient Pre _bx is normalized based on the second target parameter to obtain the first candidate coefficient of the first frame. The normalization process can be expressed by the following formula:

tmp1(i)=pre _bx (i)/(pre _bx (i)+paramx2) (5)

Wherein, tmp1(i) represents the first candidate coefficient of the horizontal dimension of the i-th sub-block after normalization, and paramx2 represents the second target parameter. Through this formula, the first candidate coefficient of the horizontal dimension of the first frame can be obtained. Similarly, the same processing is performed on the vertical dimension according to formula (4) and formula (5), and the first candidate coefficient of the vertical dimension of the first frame can be obtained. Exemplarily, the first target parameter and the second target parameter of the vertical dimension may be different from or the same as the horizontal dimension, which is not specifically limited in this embodiment.

Next, a second candidate coefficient of the first frame may be determined based on the second coefficient of the second frame and the first motion vector of the first frame. Exemplarily, the first-order variance between the second coefficient of the second frame and the first motion vector of the first frame is determined, and the second candidate coefficient of the first frame is obtained based on the first-order variance. The formula is expressed as follows:

tmp2(i)=MVmap1(i)-pre _xa (i) (6)

Wherein, tmp2(i) is the second candidate coefficient of the first frame, and pre _xa (i) is the second coefficient of the horizontal dimension of the second frame. If the second frame is the start frame of the video, then pre _xa (i)=0. MVmap1(i) represents the i-th element in the first motion vector of the first frame, where i is greater than or equal to 0 and less than or equal to m×n. The first-order variance between each element in the first motion vector of the first frame and the second coefficient of the second frame is determined as the second candidate coefficient of the first frame.

The weighting ratio of the first frame is then determined based on the first candidate coefficient and the second candidate coefficient of the first frame. Exemplarily, the weighting ratio of the first frame may be obtained by performing weighting processing on the first candidate coefficient and the second candidate coefficient of the first frame. Specifically, for the horizontal dimension, the first coefficient of the horizontal dimension of the first frame is determined based on the first candidate coefficient and the second candidate coefficient of the horizontal dimension of the first frame. The formula is expressed as follows:

cur _xb (i)=(1-tmp1(i)*pre _xb (i)) (7)

Wherein, cur _xb (i) is the first coefficient of the horizontal dimension of the ith sub-block of the first frame. The first coefficient of the first frame is determined according to the first candidate coefficient of the first frame and the first coefficient of the second frame, that is, the updated first coefficient in formula (4). In this way, the first coefficient of the second frame is added to the first coefficient of the first frame in a certain proportion, so as to enhance the correlation on the time axis, thereby improving the smoothness between the first frame and the previous frame.

The second coefficient of the horizontal dimension of the first frame is calculated according to the following formula:

cur _xa (i)=pre _xa (i)+tmp1(i)*tmp2(i) (8)

Wherein, cur _xa (i) is the second coefficient of the horizontal dimension of the first frame. The first motion vector of the first frame is integrated into the weighted ratio of the first frame according to a certain proportion, and the next frame is continued to be weighted, which can ensure the smoothness between the first frame and the next frame, thereby enhancing the video time. robustness. Similarly, for the vertical dimension, the first coefficient of the vertical dimension of the first frame can be obtained by performing the same weighting process according to the first candidate coefficient and the second candidate coefficient of the vertical dimension of the first frame.

On the basis of determining the weighting ratio of the first frame, the first motion vector determined between them is adjusted to obtain the second motion vector of the first frame. Exemplarily, the first component of the first frame may be determined based on the above-mentioned weighted ratio between the third target parameter and the first frame, the second component may be determined based on the fourth target parameter and the first motion vector of the first frame, and then the first component is combined. The one component and the second component can obtain the second motion vector. In this embodiment, the weighting ratio is used to perform weighting processing on the third target parameter, and the obtained result is fused with a certain ratio of the first motion vector to obtain the second motion vector. Adjusting the motion vector by a certain proportion of target parameters can assist the estimation of the motion vector, which is beneficial to improve the accuracy of the motion estimation. For example, the second motion vector of the first frame can be:

MVmap2(i)=pre _xa (i)*paramx3+MVmap1 _x (i)*paramx4 (9)

MVmap1 _x (i) represents the value of the horizontal dimension in the first motion vector of the first frame, and the value of the horizontal dimension of the second motion vector MVmap2(i) of the first frame can be calculated according to this formula. Wherein, paramx3 represents the third target parameter of the horizontal dimension, and the result of weighting the third target parameter by the second coefficient of the first frame is the first component, that is, pre _xa (i)*paramx3. paramx4 is the fourth target parameter in the horizontal dimension, and the result of weighting the horizontal value of the first motion vector by using the fourth target parameter is the second component, that is, MVmap1 _x (i)*paramx4. The calculation process of the vertical dimension is the same as that of the horizontal dimension, and will not be repeated here.

With continued reference to FIG. 1, in step 40, the first frame is aligned with the second frame based on the second motion vector.

Each point in the first frame can be moved according to the second motion vector to obtain an image with the same spatial perspective as the second frame. According to the above method, each frame in the captured video can be aligned, and then the aligned video is subjected to noise reduction processing to obtain a video with better image quality.

Exemplarily, after the second motion vector is obtained, the second motion vector may be further optimized in the spatial domain. Specifically, an average value of the second motion vector of the first frame is determined; then, based on the average value, the target vector value in the second motion vector is eliminated to obtain a third motion vector. For the average of the horizontal dimension, by the following formula:

Wherein, MVmap2 _x (i) is the horizontal dimension value of the i-th element in the second motion vector, and m*n is the total number of sub-blocks. By calculating the variance of each value in the second motion vector from this average, the value with the largest variance, the target vector value, can be determined. Also taking the above horizontal dimension as an example, the variance between the average value of the horizontal dimension and each horizontal dimension value in the second motion vector is calculated as follows:

x(i)=(MVmap2 _x (i)-mean _x ) ² (11)

x(i) is the variance of the i-th element in the second motion vector. The horizontal dimension value of the element with the largest variance is the value of the target vector. After obtaining the variance of each element, replace the target vector value with the mean _x value of the horizontal dimension to complete the processing of the horizontal dimension. The vertical dimension is then processed in the same way. The target vector value is replaced by the average value, so that the target vector value is eliminated to obtain a smoother motion vector, that is, the third motion vector. The first frame is then aligned with the previous frame according to the third motion vector.

The video processing method provided by this embodiment has a simple process and can improve the energy efficiency ratio of software and hardware. Moreover, the motion estimation of video frames from space and time can improve the accuracy of motion vectors and the smoothness in time and space.

Further, in the video processing method provided by the embodiment of the present application, the execution body may be a video processing apparatus. Hereinafter, the video processing apparatus provided by the embodiment of the present application is described by taking the video processing method performed in the video processing apparatus as an example.

As shown in FIG. 3 , the video processing apparatus 30 provided in this embodiment of the present application may include a first acquisition module 31 , a first determination module 32 , a first weighting module 33 , and a first alignment module 34 . Specifically, the first obtaining module 31 is configured to obtain the first frame and the second frame of the target video, wherein the second frame is the previous frame adjacent to the first frame; the first determining module 32, for determining the first motion vector and weighting ratio of the first frame according to the second frame; the first weighting module 33 is used to adjust the first motion vector according to the weighting ratio to obtain the second motion vector of the first frame; the first The alignment module 34 is configured to align the first frame with the second frame based on the second motion vector.

The video processing device provided in this embodiment determines the motion vector of the two frames before and after, determines the relationship between the two frames from the spatial dimension, and then adjusts the motion vector according to the weighting ratio of the previous frame to the next frame, so that the frame before and after is adjusted. Maintain smoothness in the time dimension. The temporal dimension combined with the spatial dimension can improve the accuracy and robustness of frame alignment. In addition, the technical solution of this embodiment has a simple process and a small data throughput, which is beneficial to reduce the power consumption pressure of the system.

In this embodiment of the present application, the first determining module 32 includes: a first dividing module, configured to divide the first frame and the second frame into M sub-blocks, where M is a positive integer; a second determining module, configured to determine the first frame and the second frame. The subblock in one frame is relative to the first local vector of the corresponding subblock in the second frame; the first filtering module is used for the jth subblock in the first frame, according to the adjacent subblocks of the jth subblock, Perform mean filtering on the first local vector of the jth subblock to obtain the second local vector of the jth subblock, 0≤j≤M; the third determination module is used for according to the second local vector of each subblock in the first frame The vector determines the first motion vector.

Exemplarily, the weighting ratio includes a first coefficient and a second coefficient, and the first weighting module 33 specifically includes: a first coefficient determination module, configured to determine the first frame based on the first target parameter and the first coefficient of the second frame. a first candidate coefficient; a second coefficient determination module for determining a second candidate coefficient of the first frame based on the second coefficient of the second frame and the first motion vector of the first frame; a second weighting module for determining the second candidate coefficient of the first frame based on the first The candidate coefficient and the second candidate coefficient determine the weighting ratio of the first frame.

In the embodiment of the present application, the first coefficient determination module includes a first update module, which is used to update the first coefficient of the second frame based on the first target parameter; the first normalization module is used to update the first coefficient of the second frame based on the second target parameter. The updated first coefficient is normalized to obtain the first candidate coefficient of the first frame.

In the embodiment of the present application, the second coefficient determination module is specifically configured to: determine the first-order variance between the second coefficient of the second frame and the first motion vector of the first frame, and obtain the second candidate coefficient of the first frame.

In this embodiment of the present application, the above-mentioned first weighting module 33 specifically includes: a first component determination module, configured to determine the first component based on the weighting ratio of the third target parameter and the first frame; a second component determination module, configured to be based on The fourth target parameter and the first motion vector of the first frame determine the second component; the first obtaining module is configured to obtain the second motion vector of the first frame based on the first component and the second component.

In the embodiment of the present application, the above-mentioned first alignment module includes: a first averaging module, used to determine the average value of the second motion vector of the first frame; a first elimination module, configured to eliminate the first frame based on the average value. The target vector in the second motion vector obtains the third motion vector of the first frame; the second alignment module is used for aligning the first frame and the second frame based on the third motion vector.

The video processing apparatus in this embodiment of the present application may be an electronic device, or may be a component in the electronic device, such as an integrated circuit or a chip. The electronic device may be a terminal, or may be other devices other than the terminal. Exemplarily, the electronic device may be a mobile phone, a tablet computer, a notebook computer, a palmtop computer, a vehicle-mounted electronic device, a Mobile Internet Device (MID), an augmented reality (AR)/virtual reality (VR) Devices, robots, wearable devices, ultra-mobile personal computers (UMPCs), netbooks or personal digital assistants (PDAs), etc., and can also be servers, network attached storages (NAS) , a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which are not specifically limited in the embodiments of the present application.

The video processing apparatus in this embodiment of the present application may be an apparatus having an operating system. The operating system may be an Android (Android) operating system, an ios operating system, or other possible operating systems, which are not specifically limited in the embodiments of the present application.

The video processing apparatus provided in this embodiment of the present application can implement each process implemented by the method embodiment in FIG. 1 , and to avoid repetition, details are not repeated here.

Optionally, as shown in FIG. 4 , an embodiment of the present application further provides an electronic device 400 , including a processor 401 and a memory 402 . The memory 402 stores a program or instruction that can be run on the processor 401. When the program or instruction is executed by the processor 401, each step of the above-mentioned video processing method embodiment can be achieved, and the same technical effect can be achieved. Repeat, and will not repeat them here.

It should be noted that the electronic devices in the embodiments of the present application include the aforementioned mobile electronic devices and non-mobile electronic devices.

FIG. 5 is a schematic diagram of a hardware structure of an electronic device implementing an embodiment of the present application.

The electronic device 500 includes but is not limited to: a radio frequency unit 501, a network module 5102, an audio output unit 503, an input unit 504, a sensor 505, a display unit 506, a user input unit 507, an interface unit 508, a memory 509, and a processor 510, etc. part.

Those skilled in the art can understand that the electronic device 500 may also include a power supply (such as a battery) for supplying power to various components, and the power supply may be logically connected to the processor 510 through a power management system, so as to manage charging, discharging, and power management through the power management system. consumption management and other functions. The structure of the electronic device shown in FIG. 5 does not constitute a limitation on the electronic device. The electronic device may include more or less components than the one shown, or combine some components, or arrange different components, which will not be repeated here. .

The processor 510 may be configured to acquire the first frame and the second frame of the target video, wherein the second frame is the previous frame adjacent to the first frame; the first frame is determined according to the second frame. The first motion vector and weighting ratio of a frame; adjusting the first motion vector according to the weighting ratio to obtain the second motion vector of the first frame; The frame is aligned with the second frame.

The input unit 504 may include a graphics processor (Graphics Processing Unit, GPU) 1041 and a microphone 5042, and the graphics processor 5041 captures images of still pictures or videos obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode data is processed. The display unit 506 may include a display panel 5061, which may be configured in the form of a liquid crystal display, an organic light emitting diode, or the like. The user input unit 507 includes at least one of a touch panel 5071 and other input devices 5072 . The touch panel 5071 is also called a touch screen. The touch panel 5071 may include two parts, a touch detection device and a touch controller. Other input devices 5072 may include, but are not limited to, physical keyboards, function keys (such as volume control keys, switch keys, etc.), trackballs, mice, and joysticks, which are not described herein again.

The memory 509 may be used to store software programs as well as various data. The memory 509 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required for at least one function (such as a sound playback function, image playback function, etc.), etc. Additionally, memory 509 may include volatile memory or non-volatile memory, or memory 509 may include both volatile and non-volatile memory. Wherein, the non-volatile memory may be Read-Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (Erasable PROM, EPROM), Erase programmable read-only memory (Electrically EPROM, EEPROM) or flash memory. Volatile memory can be random access memory (Random Access Memory, RAM), static random access memory (Static RAM, SRAM), dynamic random access memory (Dynamic RAM, DRAM), synchronous dynamic random access memory (Synchronous random access memory) DRAM, SDRAM), double data rate synchronous dynamic random access memory (Double Data Rate SDRAM, DDRSDRAM), enhanced synchronous dynamic random access memory (Enhanced SDRAM, ESDRAM), synchronous link dynamic random access memory (Synch link DRAM) , SLDRAM) and direct memory bus random access memory (Direct Rambus RAM, DRRAM). The memory 509 in this embodiment of the present application includes, but is not limited to, these and any other suitable types of memory.

The processor 510 may include one or more processing units; optionally, the processor 510 integrates an application processor and a modem processor, wherein the application processor mainly processes operations involving an operating system, a user interface, and an application program, etc. Modem processors mainly deal with wireless communication signals, such as baseband processors. It can be understood that, the above-mentioned modulation and demodulation processor may not be integrated into the processor 510.

Embodiments of the present application further provide a readable storage medium, where a program or an instruction is stored on the readable storage medium. When the program or instruction is executed by a processor, each process of the above video processing method embodiment can be achieved, and the same can be achieved. In order to avoid repetition, the technical effect will not be repeated here.

Wherein, the processor is the processor in the electronic device described in the foregoing embodiments. The readable storage medium includes a computer-readable storage medium, such as computer read-only memory ROM, random access memory RAM, magnetic disk or optical disk, and the like.

An embodiment of the present application further provides a chip, where the chip includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is configured to run a program or an instruction to implement the above video processing method embodiments. Each process can achieve the same technical effect. In order to avoid repetition, it will not be repeated here.

It should be understood that the chip mentioned in the embodiments of the present application may also be referred to as a system-on-chip, a system-on-chip, a system-on-a-chip, or a system-on-a-chip, or the like.

The embodiments of the present application provide a computer program product, the program product is stored in a storage medium, and the program product is executed by at least one processor to implement the various processes in the above video processing method embodiments, and can achieve the same technical effect , in order to avoid repetition, it will not be repeated here.

It should be noted that, herein, the terms "comprising", "comprising" or any other variation thereof are intended to encompass non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements, It also includes other elements not expressly listed or inherent to such a process, method, article or apparatus. Without further limitation, an element qualified by the phrase "comprising a..." does not preclude the presence of additional identical elements in a process, method, article or apparatus that includes the element. Furthermore, it should be noted that the scope of the methods and apparatus in the embodiments of the present application is not limited to performing the functions in the order shown or discussed, but may also include performing the functions in a substantially simultaneous manner or in the reverse order depending on the functions involved. To perform functions, for example, the described methods may be performed in an order different from that described, and various steps may also be added, omitted, or combined. Additionally, features described with reference to some examples may be combined in other examples.

From the description of the above embodiments, those skilled in the art can clearly understand that the method of the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is better implementation. Based on this understanding, the technical solutions of the present application can be embodied in the form of computer software products that are essentially or contribute to the prior art, and the computer software products are stored in a storage medium (such as ROM/RAM, magnetic disk , CD), including several instructions to make a terminal (which may be a mobile phone, a computer, a server, or a network device, etc.) execute the methods described in the various embodiments of the present application.

The embodiments of the present application have been described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific embodiments, which are merely illustrative rather than restrictive. Under the inspiration of this application, without departing from the scope of protection of the purpose of this application and the claims, many forms can be made, which all fall within the protection of this application.

Claims (10)

1. a video processing method, is characterized in that, comprises: Obtain the first frame and the second frame of the target video, wherein the second frame is the previous frame adjacent to the first frame; determining a first motion vector and a weighting ratio of the first frame according to the second frame; Adjusting the first motion vector according to the weighting ratio to obtain a second motion vector of the first frame; The first frame is aligned with the second frame based on the second motion vector. 2. The video processing method according to claim 1, wherein the determining the first motion vector of the first frame according to the second frame comprises: dividing the first frame and the second frame into M sub-blocks, where M is a positive integer; determining a first local vector of a sub-block in the first frame relative to a corresponding sub-block in the second frame; Perform mean filtering on the first local vector of the jth subblock in the first frame according to the adjacent subblocks of the jth subblock in the first frame to obtain a second local vector of the jth subblock vector, 0≤j≤M; The first motion vector is determined from the second local vector for each sub-block in the first frame. 3 . The video processing method according to claim 1 , wherein the weighting ratio includes a first coefficient and a second coefficient; and the determining the weighting ratio of the first frame according to the second frame includes: 3 . : determining a first candidate coefficient of the first frame based on the first target parameter and the first coefficient of the second frame; determining a second candidate coefficient of the first frame based on the second coefficient of the second frame and the first motion vector of the first frame; The weighting ratio of the first frame is determined based on the first candidate coefficient and the second candidate coefficient. 4. The method according to claim 3, wherein the determining the first candidate coefficient of the first frame based on the first target parameter and the first coefficient of the second frame comprises: updating the first coefficient of the second frame based on the first target parameter; The updated first coefficient is normalized based on the second target parameter to obtain the first candidate coefficient of the first frame. 5. The video processing method according to claim 1, wherein the aligning the first frame and the second frame based on the second motion vector comprises: determining the average value of the second motion vector of the first frame; Eliminate the target vector in the second motion vector based on the average to obtain the third motion vector of the first frame; The first frame is aligned with the second frame based on the third motion vector. 6. A video processing device, comprising: a first acquisition module, configured to acquire the first frame and the second frame of the target video, wherein the second frame is the previous frame adjacent to the first frame; a first determining module, configured to determine a first motion vector and a weighting ratio of the first frame according to the second frame; a first weighting module, configured to adjust the first motion vector according to the weighting ratio to obtain the second motion vector of the first frame; A first alignment module, configured to align the first frame with the second frame based on the second motion vector. 7. The video processing apparatus according to claim 6, wherein the first determining module comprises: a first dividing module, configured to divide the first frame and the second frame into M sub-blocks, where M is a positive integer; a second determining module, configured to determine a first local vector of the sub-block in the first frame relative to the corresponding sub-block in the second frame; A first filtering module, configured to perform mean filtering on the first local vector of the jth subblock in the first frame according to the adjacent subblocks of the jth subblock in the first frame, to obtain the The second local vector of the j sub-block, 0≤j≤M; A third determining module, configured to determine the first motion vector according to the second local vector of each sub-block in the first frame. 8. The video processing apparatus according to claim 6, wherein the weighting ratio comprises a first coefficient and a second coefficient, and the first weighting module comprises: a first coefficient determination module, configured to determine a first candidate coefficient of the first frame based on the first target parameter and the first coefficient of the second frame; a second coefficient determination module, configured to determine a second candidate coefficient of the first frame based on the second coefficient of the second frame and the first motion vector of the first frame; A second weighting module, configured to determine a weighting ratio of the first frame based on the first candidate coefficient and the second candidate coefficient. 9. The video processing apparatus according to claim 8, wherein the first coefficient determination module comprises: a first update module, configured to update the first coefficient of the second frame based on the first target parameter; The first normalization module is configured to perform normalization processing on the updated first coefficient based on the second target parameter to obtain the first candidate coefficient of the first frame. 10. The video processing apparatus according to claim 6, wherein the first alignment module comprises: a first averaging module for determining the average value of the second motion vector of the first frame; a first culling module for culling the target vector in the second motion vector of the first frame based on the average value to obtain the third motion vector of the first frame; A second alignment module, configured to align the first frame with the second frame based on the third motion vector.

Images