CN113507609B - Interframe image parallel coding method based on time-space domain prediction - Google Patents
Interframe image parallel coding method based on time-space domain prediction Download PDFInfo
- Publication number
- CN113507609B CN113507609B CN202111053623.1A CN202111053623A CN113507609B CN 113507609 B CN113507609 B CN 113507609B CN 202111053623 A CN202111053623 A CN 202111053623A CN 113507609 B CN113507609 B CN 113507609B
- Authority
- CN
- China
- Prior art keywords
- coding block
- coding
- motion vector
- depth
- time domain
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
- H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
- H04N19/51—Motion estimation or motion compensation
- H04N19/513—Processing of motion vectors
- H04N19/517—Processing of motion vectors by encoding
- H04N19/52—Processing of motion vectors by encoding by predictive encoding
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The invention discloses an interframe image parallel coding method based on time-space domain prediction, which relates to the technical field of image processing and comprises the following steps: setting a spatial domain reliability threshold and a temporal domain reliability threshold according to data compression requirements, and acquiring a motion vector candidate list of a current frame image coding block group; respectively acquiring a space domain reliability parameter and a time domain reliability parameter according to the motion direction of the current frame image; selecting more suitable motion vectors in the current coding block as shared motion vectors of all coding blocks in a parallel region according to the proportion of the space domain reliability parameters to the time domain reliability parameters and the size relationship between the space domain reliability parameters and the time domain reliability threshold; and carrying out parallel coding on the coding blocks in the coding block group according to the motion vector set shared by the coding blocks. The invention reduces the texture complexity of the image to be processed in the process of parallel coding by diving the depth of the coding block, so that the coding block realizes parallel coding at the same time of low bit rate loss.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to an interframe image parallel coding method based on time-space domain prediction.
Background
Motion estimation is a commonly used technique in video continuous frame image coding, such as MPEG-2, MPEG-4, h.264/AVC, h.265/HEVC, and it is also a very critical technique in video coding, and occupies half of the coding time. In the past, many motion estimation algorithms and architectures are not suitable for high-definition and ultra-high-definition video applications, and therefore, in order to realize a high-timeliness coding environment, the current motion estimation algorithms and architectures need to be optimized, and parallel processing is a feasible scheme. However, HEVC currently only supports slice (slice) level parallelism and does not support coding block (CU) level parallelism well.
The reason why the current HEVC cannot realize parallel processing at a coding block level is that there is dependency between coding blocks in the HEVC, and the HEVC removes redundant information through the interdependency between the coding blocks. Specifically, in the inter-frame prediction process of HEVC, a search center is found by an Advanced Motion Vector Prediction (AMVP) technique, and then a best matching block is found. The AMVP technique is to use Motion Vectors (MVs) available in neighboring blocks to predict the search center, and within a block, the MV of the current block can be predicted from the information of its neighboring blocks. And the current coding block can carry out coding only after the coding of the adjacent coding blocks is finished, so that the parallel coding of the current coding block and the adjacent coding blocks is difficult to carry out synchronously. Therefore, if parallel encoding is to be performed between encoding blocks, the interdependence between encoding blocks has to be eliminated.
Disclosure of Invention
In order to eliminate the dependency among the coding blocks and improve the timeliness in the process of interframe image compression coding, the invention provides an interframe image parallel coding method based on time-space domain prediction, wherein the parallel coding is the parallel coding of each coding block in a coding block group, each coding block contains part of known motion vectors and part of unknown motion vectors, and the parallel coding mainly comprises the following steps:
s1: setting a spatial domain reliability threshold and a temporal domain reliability threshold according to data compression requirements, and acquiring a motion vector candidate list of a current frame image coding block group, wherein the motion vector candidate list comprises a spatial domain candidate list and a temporal domain candidate list;
s2: extracting spatial domain motion vectors of the coding block group in the spatial domain candidate list on the directional corner in the motion direction and two corners adjacent to the directional corner according to the motion direction of the current frame image;
s3: obtaining a spatial domain reliability parameter based on depth scaling according to the depth of the current coding block and the difference value in the horizontal direction between the extracted spatial domain motion vectors;
s4: extracting time domain motion vectors at the corner-to-corner position in the direction of the current coding block in the current frame and previous frame time domain candidate lists according to the time domain motion vectors at the corner-to-corner position in the direction of the coding block group;
s5: obtaining time domain reliability parameters based on depth scaling according to the depth of the current coding block and the difference value between the horizontal component and the vertical component of the extracted time domain motion vector;
s6: judging whether the airspace reliability parameter is smaller than a airspace reliability threshold value or not and whether the time domain reliability parameter is smaller than a time domain reliability threshold value or not, if so, entering the next step, if not, entering a coding block at the next depth before the depth of the current coding block is larger than a preset threshold value, and returning to the step S2;
s7: judging whether the space domain reliability parameter is larger than or equal to a time domain reliability parameter with a preset proportion, if so, sharing the known time domain motion vector of the current coding block by all the coding blocks in the parallel region of the current coding block, and if not, sharing the known space domain motion vector of the current coding block by all the coding blocks in the parallel region of the current coding block;
s8: and carrying out parallel coding on the coding blocks in the coding block group according to the motion vector set shared by the coding blocks.
Further, in the step S6, if it is determined that the current coding block depth is greater than the preset threshold, the process proceeds to step S8.
Further, the depth of the coding block is 0 to 3, and the extracted spatial domain motion vector and the extracted temporal domain motion vector are scaled according to the depth of the current coding block based on the depth scaling, where the corresponding proportion specifically is:
when the depth is 0, the scaling factor is one, when the depth is 1, the scaling factor is one half, when the depth is 2, the scaling factor is one quarter, and when the depth is 3, the scaling factor is one eighth.
Further, the step S3 can be expressed by the following formula:
in the formula, B2 represents a spatial motion vector on a directional corner, a0 and a1 represent spatial motion vectors on one corner adjacent to the directional corner, and B0 and B1 are spatial motion vectors on the other corner adjacent to the directional corner; mvx is the horizontal component of the scaled corresponding motion vector; r is1Is the sum of the absolute values of the differences of the scaled A0 and A1 horizontal components and B2 horizontal components, r2Is the sum of the absolute values of the differences of the scaled B0 and B1 horizontal components and B2 horizontal components, respectively; rSIs a spatial domain reliability parameter.
Further, the step S5 can be expressed by the following formula:
wherein t is the time domain motion vector at the corner opposite to the current coding block direction in the current frame time domain candidate list, t-1 is the time domain motion vector at the corner opposite to the current coding block direction in the previous frame time domain candidate list, mvy is the vertical component of the scaled corresponding motion vector, RTIs a time domain reliability parameter.
Further, the step S7 can be expressed by the following formula:
in the formula, MVnSet of motion vectors, MV, known for any block in the parallel region of the current blockmSet of motion vectors, MV, shared for the arbitrary coding blockTSet of temporal motion vectors, MV, known for the current coding blockSA set of spatial motion vectors known to the current coding block,
at a predetermined ratio, TH1 is a spatial domain reliability threshold, and TH2 is a temporal domain reliability threshold.
Further, the structure of the coding block group is a 2N × 2N matrix.
Further, the next depth coding block is a quarter of the current depth coding block;
further, in step S4, when the temporal motion vector at the corner of the direction does not exist, the temporal motion vector at the center of the coding block is selected for replacement.
Compared with the prior art, the invention at least has the following beneficial effects:
(1) the invention relates to an interframe image parallel coding method based on time-space domain prediction, which takes the direction corner of a coding block group and the upper space domain motion vector of the adjacent corner as substitute quantity to substitute the motion vector of the corresponding corner of each coding block in the coding group, and then judges the consistency of the horizontal and vertical motion trends of the current coding block under the condition of depth scaling, and further selects the space domain motion vector of the coding block under the depth with higher reliability (smoother) as the sharing parameter basis of parallel coding;
(2) taking the time domain motion vector at the position corresponding to the current coding block in the time domain candidate list as a substitute for the time domain motion vector of the coding block, and further judging the consistency of the time motion trend of the current coding block under the condition of depth-based scaling, so that the time domain motion vector of the coding block under the depth with higher reliability can be selected as a shared parameter basis of parallel coding;
(3) the motion vector of a smoother (proper depth) coding block is selected by comparing the reliability parameters for parallel coding, so that the loss of the bit rate is reduced while the parallel coding is realized;
(4) by comparing the proportional relation between the spatial domain reliability parameters and the time domain reliability parameters, the motion vector with higher reliability is selected between the spatial domain reliability parameters and the time domain reliability parameters to serve as a shared parameter basis of parallel coding, and the effectiveness of the parallel coding is improved.
Drawings
FIG. 1 is a method step diagram of an interframe image parallel coding method based on time-space domain prediction;
fig. 2 is a schematic diagram of the motion vector steps in an exemplary motion direction.
Detailed Description
The following are specific embodiments of the present invention and are further described with reference to the drawings, but the present invention is not limited to these embodiments.
Example one
In the existing h.265/HEVC technology, some parallel tools can be used to improve the parallelism of motion estimation. In general, the level of parallelism within a Largest Coding Unit (LCU) can be divided into: coding block group (CU group) level and coding block (CU) level parallelism. Within the parallel area (PMER), some motion vector candidates of the coding blocks are known and some motion vector candidates of the coding blocks are not known. For CU group level parallelism, all the candidate motion vectors of the Prediction Units (PU) can be constructed in parallel within the parallel region PMER. However, when the parallelism is performed at the CU level, candidate motion vectors of prediction units inside the CU share valid candidate motion vectors outside the CU. Meanwhile, the coding parallelism is improved, the coding efficiency is also required to be balanced, and the overlarge bit rate loss is avoided. Based on the above requirement, as shown in fig. 1, the present invention provides an inter-frame image parallel coding method based on time-space domain prediction, which is used for parallel coding of coding blocks at CU level, and mainly includes the following steps:
s1: setting a spatial domain reliability threshold (set as TH 1) and a temporal domain reliability threshold (set as TH 2) according to data compression requirements, and acquiring a motion vector candidate list of a current frame image coding block group, wherein the motion vector candidate list comprises a spatial domain candidate list and a temporal domain candidate list;
s2: extracting spatial domain motion vectors of the coding block group in the spatial domain candidate list on the directional corner in the motion direction and two corners adjacent to the directional corner according to the motion direction of the current frame image;
s3: obtaining a spatial domain reliability parameter based on depth scaling according to the depth of the current coding block and the difference value in the horizontal direction between the extracted spatial domain motion vectors;
s4: extracting time domain motion vectors at the corner-to-corner position in the direction of the current coding block in the current frame and previous frame time domain candidate lists according to the time domain motion vectors at the corner-to-corner position in the direction of the coding block group;
s5: obtaining time domain reliability parameters based on depth scaling according to the depth of the current coding block and the difference value between the horizontal component and the vertical component of the extracted time domain motion vector;
s6: judging whether the airspace reliability parameter is smaller than a airspace reliability threshold value or not and whether the time domain reliability parameter is smaller than a time domain reliability threshold value or not, if so, entering the next step, if not, entering a coding block at the next depth before the depth of the current coding block is larger than a preset threshold value, and returning to the step S2;
s7: judging whether the space domain reliability parameter is larger than or equal to a time domain reliability parameter with a preset proportion, if so, sharing the known time domain motion vector of the current coding block by all the coding blocks in the parallel region of the current coding block, and if not, sharing the known space domain motion vector of the current coding block by all the coding blocks in the parallel region of the current coding block;
s8: and carrying out parallel coding on the coding blocks in the coding block group according to the motion vector set shared by the coding blocks.
It should be noted that the motion vector candidate list in step S1 is specifically a prediction unit applied to the current coding block, and the prediction unit is used for inter-image coding according to the motion vector candidate list.
Further, in the step S6, if it is determined that the current coding block depth is greater than the preset threshold, the process proceeds to step S8.
It is first to be understood that the AMVP technique in HEVC uses the correlation of motion vectors in the temporal-spatial domain to build a candidate list of motion vectors for the current coding block prediction unit. The candidate list mainly comprises two types of candidate motion vectors: spatial domain candidates and temporal domain candidates. As shown in fig. 2, the motion vectors in the candidate list are respectively: a spatial candidate list { a0, a1, B0, B1, B2}, a temporal candidate list { C0, C1}, where B2 is a spatial motion vector at a directional corner in the direction of motion, a0, a1, B0, B1 are spatial motion vectors at two corners adjacent to the directional corner, C1 is a temporal motion vector at a corner diagonal to the directional corner, and C0 is a temporal motion vector at the center point of the coding block.
Based on the theoretical basis of the prior art, it is further understood that the smoother the image texture is, the more the image texture is prone to be coded in parallel by using large blocks of coding blocks. Meanwhile, because the texture smooth image motion has a consistency relationship, the coding block parallel processing aiming at the smooth area can not only accelerate the coding, but also reduce the loss of the bit rate. In contrast, when the texture of an image has more detail, encoding tends to be performed with small blocks of encoded blocks because the motion vectors at this time have no consistency. Therefore, when the image texture is smooth, the motion vector outside the parallel region is very useful for predicting the motion vector candidates in the parallel region, and the motion vector candidates of all the encoding blocks in the parallel region can be replaced by the motion vector outside the parallel region to realize the parallel generation of the predicted motion vector.
Different from the prior art, in the motion vector candidate list established by the invention, the spatial domain candidate list comprises { A0, A1, scaled A0, scaled A1, B0, B1, B2, scaled B0, scaled B1 and scaled B2}, wherein scaled represents the scaling of the corresponding motion vector based on the depth of the coding block for the subsequent calculation of the spatial domain reliability parameter; b2 represents the spatial motion vector of the directional corner, a0 and a1 represent the spatial motion vector of the corner adjacent to the directional corner, B0 and B1 represent the spatial motion vector of the other corner adjacent to the directional corner (further defined herein, B0 and B1 represent the spatial motion vectors of the adjacent corners in the horizontal direction of B2, and a0 and a1 represent the spatial motion vectors of the adjacent corners in the vertical direction of B2. it should be noted that, here, they are only temporarily defined, and in practical application, they can be exchanged, and only need to be exchanged correspondingly according to the spatial orientation in the subsequent calculation). The establishment of the time domain candidate list in the motion vector candidate list is the time domain motion vector at the corner diagonal of the direction of the current coding block and the center point of the coding block in the extracted current frame and the previous frame time domain candidate list.
Specifically, the coding block selected by the present invention is a matrix with a 2N × 2N (64 × 64 as an example) structure, and its spatial domain candidate list is { a0, a1, B0, B1, B2}, and mvx is definedA0、mvxA1、mvxB0、mvxB1、mvxB2Respectively corresponding horizontal components of the spatial motion vector after scaling, the invention introduces a new parameter RSThe reliability of these spatial motion vectors is expressed, and the following formula is defined:
wherein r is1Is the sum of the absolute values of the differences of the scaled A0 and A1 horizontal components and B2 horizontal components, r2Is the sum of the absolute values of the differences of the scaled B0 and B1 horizontal components and B2 horizontal components, respectively; rSIs a spatial domain reliability parameter. r is1And r2The definition formula of (1) is as follows:
in the formula, mvx is a horizontal component of the scaled corresponding motion vector.
In the non-scaled state (i.e. in the initial depth state, the initial depth is set to 0), the sum r of the differences between B2 and B0 and B1 respectively is obtained by comparing the coding block groups in the horizontal direction1The consistency of the motion trend of the coding block group in the horizontal direction is obtained by comparing the sum r of the difference values of B2 and A0 and A1 in the vertical direction of the coding block group2The consistency of the motion trend of the coding block group in the vertical direction is obtained, the two are added and compared with the spatial domain reliability threshold value, and the consistency of the motion trend of the current coding block group in the spatial domain, namely the smoothness of the spatial domain, is judged. When smoothness is low (R)SGreater than or equal to TH 1), i.e., more texture, the next depth (depth value +1, until the depth reaches 3,the coding block size corresponding to the depth 3 is 8 multiplied by 8), and the corresponding motion vector is scaled and calculated to obtain the spatial domain reliability parameter of the coding block under the depth. When the space smoothness reaches the preset requirement (R)SLess than TH 1), the Prediction Units (PUs) of all the coding blocks in the parallel region of the current coding block can share the known spatial motion vector of the current coding block.
And all the coding blocks in the parallel region are composed of all the sub-coding blocks which are split when the current coding block enters the next depth.
It should be noted that, the next depth coding block in the present invention is a quarter of the current depth coding block, the depth of the coding block includes 0 to 3, and scaling the extracted spatial domain motion vector and temporal domain motion vector according to the depth of the current coding block is based on depth scaling, where the corresponding scaling specifically is:
when the depth is 0, the scaling factor is one, when the depth is 1, the scaling factor is one half, when the depth is 2, the scaling factor is one quarter, and when the depth is 3, the scaling factor is one eighth.
Further, for most video continuous frame images, the image itself is not only smooth, but the image motion level is consistent. Therefore, there is close continuity between the temporal motion vector of the current frame coding block and the motion information of the co-located (co-located) reference coding block of the previous frame. The time domain motion vectors at the opposite angles of the direction corners of the current frame and the previous frame of the coding block at the same position are respectively set as t and t-1, when the image motion levels are consistent, the time domain motion vectors can be shared by the Prediction Units (PU) of the coding blocks in the parallel area by utilizing the consistency of the motion vectors, and the parallel processing is conveniently realized during motion estimation.
According to this, the invention introduces a temporal reliability parameter R for a coding block (size 64 × 64) in a parallel regionTTo express the reliability of the temporal candidate motion vector, the formula is defined as follows:
wherein t is the time domain motion vector at the corner opposite to the current coding block direction in the current frame time domain candidate list, t-1 is the time domain motion vector at the corner opposite to the current coding block direction in the previous frame time domain candidate list, mvy is the vertical component of the scaled corresponding motion vector, RTIs a time domain reliability parameter.
Specifically, the consistency judgment of the time domain motion vectors of the previous and next frames is used for acquiring the time domain reliability parameter, the lower the parameter is, the higher the motion consistency of the previous and next frame images is, and the time domain reliability parameter is lower than a time domain reliability threshold (R)TLess than TH 2), then the current coding block shares its time domain motion vector in the time domain to the coding block in the parallel region. Similarly, if the temporal reliability parameter is greater than the temporal reliability threshold (R)TGreater than or equal to TH 2), the next depth is entered, and the corresponding motion vector is scaled and calculated to obtain the temporal reliability parameter of the coding block at the depth.
It should be noted that if the time domain motion vector at the scaling position corresponding to the time domain motion vector at the diagonal position of the direction corner does not exist, the time domain motion vector corresponding to the scaling position at the central point of the coding block may be used instead.
From the above analysis, it can be seen that motion information in both the time domain and the spatial domain is important to achieve parallel processing at the coding block level. Because the texture content of the image is rich, some images are closely related to adjacent coding blocks in a spatial domain, and some images are closely related to a reference image in a temporal domain. Therefore, in order to avoid the reduction of the coding efficiency, more conditional restrictions need to be added to the parallel condition of the spatial and temporal candidate motion vectors. Based on the above, the present invention further provides a parallel generation method of candidate motion vectors based on the time-space domain, which is used to balance the efficiency of good coding and the parallelism generated by the candidate motion vectors (the space-domain candidate motion vector and the time-domain candidate motion vector).
The parallelism of candidate motion vector generation is represented by SP, and the formula is defined as follows:
wherein
（
) The preset proportion in the method step is a space domain, which is also a weight value of the time domain reliability parameter, based on TH1 and TH2, which are the space domain reliability threshold and the time domain reliability parameter threshold,
the definition formula of (1) is as follows:
the invention herein described is achieved by
And eliminating the proportion difference between the spatial domain reliability threshold and the time domain reliability threshold at the beginning of setting. It is apparent that when SP is 0 or more, R isSIs greater than or equal to
The parallel space is in the time domain; when SP is less than 0, RSIs less than
The parallel space is in space domain. The motion vector selection for sharing after comparison can be expressed as the following formula:
in the formula, MVnSet of motion vectors, MV, known for any block in the parallel region of the current blockmIs thatSet of motion vectors, MV, after sharing of arbitrary coding blocksTSet of temporal motion vectors, MV, known for the current coding blockSA set of spatial motion vectors known to the current coding block.
The invention reduces the texture complexity of the image to be processed in the process of parallel coding by the descending of the depth of the coding block for the low-depth coding block which can not be coded in parallel, thereby realizing the parallel coding of the coding block while ensuring the low bit rate and the low loss.
For setting the spatial domain reliability threshold and the time domain reliability threshold, the invention discovers that the reliability parameters obey the standard normal distribution by carrying out simulation experiments on test sequences with different resolutions. A number of simulations were performed on reference sequences of different resolution and different characteristics to select the appropriate thresholds TH1 and TH 2. The difference in coding efficiency is small as TH1 approaches 21, and the variation in coding efficiency is small as TH2 approaches 8. However, as the values of TH1 and TH2 become larger, the coding efficiency also increases. Therefore, in the present invention, the threshold values TH1=21 and TH2=8 are selected, which are good for balancing good parallelism and coding efficiency.
In summary, the interframe image parallel coding method based on time-space domain prediction according to the present invention substitutes the motion vectors of the corners corresponding to the coding blocks in the coding group by using the spatial domain motion vectors on the corners in the direction of the coding block group and the adjacent corners as the substitute quantity, and further judges the consistency of the horizontal and vertical motion trends of the current coding block based on the depth reduction, so that the spatial domain motion vector of the coding block at the depth with higher reliability (smoother) can be selected as the shared parameter basis of the parallel coding; and taking the time domain motion vector at the position corresponding to the current coding block in the time domain candidate list as a substitute of the time domain motion vector of the coding block, and further judging the consistency of the time domain motion trend of the current coding block based on depth reduction, so that the time domain motion vector of the coding block at the depth with higher reliability can be selected as a shared parameter basis of parallel coding.
The motion vector of the smooth (proper depth) coding block is selected by comparing the reliability parameters for parallel coding, and the loss of the bit rate is reduced while the parallel coding is realized. By comparing the proportional relation between the spatial domain reliability parameters and the time domain reliability parameters, the motion vectors with better reliability are selected between the spatial domain reliability parameters and the time domain reliability parameters to serve as the shared parameter basis of the parallel coding, and the effectiveness of the parallel coding is improved.
Example two
In order to better prove the effectiveness of the present invention, this embodiment demonstrates the technical effect of the present invention by exemplifying a specific simulation experiment, and verifies the performance of the algorithm by comparing the rate distortion and the computational complexity of the algorithm proposed by the present invention and the h.265/HEVC reference software, the experiment test uses a standard h.265/HEVC video sequence, which has different resolutions and image texture characteristics, and the configuration of the simulated experiment environment is shown in table 1.
Table 1: simulation experiment environment
In order to evaluate the performance of the algorithm provided by the invention, the evaluation standard of the performance of the algorithm adopts BDBR and BDPSNR, and the average coding time TS calculation method is as follows:
whereinT HM (QP i ) AndT pro (QP i ) The coding time of reference software and the algorithm coding time provided in this chapter are respectively under different quantization parameters QP.
The experimental results of the parallel improved algorithm based on the time-space domain prediction provided by the embodiment of the invention are shown in table 2. When the parallel region is on the space region, the coding blocks in the parallel region share the space candidate motion vector set of the current coding block. When the parallel region is over a temporal region, temporal candidates within the parallel region are replaced with temporal motion vectors of the coding blocks at their corresponding positions. The algorithm takes the image characteristics in space and time into consideration, and compared with the prior algorithm based on space prediction and time prediction, the algorithm based on time prediction has higher coding efficiency.
Table 2: parallel improved algorithm experimental result based on time-space domain prediction
Compared with the prior method in performance, the comparison method selected by the embodiment is a parallel algorithm based on a directed graph model, which is proposed by Yan and the like, and the method has good representativeness. In terms of performance, the BDBR of the joint time-space domain prediction algorithm proposed herein is only 0.01% higher. Whereas in the Yan method, MER is 8 × 8 and 16 × 16, respectively, and BDBR increase is 0.1% and 0.5%, respectively, the joint time-space domain prediction algorithm proposed herein suffers little BDBR loss.
It should be noted that all the directional indicators (such as up, down, left, right, front, and rear … …) in the embodiment of the present invention are only used to explain the relative position relationship between the components, the movement situation, etc. in a specific posture (as shown in the drawing), and if the specific posture is changed, the directional indicator is changed accordingly.
Moreover, descriptions of the present invention as relating to "first," "second," "a," etc. are for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicit ly indicating a number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, "a plurality" means at least two, e.g., two, three, etc., unless specifically limited otherwise.
In the present invention, unless otherwise expressly stated or limited, the terms "connected," "secured," and the like are to be construed broadly, and for example, "secured" may be a fixed connection, a removable connection, or an integral part; can be mechanically or electrically connected; they may be directly connected or indirectly connected through intervening media, or they may be connected internally or in any other suitable relationship, unless expressly stated otherwise. The specific meanings of the above terms in the present invention can be understood by those skilled in the art according to specific situations.
In addition, the technical solutions in the embodiments of the present invention may be combined with each other, but it must be based on the realization of those skilled in the art, and when the technical solutions are contradictory or cannot be realized, such a combination of technical solutions should not be considered to exist, and is not within the protection scope of the present invention.
Claims (6)
1. An interframe image parallel coding method based on time-space domain prediction is characterized in that the parallel coding is the parallel coding of each coding block in a coding block group, each coding block contains a part of known motion vectors and a part of unknown motion vectors, and the parallel coding mainly comprises the following steps:
s1: setting a spatial domain reliability threshold and a temporal domain reliability threshold according to data compression requirements, and acquiring a motion vector candidate list of a current frame image coding block group, wherein the motion vector candidate list comprises a spatial domain candidate list and a temporal domain candidate list;
s2: extracting spatial domain motion vectors of the coding block group in the spatial domain candidate list on the directional corner in the motion direction and two corners adjacent to the directional corner according to the motion direction of the current frame image;
s3: obtaining a spatial domain reliability parameter based on depth scaling according to the depth of the current coding block and the difference value in the horizontal direction between the extracted spatial domain motion vectors;
s4: extracting time domain motion vectors at the corner-to-corner position in the direction of the current coding block in the current frame and previous frame time domain candidate lists according to the time domain motion vectors at the corner-to-corner position in the direction of the coding block group;
s5: obtaining time domain reliability parameters based on depth scaling according to the depth of the current coding block and the difference value between the horizontal component and the vertical component of the extracted time domain motion vector;
s6: judging whether the airspace reliability parameter is smaller than a airspace reliability threshold value or not and whether the time domain reliability parameter is smaller than a time domain reliability threshold value or not, if so, entering the next step, if not, entering a coding block at the next depth before the depth of the current coding block is larger than a preset threshold value, and returning to the step S2;
s7: judging whether the space domain reliability parameter is larger than or equal to a time domain reliability parameter with a preset proportion, if so, sharing the known time domain motion vector of the current coding block by all the coding blocks in the parallel region of the current coding block, and if not, sharing the known space domain motion vector of the current coding block by all the coding blocks in the parallel region of the current coding block;
s8: according to the motion vector set shared by the coding blocks, parallel coding of the coding blocks in the coding block group is carried out;
the depth of the coding block comprises 0 to 3, and the extracted space domain motion vector and the time domain motion vector are scaled according to the depth of the current coding block based on the depth scaling, wherein the corresponding proportion specifically comprises the following steps:
when the depth is 0, the scaling factor is one, when the depth is 1, the scaling factor is one half, when the depth is 2, the scaling factor is one quarter, and when the depth is 3, the scaling factor is one eighth;
the step S3 can be expressed by the following formula:
wherein B2 represents the airspace motion vector at the directional corner, A0 and A1 represent the corner adjacent to the directional cornerB0 and B1 are spatial motion vectors on another corner adjacent to the direction corner; mvx is the horizontal component of the scaled corresponding motion vector; r is1Is the sum of the absolute values of the differences of the scaled A0 and A1 horizontal components and B2 horizontal components, r2Is the sum of the absolute values of the differences of the scaled B0 and B1 horizontal components and B2 horizontal components, respectively; rSIs a spatial domain reliability parameter;
the step S5 can be expressed by the following formula:
wherein t is the time domain motion vector at the corner opposite to the current coding block direction in the current frame time domain candidate list, t-1 is the time domain motion vector at the corner opposite to the current coding block direction in the previous frame time domain candidate list, mvy is the vertical component of the scaled corresponding motion vector, RTIs a time domain reliability parameter.
2. The method as claimed in claim 1, wherein in step S6, if it is determined that the current coding block depth is greater than the predetermined threshold, the method proceeds to step S8.
3. The method of claim 1, wherein the step S7 is expressed by the following formula:
in the formula, MVnMotion vector known to any coding block in parallel region of current coding blockCollection, MVmSet of motion vectors, MV, shared for the arbitrary coding blockTSet of temporal motion vectors, MV, known for the current coding blockSA set of spatial motion vectors known to the current coding block,
at a predetermined ratio, TH1 is a spatial domain reliability threshold, and TH2 is a temporal domain reliability threshold.
4. The method as claimed in claim 1, wherein the coding block set has a structure of a 2N x 2N matrix.
5. The method of claim 4, wherein the next depth coding block is a quarter of the current depth coding block.
6. The method as claimed in claim 1, wherein in step S4, when the temporal motion vector at the corner of the direction does not exist, the temporal motion vector at the center of the coding block is used instead.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111053623.1A CN113507609B (en) | 2021-09-09 | 2021-09-09 | Interframe image parallel coding method based on time-space domain prediction |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111053623.1A CN113507609B (en) | 2021-09-09 | 2021-09-09 | Interframe image parallel coding method based on time-space domain prediction |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113507609A CN113507609A (en) | 2021-10-15 |
| CN113507609B true CN113507609B (en) | 2021-11-19 |
Family
ID=78016827
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111053623.1A Active CN113507609B (en) | 2021-09-09 | 2021-09-09 | Interframe image parallel coding method based on time-space domain prediction |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113507609B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115297333B (en) * | 2022-09-29 | 2023-03-24 | 北京达佳互联信息技术有限公司 | Inter-frame prediction method and device of video data, electronic equipment and storage medium |
| CN116456100B (en) * | 2023-06-16 | 2023-08-29 | 深流微智能科技(深圳)有限公司 | Inter-frame coding tree unit division method and device |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101917615A (en) * | 2010-06-03 | 2010-12-15 | 北京邮电大学 | An Enhanced Bidirectional Motion Vector Prediction Method in a Hybrid Video Coding Framework |
| US20150271516A1 (en) * | 2014-03-18 | 2015-09-24 | Panasonic Intellectual Property Management Co., Ltd. | Video coding apparatus and video coding method |
| CN110545424A (en) * | 2019-08-21 | 2019-12-06 | 浙江大华技术股份有限公司 | Inter-frame prediction method based on MMVD (multimedia MediaVision video) mode, video coding method, related device and equipment |
| CN112995669A (en) * | 2021-04-27 | 2021-06-18 | 浙江华创视讯科技有限公司 | Inter-frame prediction method and device, electronic equipment and storage medium |
-
2021
- 2021-09-09 CN CN202111053623.1A patent/CN113507609B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101917615A (en) * | 2010-06-03 | 2010-12-15 | 北京邮电大学 | An Enhanced Bidirectional Motion Vector Prediction Method in a Hybrid Video Coding Framework |
| US20150271516A1 (en) * | 2014-03-18 | 2015-09-24 | Panasonic Intellectual Property Management Co., Ltd. | Video coding apparatus and video coding method |
| CN110545424A (en) * | 2019-08-21 | 2019-12-06 | 浙江大华技术股份有限公司 | Inter-frame prediction method based on MMVD (multimedia MediaVision video) mode, video coding method, related device and equipment |
| CN112995669A (en) * | 2021-04-27 | 2021-06-18 | 浙江华创视讯科技有限公司 | Inter-frame prediction method and device, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113507609A (en) | 2021-10-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP7358436B2 (en) | Motion vector refinement for multi-reference prediction | |
| TWI432033B (en) | Apparatus for decoding motion vector | |
| CN110519600B (en) | Intra-frame and inter-frame joint prediction method and device, coder and decoder and storage device | |
| CN103079070B (en) | Video encoder | |
| CN103222265B (en) | Dynamic image encoding device, dynamic image decoding device, dynamic image encoding method, and dynamic image decoding method | |
| CN102833540B (en) | motion prediction method | |
| US10148947B2 (en) | Method and device for determining parameters for encoding or decoding of an image of a video sequence | |
| CN102598670B (en) | With reference to multiple frame, image is carried out to the method and apparatus of coding/decoding | |
| CN110312130B (en) | Inter-frame prediction and video coding method and device based on triangular mode | |
| CN115002481B (en) | Method and apparatus for encoding or decoding video data | |
| US11849142B2 (en) | Method and apparatus for encoding or decoding video data with sub-pixel motion vector refinement | |
| US20080002772A1 (en) | Motion vector estimation method | |
| CN113507609B (en) | Interframe image parallel coding method based on time-space domain prediction | |
| CN102131094A (en) | motion prediction method | |
| TW202007154A (en) | Improvement on inter-layer prediction | |
| US20170257631A1 (en) | Block size determining method and program recording medium | |
| US20150036751A1 (en) | Method for storing movement prediction-related information in an interscreen prediction method, and method for calculating the movement prediction-related information in the inter-screen prediction method | |
| CN112449180A (en) | Encoding and decoding method, device and equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |