CN113301347A - Optimization method of HEVC high-definition video coding - Google Patents
Optimization method of HEVC high-definition video coding Download PDFInfo
- Publication number
- CN113301347A CN113301347A CN202110501572.8A CN202110501572A CN113301347A CN 113301347 A CN113301347 A CN 113301347A CN 202110501572 A CN202110501572 A CN 202110501572A CN 113301347 A CN113301347 A CN 113301347A
- Authority
- CN
- China
- Prior art keywords
- coding unit
- block
- image
- prediction
- current frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 51
- 238000005457 optimization Methods 0.000 title claims abstract description 17
- 238000013139 quantization Methods 0.000 claims abstract description 5
- 239000011159 matrix material Substances 0.000 claims description 77
- 230000009466 transformation Effects 0.000 claims description 42
- 238000013507 mapping Methods 0.000 claims description 10
- 238000005070 sampling Methods 0.000 claims description 6
- 239000013598 vector Substances 0.000 claims description 5
- 230000002123 temporal effect Effects 0.000 claims description 4
- 230000005540 biological transmission Effects 0.000 abstract description 4
- 230000008859 change Effects 0.000 abstract description 2
- 238000005192 partition Methods 0.000 description 14
- 238000011426 transformation method Methods 0.000 description 7
- 238000000354 decomposition reaction Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 5
- 230000006835 compression Effects 0.000 description 4
- 238000007906 compression Methods 0.000 description 4
- 230000003321 amplification Effects 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 3
- 230000007547 defect Effects 0.000 description 3
- 238000003199 nucleic acid amplification method Methods 0.000 description 3
- 229920002635 polyurethane Polymers 0.000 description 3
- 238000004422 calculation algorithm Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 239000004814 polyurethane Substances 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 230000001131 transforming effect Effects 0.000 description 2
- 208000037170 Delayed Emergence from Anesthesia Diseases 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000000903 blocking effect Effects 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 230000002441 reversible effect Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/46—Embedding additional information in the video signal during the compression process
- H04N19/463—Embedding additional information in the video signal during the compression process by compressing encoding parameters before transmission
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/124—Quantisation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/157—Assigned coding mode, i.e. the coding mode being predefined or preselected to be further used for selection of another element or parameter
- H04N19/159—Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
- H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
- H04N19/625—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding using discrete cosine transform [DCT]
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- Discrete Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
一种HEVC高清视频编码的优化方法，包括：原始视频图像划分为多个互相独立的编码单元，即原始图像块；进行预测编码，包括根据视频图像时间和空间冗余，分别采用帧内预测和帧间预测的方式去除冗余信息并获得预测图像块；基于预测块、原始图像块的图像值的差值获得预测残差，对预测残差进行离散余弦变换和量化；将量化的离散余弦变换系数进行熵编码得到视频序列的压缩码流，输出压缩码流；对压缩码流进行解码后得到高分辨率视频序列，对高分辨率视频序列进行图像插值以使高分辨视频序列的图像大小还原至原始图像大小。实现在视频质量变化不大的情况下，得到编码效率更高、性能更好、计算复杂度更低的编码方案，更有利于应用到今后的视频传输和存储。
An optimization method for HEVC high-definition video coding, comprising: dividing an original video image into a plurality of mutually independent coding units, namely original image blocks; performing predictive coding, including using intra-frame prediction and The method of inter-frame prediction removes redundant information and obtains a predicted image block; obtains a prediction residual based on the difference between the image values of the predicted block and the original image block, and performs discrete cosine transform and quantization on the prediction residual; the quantized discrete cosine transform Entropy coding the coefficients to obtain the compressed code stream of the video sequence, and output the compressed code stream; after decoding the compressed code stream, a high-resolution video sequence is obtained, and image interpolation is performed on the high-resolution video sequence to restore the image size of the high-resolution video sequence. to the original image size. In the case where the video quality does not change much, a coding scheme with higher coding efficiency, better performance and lower computational complexity can be obtained, which is more conducive to application to future video transmission and storage.
Description
Technical Field
The invention relates to the technical field of video coding, in particular to an optimization method of HEVC high-definition video coding.
Background
High Efficiency Video Coding (HEVC), also known as h.265, can achieve twice as much compression by the blu-ray best video compression method; however, the existing HEVC coding technology has many defects, such as the following defects:
firstly, the complexity of HEVC coding is much higher than other video coding standards, so the performance is not good under the condition of low code rate, and the decoded video has more serious distortion;
secondly, although the compression method based on image downsampling greatly reduces the calculated amount of coding and relieves the pressure on wireless network transmission, the method can be smoother when interpolating a flat area in an image, but the interpolation effect on detailed parts such as the edge and texture of the image is general, and serious fuzzy and sawtooth effects exist, so that the visual effect of the reconstructed image is poor;
third, the transform method adopted in HEVC is implemented by performing two basic one-dimensional transforms in the horizontal and vertical directions separately in the actual coding. However, for two-dimensional images where the dominant direction of the internal texture is not horizontal or vertical, conventional two-dimensional transforms do not compress their energy best. Therefore, neither the DCT nor the DST transform used in HEVC can take into account the detail content of the image very well.
Disclosure of Invention
The invention aims to provide an optimization method of HEVC high-definition video coding aiming at the defects in the background art, so that a coding scheme with higher coding efficiency, better performance and lower computational complexity is obtained under the condition of small video quality change, and the method is more favorable for video transmission and storage in the future.
In order to achieve the purpose, the invention adopts the following technical scheme:
an optimization method for HEVC high definition video coding comprises the following steps:
step A: inputting an original video sequence at an encoding end and performing downsampling to obtain a degraded low-resolution video sequence;
and B: an HEVC encoder encodes a low resolution video sequence, comprising:
each frame of input original video image is divided into a plurality of mutually independent coding units, namely original image blocks;
performing predictive coding, including removing redundant information and obtaining a predictive image block by adopting intra-frame prediction and inter-frame prediction modes respectively according to the time redundancy and the spatial redundancy of the video image;
obtaining a prediction residual based on the difference value of the image values of the prediction block and the original image block, and performing discrete cosine transform and quantization on the prediction residual;
entropy coding is carried out on the quantized discrete cosine transform coefficient to obtain a compressed code stream of the video sequence, and the compressed code stream is output;
and C: and the decoder decodes the compressed code stream to obtain a high-resolution video sequence, and performs image interpolation on the high-resolution video sequence to restore the image size of the high-resolution video sequence to the original image size.
Preferably, in the step B, selecting a PU prediction mode with a minimum rate-distortion cost value at a maximum probability of a current frame coding unit according to a size relationship between the current frame coding unit and a corresponding frame coding unit, specifically includes:
when the side length of the current frame coding unit is smaller than the side length of the corresponding frame coding unit, the method comprises the following steps:
when the side length of the current frame coding unit is half of the side length of the corresponding frame coding unit, if the PU prediction mode of the partition of the corresponding frame coding unit is nL multiplied by 2N, the PU prediction mode of the partition of the current frame coding unit is selected to be Nmultiplied by 2N; if the PU prediction mode of the partition corresponding to the frame coding unit is 2 NxnU; selecting the PU prediction mode of the block of the current frame coding unit as 2 NxN;
when the side length of the current frame coding unit is one fourth of the side length of the corresponding frame coding unit, selecting a PU (polyurethane) prediction mode of a block of the current frame coding unit to be 2 Nx 2N;
when the side length of the current frame coding unit is less than one fourth of the side length of the corresponding frame coding unit, the partition of the current frame coding unit does not select the PU prediction mode.
Preferably, when the side length of the current frame coding unit is greater than the side length of the corresponding frame coding unit, the judgment is carried out according to the size of each block of the corresponding frame coding unit and the PU prediction mode, and the PU prediction module is selected according to the distribution condition of the blocks of the corresponding frame coding unit and the prediction units thereof;
and when the side length of the current frame coding unit is equal to the side length of the corresponding frame coding unit, the coding unit of the current frame selects the PU prediction mode which is the same as the partition of the corresponding frame coding unit.
Preferably, in the step B, the discrete cosine transforming and quantizing the prediction residual includes:
the method comprises the steps of establishing intra-frame prediction models in different modes according to the HEVC standard, obtaining an initial pixel covariance matrix according to a direction-based rotation ellipse model, updating a residual pixel expression according to the initial pixel covariance matrix to obtain a residual pixel covariance matrix, decomposing and adjusting the residual pixel covariance matrix through KLT, and extracting a transformation matrix.
Preferably, the obtaining the initial pixel covariance matrix according to the direction-based ellipse model comprises:
a rotation ellipse model based on the image texture direction is established,
obtaining an offset angle corresponding to a PU prediction mode of a current frame coding unit, namely an image texture direction corresponding to the rotation ellipse model;
obtaining the correlation between pixel points A (a, B) and B (c, d) in a current frame coding unit, wherein the correlation is as follows:
wherein:
r represents the ratio of the major and minor axes;
theta represents the direction of texture in the current frame coding unit, namely an angle value;
ρ represents the correlation strength between pixels;
d1(θ)，d2(theta) represents the projection coordinates of the pixel points A and B in the rotating ellipse model;
the mapping relation between the projection coordinates and the real coordinates A (a, B) and B (c, d) is as follows:
and sleeving the pixel value of the current frame coding unit and the reference pixel value of the boundary of the adjacent coding unit into the rotation ellipse model, acquiring the correlation between the current frame coding unit and the adjacent coding unit thereof, and acquiring an initial pixel covariance matrix.
Preferably, the establishing of the intra prediction model includes:
and mapping the reference pixel: mapping all reference pixels required by a current frame coding unit into a row or a column;
obtaining a predicted pixel value P according toi，j：
Pi，j＝((32-ω)×Mon0，pla+ω×Mon0，pla+1+16)；
Wherein:
omega represents the weight of interpolation operation;
Mon0，plaand Mon0，pla+1Representing a reference pixel mapped by the current prediction pixel;
pla represents the corresponding reference pixel location;
offset [ P ] represents the offset corresponding to the current mode;
(x, y) represents the coordinates of the current predicted pixel.
Preferably, obtaining the residual pixel covariance matrix comprises:
for a residual pixel block with a size of N × N and selected intra-frame prediction mode in intra-frame coding, transposing the residual pixel block into a one-dimensional vector form
With a covariance matrix of R (size N)2×N2)：
Wherein each element Ra，bComprises the following steps:
Rb，a＝Ra，b＝E{ea，eb}，a，b＝1，2，…，N2；
extracting the transformation matrix includes:
solving an initial KLT transformation matrix based on the residual pixel covariance matrix, and adjusting the amplification factor and the scanning sequence of an integer matrix in transformation coding of the initial KLT transformation matrix;
for coding units whose scanning order is horizontal scanning, the magnification and scanning order of the integer matrix of transform coding are not adjusted;
for coding units with vertical scanning sequence, arranging the characteristic vectors in the initial KLT transformation matrix according to the sequence consistent with the energy arrangement in the HEVC standard;
for coding units with a scanning sequence of diagonal scanning, arranging the initial KLT transformation matrixes according to the same energy sequence in the HEVC standard;
preferably, in step C, the method includes interpolating the low-resolution video image to be restored by using the intra-frame redundant similarity structure to obtain the high-resolution prediction image block, and specifically includes:
expanding a search area by utilizing similarity, and increasing the number of image blocks which can be referred to, the method comprises the following specific steps:
selecting an original image block Pi to be interpolated in a video image of a current frame as a target center, and establishing a window with radius r in the current video image;
determining a search frame with the same position and size as a window in each L frame of video images before and after the video image of the current frame, and using the search frame as a search area of a similar neighbor block of an original image block Pi;
respectively putting the image blocks in the search area into a down-sampling grid, and sequentially calculating the similarity between the image blocks and the original image blocks Pi;
selecting the most similar N image blocks to fit the original image blocks Pi to obtain image blocks similar to the original image blocks Pi, and marking the similar image blocks respectively;
down-sampling the marked image blocks again and splicing the marked image blocks into high-resolution prediction image blocks;
and continuously and iteratively updating the prediction image block to ensure that the approximation degree of the prediction image block and the original image block to be interpolated reaches the highest.
Preferably, the determining the similarity between the image block of the search area and the original image block includes:
acquiring a difference value of the alignment gray value of an original image block to be interpolated and each image block in a search area;
accumulating and summing the absolute values of the difference values;
and selecting the image blocks corresponding to the first K minimum numerical values to linearly represent the original image blocks to be interpolated, wherein the image blocks corresponding to the first K minimum numerical values have the highest similarity with the original image blocks to be interpolated.
Preferably, in the step C, the method further includes reducing the radius of the search box and reducing the number of iterations, so as to improve the similarity between the prediction image block and the original image block to be interpolated.
Compared with the prior art, the invention has the following beneficial effects:
firstly, in the aspect of intra-frame prediction, the video image is firstly downsampled by utilizing the similarity prior information between the images, then compression coding is carried out, and image interpolation is carried out on the video data after decoding so as to restore the video data to the original resolution. The method expands the search area of the image block to be interpolated in the current video frame to the adjacent multi-frame video image, expands the search area, increases the number of the image blocks which can be referred to, has stronger fitting capability and clearer synthetic image; meanwhile, the data size of the code can be greatly reduced, and higher video quality can be output under the condition of low code rate;
secondly, in the aspect of inter-frame prediction, the temporal correlation between the CU partition mode and the PU prediction mode between adjacent frames is utilized to skip some CU partition modes and PU prediction modes of the current block in a targeted manner, and then the difference value is calculated and transmitted. This has the advantage of being simpler and more efficient than all previous methods for computing retransmissions, thereby reducing the coding computational complexity of HEVC.
Thirdly, in the aspect of transformation, a rotation ellipse model considering the directional information of the image is established, and a series of improved transformation methods are designed based on the principle of KLT transformation. Then, different transformation methods with fixed forms are respectively extracted according to different intra-frame prediction modes, and the different transformation methods are correspondingly put into the HEVC standard to replace the original transformation and then applied.
Drawings
Fig. 1 is an HEVC encoding flow diagram of one embodiment of this disclosure;
FIG. 2 is a diagram illustrating the prediction modes of nL × 2N PU according to an embodiment of the present invention;
FIG. 3 is a diagram illustrating a PU prediction mode of 2 NxnU according to an embodiment of the present invention;
FIG. 4 is a diagram of PU prediction modes in the prior art;
FIG. 5 is a flow diagram of an improved transformation module of one embodiment of the present invention.
Detailed Description
Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the accompanying drawings are illustrative only for the purpose of illustrating the present invention and are not to be construed as limiting the present invention;
any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps of the process, and alternate implementations are included within the scope of the preferred embodiment of the present invention in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the present invention.
The technical scheme of the invention is further explained by the specific implementation mode in combination with the attached drawings.
The invention provides an optimization method of HEVC high-definition video coding, which comprises the steps of adopting a video image interpolation scheme based on self-similarity, an inter-frame mode quick selection mode based on time domain correlation and a high-definition video coding scheme based on a transformation method of a rotation ellipse model and KLT (Kelvin transform);
the method comprises the steps of firstly carrying out down-sampling processing on an original video sequence at a coding end so as to obtain a reduced low-resolution small video, and then coding the small video by using an HEVC coder. Secondly, a method for quickly selecting an inter-frame mode based on time domain correlation is established, and a prediction mode is quickly determined according to the side length of a coding unit of a current frame. Finally, a rotation ellipse model considering the directional information of the image is established, transformation methods under different prediction modes are designed based on the KLT transformation principle, the transformation methods are correspondingly put into the HEVC standard to replace the original transformation and then tested, the transmission is carried out after the coding is finished, and finally, a decoder carries out image interpolation on the decoded small video to restore the size of the initial image;
the method specifically comprises the following steps:
step A: inputting an original video sequence at an encoding end and performing downsampling to obtain a degraded low-resolution video sequence;
and B: an HEVC encoder encodes a low resolution video sequence, comprising:
each frame of input original video image is divided into a plurality of mutually independent coding units, namely original image blocks;
performing predictive coding, including removing redundant information and obtaining a predictive image block by adopting intra-frame prediction and inter-frame prediction modes respectively according to the time redundancy and the spatial redundancy of the video image;
obtaining a prediction residual based on the difference value of the image values of the prediction block and the original image block, and performing discrete cosine transform and quantization on the prediction residual;
entropy coding is carried out on the quantized discrete cosine transform coefficient to obtain a compressed code stream of the video sequence, and the compressed code stream is output;
and C: and the decoder decodes the compressed code stream to obtain a high-resolution video sequence, and performs image interpolation on the high-resolution video sequence to restore the image size of the high-resolution video sequence to the original image size.
The inter prediction modes include: 2 nx 2N, N xn, 2 nx N, N X2N, nL X2N, nR X2N, 2 nxnu and 2 nxnd, as shown in fig. 4, each mode is divided in a format of (width [2 nx ] height [2N ]), and when each coding unit is divided into upper, lower, and upper large blocks, the height [2N ] may be expressed as nU and N U's equal to 2N, and similarly, when each coding unit is divided into left, lower, and right large blocks, the width [2N ] may be expressed as nL and N's equal to 2N;
in this embodiment, regarding that the PU prediction mode for selecting the minimum rate-distortion cost value of the maximum probability of the current frame coding unit is a fast inter-frame mode selection mode based on the temporal correlation, the PU mode with the highest probability of occurrence in inter-frame prediction is found out according to the distribution of different PUs under various sizes of CUs by first counting and comprehensively analyzing the temporal correlation between the corresponding frame Coding Unit (CU) and Prediction Unit (PU) between frames. The method skips some high-similarity CU partition layers and PU prediction modes, and the CU layers only take two PU prediction modes at most, thereby greatly reducing the number of rate-distortion cost calculation required. With little loss in bit rate and video quality, nearly half of the encoding time can be saved. Then, according to the size relationship between the current frame coding unit and the corresponding frame coding unit, three cases are divided for discussion, each type respectively uses a set method to calculate the PU prediction mode of the current image block with the maximum probability of the minimum rate-distortion cost value according to the prediction mode of the prediction unit, and lists the inter-frame mode methods under 3 cases according to the statistical result, which is specifically as follows:
when the side length of the current frame coding unit is smaller than the side length of the corresponding frame coding unit, the method comprises the following steps:
when the side length of the current frame coding unit is half of the side length of the corresponding frame coding unit, if the PU prediction mode of the partition of the corresponding frame coding unit is nL multiplied by 2N, the PU prediction mode of the partition of the current frame coding unit is selected to be Nmultiplied by 2N; if the PU prediction mode of the partition corresponding to the frame coding unit is 2 NxnU; selecting the PU prediction mode of the block of the current frame coding unit as 2 NxN;
as shown in fig. 2, when the side length of the current CU is half of the corresponding block, the block P may correspond to a position of C, D, E or F. When the PU mode of the block corresponding to the previous frame is nL multiplied by 2N, selecting Nmultiplied by 2N for the PU mode of the CU block of the current frame; as shown in fig. 3, if the PU mode is 2N × nU, then 2N × N is selected as the PU mode of the current CU block;
when the side length of the current frame coding unit is one fourth of the side length of the corresponding frame coding unit, selecting a PU (polyurethane) prediction mode of a block of the current frame coding unit to be 2 Nx 2N;
when the side length of the current frame coding unit is less than one fourth of the side length of the corresponding frame coding unit, the partition of the current frame coding unit does not select the PU prediction mode.
Preferably, when the side length of the current frame coding unit is greater than the side length of the corresponding frame coding unit, the judgment is carried out according to the size of each block of the corresponding frame coding unit and the PU prediction mode, and the PU prediction module is selected according to the distribution condition of the blocks of the corresponding frame coding unit and the prediction units thereof;
in addition to the mode 2N × 2N, other modes must be determined by the size of each CU partition in the corresponding block and the PU prediction mode. Through a large number of experimental statistics, 6 modes (2N × 2N, 2N × N, N × 2N, nL × 2N, nR × 2N, 2N × nU and 2N × nD) are divided into 2 types, namely vertical and horizontal, and which prediction mode is selected is determined according to the distribution of specific coding units and prediction units in a corresponding block.
And when the side length of the current frame coding unit is equal to the side length of the corresponding frame coding unit, the coding unit of the current frame selects the PU prediction mode which is the same as the partition of the corresponding frame coding unit.
The optimized transformation scheme based on different intra and inter prediction modes in this embodiment is specifically as follows:
as shown in fig. 5, intra-frame prediction models in different modes are first established according to the HEVC standard, then a residual pixel expression is updated by using a pixel covariance matrix obtained by an ellipse model based on a direction, then a residual pixel covariance matrix is obtained, and then a transformation matrix is extracted through KLT decomposition and adjustment. The improved transform method may improve the overall coding efficiency on the luminance component and the chrominance component.
Preferably, in the step B, the discrete cosine transforming and quantizing the prediction residual includes:
the method comprises the steps of establishing intra-frame prediction models in different modes according to the HEVC standard, obtaining an initial pixel covariance matrix according to a direction-based rotation ellipse model, updating a residual pixel expression according to the initial pixel covariance matrix to obtain a residual pixel covariance matrix, decomposing and adjusting the residual pixel covariance matrix through KLT, and extracting a transformation matrix.
And establishing an elliptical model based on the texture direction information to obtain a covariance matrix of the initial pixel. After the image frame is subjected to the blocking operation, the encoding unit has a dominant direction. The traditional circular model ignores that the pixel points in the dominant direction are more relevant than the rest. In order to fully utilize the original directional information of the image and improve the accuracy of prediction.
Preferably, the obtaining the initial pixel covariance matrix according to the direction-based ellipse model comprises:
a rotation ellipse model based on the image texture direction is established,
obtaining an offset angle corresponding to a PU prediction mode of a current frame coding unit, namely an image texture direction corresponding to the rotation ellipse model;
obtaining the correlation between pixel points A (a, B) and B (c, d) in a current frame coding unit, wherein the correlation is as follows:
wherein:
r represents the ratio of the major and minor axes;
theta represents the direction of texture in the current frame coding unit, namely an angle value;
ρ represents the correlation strength between pixels;
d1(θ)，d2(theta) represents the projection coordinates of the pixel points A and B in the rotating ellipse model;
the mapping relation between the projection coordinates and the real coordinates A (a, B) and B (c, d) is as follows:
and sleeving the pixel value of the current frame coding unit and the reference pixel value of the boundary of the adjacent coding unit into the rotation ellipse model, acquiring the correlation between the current frame coding unit and the adjacent coding unit thereof, and acquiring an initial pixel covariance matrix.
Preferably, the establishing of the intra prediction model includes:
and mapping the reference pixel: mapping all reference pixels required by a current frame coding unit into a row or a column;
obtaining a predicted pixel value P according toi，j：
Pi，j＝((32-ω)×Mon0，pla+ω×M0n0，pla+1+16)；
Wherein:
omega represents the weight of interpolation operation;
Mon0，plaand Mon0，pla+1Representing a reference pixel mapped by the current prediction pixel;
pla represents the corresponding reference pixel location;
offset [ P ] represents the offset corresponding to the current mode;
(x, y) represents the coordinates of the current predicted pixel.
And establishing an intra-frame prediction model. According to the HEVC standard, intra prediction pixels are calculated differently in different modes. Firstly, mapping processing is carried out on reference pixels: in the intra prediction of HEVC, all reference pixels needed by the current block are mapped into a line (denoted as Mon) or a column according to the prediction mode, and the pixel values in the subsequent TU are derived from the reconstructed reference pixels. And then, calculating a predicted pixel value, wherein after the one-dimensional reference pixel set corresponding to the mode is obtained, each predicted pixel value is obtained by performing one-time interpolation operation on two reference pixels corresponding to the coordinates.
When the corresponding offset value is 0 or 32 and the corresponding ω is 0, each pixel of the current block is only related to one reference pixel in the interpolation operation of the prediction value in the mode. Otherwise, the pixel point is related to two reference pixel points;
a covariance matrix of the residuals is calculated. The operation object of the transformation is residual data, and in order to obtain the improved transformation matrix, the covariance matrix of the residual block in different prediction modes must be calculated. The value of the prediction residual is equal to the difference between the true value of the current pixel and its predicted value. Firstly, according to any two residual pixels in the predicted residual pixel block
Obtaining the correlation size of the two, combining the coordinates of the two residual pixels in the current transformation unit and the position of the reference pixel, and combining the initial pixel set and the reference pixelAnd (4) solving the correlation between two predicted residual pixels by the relationship between every two pixel sets.
Preferably, obtaining the residual pixel covariance matrix comprises:
for a residual pixel block with a size of N × N and selected intra-frame prediction mode in intra-frame coding, transposing the residual pixel block into a one-dimensional vector form
With a covariance matrix of R (size N)2×N2)：
Wherein each element Ra，bComprises the following steps:
Rb，a＝Ra，b＝E{ea，eb}，a，b＝1，2，…，N2；
extracting the transformation matrix includes:
solving an initial KLT transformation matrix based on the residual pixel covariance matrix, and adjusting the amplification factor and the scanning sequence of an integer matrix in transformation coding of the initial KLT transformation matrix;
an improved transformation matrix is solved. The KLT transform has transform performance that cannot be achieved by other transform methods, and in order to apply the transform matrix of the method to the actual HEVC coding standard, the transform matrix needs to be adjusted by referring to the integer transform method in HEVC, one is to adjust the magnification to keep the transform coding precision unchanged and the scanning order to keep the energy distribution unchanged, and the two operations are not in sequence. Firstly, solving a covariance matrix of a residual error pixel block under any intra-frame prediction mode by combining a rotation ellipse model and an intra-frame prediction model, and then solving an initial KLT transformation matrix; adjusting the initial KLT matrix results in an improved transform matrix.
The initially decomposed transformation matrix is sequentially arranged in a descending order from top to bottom according to the eigenvectors corresponding to the eigenvalues. To obtain the KLT transform matrix, a decomposition of the eigenvalues of the standard is required. Intra prediction for a NxN size selectionResidual pixel blocks of pattern P. Similar to the amplification factor of integer matrix in transform coding in HEVC standard, for a transform unit with size N × N, the corresponding transform matrix needs to be amplified when performing integer operation
And (4) doubling. In order to keep the order of the improved transformed transform coefficients and the transform coding in HEVC unchanged and keep the frame of the transform in HEVC basically unchanged, the transform method proposed in this patent is inseparable and therefore can be derived with only one multiplication. The improved transform matrix needs to be scaled up 4096N times on the basis of the initial KLT decomposition matrix and then rounded up.
Since the initial KLT transform matrix obtained by this patent is not suitable for other prediction modes using vertical scan order or diagonal scan order. Therefore, we need to make corresponding adjustment to the transform matrix based on the intra prediction mode corresponding to the current coefficient block:
for image blocks with a prediction mode of 22-30, a scanning sequence adopted in HEVC is horizontal scanning, and in order to keep the coefficient energy distribution consistent with the arrangement of an initial KLT matrix, the improved transform coding does not need to be adjusted;
for image blocks with prediction modes of 6-14, a vertical scanning sequence is adopted on an HEVC frame, and at the moment, an improved transformation method needs to put feature vectors in a KLT matrix in a sequence consistent with energy arrangement in HEVC.
For the image blocks of the remaining prediction modes. The adjustment of the transformation matrix is arranged according to the same energy sequence by adopting a diagonal scanning method.
The transformation matrix in the project is replaced and applied. Once the transform unit TU and the prediction mode in intra-frame predictive coding are determined, the covariance matrix is obtained, the eigenvalue decomposition is performed on the covariance matrix, and the eigenvalue decomposition is adjusted according to the property of the transform matrix in the HEVC standard and the rule of transform coefficient entropy coding, so that the improved transform matrix can be obtained. Due to the limitation of the size of the transformation matrix, only residual transformation with the sizes of 4 × 4 and 8 × 8 of the intra-frame coding part in HEVC is changed, namely, the original DCT transformation in the HEVC standard is replaced by improved transformation kernel matrixes with the sizes of 16 × 16 and 64 × 64, each block obtains a corresponding transformation matrix, and then the series of transformation matrixes are numbered and put into engineering, and the transformation matrixes are selected and replaced according to the size of TU (transformation unit) and the prediction mode.
The video is a set of a static image, and usually, the chrominance information of pixel points at the same position in adjacent images is basically unchanged, so that the NPCI algorithm can be widened to the interpolation of the video image, namely, the low-resolution video image to be restored is interpolated by utilizing the similarity structure of the inter-frame redundancy and intra-frame redundancy of the video.
Preferably, in step C, the method includes interpolating the low-resolution video image to be restored by using the intra-frame redundant similarity structure to obtain the high-resolution prediction image block, and specifically includes:
expanding a search area by utilizing similarity, and increasing the number of image blocks which can be referred to, the method comprises the following specific steps:
selecting an original image block Pi to be interpolated in a video image of a current frame as a target center, and establishing a window with radius r in the current video image;
determining a search frame with the same position and size as a window in each L frame of video images before and after the video image of the current frame, and using the search frame as a search area of a similar neighbor block of an original image block Pi;
respectively putting the image blocks in the search area into a down-sampling grid, and sequentially calculating the similarity between the image blocks and the original image blocks Pi;
selecting the most similar N image blocks to fit the original image blocks Pi to obtain image blocks similar to the original image blocks Pi, and marking the similar image blocks respectively;
down-sampling the marked image blocks again and splicing the marked image blocks into high-resolution prediction image blocks;
and continuously and iteratively updating the prediction image block to ensure that the approximation degree of the prediction image block and the original image block to be interpolated reaches the highest.
The image block that is continuously updated will be closer to the image block to be interpolated. However, the number of iterations is not too large, so that overfitting is easy to occur, and the calculation amount is too large.
Preferably, the determining the similarity between the image block of the search area and the original image block includes:
acquiring a difference value of the alignment gray value of an original image block to be interpolated and each image block in a search area;
accumulating and summing the absolute values of the difference values;
and selecting the image blocks corresponding to the first K minimum numerical values to linearly represent the original image blocks to be interpolated, wherein the image blocks corresponding to the first K minimum numerical values have the highest similarity with the original image blocks to be interpolated.
Preferably, in the step C, the method further includes reducing the radius of the search box and reducing the number of iterations, so as to improve the similarity between the prediction image block and the original image block to be interpolated.
The radius of a search window and the iteration times are reduced to prevent the quality of the interpolation image from being reduced, and meanwhile, the running speed is higher. The search window is mainly used for searching non-local similar neighbor blocks, so the larger the window is, the more similar blocks are searched, and the more accurate the image restored finally. However, an excessively large window increases the amount of unnecessary computation, and degrades the performance of interpolation. The choice of the value of K directly affects the quality of the interpolated image. Images with different characteristics preferably have different K values.
Although the numerical value difference between the values of the parameters (such as the size of a search window of an adjacent area and the size of an image block) is small, the calculation amount of the algorithm is greatly increased, so that an appropriate value needs to be set according to a specific scene;
the technical principle of the present invention is described above in connection with specific embodiments. The description is made for the purpose of illustrating the principles of the invention and should not be construed in any way as limiting the scope of the invention. Based on the explanations herein, those skilled in the art will be able to conceive of other embodiments of the present invention without inventive effort, which would fall within the scope of the present invention.
Claims (10)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110501572.8A CN113301347B (en) | 2021-05-08 | 2021-05-08 | HEVC high definition video coding optimization method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110501572.8A CN113301347B (en) | 2021-05-08 | 2021-05-08 | HEVC high definition video coding optimization method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113301347A true CN113301347A (en) | 2021-08-24 |
| CN113301347B CN113301347B (en) | 2023-05-05 |
Family
ID=77321187
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110501572.8A Active CN113301347B (en) | 2021-05-08 | 2021-05-08 | HEVC high definition video coding optimization method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113301347B (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114554226A (en) * | 2022-02-25 | 2022-05-27 | 北京市商汤科技开发有限公司 | Image processing method and device, electronic equipment and storage medium |
| CN115209147A (en) * | 2022-09-15 | 2022-10-18 | 深圳沛喆微电子有限公司 | Camera video transmission bandwidth optimization method, device, equipment and storage medium |
| WO2023028965A1 (en) * | 2021-09-02 | 2023-03-09 | Nvidia Corporation | Hardware codec accelerators for high-performance video encoding |
| CN116600119A (en) * | 2023-07-18 | 2023-08-15 | 腾讯科技（深圳）有限公司 | Video encoding method, video decoding method, video encoding device, video decoding device, computer equipment and storage medium |
| CN116962685A (en) * | 2023-09-21 | 2023-10-27 | 杭州爱芯元智科技有限公司 | Video encoding method, video encoding device, electronic equipment and storage medium |
| US11871018B2 (en) | 2021-09-02 | 2024-01-09 | Nvidia Corporation | Parallel processing of video frames during video encoding |
| US12184843B2 (en) | 2021-09-06 | 2024-12-31 | Nvidia Corporation | Parallel encoding of video frames without filtering dependency |
| US12238335B2 (en) | 2023-04-18 | 2025-02-25 | Nvidia Corporation | Efficient sub-pixel motion vector search for high-performance video encoding |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102196272A (en) * | 2010-03-11 | 2011-09-21 | 中国科学院微电子研究所 | A P-frame encoding method and device |
| CN103546749A (en) * | 2013-10-14 | 2014-01-29 | 上海大学 | Method of Optimizing HEVC Residual Coding Using Residual Coefficient Distribution Characteristics and Bayes Theorem |
| CN103634608A (en) * | 2013-12-04 | 2014-03-12 | 中国科学技术大学 | High-performance video coding lossless mode residual error transform method |
| CN106791879A (en) * | 2016-11-30 | 2017-05-31 | 北京工业大学 | A kind of direction transformation method based on Video Encoding Mode |
| WO2018014301A1 (en) * | 2016-07-21 | 2018-01-25 | 华为技术有限公司 | Video coding method and device |
| US20180220148A1 (en) * | 2015-09-25 | 2018-08-02 | Huawei Technologies Co., Ltd. | Apparatus and method for video motion compensation with selectable interpolation filter |
| CN109429071A (en) * | 2017-08-23 | 2019-03-05 | 富士通株式会社 | Picture coding device, picture decoding apparatus and image processing method |
| CN109547783A (en) * | 2018-10-26 | 2019-03-29 | 西安科锐盛创新科技有限公司 | Video-frequency compression method and its equipment based on intra prediction |
| CN111355959A (en) * | 2018-12-22 | 2020-06-30 | 华为技术有限公司 | Image block division method and device |
| CN111711817A (en) * | 2019-03-18 | 2020-09-25 | 四川大学 | An optimization research on HEVC intra-frame coding compression performance combined with convolutional neural network |
-
2021
- 2021-05-08 CN CN202110501572.8A patent/CN113301347B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102196272A (en) * | 2010-03-11 | 2011-09-21 | 中国科学院微电子研究所 | A P-frame encoding method and device |
| CN103546749A (en) * | 2013-10-14 | 2014-01-29 | 上海大学 | Method of Optimizing HEVC Residual Coding Using Residual Coefficient Distribution Characteristics and Bayes Theorem |
| CN103634608A (en) * | 2013-12-04 | 2014-03-12 | 中国科学技术大学 | High-performance video coding lossless mode residual error transform method |
| US20180220148A1 (en) * | 2015-09-25 | 2018-08-02 | Huawei Technologies Co., Ltd. | Apparatus and method for video motion compensation with selectable interpolation filter |
| WO2018014301A1 (en) * | 2016-07-21 | 2018-01-25 | 华为技术有限公司 | Video coding method and device |
| CN106791879A (en) * | 2016-11-30 | 2017-05-31 | 北京工业大学 | A kind of direction transformation method based on Video Encoding Mode |
| CN109429071A (en) * | 2017-08-23 | 2019-03-05 | 富士通株式会社 | Picture coding device, picture decoding apparatus and image processing method |
| CN109547783A (en) * | 2018-10-26 | 2019-03-29 | 西安科锐盛创新科技有限公司 | Video-frequency compression method and its equipment based on intra prediction |
| CN111355959A (en) * | 2018-12-22 | 2020-06-30 | 华为技术有限公司 | Image block division method and device |
| CN111711817A (en) * | 2019-03-18 | 2020-09-25 | 四川大学 | An optimization research on HEVC intra-frame coding compression performance combined with convolutional neural network |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023028965A1 (en) * | 2021-09-02 | 2023-03-09 | Nvidia Corporation | Hardware codec accelerators for high-performance video encoding |
| US11871018B2 (en) | 2021-09-02 | 2024-01-09 | Nvidia Corporation | Parallel processing of video frames during video encoding |
| US12170757B2 (en) | 2021-09-02 | 2024-12-17 | Nvidia Corporation | Hardware codec accelerators for high-performance video encoding |
| US12184843B2 (en) | 2021-09-06 | 2024-12-31 | Nvidia Corporation | Parallel encoding of video frames without filtering dependency |
| CN114554226A (en) * | 2022-02-25 | 2022-05-27 | 北京市商汤科技开发有限公司 | Image processing method and device, electronic equipment and storage medium |
| CN115209147A (en) * | 2022-09-15 | 2022-10-18 | 深圳沛喆微电子有限公司 | Camera video transmission bandwidth optimization method, device, equipment and storage medium |
| CN115209147B (en) * | 2022-09-15 | 2022-12-27 | 深圳沛喆微电子有限公司 | Camera video transmission bandwidth optimization method, device, equipment and storage medium |
| US12238335B2 (en) | 2023-04-18 | 2025-02-25 | Nvidia Corporation | Efficient sub-pixel motion vector search for high-performance video encoding |
| CN116600119A (en) * | 2023-07-18 | 2023-08-15 | 腾讯科技（深圳）有限公司 | Video encoding method, video decoding method, video encoding device, video decoding device, computer equipment and storage medium |
| CN116600119B (en) * | 2023-07-18 | 2023-11-03 | 腾讯科技（深圳）有限公司 | Video encoding method, video decoding method, video encoding device, video decoding device, computer equipment and storage medium |
| CN116962685A (en) * | 2023-09-21 | 2023-10-27 | 杭州爱芯元智科技有限公司 | Video encoding method, video encoding device, electronic equipment and storage medium |
| CN116962685B (en) * | 2023-09-21 | 2024-01-30 | 杭州爱芯元智科技有限公司 | Video encoding method, video encoding device, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113301347B (en) | 2023-05-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113301347A (en) | Optimization method of HEVC high-definition video coding | |
| CN107005712B (en) | Method and apparatus for performing graph-based prediction using optimization function | |
| JP4572010B2 (en) | Methods for sprite generation for object-based coding systems using masks and rounded averages | |
| CN101496412A (en) | Mesh-based video compression with domain transformation | |
| CN103220527B (en) | Method for encoding images and device and its coding/decoding method and device | |
| CN101860748B (en) | System and method for generating side information based on distributed video coding | |
| US20220217337A1 (en) | Method, codec device for intra frame and inter frame joint prediction | |
| CN103327325B (en) | The quick self-adapted system of selection of intra prediction mode based on HEVC standard | |
| JP3655651B2 (en) | Data processing device | |
| JPH08265780A (en) | Method and apparatus for coding/decoding video signal | |
| JP6636615B2 (en) | Motion vector field encoding method, decoding method, encoding device, and decoding device | |
| CN101184233B (en) | A method of digital video compression coding based on CFRFS | |
| JPH08242458A (en) | Movement vector detecting method | |
| JPH08307874A (en) | Video signal encoding device | |
| CN100591136C (en) | A Video Intra-Frame Coding Method Based on Spatial Domain Decomposition | |
| CN102075757B (en) | Video foreground object coding method by taking boundary detection as motion estimation reference | |
| Li et al. | Deep image compression based on multi-scale deformable convolution | |
| CN113079378A (en) | Image processing method and device and electronic equipment | |
| Jeong et al. | An overhead-free region-based JPEG framework for task-driven image compression | |
| CN206698375U (en) | One kind slides block of pixels integer DCT kernel matrixs conversion motion compensator | |
| CN117319652A (en) | Video coding and decoding model processing, video coding and decoding methods and related equipment | |
| CN102263954B (en) | An Object-Based Fast Fractal Video Compression and Decompression Method | |
| JP4490351B2 (en) | Inter-layer prediction processing method, inter-layer prediction processing apparatus, inter-layer prediction processing program, and recording medium therefor | |
| JP5871714B2 (en) | Distributed video encoding method and system, and decoding apparatus | |
| CN109600608B (en) | Dual-mode selection prediction method for complex texture in bandwidth compression |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |