WO2020184264A1 - Image decoding device, image decoding method, and program - Google Patents

Abstract

An image decoding device 200 includes: sub-block division units 101A/101B for acquiring a first sub-block and a second sub-block by dividing a prediction target block at different dividing positions; motion vector generation units 101D1/101D2 for outputting motion vectors obtained by affine transformations corresponding to the first sub-block and the second sub-block; motion compensation prediction units 101F1/101F2 for generating predicted images of the first sub-block and the second sub-block on the basis of the motion vectors; and an interpolation unit 101G for generating a predicted image of the prediction target block on the basis of the predicted image of the first sub-block and the predicted image of the second sub-block.

Description

Image decoding device, image decoding method and program

The present invention relates to an image decoding device, an image decoding method, and a program.

Conventionally, an image coding method using intra-prediction or inter-prediction, conversion / quantization of a prediction residual signal, and entropy coding has been proposed (see, for example, Non-Patent Document 1).

Further, as one of the inter-prediction methods, an affine motion compensation prediction method has been proposed in the next-generation moving image coding method WC (see, for example, Non-Patent Document 2). The affine motion compensation prediction method is a kind of motion compensation prediction method, and is configured to generate a prediction image by a motion vector and a translation model.

As shown in FIG. 8, in the affine motion compensation prediction method, the block size to which the translation model is applied is a subblock of 4 × 4 pixels, while a different motion vector is derived for each subblock by affine transformation.

Here, the affine transformation parameters are expressed as motion vectors at the vertices of the target block for which affine motion compensation prediction is performed in order to be consistent with the existing image coding method. The motion vector at each vertex is called a "control point".

The motion vector (mv _x , mv _y ) at an arbitrary pixel position in the case of the 4-parameter model shown in FIG. 9 (a) is derived by (Equation 1), and in the case of the 6-parameter model shown in FIG. 9 (b). The motion vector (mv _x , mv _y ) at an arbitrary pixel position is derived by (Equation 2).

Here, (mv _0x , mv _0y ) is the control point (movement vector) of the upper left vertex of the target block (subblock), and (mv _1x , mv _1y ) is the control point of the upper right vertex of the target block. , (Mv _2x , mv _2y ) are control points of the lower left vertex of the target block. Further, W is the width of the target block, and H is the height of the target block.

Further, the formula for deriving such motion vector _(mv x, mv _y) is but can be applied to any pixel position, as described above, sharing one motion vector in sub-blocks of 4 × 4 pixels.

ITU-T H.265 High Efficiency Video Coding Versatile Video Coding (Draft 4)

However, in the above-mentioned technology, in order to prioritize consistency with the existing motion compensation prediction method, affine motion compensation prediction can be performed only in units of 4 × 4 pixel subblocks, and the prediction performance is low. There was a problem.

Therefore, the present invention has been made in view of the above-mentioned problems, and provides an image decoding device, an image decoding method, and a program capable of improving prediction performance while keeping the rate of increase in the number of subblocks constant. The purpose is.

The first feature of the present invention is an image decoding device, which is a subblock division unit configured to acquire a first subblock and a second subblock by dividing a prediction target block at different division positions. And the motion vector generator configured to output the motion vector obtained by the affine transformation corresponding to the first subblock and the second subblock, and the first sub based on the motion vector. The prediction target block is based on the motion compensation prediction unit configured to generate the prediction image of the block and the second subblock, the prediction image of the first subblock, and the prediction image of the second subblock. It is a gist to have an interpolation unit configured to generate a predicted image of.

The second feature of the present invention is an image decoding method, in which a step A of acquiring a first subblock and a second subblock by dividing a prediction target block at different division positions, and the first subblock and the first subblock A step B for outputting a motion vector obtained by an affine transformation corresponding to the second subblock, and a step C for generating a predicted image of the first subblock and the second subblock based on the motion vector. It is a gist to have a step D of generating a predicted image of the prediction target block based on the predicted image of the first subblock and the predicted image of the second subblock.

A third feature of the present invention is a program that causes a computer to function as an image decoding device, in which the image decoding device divides the prediction target block at different division positions to form a first subblock and a second subblock. A subblock dividing unit configured to acquire the above, and a motion vector generating unit configured to output a motion vector obtained by an affine transformation corresponding to the first subblock and the second subblock. A motion compensation prediction unit configured to generate a prediction image of the first subblock and the second subblock based on the motion vector, a prediction image of the first subblock, and the second subblock. The gist is to have an interpolation unit configured to generate a predicted image of the predicted target block based on the predicted image of the sub-block.

According to the present invention, it is possible to provide an image decoding device, an image decoding method and a program capable of improving the prediction performance while suppressing the increase rate of the number of subblocks to a constant level.

It is a figure which shows an example of the structure of the image processing system 100 which concerns on one Embodiment. It is a figure which shows an example of the functional block of the image coding apparatus 100 which concerns on one Embodiment. It is a figure which shows an example of the functional block of the image decoding apparatus 200 which concerns on one Embodiment. It is a figure which shows an example of the functional block of the inter-prediction unit 101 of the image coding apparatus 100 and the inter-prediction unit 203 of the image decoding apparatus 200 which concerns on one Embodiment. Sub-blocks divided by the additional sub-block division section 101B of the inter-prediction unit 101 of the image coding device 100 and the inter-prediction unit 203 of the image decoding device 200 according to one embodiment, and sub-blocks divided by the normal sub-block division section 101A. It is a figure which shows an example of a block. It is a flowchart which shows an example of the operation of the image decoding apparatus 200 which concerns on one Embodiment. Sub-blocks divided by the additional sub-block division section 101B of the inter-prediction unit 101 of the image coding device 100 and the inter-prediction unit 203 of the image decoding device 200 according to one embodiment, and sub-blocks divided by the normal sub-block division section 101A. It is a figure which shows an example of a block. It is a figure for demonstrating the prior art. It is a figure for demonstrating the prior art.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. The components in the following embodiments can be replaced with existing components as appropriate, and various variations including combinations with other existing components are possible. Therefore, the description of the following embodiments does not limit the content of the invention described in the claims.

(First Embodiment)
FIG. 1 is a diagram showing an example of a functional block of the image processing system 1 according to the first embodiment of the present invention. The image processing system 1 includes an image coding device 100 that encodes a moving image and generates coded data, and an image decoding device 200 that decodes the coded data generated by the image coding device 100. The above-mentioned coded data is transmitted and received between the image coding device 100 and the image decoding device 200, for example, via a transmission line.

FIG. 2 is a diagram showing an example of a functional block of the image coding device 100. As shown in FIG. 2, the image coding device 100 includes an inter prediction unit 101, an intra prediction unit 102, a conversion / quantization unit 103, an entropy coding unit 104, and an inverse conversion / inverse quantization unit 105. , An in-loop filter 106 and a frame buffer 107.

The inter-prediction unit 101 is configured to perform inter-prediction using the input image and the filtered locally decoded image (described later) input from the frame buffer 109 to generate and output the inter-prediction image.

The intra prediction unit 102 is configured to perform intra prediction using an input image and a locally decoded image before filtering (described later) to generate and output an intra prediction image.

The conversion / quantization unit 103 performs orthogonal conversion processing on the residual signal input from the subtraction unit 106, and performs quantization processing on the conversion coefficient obtained by such orthogonal conversion processing. It is configured to output the level value.

The entropy encoding unit 104 is configured to entropy-encode the quantization coefficient level value, conversion unit size and conversion size, motion compensation method, etc. input from the conversion / quantization unit 103 and output them as encoded data. There is.

The inverse conversion / inverse quantization unit 105 performs an inverse quantization process on the quantization coefficient level value input from the conversion / quantization unit 103, and reverses the conversion coefficient obtained by the inverse quantization process. It is configured to output the residual signal obtained by performing the orthogonal conversion process.

The subtraction unit 106 is configured to output a residual signal which is a difference between the input image and the intra prediction image or the inter prediction image.

The addition unit 107 is configured to output a pre-filter local decoding image obtained by adding the residual signal input from the inverse transformation / inverse quantization unit 105 and the intra-prediction image or the inter-prediction image.

The in-loop filter unit 108 applies in-loop filter processing such as deblocking filter processing to the pre-filter local decoding image input from the addition unit 107 to generate and output the post-filter local decoding image. It is configured.

The frame buffer 109 accumulates the filtered locally decoded image and appropriately supplies it to the inter-prediction unit as the filtered locally decoded image.

FIG. 3 is a block diagram of the image decoding device 200. As shown in FIG. 3, the image decoding device 200 includes an entropy decoding unit 201, an inverse conversion / inverse quantization unit 202, an inter prediction unit 203, an intra prediction unit 204, an addition unit 205, and an in-loop filter 206. And a frame buffer 207.

The entropy decoding unit 201 is configured to entropy decode the encoded data, derive the quantization coefficient level value, the conversion unit size and the conversion size, the motion compensation method, and the like, and output the coded data.

The inverse conversion / inverse quantization unit 202 performs an inverse quantization process on the quantization coefficient level value input from the entropy decoding unit 201, and an inverse orthogonal conversion process on the result obtained by the inverse quantization process. Is configured to be output as a residual signal.

The inter-prediction unit 203 is configured to perform inter-prediction using the filtered locally decoded image input from the frame buffer 207 to generate and output the inter-prediction image.

The intra prediction unit 204 is configured to perform intra prediction using the pre-filter locally decoded image input from the addition unit 205 to generate and output an intra prediction image.

The addition unit 205 combines the residual signal input from the inverse transformation / inverse quantization unit 202 with the prediction image (inter prediction image input from the inter prediction unit 203 or intra prediction image input from the intra prediction unit 204). It is configured to output the pre-filter locally decoded image obtained by addition.

Here, the prediction image is a prediction with the highest expected coding performance obtained by entropy decoding among the inter prediction image input from the inter prediction unit 203 and the intra prediction image input from the intra prediction unit 204. It is a predicted image calculated by the method.

The in-loop filter 206 is configured to apply in-loop filter processing such as deblocking filter processing to the pre-filter locally decoded image input from the addition unit 205 to generate and output the post-filtered locally decoded image. ing.

The frame buffer 207 is configured to accumulate the filtered locally decoded image input from the in-loop filter 206, appropriately supply it to the inter-prediction unit 203 as the filtered locally decoded image, and output it as a decoded image. There is.

The inter-prediction unit 101 of the image coding device 100 and the inter-prediction unit 203 of the image decoding device 200 will be described with reference to FIGS. 4 to 6. Since the functions of the inter-prediction unit 101 of the image coding device 100 and the functions of the inter-prediction unit 203 of the image decoding device 200 are basically the same, the functions of the inter-prediction unit 203 of the image decoding device 200 will be taken as an example below. I will explain it by listing it.

As shown in FIG. 4, the inter-prediction unit 203 of the image decoding apparatus 200 includes a normal sub-block division unit 101A, an additional sub-block division unit 101B, a motion vector generation unit 101C, and an affine motion vector generation unit 101D1 / 101D2. , The frame buffer 101E, the motion compensation prediction units 101F1 to 101F3, the addition unit 101G, and the selection unit 101H are provided.

Normally, the sub-block division unit 101A is configured to acquire the first sub-block by dividing the prediction target block at the first division position.

Specifically, the normal sub-block division unit 101A divides the prediction target block based on the input coordinates and block size of the prediction target block to acquire the first sub-block, and obtains the first sub-block, and the coordinates of the first sub-block and It is configured to output the block size.

For example, as shown in FIG. 5B, the normal subblock division unit 101A divides the prediction target block at the first division position X, and divides 16 subblocks (blocks composed of 4 × 4 pixels) B1. It may be configured to acquire.

The additional sub-block division unit 101B is configured to acquire the second sub-block by dividing the prediction target block at the second division position, which is a position deviated from the first division position by a predetermined pixel.

Specifically, the additional sub-block division unit 101B grasps the first division position from the input coordinates and block size of the prediction target block, and is a position shifted by a predetermined pixel from the first division position. It is configured to divide the prediction target block at the division position, acquire the second subblock, and output the coordinates and block size of the second subblock.

Here, the second division position may be a position shifted in the vertical direction and the horizontal direction by half the pixels of the first subblock from the first division position.

For example, as shown in FIG. 5A, the additional sub-block division unit 101B divides the prediction target block at the second division position Y shifted in the vertical and horizontal directions by two pixels from the first division position X. , Nine second subblocks (blocks composed of 4 × 4 pixels) B2 may be acquired.

Since the affine motion compensation prediction is performed in units of 4 × 4 pixel subblocks, in the example of FIG. 5, the additional subblock division 101B has the coordinates of nine second subblocks composed of 4 × 4 pixels and the coordinates and It is configured to output the block size.

Here, the normal subblock division unit 101A and the additional subblock division unit 101B are configured to acquire the first subblock and the second subblock by dividing the prediction target block at different division positions. It constitutes a division part.

The motion vector generation unit 101C is configured to acquire the input coordinates and control points of the prediction target block and output the control points at the upper right vertices of the prediction target block as motion vectors.

The affine motion vector generation unit 101D1 is configured to output the motion vector obtained by the affine transformation corresponding to the first subblock. Similarly, the affine motion vector generation unit 101D2 is configured to output the motion vector obtained by the affine transformation corresponding to the second subblock.

Specifically, the affine motion vector generation units 101D1 / 101D2 perform affine transformation using the input coordinates, block size and control points of the first subblock and the second subblock, and perform the affine transformation, and the first subblock and the second subblock and the second subblock. It is configured to output the motion vector corresponding to the subblock.

The frame buffer 101E is configured to output a fractional part of the reference pixel and motion vector indicated by the reference pixel subscript based on the input reference image subscript and motion vector of the block (or subblock). .. The frame buffer 101E may be substituted by the frame buffer 109/207.

The motion compensation prediction unit 101F1 / 101F2 is configured to generate prediction images (interpolated images) of the first subblock and the second subblock based on the above-mentioned motion vector.

Specifically, the motion compensation prediction unit 101F1 is configured to generate a prediction image of the second subblock based on the motion vector corresponding to the second subblock output from the frame buffer 101E.

More specifically, the motion compensation prediction unit 101F1 is configured to generate a prediction image of the second subblock from the reference pixel output from the frame buffer 101E and the fractional part of the motion vector corresponding to the second subblock. Has been done.

Further, the motion compensation prediction unit 101F2 is configured to generate a prediction image of the first subblock based on the motion vector corresponding to the first subblock output from the frame buffer 101E.

More specifically, the motion compensation prediction unit 101F2 is configured to generate a prediction image of the first subblock from a reference pixel output from the frame buffer 101E and a fractional part of the motion vector corresponding to the first subblock. Has been done.

Further, the motion compensation prediction unit 101F3 is configured to generate a prediction image of the prediction target block from the reference pixel output from the frame buffer 101E and a fractional part of the motion vector corresponding to the prediction target block.

The addition unit 101G constitutes an interpolation unit configured to generate a prediction image of the prediction target block based on the prediction image of the first subblock and the prediction image of the second subblock.

Specifically, the addition unit 101G is configured to generate a prediction image of the prediction target block by weighted averaging the prediction image of the first subblock and the prediction image of the second subblock at a constant ratio. ing.

The selection unit 101H responds to the inter-prediction mode from the prediction image of the prediction target block output from the addition unit 101G and the prediction image of the prediction target block (when no subblock is used) output from the motion compensation prediction unit 101F3. It is configured to select and output a predicted image. The selection is determined by a control unit (not shown), and is signaled from the image coding device 100 to the image decoding device 200 as a motion compensation method.

An example of the operation of the image decoding device 200 according to the present embodiment will be described with reference to FIG.

As shown in FIG. 6, in step S101, the image decoding device 200 acquires the first subblock and the second subblock by dividing the prediction target block.

In step S102, the image decoding device 200 generates a motion vector corresponding to the first subblock, a motion vector corresponding to the second subblock, and a motion vector corresponding to the prediction target subblock.

In step S103, the image decoding device 200 generates a predicted image of the first subblock based on the motion vector corresponding to the first subblock, and generates a predicted image of the second subblock based on the motion vector corresponding to the second subblock. A prediction image is generated, and a prediction image (when no subblock is used) of the prediction target block is generated based on the motion vector corresponding to the prediction target block.

In step S104, the image decoding device 200 generates a predicted image of the prediction target block by weighted averaging the predicted image of the first subblock and the predicted image of the second subblock at a constant ratio.

In step S105, the image decoding device 200 selects a prediction image according to the inter-prediction mode from the prediction image of the prediction target block generated in step S104 and the prediction image of the prediction target block generated in step S103.

According to the image processing system 1 according to the present embodiment, the area of the reference pixel of the second subblock is not wider than the area of the reference pixel of the first subblock, so that each subblock does not increase the memory bandwidth. Prediction performance can be improved while maintaining the processing unit of.

(Second Embodiment)
Hereinafter, the image processing system 1 according to the second embodiment of the present invention will be described with reference to FIG. 7, focusing on the differences from the image processing system 1 according to the first embodiment described above.

In the present embodiment, the additional sub-block division unit 101B is configured to output a part of the second sub-block.

Specifically, the additional subblock division unit 101B outputs the second subblock only to a certain ratio (for example, half, etc.) or less of the number of the first subblocks normally acquired by the subblock division unit 101A. It is configured as follows.

For example, as shown in FIG. 7B, 16 first subblocks are normally generated by the subblock dividing unit 101A, and 9 additional subblocks 101B are generated by the additional subblock dividing unit 101B as shown in FIG. 7A. A second subblock is generated.

When the number of the second subblocks is half or less of the number of the first subblocks, the additional subblock division portion 101B is a shaded portion excluding a part of the second subblocks as shown in FIG. 7A. It is configured to output eight second subblocks.

Similarly, when 64 first subblocks are output by the normal block dividing unit 101A, the additional subblock dividing unit 101B outputs 49 second subblocks in the above-described first embodiment. In this embodiment, it may be configured to output 32 or less second subblocks.

For example, the additional sub-block dividing unit 101B is configured to output 25 second sub-blocks when the second sub-block is reduced so as to have a checkered pattern.

Alternatively, the additional subblock dividing unit 101B is configured to output the first n second subblocks of the arbitrarily ordered blocks. Here, n is an integer, and is "32" when the rate of increase is 50%.

According to the image processing system 1 according to the present embodiment, since the number of the second subblocks is suppressed to a certain ratio, the prediction accuracy is predicted even though the additional calculation cost increases by a certain ratio even in the worst case. Can be improved.

1 ... Image processing system 100 ... Image coding devices 101, 203 ... Inter-prediction unit 101A ... Normal sub-block division unit 101B ... Additional sub-block division 101C ... Motion vector generation units 101D1, 101D2 ... Affine motion vector generation units 101F1 to 101F3 ... Motion compensation prediction unit 102, 204 ... Intra-prediction unit 103 ... Conversion / quantization unit 104 ... Entropy coding unit 105, 202 ... Inverse conversion / inverse quantization unit 106 ... Subtraction unit 101G, 107, 205 ... Addition unit 108, 206 ... In-loop filters 101E, 109, 207 ... Frame buffer 200 ... Entropy decoding unit

Claims (6)

A subblock division unit configured to acquire the first subblock and the second subblock by dividing the prediction target block at different division positions, and
A motion vector generator configured to acquire motion vectors corresponding to the first subblock and the second subblock by affine transformation, and
A motion compensation prediction unit configured to generate a prediction image of the first subblock and the second subblock based on the motion vector.
An image decoding device comprising an interpolation unit configured to generate a predicted image of the predicted target block based on the predicted image of the sub-block. The sub-block division portion
A normal subblock division unit configured to acquire the first subblock by dividing the prediction target block at the first division position, and
It has an additional sub-block division unit configured to acquire a second sub-block by dividing the prediction target block at a second division position which is a position deviated by a predetermined pixel from the first division position. The image decoding apparatus according to claim 1. The image decoding apparatus according to claim 2, wherein the second division position is a position deviated from the first division position by half a pixel of the first subblock in the vertical direction and the horizontal direction. The image decoding device according to claim 2 or 3, wherein the additional sub-block dividing unit is configured to output a part of the second sub-block. Step A to acquire the first subblock and the second subblock by dividing the prediction target block at different division positions, and
Step B that outputs the motion vector obtained by the affine transformation corresponding to the first subblock and the second subblock, and
Step C of generating a predicted image of the first subblock and the second subblock based on the motion vector, and
An image decoding method comprising a step D of generating a predicted image of the predicted target block based on the predicted image of the first subblock and the predicted image of the second subblock. A program that makes a computer function as an image decoding device.
The image decoding device is
A subblock division unit configured to acquire the first subblock and the second subblock by dividing the prediction target block at different division positions, and
A motion vector generator configured to output a motion vector obtained by an affine transformation corresponding to the first subblock and the second subblock, and a motion vector generator.
A motion compensation prediction unit configured to generate a prediction image of the first subblock and the second subblock based on the motion vector.
A program characterized by having an interpolation unit configured to generate a predicted image of the predicted target block based on the predicted image of the first subblock and the predicted image of the second subblock.

Images