JP7544722B2 - Encoding device, decoding device, and program - Google Patents

Description

The present invention relates to an encoding device, a decoding device, and a program.

Video coding methods that perform coding processing on a block-by-block basis, such as the series of standards recommended by the Moving Picture Experts Group (MPEG) committee, have the characteristic that the quality between blocks is not uniform because coding is performed under different conditions for each block.

This feature has the advantage that it allows for coding control that does not cause visual degradation of image quality by controlling the coding according to the signal characteristics of partial regions of the image, for example by coding areas with signal characteristics that are easy for humans to detect into a high-quality signal and areas that are difficult to detect into a low-quality signal. On the other hand, because coding is controlled on a block-by-block basis, quality differences in image quality become apparent at block boundaries, and block-like distortion may be detected.

To reduce this type of signal degradation, deblocking filter processing is commonly used in recent coding methods. A deblocking filter is a filter that reduces signal gaps, so it generally has low-pass filter characteristics.

When a deblocking filter is applied in a fixed manner, the signal degradation caused by the filter process may have a greater effect than the quality improvement at the boundaries between high-quality regions, and conversely, the filter may not be effective enough at the boundaries between regions that have been coded at excessively low quality.

In view of this situation, in conventional video coding methods, the filter strength of the deblocking filter is controlled based on a quantization parameter (specifically, the average value of the quantization parameters of blocks on either side of a boundary) (see, for example, Non-Patent Document 1). Here, the quantization parameter refers to a parameter for which one value is set for one block and which specifies the quantization coarseness (step size) of the block.

Recommendation ITU-T H. 265, (12/2016), “High efficiency video coding”, International Telecommunication Union

In addition to the quantization parameters mentioned above, parameters that control the quality within the blocks into which an image is divided include quantization matrices (sometimes called scaling lists or scaling factors) that control the frequency characteristics within the blocks in order to control quality according to the intensity of visual perception.

A quantization matrix is a matrix consisting of values (weighting coefficients) that are set for each i x j element component depending on the block size, and is used to adjust the quantization coarseness for each component ranging from low to high frequencies of the transform coefficients.

However, while conventional video coding methods control the filter strength of the deblocking filter by taking into account the quantization parameter, which is a unique parameter that is set on a block-by-block basis, they do not take the quantization matrix into account when controlling the filter strength.

As a result, there is no way to control the filter strength of the deblocking filter to prevent block noise caused by the quantization matrix, and the effect of the deblocking filter in reducing block noise is insufficient.

Therefore, the present invention aims to provide an encoding device, a decoding device, and a program that improve image quality and encoding efficiency by appropriately controlling a deblocking filter.

The coding device according to the first aspect is a coding device that divides an image into blocks and codes the image in blocks, and includes a transform/quantization unit that performs a transform process and a quantization process on a prediction residual, which is a difference between a block to be coded obtained by dividing the image into blocks and a prediction block generated by predicting the block to be coded; an inverse quantization/inverse transform unit that restores the prediction residual by performing an inverse quantization process and an inverse transform process on the transform coefficient generated by the transform/quantization unit; a synthesis unit that restores the block to be coded by synthesizing the restored prediction residual and the prediction block; a deblocking filter that performs a filter process on the boundary between the restored block to be coded and an adjacent block adjacent to the block to be coded; and a filter control unit that controls the filter strength of the deblocking filter based on a quantization parameter and a quantization matrix used in the quantization process and the inverse quantization process, wherein the quantization parameter is a parameter for which one value is set for one block, and the quantization matrix is a matrix consisting of values set for each component in the one block.

The decoding device according to the second aspect is a decoding device that performs decoding on a block-by-block basis generated by dividing an image, and includes an entropy decoding unit that decodes an encoded stream to output transform coefficients corresponding to a block to be decoded, an inverse quantization/inverse transform unit that performs inverse quantization and inverse transform processing on the transform coefficients output by the entropy decoding unit to restore a prediction residual, a synthesis unit that synthesizes the restored prediction residual and a prediction block generated by predicting the block to be decoded to restore the block to be decoded, a deblocking filter that performs a filter process on the boundary between the restored block to be decoded and an adjacent block adjacent to the block to be decoded, and a filter control unit that controls the filter strength of the deblocking filter based on a quantization parameter and a quantization matrix used in the inverse quantization process, wherein the quantization parameter is a parameter for which one value is set for one block, and the quantization matrix is a matrix consisting of values set for each component in the one block.

The program of the third aspect has the purpose of causing a computer to function as an encoding device of the first aspect.

The program relating to the fourth aspect has the purpose of causing a computer to function as a decryption device relating to the second aspect.

The present invention provides an encoding device, a decoding device, and a program that improve image quality and encoding efficiency by appropriately controlling a deblocking filter.

FIG. 1 is a diagram illustrating a configuration of an encoding device according to an embodiment. FIG. 2 is a diagram illustrating an example of a quantization matrix. FIG. 11 is a diagram illustrating an example of the operation of a deblocking filter according to the embodiment. FIG. 2 is a diagram illustrating a configuration of a filter control unit of the encoding device according to the embodiment. FIG. 2 is a diagram illustrating a configuration of a decoding device according to an embodiment. FIG. 2 is a diagram illustrating a configuration of a filter control unit of a decoding device according to an embodiment. FIG. 4 is a diagram illustrating an example of an operation flow of a filter control unit according to the embodiment. FIG. 13 is a diagram illustrating a configuration of a filter control unit of an encoding device according to a modified example. FIG. 13 is a diagram illustrating a configuration of a filter control unit of a decoding device according to a modified example. FIG. 11 is a diagram illustrating an example of an operation flow of a filter control unit according to a modified example.

The encoding device and the decoding device according to the embodiment will be described with reference to the drawings. The encoding device and the decoding device according to the embodiment respectively encode and decode moving images as represented by MPEG. In the following description of the drawings, the same or similar parts are given the same or similar reference symbols.

<Structure of the Encoding Device>
First, the configuration of the encoding device according to this embodiment will be described. Fig. 1 is a diagram showing the configuration of an encoding device 1 according to this embodiment. The encoding device 1 is a device that performs encoding in units of blocks obtained by dividing an image.

As shown in FIG. 1, the encoding device 1 has a block division unit 100, a subtraction unit 110, a transformation/quantization unit 120, an entropy encoding unit 130, an inverse quantization/inverse transform unit 140, a synthesis unit 150, a deblocking filter 160, a filter control unit 161, a memory 170, and a prediction unit 180.

The block division unit 100 divides an input image of a frame (or picture) constituting a moving image into a plurality of image blocks, and outputs the image blocks obtained by division to the subtraction unit 110. The size of the image block is, for example, 32x32 pixels, 16x16 pixels, 8x8 pixels, or 4x4 pixels. The shape of the image block is not limited to a square, but may be a rectangle (non-square). The image block is the unit (block to be coded) that the coding device 1 codes, and is the unit (block to be coded) that the decoding device codes. Such an image block is sometimes called a CU (Coding Unit).

The block division unit 100 performs block division on the luminance signal and the color difference signal. In the following, the case where the shape of the block division is the same for the luminance signal and the color difference signal will be mainly described, but it may be possible to control the division independently for the luminance signal and the color difference signal. When there is no particular distinction between the luminance block and the color difference block, they are simply referred to as the block to be coded.

The subtraction unit 110 calculates a prediction residual that represents the difference (error) between the block to be coded output from the block division unit 100 and a predicted block obtained by predicting the block to be coded by the prediction unit 180. The subtraction unit 110 calculates the prediction residual by subtracting each pixel value of the predicted block from each pixel value of the block, and outputs the calculated prediction residual to the transformation/quantization unit 120.

The transform/quantization unit 120 performs transform processing and quantization processing on a block-by-block basis. The transform/quantization unit 120 has a transform unit 121 and a quantization unit 122.

The transform unit 121 performs a transform process on the prediction residuals output from the subtraction unit 110 to calculate transform coefficients for each frequency component, and outputs the calculated transform coefficients to the quantization unit 122. The transform process (transformation) refers to a process of converting a pixel domain signal into a frequency domain signal, such as discrete cosine transform (DCT), discrete sine transform (DST), Karhunen-Loeb transform (KLT), and transforms that convert these into integers. The transform process may also include a transform skip that adjusts the pixel domain signal by scaling or the like without converting it into a frequency domain signal.

The quantization unit 122 quantizes the transformation coefficients output from the transformation unit 121 using the quantization parameters and the quantization matrix, and outputs the quantized transformation coefficients to the entropy coding unit 130 and the inverse quantization/inverse transform unit 140. The quantization unit 122 also outputs information related to the quantization process (specifically, information on the quantization parameters and the quantization matrix used in the quantization process) to the entropy coding unit 130, the inverse quantization unit 141, and the filter control unit 161.

Here, the quantization parameter is a parameter for which one value is set for one block. Specifically, the quantization parameter is a parameter that is commonly applied to each transform coefficient in a block, and is a parameter that determines the coarseness (step size) of quantization.

A quantization matrix is a matrix of values that are set for each component in a block. Specifically, a quantization matrix is a matrix of values (weighting coefficients) that are set for each component of i x j elements according to the block size, and is used to adjust the quantization coarseness for each component ranging from low to high frequencies of the transform coefficients.

Figure 2 is a diagram showing an example of a quantization matrix. In Figure 2, an example where i x j = 4 x 4 is shown. In the example shown in Figure 2, the quantization matrix has larger values as the horizontal and vertical orders increase. In such a quantization matrix, the conversion coefficients associated with elements arranged further to the bottom right, i.e., the higher the frequency, the lower the quantization accuracy. Therefore, by utilizing the human visual characteristic that the lower the frequency, the more sensitive the spatial changes in shading and hue are, it is permissible to reduce the amount of information in the high frequency range by quantization without degrading the subjective image quality.

The quantization matrix can be set for each combination of the prediction mode of the block to be encoded (intra prediction or inter prediction), block size (e.g., 2x2, 4x4, 8x8, 16x16, 32x32, 64x64), and one luminance and two color difference signals (in the case of an RGB signal, the R signal, the B signal, and the G signal).

The entropy coding unit 130 performs entropy coding on the transform coefficients output from the quantization unit 122, compresses the data, generates a coded stream (bit stream), and outputs the coded stream to the outside of the coding device 1. For entropy coding, Huffman codes, CABAC (Context-based Adaptive Binary Arithmetic Coding), etc. can be used.

In addition, the entropy coding unit 130 obtains information such as the size and shape of each block to be coded from the block division unit 100, obtains information regarding the quantization process from the quantization unit 122, and obtains information regarding prediction (e.g., prediction mode and motion vector information) from the prediction unit 180, and also codes this information.

The inverse quantization and inverse transform unit 140 performs inverse quantization and inverse transform processing on a block-by-block basis. The inverse quantization and inverse transform unit 140 includes an inverse quantization unit 141 and an inverse transform unit 142.

The inverse quantization unit 141 performs an inverse quantization process corresponding to the quantization process performed by the quantization unit 122. Specifically, the inverse quantization unit 141 restores the transform coefficients by inverse quantizing the transform coefficients output from the quantization unit 122 using a quantization parameter and a quantization matrix, and outputs the restored transform coefficients to the inverse transform unit 142.

The inverse transform unit 142 performs an inverse transform process corresponding to the transform process performed by the transform unit 121. For example, if the transform unit 121 performs a DCT, the inverse transform unit 142 performs an inverse DCT. The inverse transform unit 142 performs an inverse transform process on the transform coefficients output from the inverse quantization unit 141 to restore the prediction residual, and outputs the restored prediction residual, which is the restored prediction residual, to the synthesis unit 150.

The synthesis unit 150 synthesizes the reconstructed prediction residual output from the inverse transform unit 142 with the predicted block output from the prediction unit 180 on a pixel-by-pixel basis. The synthesis unit 150 adds each pixel value of the reconstructed prediction residual to each pixel value of the predicted block to reconstruct (decode) the block to be coded, and outputs the reconstructed decoded image (reconstructed block) on a block-by-block basis to the deblocking filter 160.

The deblocking filter 160 performs filtering on the block boundary between two blocks consisting of a restored block and an adjacent block adjacent to the restored block, and outputs the filtered restored block to the memory 170. The filtering is a process for reducing signal degradation caused by block-by-block processing, and is a filtering process for smoothing signal gaps at the block boundary between two adjacent blocks. The deblocking filter 160 is generally configured as a low-pass filter that smoothes signal fluctuations.

Figure 3 is a diagram showing an example of the operation of the deblocking filter 160 according to this embodiment. In the example shown in Figure 3, the deblocking filter 160 performs filtering on block boundaries for each block of 8x8 pixels. The deblocking filter 160 also performs filtering in units of 4 rows or 4 columns. Blocks P and Q shown in Figure 3 are one unit of filtering by the deblocking filter 160, and show an example in which the block size is 4x4 pixels. Each of blocks P and Q may be referred to as a sub-block. Details of the operation of the deblocking filter 160 will be described later.

The filter control unit 161 controls the deblocking filter 160. Specifically, the filter control unit 161 controls the boundary strength (Bs) indicating whether or not to perform filter processing on the block boundary of the target block pair, and the filter strength of the deblocking filter 160. The boundary strength Bs refers to a parameter for determining whether or not to apply filter processing and the type of filter processing. Note that the control of whether or not to perform filter processing can be regarded as the control of whether to set the boundary strength Bs to 1 or more or to zero.

The filter control unit 161 controls the deblocking filter 160 based on the fluctuations in pixel values in the area near the boundary of the target block pair, the prediction mode, the quantization parameter, and the value of the motion vector used for motion compensation prediction (inter prediction).

Although details will be described later, the filter control unit 161 in this embodiment controls the filter strength of the deblocking filter 160 based on the quantization parameter and quantization matrix used in the quantization process in the quantization unit 120 and the inverse transformation process in the inverse quantization unit 141.

In this way, by controlling the filter strength of the deblocking filter taking into consideration not only the quantization parameter, which is a unique parameter set on a block-by-block basis, but also the quantization matrix, it becomes possible to control the filter strength of the deblocking filter 160 against the occurrence of block noise caused by the quantization matrix, thereby improving the effect of reducing block distortion by the deblocking filter 160.

The memory 170 accumulates the reconstructed blocks output from the deblocking filter 160 as decoded images on a frame-by-frame basis. The memory 170 outputs the stored decoded images to the prediction unit 180.

The prediction unit 180 performs prediction processing on a block-by-block basis to generate a prediction block corresponding to the block to be coded, and outputs the generated prediction block to the subtraction unit 110 and the synthesis unit 150. The prediction unit 180 has an inter prediction unit 181, an intra prediction unit 182, and a switching unit 183.

The inter prediction unit 181 uses the decoded image stored in the memory 170 as a reference image to calculate a motion vector by a method such as block matching, predicts the block to be coded to generate an inter prediction block, and outputs the generated inter prediction block to the switching unit 183. The inter prediction unit 181 selects an optimal inter prediction method from inter prediction using multiple reference images (typically bi-prediction) and inter prediction using one reference image (unidirectional prediction), and performs inter prediction using the selected inter prediction method. The inter prediction unit 181 outputs information related to the inter prediction (motion vector, etc.) to the entropy coding unit 130 and the filter control unit 161.

The intra prediction unit 182 selects an optimal intra prediction mode to be applied to the block to be coded from among multiple intra prediction modes, and predicts the block to be coded using the selected intra prediction mode. The intra prediction unit 182 generates an intra prediction block by referring to decoded pixel values adjacent to the block to be coded in the decoded image stored in the memory 170, and outputs the generated intra prediction block to the switching unit 183. The intra prediction unit 182 also outputs information related to the selected intra prediction mode to the entropy coding unit 130 and the filter control unit 161.

The switching unit 183 switches between the inter prediction block output from the inter prediction unit 181 and the intra prediction block output from the intra prediction unit 182, and outputs one of the prediction blocks to the subtraction unit 110 and the synthesis unit 150.

Thus, the encoding device 1 according to this embodiment has a transform/quantization unit 120 that performs transform processing and quantization processing on a prediction residual representing the difference between the block to be encoded and a prediction block obtained by predicting the block to be encoded, an inverse quantization/inverse transform unit 140 that performs inverse quantization processing and inverse transform processing on the transform coefficients obtained by the transform/quantization unit to restore the prediction residual, a synthesis unit 150 that synthesizes the restored prediction residual and the prediction block to restore the block to be encoded, a deblocking filter 160 that performs filtering processing on the boundary between the restored block to be encoded and an adjacent block adjacent to the block to be encoded, and a filter control unit 161 that controls the filter strength of the deblocking filter 160 based on the quantization parameter and quantization matrix used in the quantization processing and inverse quantization processing.

Next, the configuration of the filter control unit 161 according to this embodiment will be described. The filter control unit 161 according to this embodiment controls the filter strength of the deblocking filter 160 based on the quantization parameters and quantization matrix used in the quantization process and inverse quantization process for the block to be coded, and the quantization parameters and quantization matrix used in the quantization process and inverse quantization process for the adjacent block. In the following, it is assumed that the block to be coded or its sub-block is block P, and the adjacent block or its sub-block is block Q.

Figure 4 is a diagram showing the configuration of the filter control unit 161 according to this embodiment. As shown in Figure 4, the filter control unit 161 according to this embodiment has a boundary strength determination unit 161a, a representative value derivation unit 161b, an offset calculation unit 161c, a threshold derivation unit 161d, and a filter strength control unit 161e.

The boundary strength determination unit 161a determines the boundary strength Bs, for example, based on Table 1 below, and outputs the determined value of boundary strength Bs to the threshold derivation unit 161d and the filter strength control unit 161e. In this embodiment, the value of boundary strength Bs is set to 0, 1, or 2. Note that the boundary strengths for the luminance signal and color difference signal blocks may be calculated separately, or the combination of the boundary strengths for the luminance signal and color difference signal blocks may be determined as one boundary strength.

As shown in Table 1, the boundary strength determination unit 161a sets the value of Bs to 2 when intra prediction is applied to at least one of blocks P and Q.

On the other hand, the boundary strength determination unit 161a sets the Bs value to 1 when inter prediction is applied to both blocks P and Q and at least one of the following conditions (a) to (d) is satisfied, and sets the Bs value to 0 otherwise.

(a) at least one of blocks P and Q contains significant transform coefficients (i.e., non-zero transform coefficients);

(b) The number of motion vectors or reference images of blocks P and Q are different.

(c) The absolute value of the difference between the motion vectors of blocks P and Q is greater than or equal to a threshold value (e.g., one pixel).

When the determined boundary strength Bs value is 0, the boundary strength determination unit 161a controls the deblocking filter 160 so as not to perform filtering on the boundaries of blocks P and Q.

The representative value derivation unit 161b derives a representative value m _pp of the quantization matrix used in the quantization process and inverse quantization process for the block P, and also derives a representative value m _pq of the quantization matrix used in the quantization process and inverse quantization process for the block Q, and outputs the derived representative values m _pp and m _pq to the offset calculation unit 161c. Hereinafter, when there is no particular distinction between the representative values m _pp and m _pq , they will simply be referred to as the representative value m _p .

The representative value _mp derived by the representative value derivation unit 161b is any one of the following 1) to 3).

1) The representative value m _p is the average value of a plurality of values constituting the low-frequency components of the quantization matrix.

In the case where the orthogonally transformed prediction residual is quantized by a quantization matrix, m _p is derived by limiting it to the low-frequency component that most affects the block distortion. Specifically, the representative value derivation unit 161b determines the average value of m[j][i] that satisfies the low frequency, i<s, j<t (for example, s=t=4), as the representative value m _p , as shown in the following formula (1).

Generally, the closer i and j are to 0, the lower the weighting coefficient for the low frequency range. Taking into account the human visual characteristics, the lower the weighting coefficient, but the larger i and j are, the higher the weighting coefficient (see FIG. 2). In addition, generally, the closer i and j are to 0, the larger the conversion coefficient is, and the larger i and j are, the smaller the conversion coefficient is. For this reason, _mp is derived by limiting it to the low frequency components.

2) The representative value m _p is a single value that corresponds to the lowest range of the quantization matrix.

In the case of quantizing the orthogonally transformed prediction residuals using a quantization matrix, the representative value derivation unit 161b sets the lowest frequency coefficient m[0][0] that has the greatest impact on block distortion as the representative value m _p , as shown in the following equation (2).

m _p = m[0][0] (2)
This method considers only the impact on lowest-frequency blocking distortion, and has less impact on deblocking filter control than method 1) above. However, since it can be implemented by only substituting one coefficient, it is a very lightweight implementation method.

3) The representative value m _p is a weighted average value of a plurality of values constituting all the components of the quantization matrix.

The representative value derivation unit 161b derives the representative value m _p in consideration of the quantization matrix of all the coefficients of the quantization matrix. By using all the coefficients, cases in which many coefficients occur in the high frequency range can be effectively reflected in the deblocking filter control. Since a large weighting coefficient is generally used for the high frequency range in the quantization matrix (MxN), the value of the representative value m _p tends to become extremely large when averaging is performed including the wide frequency range coefficients. Therefore, the representative value derivation unit 161b takes into consideration the visual characteristics and averages the low frequency components by increasing the weighting of the quantization matrix as shown in the following formula (3).

Here, c[j][i] is a decimal value less than or equal to 1.0, and generally has a weighting approaching 0 in the high frequency range. The specific value is designed depending on the system.

The above-mentioned methods 1) and 3) may be combined. Specifically, the representative value derivation unit 161b may derive m _p limited to the low-frequency components of the quantization matrix, and perform a weighted average in which the weight of the lowest-frequency coefficient of the quantization matrix is increased.

The offset calculation unit 161c calculates offset values for each of the blocks P and Q based on the representative values m _pp and m _pq derived by the representative value derivation unit 161b.

Specifically, the offset calculation unit 161c uses the representative value _mpp to calculate an offset value QppOffset for the block P, and outputs the calculated offset value QppOffset to the threshold derivation unit 161d. For example, the offset calculation unit 161c calculates the offset value QppOffset by the following equation (4).

QppOffset=log ₂ (m _pp /m _d )×6 (4)
Here, _md is a value defined in the system as a default quantization matrix value. In this example, _md is set to 16.

Similarly, the offset calculation unit 161c uses the representative value m _pq to calculate an offset value QpqOffset for the block Q, and outputs the calculated offset value QpqOffset to the threshold derivation unit 161d. For example, the offset calculation unit 161c calculates the offset QpqOffset by the following equation (5).

QpqOffset=log ₂ (m _pq /m _d )×6 (5)
In equations (4) and (5), the quantization parameter Qp is a parameter that increases and decreases according to a logarithmic function, and the representative value m _p is divided by m _d to take its logarithm.

In addition, in the case of hardware implementation, the offset calculation unit 161c may implement the formulas (4) and (5) using a lookup table instead of division or log ₂ function processing.

For example, instead of processing equation (4), when the lookup table LUT is LUT[ ]={18, 21, 23, 26, 29, 32, 36, 41, 46, 51, 58, 64, 72, 81, 91, 102, 115, 128, 144, 162, 182, 204, 229, 256}, QppOffset may be calculated by the following processing.
if (m _{p p} < m _d )
{
sign = -1
M = _md × _md / _mpp
}
else
{
sign = 1
M = _mpp
}
Q = 0
while (M >= LUT[Q])
{
Q++
}
QppOffset = sign × Q

Similarly, instead of the process of equation (5), QpqOffset may be calculated by the following process.
if (m _pq < m _d )
{
sign = -1
M = _md × _md / _mpq
}
else
{
sign = 1
M = m _pq
}
Q = 0
while (M >= LUT[Q])
{
Q++
}
QpqOffset = sign × Q

Note that the above is one example of calculating QppOffset and QpqOffset using a lookup table, and is not limited to the above example as long as division by _md and logarithmic processing are calculated using an LUT based on representative values _mpp and _mqq of the quantization matrix.

The threshold derivation unit 161d derives thresholds β and tC for controlling the filter strength of the deblocking filter 160, based on the quantization parameter _QpP used in the quantization process and inverse quantization process for the block P, the quantization parameter _QpQ used in the quantization process and inverse quantization process for the block Q, and the offset values QppOffset and _QpqOffset calculated by the offset calculation unit 161c, and outputs the derived thresholds β and _tC to the filter strength control unit 161e.

First, the threshold derivation unit 161d calculates the variable qP, for example, using the following equation (6).

qP=((Qp _Q +QpqOffset+Qp _P +QppOffset+1)>>1) (6)
Here, “>>” indicates a shift operator (right shift operation). Equation (6) is basically a quantization parameter Qp _P of block P and a quantization parameter Qp _Q of block Q. The formula for calculating the average includes offset values QppOffset and QpqOffset based on the quantization matrix. Note that the formula (6) is merely an example. Other calculation formulas may be used as long as they reflect the offset values QppOffset and QpqOffset based on the matrix.

Secondly, the threshold derivation unit 161d calculates Q for deriving threshold β, for example, using the following formula (7).

Q=Clip3(0,63,qP+(slice_beta_offset_div2<<1)) (7)
Here, "<<" indicates a shift operator (left shift operation). "Clip3(X,Y,Z)" returns X if Z is less than X, and returns Y if Z is greater than Y. otherwise, it returns Z. slice_beta_offset_div2 is one of the parameters signaled to the decoding device 2.

Furthermore, the threshold derivation unit 161d calculates Q for deriving the threshold _tC , for example, by the following formula (8).

Q=Clip3(0,65,qP+2*(bS?1)+(slice_tc_offset_div2<<1)) (8)
Here, “bS” is the Bs value output by the boundary strength determining unit 161 a. slice_tc_offset_div2 is one of the parameters signaled to the decoding device 2.

Thirdly, the threshold derivation unit 161d derives thresholds β and _tC from the calculated Q according to Table 2 below.

The filter strength control unit 161e controls the filter strength of the deblocking filter 160 based on the value of the boundary strength Bs output by the boundary strength determination unit 161a and the thresholds β and _tC output by the threshold derivation unit 161d.

When the value of the boundary strength Bs is 1 or 2, the filter strength control unit 161e may control the deblocking filter 160 to perform filtering only when the following equation (9) is satisfied (see Figure 3).

In addition, when performing filter processing, the filter strength control unit 161e may apply a strong filter when all of the following conditional expressions (10) to (15) are satisfied, and apply a weak filter in other cases (see Figure 3).

In this way, the filter control unit 161 according to this embodiment controls the filter strength of the deblocking filter 160 taking into consideration not only the quantization parameter, which is a unique parameter set for each block, but also the quantization matrix. This makes it possible to control the filter strength of the deblocking filter 160 in response to the occurrence of block noise caused by the quantization matrix, thereby improving the effect of reducing block distortion by the deblocking filter 160.

<Configuration of Decoding Device>
Next, the configuration of a decoding device according to this embodiment will be described, focusing mainly on the differences from the configuration of the encoding device described above. Fig. 5 is a diagram showing the configuration of a decoding device 2 according to this embodiment. The decoding device 2 is a device that decodes a current block from an encoded stream.

As shown in FIG. 5, the decoding device 2 has an entropy decoding unit 200, an inverse quantization/inverse transform unit 210, a synthesis unit 220, a deblocking filter 230, a filter control unit 231, a memory 240, and a prediction unit 250.

The entropy decoding unit 200 decodes the encoded stream generated by the encoding device 1 and decodes various signaling information. Specifically, the entropy decoding unit 200 acquires information related to the quantization process applied to the block to be decoded, and outputs the acquired information to the inverse quantization unit 211 and the filter control unit 231. The entropy decoding unit 200 also acquires information related to the prediction applied to the block to be decoded (e.g., prediction type information, motion vector information), and outputs the acquired information to the prediction unit 250 and the filter control unit 231.

In addition, the entropy decoding unit 200 decodes the encoded stream, obtains quantized transform coefficients, and outputs the obtained transform coefficients to the inverse quantization/inverse transform unit 210 (inverse quantization unit 211).

The inverse quantization and inverse transform unit 210 performs inverse quantization and inverse transform processing on a block-by-block basis. The inverse quantization and inverse transform unit 210 has an inverse quantization unit 211 and an inverse transform unit 212.

The inverse quantization unit 211 performs an inverse quantization process corresponding to the quantization process performed by the quantization unit 122 of the encoding device 1. The inverse quantization unit 211 restores the transform coefficients of the block to be decoded by inverse quantizing the quantized transform coefficients output from the entropy decoding unit 200 using a quantization parameter and a quantization matrix, and outputs the restored transform coefficients to the inverse transform unit 212.

The inverse transform unit 212 performs an inverse transform process corresponding to the transform process performed by the transform unit 121 of the encoding device 1. The inverse transform unit 212 performs an inverse transform process on the transform coefficients output from the inverse quantization unit 211 to restore the prediction residual, and outputs the restored prediction residual (restored prediction residual) to the synthesis unit 220.

The synthesis unit 220 reconstructs (decodes) the block to be decoded by synthesizing the prediction residual output from the inverse transform unit 212 and the prediction block output from the prediction unit 250 on a pixel-by-pixel basis, and outputs the reconstructed block to the deblocking filter 230.

The deblocking filter 230 performs an operation similar to that of the deblocking filter 160 of the encoding device 1. The deblocking filter 230 performs a filter process on the boundary of a block pair (blocks P and Q) consisting of a reconstructed block output from the synthesis unit 220 and a block adjacent to the reconstructed block, and outputs the reconstructed block after the filter process to the memory 240.

The filter control unit 231 performs operations similar to those of the filter control unit 161 of the encoding device 1 based on the information output from the entropy decoding unit 200. The filter control unit 231 selects the boundary strength Bs, for example, by the method shown in Table 1, and controls the filter strength of the deblocking filter 160 according to equations (1) to (15).

The memory 240 stores the reconstructed blocks output from the deblocking filter 230 as decoded images on a frame-by-frame basis. The memory 240 outputs the decoded images on a frame-by-frame basis to the outside of the decoding device 2.

The prediction unit 250 performs prediction on a block-by-block basis. The prediction unit 250 has an inter prediction unit 251, an intra prediction unit 252, and a switching unit 253.

The inter prediction unit 251 predicts the block to be decoded by inter prediction using the decoded image stored in the memory 240 as a reference image. The inter prediction unit 251 generates an inter prediction block by performing inter prediction using the motion vector information output from the entropy decoding unit 200, and outputs the generated inter prediction block to the switching unit 253.

The intra prediction unit 252 refers to reference pixels adjacent to the block to be decoded in the decoded image stored in the memory 240, and predicts the block to be decoded by intra prediction based on the information output from the entropy decoding unit 200. The intra prediction unit 252 then generates an intra prediction block and outputs the generated intra prediction block to the switching unit 253.

The switching unit 253 switches between the inter prediction block output from the inter prediction unit 251 and the intra prediction block output from the intra prediction unit 252, and outputs one of the prediction blocks to the synthesis unit 220.

Thus, the decoding device 2 according to this embodiment includes an entropy decoding unit 200 that outputs transform coefficients corresponding to a block to be decoded by decoding an encoded stream, an inverse quantization/inverse transform unit 210 that performs inverse quantization and inverse transform processing on the transform coefficients output by the entropy decoding unit 200 to restore a prediction residual, a synthesis unit 220 that synthesizes the restored prediction residual and a prediction block obtained by predicting the block to be decoded to restore the block to be decoded, a deblocking filter 230 that performs filtering processing on the boundary between the restored block to be decoded and an adjacent block adjacent to the block to be decoded, and a filter control unit 231 that controls the filter strength of the deblocking filter 230 based on the quantization parameter and quantization matrix used in the inverse quantization processing.

Next, the configuration of the filter control unit 231 according to this embodiment will be described. FIG. 6 is a diagram showing the configuration of the filter control unit 231 according to this embodiment. As shown in FIG. 6, the filter control unit 231 according to this embodiment has a boundary strength determination unit 231a, a representative value derivation unit 231b, an offset calculation unit 231c, a threshold derivation unit 231d, and a filter strength control unit 231e.

The boundary strength determination unit 231a, the representative value derivation unit 231b, the offset calculation unit 231c, the threshold derivation unit 231d, and the filter strength control unit 231e perform operations similar to those of the boundary strength determination unit 161a, the representative value derivation unit 161b, the offset calculation unit 161c, the threshold derivation unit 161d, and the filter strength control unit 161e of the encoding device 1, respectively.

<Example of operation of filter control section>
Next, an example of the operation of the filter control unit 161 and the filter control unit 231 according to this embodiment will be described. Since the filter control unit 161 and the filter control unit 231 perform the same operation, the filter control unit 231 will be taken as an example for description here. Fig. 7 is a diagram showing an example of the operation flow of the filter control unit 231 according to this embodiment.

As shown in FIG. 7, in step S1, the boundary strength determination unit 231a determines the boundary strength Bs using the method shown in Table 1, and outputs the determined value of boundary strength Bs to the threshold derivation unit 231d and the filter strength control unit 231e. If the determined value of boundary strength Bs is 0, the processing from step S2 onwards is not performed, and filter processing is not performed on the boundaries of blocks P and Q.

In step S2, the representative value derivation unit 231b derives a representative value m _pp of the quantization matrix used in the inverse quantization process for block P, and derives a representative value m _pq of the quantization matrix used in the inverse quantization process for block Q, and outputs the derived representative values m _pp and m _pq to the offset calculation unit 231c. As the representative value m _p derived by the representative value derivation unit 231b, any of 1) to 3) described above is used.

In step S3, the offset calculation unit 231c calculates offset values for each of blocks P and Q based on the representative values m _pp and m _pq derived by the representative value derivation unit 231b. Specifically, the offset calculation unit 231c uses the representative value m _pp to calculate an offset value QppOffset for block P according to the above-mentioned equation (4), and outputs the calculated offset value QppOffset to the threshold derivation unit 231d. Similarly, the offset calculation unit 231c uses the representative value m _pq to calculate an offset value QpqOffset for block Q according to the above-mentioned equation (5), and outputs the calculated offset value QpqOffset to the threshold derivation unit 231d.

In step S4, the threshold derivation unit 231d calculates the variable qP by the above-mentioned equation (6) based on the quantization parameter _QpP used in the inverse quantization process for the block P, the quantization parameter _QpQ used in the inverse quantization process for the block Q, and the offset values QppOffset and QpqOffset calculated by the offset calculation unit 231c.

In step S5, the threshold derivation unit 231d calculates the variable Q by the above-mentioned formulas (7) and (8) based on the variable qP, and derives the thresholds β and _tC from the calculated variable Q according to Table 2.

In step S6, the filter strength control unit 231e controls the filter strength of the deblocking filter 230 based on the value of the boundary strength Bs output by the boundary strength determination unit 231a and the thresholds β and _tC output by the threshold derivation unit 231d.

In this way, the filter control unit 231 according to this embodiment controls the filter strength of the deblocking filter 230 taking into consideration not only the quantization parameter, which is a unique parameter set for each block, but also the quantization matrix. This makes it possible to control the filter strength of the deblocking filter 230 in response to the occurrence of block noise caused by the quantization matrix, thereby improving the effect of reducing block distortion by the deblocking filter 230.

<Example of change>
Next, a modified example of the embodiment will be described, focusing mainly on the differences from the above-described embodiment.

First, the encoding device 1 according to this modified example will be described. In the encoding device 1 according to this modified example, the entropy encoding unit 130 transmits an array (hereinafter referred to as the "offset array") consisting of offset values defined for each of a plurality of quantization matrix candidates to the decoding device, including the array in the encoding stream. Specifically, the offset array stores offset values for each quantization matrix set according to a combination of the prediction mode, block size, and signal type (one luminance and two chrominance signals) of the block to be encoded. The offset values are used to control the filter strength in the deblocking filter process, as will be described in detail later.

Figure 8 is a diagram showing the configuration of the filter control unit 161 according to this modified example. As shown in Figure 8, the filter control unit 161 according to this modified example has a boundary strength determination unit 1161a, an offset derivation unit 1161b, a threshold derivation unit 1161c, and a filter strength control unit 1161d.

The boundary strength determination unit 1161a determines the boundary strength Bs, for example, based on Table 1 above, and outputs the determined value of boundary strength Bs to the threshold derivation unit 1161c and the filter strength control unit 1161d. In this modified example, the value of boundary strength Bs is set to 0, 1, or 2. Note that the boundary strengths for the luminance signal and color difference signal blocks may be calculated separately, or the combination of the boundary strengths of the luminance signal and color difference signal blocks may be determined as one boundary strength.

As shown in Table 1, the boundary strength determination unit 1161a sets the value of Bs to 2 when intra prediction is applied to at least one of blocks P and Q.

On the other hand, the boundary strength determination unit 1161a sets the Bs value to 1 when inter prediction is applied to both blocks P and Q and at least one of the following conditions (a) to (d) is satisfied, and sets the Bs value to 0 otherwise.

(a) at least one of blocks P and Q contains significant transform coefficients (i.e., non-zero transform coefficients);

(b) The number of motion vectors or reference images of blocks P and Q are different.

(c) The absolute value of the difference between the motion vectors of blocks P and Q is greater than or equal to a threshold value (e.g., one pixel).

When the determined boundary strength Bs value is 0, the boundary strength determination unit 1161a controls the deblocking filter 160 so as not to perform filtering on the boundaries of blocks P and Q.

Based on the above-mentioned offset array, the offset derivation unit 1161b derives an offset value QppOffset corresponding to the quantization matrix used in the quantization process and inverse quantization process for block P, and an offset value QpqOffset corresponding to the quantization matrix used in the quantization process and inverse quantization process for block Q.

For example, when the number of the quantization matrix used in the quantization process and the inverse quantization process for the block P is k, the offset derivation unit 1161b obtains an offset value m[k] in the offset array, and sets the obtained offset value m[k] as QppOffset. Here, k may be an index value indicating a block type determined according to a combination of a prediction mode, a block size, and a signal type. The offset derivation unit 1161b may convert the offset value m[k] according to the following formula (16) to obtain QppOffset.

QppOffset=log ₂ (m _pp /m _d )×6 (16)
Here, _md is a value defined in the system as the default quantization matrix value. In this example, _md is set to 16.

Similarly, when the number of the quantization matrix used in the quantization process and the inverse quantization process for block Q is l, the offset derivation unit 1161b obtains an offset value m[l] in the offset array, and sets the obtained offset value m[l] as QpqOffset. Here, l may be an index value indicating a block type determined according to a combination of a prediction mode, a block size, and a signal type. The offset derivation unit 1161b may convert the offset value m[l] according to the following formula (17) to obtain QpqOffset.

QpqOffset=log ₂ (m _pq /m _d )×6 (17)
In equations (16) and (17), the quantization parameter Qp is a parameter that increases and decreases according to a logarithmic function, and the representative value m _p is divided by m _d to take the logarithm.

In addition, in the case of hardware implementation, the offset derivation unit 1161b may implement equations (16) and (17) using a lookup table instead of division or log ₂ function processing.

For example, instead of processing equation (16), when the lookup table LUT is LUT[ ]={18, 21, 23, 26, 29, 32, 36, 41, 46, 51, 58, 64, 72, 81, 91, 102, 115, 128, 144, 162, 182, 204, 229, 256}, QppOffset may be calculated by the following processing.
if (m _{p p} < m _d )
[
sign = -1
M = _md × _md / _mpp
]
else
[
sign = 1
M = _mpp
]
Q = 0
while (M >= LUT[Q])
[
Q++
]
QppOffset = sign × Q

Similarly, instead of the process of equation (17), QpqOffset may be calculated by the following process.
if (m _pq < m _d )
[
sign = -1
M = _md × _md / _mpq
]
else
[
sign = 1
M = m _pq
]
Q = 0
while (M >= LUT[Q])
[
Q++
]
QpqOffset = sign × Q

Note that the above is an example of calculating QppOffset and QpqOffset using a lookup table, and is not limited to the above example.

The threshold derivation unit 1161c derives thresholds β and tC for controlling the filter strength of the deblocking filter 160 based on the quantization parameter QpP used in the quantization process and inverse quantization process for the block _P , the quantization parameter _QpQ used in the quantization process and inverse quantization process for the block Q, and the offset values QppOffset and _QpqOffset derived by the offset derivation unit 1161b, and outputs the derived thresholds β and _tC to the filter strength control unit 1161d.

First, the threshold derivation unit 1161c calculates the variable qP, for example, using the following equation (18).

qP=((Qp _Q +QpqOffset+Qp _P +QppOffset+1)>>1) (18)
Here, “>>” indicates a shift operator (right shift operation). Equation (18) basically expresses the relationship between the quantization parameter Qp _P of block P and the quantization parameter Qp _Q of block Q. The formula for calculating the average includes offset values QppOffset and QpqOffset based on the quantization matrix. Note that the formula (18) is merely an example. Other calculation formulas may be used as long as they reflect the offset values QppOffset and QpqOffset based on the matrix.

Secondly, the threshold derivation unit 1161c calculates Q for deriving threshold β, for example, using the following formula (19).

Q=Clip3(0,63,qP+(slice_beta_offset_div2<<1)) (19)
Here, "<<" indicates a shift operator (left shift operation). "Clip3(X,Y,Z)" returns X if Z is less than X, and returns Y if Z is greater than Y. otherwise, it returns Z. slice_beta_offset_div2 is one of the parameters signaled to the decoding device 2.

Furthermore, the threshold derivation unit 1161c calculates Q for deriving the threshold t _C , for example, by the following equation (20).

Q=Clip3(0,65,qP+2*(bS-1)+(slice_tc_offset_div2<<1)) (20)
Here, “bS” is the Bs value output by the boundary strength determining unit 1161 a. slice_tc_offset_div2 is one of the parameters signaled to the decoding device 2.

Thirdly, the threshold derivation unit 1161c derives thresholds β and t _C from the calculated Q according to Table 2 above.

The filter strength control unit 1161d controls the filter strength of the deblocking filter 160 based on the value of the boundary strength Bs output by the boundary strength determination unit 1161a and the thresholds β and t _C output by the threshold derivation unit 1161c.

When the value of the boundary strength Bs is 1 or 2, the filter strength control unit 1161d may control the deblocking filter 160 to perform filtering only when the above equation (9) is satisfied (see Figure 3).

In addition, when performing filter processing, the filter strength control unit 1161d may apply a strong filter when all of the above conditional expressions (10) to (15) are satisfied, and apply a weak filter in other cases (see Figure 3).

In this way, the filter control unit 161 according to this modified example controls the filter strength of the deblocking filter 160 taking into consideration not only the quantization parameter, which is a unique parameter set for each block, but also the quantization matrix. This makes it possible to control the filter strength of the deblocking filter 160 in response to the occurrence of block noise caused by the quantization matrix, thereby improving the effect of reducing block distortion by the deblocking filter 160.

Next, we will explain the decoding device 2 related to this modified example.

In the decoding device 2 according to this modified example, the entropy decoding unit 200 decodes the encoded stream generated by the encoding device 1 and decodes various signaling information. Specifically, the entropy decoding unit 200 acquires information related to the quantization process applied to the block to be decoded, and outputs the acquired information to the inverse quantization unit 211 and the filter control unit 231. The entropy decoding unit 200 also acquires information related to the prediction applied to the block to be decoded (e.g., prediction type information, motion vector information), and outputs the acquired information to the prediction unit 250 and the filter control unit 231. Furthermore, the entropy decoding unit 200 acquires an offset array, and outputs the acquired offset array to the filter control unit 231.

In this way, the decoding device 2 according to this modified example has an entropy decoding unit 200 that decodes the encoded stream and outputs an array consisting of offset values specified for each of a plurality of quantization matrix candidates and a transform coefficient corresponding to the block to be decoded, an inverse quantization/inverse transform unit 210 that performs inverse quantization processing and inverse transform processing on the transform coefficients output by the entropy decoding unit 200 to restore a prediction residual, a synthesis unit 220 that synthesizes the restored prediction residual and a prediction block obtained by predicting the block to be decoded to restore the block to be decoded, a deblocking filter 230 that performs a filter processing on the boundary between the restored block to be decoded and an adjacent block adjacent to the block to be decoded, and a filter control unit 231 that controls the filter strength of the deblocking filter 230 based on the quantization parameter and quantization matrix used in the inverse quantization processing and the offset array.

Next, the configuration of the filter control unit 231 according to this modified example will be described. FIG. 9 is a diagram showing the configuration of the filter control unit 231 according to this modified example. As shown in FIG. 9, the filter control unit 231 according to this modified example has a boundary strength determination unit 2231a, an offset derivation unit 2231b, a threshold derivation unit 2231c, and a filter strength control unit 2231d.

The boundary strength determination unit 2231a, the offset derivation unit 2231b, the threshold derivation unit 2231c, and the filter strength control unit 2231d perform operations similar to those of the boundary strength determination unit 1161a, the offset derivation unit 1161b, the threshold derivation unit 1161c, and the filter strength control unit 1161d of the encoding device 1, respectively.

Next, an example of the operation of the filter control unit 161 and the filter control unit 231 according to this modified example will be described. Since the filter control unit 161 and the filter control unit 231 perform the same operation, the filter control unit 231 will be described as an example here. Figure 10 is a diagram showing an example of the operation flow of the filter control unit 231 according to this modified example.

As shown in FIG. 10, in step S11, the boundary strength determination unit 2231a determines the boundary strength Bs using the method shown in Table 1, and outputs the determined value of boundary strength Bs to the threshold derivation unit 2231c and the filter strength control unit 2231d. If the determined value of boundary strength Bs is 0, the processing from step S12 onwards is not performed, and filter processing is not performed on the boundaries of blocks P and Q.

In step S12, the offset derivation unit 2231b calculates an offset value QppOffset corresponding to the quantization matrix used in the inverse quantization process for block P based on the offset array transmitted from the encoding device 1, and calculates an offset value QpqOffset corresponding to the quantization matrix used in the inverse quantization process for block Q, and outputs the offset values QppOffset and QpqOffset calculated by the offset derivation unit 2231b to the threshold derivation unit 2231c.

In step S13, the threshold derivation unit 2231c calculates the variable qP using the above-mentioned equation (18) based on the quantization parameter Qp _P used in the inverse quantization process for block P, the quantization parameter Qp _Q used in the inverse quantization process for block Q, and the offset values QppOffset and QpqOffset derived by the offset derivation unit 2231b.

In step S14, the threshold derivation unit 2231c calculates the variable Q by the above-mentioned equations (19) and (20) based on the variable qP, and derives the thresholds β and t _C from the calculated variable Q according to Table 2.

In step S15, the filter strength control unit 2231d controls the filter strength of the deblocking filter 230 based on the value of the boundary strength Bs output by the boundary strength determination unit 2231a and the thresholds β and t _C output by the threshold derivation unit 2231c.

In this way, the filter control unit 231 according to this modified example controls the filter strength of the deblocking filter 230 taking into consideration not only the quantization parameter, which is a unique parameter set for each block, but also the quantization matrix. This makes it possible to control the filter strength of the deblocking filter 230 in response to the occurrence of block noise caused by the quantization matrix, thereby improving the effect of reducing block distortion by the deblocking filter 230.

<Other embodiments>
In the encoding device 1 and decoding device 2 according to the above-described embodiment, the filter control units 161 and 231 are configured to calculate offset values QppOffset and QpqOffset for each target block pair using a representative value of the quantization matrix applied to block P and a representative value of the quantization matrix applied to block Q.

However, to simplify processing, offset values may be calculated and stored in memory for the quantization matrix set for each combination of prediction mode (intra prediction or inter prediction), block size (e.g., 2x2, 4x4, 8x8, 16x16, 32x32, 64x64), and one luminance and two chrominance signals (in the case of RGB signals, the R signal, the B signal, and the G signal), and the offset values may be called from memory based on the prediction mode and block size, whether each block of the target block pair is a luminance signal or two chrominance signals.

A program may be provided that causes a computer to execute each process performed by the encoding device 1. A program may be provided that causes a computer to execute each process performed by the decoding device 2. The program may be recorded on a computer-readable medium. Using the computer-readable medium, it is possible to install the program on a computer. Here, the computer-readable medium on which the program is recorded may be a non-transient recording medium. The non-transient recording medium is not particularly limited, and may be, for example, a recording medium such as a CD-ROM or a DVD-ROM.

The circuits that execute the processes performed by the encoding device 1 may be integrated, and the encoding device 1 may be configured as a semiconductor integrated circuit (chip set, SoC). The circuits that execute the processes performed by the decoding device 2 may be integrated, and the decoding device 2 may be configured as a semiconductor integrated circuit (chip set, SoC).

The above describes the embodiments in detail with reference to the drawings, but the specific configuration is not limited to that described above, and various design changes can be made without departing from the spirit of the invention.

This application claims priority to Japanese Patent Application No. 2019-159971 (filed September 2, 2019) and Japanese Patent Application No. 2019-160251 (filed September 3, 2019), the entire contents of which are incorporated herein by reference.

Claims (15)

An encoding device that divides an image into blocks and encodes the image in blocks, comprising:
a transform/quantization unit that performs a transform process and a quantization process on a prediction residual that is a difference between a current block to be coded, which is obtained by dividing the image into blocks, and a predicted block generated by predicting the current block to be coded;
an inverse quantization and inverse transform unit that performs inverse quantization and inverse transform processing on the transform coefficients generated by the transform and quantization unit to restore the prediction residual;
a synthesis unit that synthesizes the reconstructed prediction residual and the prediction block to reconstruct the encoding target block;
a deblocking filter that performs filtering on a boundary between the restored encoding target block and an adjacent block adjacent to the encoding target block;
A filter control unit that controls a filter strength of the deblocking filter based on a quantization parameter and a quantization matrix used in the quantization process and the inverse quantization process,
the quantization parameter is a parameter in which one value is set for one block, and the quantization matrix is a matrix of values set for each component in the one block;
the filter control unit has a representative value derivation unit that derives a representative value of the quantization matrix,
the representative value is a weighted average of a plurality of values constituting only low-frequency components of the quantization matrix, and in the weighting, a weight of one value corresponding to a lowest frequency component of the quantization matrix among the plurality of values is greater than weights of other values;
The encoding device , wherein the filter control unit controls the filter strength based on the representative value . The encoding device according to claim 1, characterized in that the filter control unit controls the filter strength based on the quantization parameter and the quantization matrix used in the quantization process and the inverse quantization process for the encoding target block, and the quantization parameter and the quantization matrix used in the quantization process and the inverse quantization process for the adjacent block. The representative value derivation unit
The encoding device according to claim 2, characterized in that the representative value of the quantization matrix used in the quantization process and the inverse quantization process for the block to be encoded and the representative value of the quantization matrix used in the quantization process and the inverse quantization process for the adjacent block are derived. The filter control unit includes:
an offset calculation unit that calculates an offset value for each of the encoding target block and the neighboring blocks based on the representative value derived by the representative value derivation unit;
the encoding device according to claim 3, further comprising a threshold derivation unit that derives a threshold for controlling the filter strength based on the quantization parameter used in the quantization process and the inverse quantization process for the block to be encoded, the quantization parameter used in the quantization process and the inverse quantization process for the adjacent block, and the offset value calculated by the offset calculation unit. an entropy coding unit that codes the transform coefficients generated by the transform/quantization unit and outputs a coded stream;
the filter control unit controls a filter strength of the deblocking filter based on the quantization parameter and the quantization matrix used in the quantization process and the inverse quantization process, and an array of offset values defined for each of a plurality of quantization matrix candidates;
The encoding device according to claim 1 , wherein the entropy encoding unit transmits the sequence by including it in the encoded stream. The encoding device according to claim 5, characterized in that the filter control unit controls the filter strength based on the quantization parameter and the quantization matrix used in the quantization process and the inverse quantization process for the block to be encoded, and the quantization parameter and the quantization matrix used in the quantization process and the inverse quantization process for the adjacent block. The filter control unit includes:
The encoding device according to claim 6, further comprising an offset derivation unit that derives, based on the array, an offset value corresponding to the quantization matrix used in the quantization process and the inverse quantization process for the block to be encoded, and an offset value corresponding to the quantization matrix used in the quantization process and the inverse quantization process for the adjacent block. The filter control unit includes:
The encoding device according to claim 7, further comprising a threshold derivation unit that derives a threshold for controlling the filter strength based on the quantization parameter used in the quantization process and the inverse quantization process for the block to be encoded, the quantization parameter used in the quantization process and the inverse quantization process for the adjacent block, and the offset value derived by the offset derivation unit. A decoding device that performs decoding on a block-by-block basis by dividing an image,
an entropy decoding unit that decodes the encoded stream to output a transform coefficient corresponding to a block to be decoded;
an inverse quantization and inverse transform unit that performs inverse quantization processing and inverse transform processing on the transform coefficients output by the entropy decoding unit to restore a prediction residual;
a synthesis unit that synthesizes the reconstructed prediction residual and a prediction block generated by predicting the block to be decoded to reconstruct the block to be decoded;
a deblocking filter that performs a filtering process on a boundary between the restored decoding target block and an adjacent block adjacent to the decoding target block;
A filter control unit that controls a filter strength of the deblocking filter based on a quantization parameter and a quantization matrix used in the inverse quantization process,
the quantization parameter is a parameter in which one value is set for one block, and the quantization matrix is a matrix of values set for each component in the one block;
the filter control unit controls the filter strength based on a representative value of the quantization matrix;
A decoding device characterized in that the representative value is a weighted average of multiple values that constitute only the low-frequency components of the quantization matrix, and in the weighting, the weight of one value among the multiple values that corresponds to the lowest frequency component of the quantization matrix is greater than the weight of the other values . the entropy decoding unit further outputs an array including offset values defined for each of a plurality of quantization matrix candidates by decoding the encoded stream;
10. The decoding device according to claim 9, wherein the filter control unit controls a filter strength of the deblocking filter based on the quantization parameter and the quantization matrix used in the inverse quantization process, and the array. The decoding device according to claim 10, characterized in that the filter control unit controls the filter strength based on the quantization parameter and the quantization matrix used in the inverse quantization process for the block to be decoded and the quantization parameter and the quantization matrix used in the inverse quantization process for the adjacent block. The filter control unit includes:
The decoding device according to claim 11, further comprising an offset derivation unit that derives, based on the array, an offset value corresponding to the quantization matrix used in the inverse quantization process for the block to be decoded and an offset value corresponding to the quantization matrix used in the inverse quantization process for the adjacent block. The filter control unit includes:
The decoding device according to claim 12, further comprising a threshold derivation unit that derives a threshold for controlling the filter strength based on the quantization parameter used in the inverse quantization process for the block to be decoded, the quantization parameter used in the inverse quantization process for the adjacent block, and the offset value derived by the offset derivation unit. A program that causes a computer to function as the encoding device described in claim 1. A program for causing a computer to function as the decoding device according to claim 9 .

Images