JP4793424B2 - Image encoding apparatus, image encoding method, image decoding apparatus, image decoding method, and communication apparatus - Google Patents

Description

  The present invention relates to an image code that predicts an image to be encoded or an image to be decoded from an existing image and encodes or decodes a prediction error by addition with the prediction error in high-efficiency encoding or decoding of an image. The present invention relates to an encoding device, an image encoding method, an image decoding device, an image decoding method, and a communication device including at least one of these image encoding device and image decoding device.

  Conventionally, standard video coding schemes such as MPEG and ITU-T H.26x are called macroblocks, which are framed into square blocks composed of 16 × 16 pixels of luminance signals (+ 8 × 8 pixels of corresponding color difference signals). The screen is divided, the motion from the reference frame is estimated by motion compensated prediction in that unit, and the signal for the estimation error (prediction residual signal) and the motion vector information are encoded. In MPEG-2, the macroblock is divided into two field areas and motion prediction is performed for each field. In H.263 and MPEG-4, the macroblock is further divided into four 8 × 8 pixel block sizes. A technique for performing motion prediction in units of sub-blocks has been introduced. In particular, adaptation of the motion prediction block size in MPEG-4 increases the coding amount of the motion vector, but improves the followability to more intense and fine motion, and can improve performance by selecting an appropriate mode. It is known.

  Another technical aspect of motion compensation prediction is the accuracy of motion vectors. Originally, because it is digital image data, there is only discrete pixel information (hereinafter referred to as integer pixels) generated by sampling, but virtual samples are created by interpolation between integer pixels, and this is predicted image The technology used as is widely used. This technology is known to have two effects: improvement of prediction accuracy by increasing the number of candidate points for prediction, and improvement of prediction efficiency by reducing singular points of the predicted image due to the filter effect accompanying interpolation. ing. On the other hand, when the accuracy of the virtual sample is improved, it is necessary to increase the accuracy of the motion vector expressing the amount of motion, so it is necessary to pay attention to the increase in the code amount.

  MPEG-1 and MPEG-2 employ half-pixel prediction that allows the accuracy of this virtual sample to be up to 1/2 pixel accuracy. FIG. 17 shows a state of generating a sample with 1/2 pixel accuracy. In the figure, A, B, C, and D are integer pixels, and e, f, g, h, and i are half-pixel precision virtual samples generated from A to D, respectively.

e = (A + B) // 2
f = (C + D) // 2
g = (A + C) // 2
h = (B + D) // 2
i = (A + B + C + D) // 2
(However, // indicates division with rounding.)
When this half-pixel precision virtual sample generation procedure is applied to a predetermined block, extra data for one integer pixel from the end of the block is required. This is because it is necessary to calculate a virtual sample that is half a pixel outside the end point (integer pixel) of the block.

In MPEG-4, 1/4 pixel accuracy prediction using virtual samples up to 1/4 pixel accuracy is employed. In the 1/4 pixel accuracy prediction, after a half pixel sample is generated, a 1/4 pixel accuracy sample is generated using them. In order to suppress excessive smoothing at the time of half-pixel sample generation, the frequency component of the original signal is designed to be kept as much as possible even when a filter having a large number of taps is used. For example, in the 1/4 pixel accuracy prediction of MPEG-4, the half pixel accuracy virtual sample a created for generating the 1/4 pixel accuracy virtual sample uses the surrounding 8 pixels as follows, Generated. The following expression shows only the case of horizontal processing, and a half-pixel precision virtual sample a created for generating a 1/4 pixel precision virtual sample and an X component X ₋₄ of an integer pixel in the following expression relationship with to X ₄ are in a positional relationship shown in FIG. 18.

a = (COE ₁ * X ₁ + COE ₂ * X ₂ + COE ₃ * X ₃ + COE ₄ * X ₄ + COE _-1 * X _-1 + COE _-2 * X _-2 + COE _-3 * X _-3 + COE _-4 * X _-4 ) // 256
(However, COE _k : filter coefficient (coefficient sum is 256). // indicates rounding division.)
When this 1/4 pixel-accurate virtual sample generation procedure is applied to a predetermined block, extra data for 4 integer pixels from the end point of the block is required. This is because it is necessary to calculate a virtual sample that is 1/4 pixel outside the end point (integer pixel) of the block.

FDAM for ISO / IEC 14496-2

  However, since the number of neighboring pixels of the prediction target block corresponding to the number of filter taps is necessary for the filter operation at the end point of the prediction target block, the memory bandwidth required for generating the predicted image increases depending on the number of taps. There is a problem.

  In particular, in the 1/4 pixel accuracy prediction of MPEG-4, in order to avoid this problem, there is a contrivance to suppress the number of newly read pixels necessary for predictive image generation by folding back the end pixel of the prediction target block. As a result, there is a problem that natural filtering at the boundary of the prediction target block is hindered, which is not preferable in terms of coding efficiency.

  Therefore, the present invention provides an image coding that can improve the coding efficiency while suppressing the memory bandwidth even when a motion compensation prediction is performed by dividing a video frame such as a macro block into small regions. It is an object to provide a device, an image encoding method, an image decoding device, an image decoding method, and a communication device including at least one of these image encoding device and image decoding device.

In order to solve the above problems, the present invention provides:
A moving image that generates a coded bitstream obtained by compressing and encoding a difference signal between a predicted image generated by performing motion compensation prediction in units of regions obtained by dividing each frame of the moving image signal by a predetermined method and the moving image signal An encoding device comprising:
A frame memory group for storing a reference image used to generate a predicted image;
A motion compensation unit that detects a motion vector between an encoding target frame and a reference image stored in the frame memory group and generates a motion compensated prediction image;
The difference signal between the encoding target frame and the prediction image generated by the motion compensation unit is subjected to orthogonal transform / quantization to perform variable length encoding, and the encoding parameter including the motion vector and variable length encoding of the difference signal An encoding unit that multiplexes data into a bitstream,
The motion compensation unit is configured to switch between a virtual pixel accuracy in which a motion vector to be detected can be described and an interpolation filtering method used when generating a predicted image including the virtual pixel accuracy according to a shape of a region serving as a unit of motion compensated prediction. With
The motion vector is detected according to the virtual pixel accuracy based on the mechanism and the interpolation filtering method by referring to a plurality of reference images on the frame memory group in the unit of the area obtained by dividing the frame, and corresponding to each reference image Generate multiple predicted images,
The encoding unit is information indicating the virtual pixel accuracy and interpolation filtering method based on the mechanism, and multiplexes shape information indicating a shape of a region serving as a unit of motion compensation prediction into a bitstream,
The motion vector detected in accordance with the virtual pixel accuracy based on the mechanism and the interpolation filtering method is encoded, and the macroblock is most often selected from among a plurality of pieces of information indicating which reference image is used to generate a predicted image. The moving picture coding apparatus is characterized in that the information that can be efficiently coded is coded and multiplexed into a bit stream .

The present invention also provides:
A moving image decoding apparatus that restores a moving image signal based on a coded bitstream that is compression-coded by performing motion compensation prediction in units of regions obtained by dividing each frame of the moving image signal by a predetermined method,
A frame memory group for storing a plurality of reference images used for generating a predicted image;
Based on the encoded bitstream, the difference signal, the motion vector, and information indicating a reference image that the motion vector refers to when generating a predicted image are decoded in units of regions obtained by dividing the frame, and virtual pixel accuracy and virtual A decoding unit that decodes shape information indicating a shape of a region serving as a unit of motion compensation prediction, which is instruction information indicating an interpolation filtering method used when generating a prediction image including pixels;
Based on the motion vector decoded by the decoding unit and information indicating a reference image that the motion vector refers to when generating a predicted image, the predicted image is generated with reference to a reference image stored in the frame memory group And a motion compensation unit that
The decoding unit is instruction information representing virtual pixel accuracy and an interpolation filtering method used when generating a predicted image including the virtual pixel, and is based on the shape information indicating a shape of a region serving as a unit of motion compensation prediction. , Switching the accuracy of the virtual pixel that is a component of the predicted image, decoding the motion vector based on the switched accuracy,
The motion compensation unit generates a predicted image using an interpolation filter selected from a plurality of the interpolation filtering methods based on the shape information when the virtual pixel accuracy is switched,
The moving image decoding apparatus restores a moving image signal by adding the difference signal decoded by the decoding unit and the predicted image generated by the motion compensation unit .

  As described above, according to the image coding apparatus and method according to the present invention, the predicted image is switched by switching the accuracy of the virtual pixel that is a component of the predicted image in accordance with the shape of the region that is the unit of motion compensation prediction. And generating a motion vector that gives a predicted image with high prediction efficiency among the plurality of predicted image candidates, and predicting according to the shape of the region that is a unit of motion compensated prediction based on the generated motion vector Since the prediction image is generated by switching the accuracy of the virtual pixel that is a component of the image, the memory bandwidth is suppressed by switching the accuracy of the motion compensation prediction according to the shape of the region that is the motion compensation prediction unit. It becomes possible to perform compression encoding with improved encoding efficiency.

  In addition, the accuracy of motion compensation prediction is switched according to the shape of the region serving as the motion compensation prediction unit, and the motion vector predictive coding method is also switched according to the shape of the region serving as the motion compensation prediction unit to adaptively switch. By changing the encoding, for example, it is possible to allocate a larger amount of code to the motion vector as much as the encoding efficiency is improved while suppressing the memory bandwidth, and maintaining the image quality while suppressing the memory bandwidth, etc. You can also do it.

  Further, according to the image decoding apparatus according to the present invention, the encoded bitstream is input and the difference signal, the motion vector, and shape information indicating the shape of the region serving as a unit of the motion compensation prediction are decoded, Based on the shape information indicating the shape of the region serving as a unit of the motion compensated prediction, the accuracy of the virtual pixel that is a constituent element of the predicted image is switched, and the reference image is used using the decoded motion vector according to the switched accuracy. The prediction image is generated by referring to the decoded difference signal and the prediction image generated by the motion compensation to restore the moving image signal. It is possible to decode the encoded bitstream that has been subjected to compression encoding while improving the conversion efficiency.

  In particular, the memory bandwidth reduction is achieved by simplifying the implementation of video decoding processing, especially when implementing a video encoding device such as a mobile phone or a personal digital assistant, or hardware of a video decoding device, mainly for video playback. Since a remarkable effect is achieved in terms of power consumption, it is possible to provide a video encoding device and a video decoding device with high transmission / recording efficiency while suppressing the mounting cost of these encoding devices and decoding devices.

Embodiment 1 FIG.
In the first embodiment, each frame image of a video is divided into macroblock units, and further, the macroblock is divided into sub-blocks of a plurality of shapes, and motion compensation prediction means capable of individually performing motion compensation prediction A video encoding / decoding device having the above will be described. The video encoding / decoding device according to the first embodiment is characterized in that the accuracy of virtual samples described in the conventional example is switched according to the shape and size of a region (block) that is a unit of motion compensation prediction. Accordingly, the motion vector encoding / decoding method is switched accordingly. The configuration of the video encoding device and the decoding device according to the first embodiment is shown in FIGS.

  FIG. 1 shows the configuration of the video encoding apparatus according to the first embodiment. As shown in FIG. 3, the video encoding apparatus includes a subtracter 10, an encoding mode determination unit 12, a quantization unit 16, an inverse quantization unit 18, an inverse orthogonal transform unit 19, a switch 52, an adder 53, The frame memory 3, the motion detector 2, the motion compensator 7, the variable length encoder 6, the transmission buffer 24, and the encoding controller 22 are included.

Next, the operation of the video encoding apparatus shown in FIG. 3 will be described.
[1] Outline of Operation of Encoding Device In the encoding device of FIG. 1, an input video signal 1 is input in units obtained by dividing individual video frames into macroblocks. A motion vector 5 is detected for each macroblock using the reference image 4 stored in the frame memory 3. A predicted image 8 is obtained in the motion compensation unit 7 based on the motion vector 5, and a predicted residual signal 9 is obtained by taking the difference between the predicted image 8 and the input signal 1 in the subtractor 10.

  The encoding mode determination unit 12 selects the macroblock from a plurality of modes that specify the encoding method of the macroblock, such as a motion prediction mode that encodes the prediction residual signal 9 and an intra mode that encodes the inside of the frame. Is selected in such a manner that it can be encoded most efficiently. The encoding mode information 13 is output to the variable length encoding unit 6 as encoding target information. Here, when the motion prediction mode is selected as the encoding mode information 13 in the encoding mode determination unit 12, the motion vector 5 is transferred to the variable length encoding unit 6 as encoding target information.

  The encoding target signal selected by the encoding mode determination unit 12 is passed to the variable length encoding unit 6 as the orthogonal transform coefficient data 17 via the orthogonal transform unit 15 and the quantization unit 16, while the orthogonal The transform coefficient data 17 is output to the switch 52 after passing through the inverse quantization unit 18 and the inverse orthogonal transform unit 19.

  In the switch 52, when the coding mode information 13 indicates the motion prediction mode according to the coding mode information 13, the orthogonal transform coefficient data 17 subjected to inverse quantization and inverse orthogonal transform, and the motion compensation unit 7 Is output to the frame memory 3 as a locally decoded image 21 by adding to the predicted image 8 from the above or when the encoding mode information 13 indicates an intra mode, the orthogonalization is performed by inverse quantization and inverse orthogonal transform. The transform coefficient data 17 is output as the local decoded image 21 as it is. The locally decoded image 21 is stored in the frame memory 3 as reference image data because it is used for motion prediction of subsequent frames.

  The quantization unit 16 quantizes the orthogonal transform coefficient data with the quantization accuracy given by the quantization step parameter 23 determined by the encoding control unit 22. By adjusting the quantization step parameter 23, the output encoding rate and quality are balanced. In general, after the variable length coding, the occupation amount of the coded data accumulated in the transmission buffer 24 immediately before transmission is checked at regular intervals, and parameter adjustment is performed according to the buffer remaining amount 25. More specifically, for example, when the remaining buffer capacity 25 is small, the rate is reduced, and when the remaining buffer capacity 25 has room, the rate is increased to improve the quality. The quantization step parameter 23 determined by the encoding control unit 22 is also output to the variable length encoding unit 6.

  The variable length coding unit 6 performs entropy coding of data to be coded such as the motion vector 5, the quantization step parameter 23, the coding mode information 13, and the orthogonal transform coefficient data 17, and compresses the video via the transmission buffer 24. The data 26 is transmitted.

  FIG. 2 shows the configuration of the video decoding apparatus according to the first embodiment. As shown in FIG. 2, the video decoding apparatus includes a variable length decoding unit 27, an inverse quantization unit 18, an inverse orthogonal transform unit 19, an adder 55, a switch 54, a motion compensation unit 7, and a frame memory 3. ing.

[2] Outline of Operation of Decoding Device Next, the operation of the video decoding device according to Embodiment 1 shown in FIG. 2 will be described.
In the decoding apparatus shown in FIG. 2, when the compressed video data 26 is received, an entropy decoding process, which will be described later, is performed by the variable length decoding unit 27, and the motion vector 5, the encoding mode information 13, the orthogonal transform coefficient data 17, The quantization step parameter 23 and the like are restored.

  The orthogonal transform coefficient data 17 and the quantization step parameter 23 are decoded by the same inverse quantization unit 18 and inverse orthogonal transform unit 19 as those on the encoding side.

  Further, when the coding mode information 13 indicates the motion prediction mode, the switch 54 restores the predicted image 8 based on the motion vector 5 decoded by the motion compensation unit 7 and the coding mode information 13. On the other hand, when the intra mode is indicated, 0 is output.

  Then, the output from the switch 54 is added to the decoded signal which is the output of the inverse orthogonal transform unit 19 by the adder 55, whereby the decoded image 21 is obtained. The decoded image 21 is stored in the frame memory 3 because it is used to generate a predicted image of a subsequent frame.

[3] Detailed Operation of Motion Compensated Prediction Next, motion compensation prediction processing performed using the motion detection unit 2, the motion compensation unit 7, and the frame memory 3 of the encoding device, and the motion compensation unit 7 of the decoding device, Each of the motion compensation processes performed using the frame memory 3 will be described.

[3] -1 Motion Compensation Prediction Processing Procedure in Encoding Device FIG. 3 shows a flowchart of motion compensation prediction processing in the encoding device. Hereinafter, each step will be described.

[3] -1-1 Determination of virtual sample accuracy (step S1)
FIG. 4 shows a configuration of a motion vector detection unit region in the first embodiment. In the figure, 16 × 16 MC uses a macro block itself as a motion vector detection unit. 16 × 8 MC is an area divided into two in the vertical direction, and 8 × 16 MC is an area divided into two in the horizontal direction. In 8 × 8 MC, a macroblock is equally divided into four areas, each of which is used as a motion vector detection unit. Further, in the case of the first embodiment, in 8 × 8 MC, each divided region is further divided into two vertically (8 × 4 MC), two horizontally (4 × 8 MC), and four (4 × 4 MC). 4 MC) area division is possible, and each can be used as a motion vector detection unit.

  In general, fine division can improve prediction efficiency when complex motion exists inside a macroblock, but a lot of motion vector information needs to be transmitted. When the shape of the motion vector detection unit region can be adapted in various ways inside the macroblock in this way, encoding can be executed while selecting and detecting the optimal divided shape and motion vector locally. Because it can.

  Now, in the detection of the motion vector of each area, motion compensation prediction using virtual samples is used as shown in the conventional example. However, unlike the conventional standard video encoding method and the like, in the first embodiment, for example, as shown in FIG. 4, a virtual image is locally associated with the shape and size of each motion vector detection unit region. Determine sample accuracy and motion vector predictive coding method.

  In the encoding apparatus according to the first embodiment, the shape information indicating the shape and size of the region of the motion vector detection unit, which is a unit of motion compensation prediction, is used as the motion prediction mode in the encoding mode information 13. As a part, it is encoded by the variable length encoding unit 6 and transmitted to the decoding device.

  Therefore, in the decoding apparatus according to the first embodiment, only the coding mode information 13 is used to predict motion prediction in the coding mode information 13 in addition to the coding mode whether the motion prediction mode or the intra coding mode. The shape and size of the motion vector detection unit area, which is the unit of motion compensation prediction, and the accuracy of the virtual sample and the motion vector prediction code that are uniquely determined from the shape and size based on the shape information included as part of the mode Therefore, no additional information is required for switching between the virtual sample accuracy and the motion vector predictive encoding method.

  In the first embodiment, the decision rule uses a virtual sample with half-pixel accuracy in a motion vector detection unit area smaller than 8 × 8 MC, for example, 8 × 4, 4 × 8, or 4 × 4 size. In a motion vector detection unit area of a larger size, a 1/4 pixel precision virtual sample is used.

  The reason for applying this rule is how the shape of the motion vector detection unit region is selected. That is, generally, in a region where the motion is uniform and the motion speed is slow, the spatial resolution of the screen is maintained, and the visibility with respect to the texture is improved. In these areas, the motion vector detection area is made as uniform as possible by using a large motion vector detection area, thereby avoiding discontinuity between areas due to subdivision of the motion area, improving the reproducibility of the signal, and improving the accuracy of the virtual samples and predicting It is desirable to increase efficiency. On the other hand, in a region where the movement is complicated or the movement speed is difficult to visually perceive, the detailed texture of the screen is not saved, and the spatial resolution is visually felt low. In such a region, it is desirable to increase the number of motion vectors to improve the prediction efficiency even at the expense of signal reproducibility to some extent. However, since the spatial resolution of the signal is low and the amount of motion vector information is large, it can be considered that there is no problem from the viewpoint of overall coding efficiency even if the accuracy of the virtual samples is set low.

  By enabling such local setting of virtual sample accuracy, as shown in FIG. 5, the memory bandwidth required for virtual sample generation for each mode of 8 × 4, 4 × 8, and 4 × 4 MC This is effective in simplifying the apparatus. In the figure, for the mode state in the middle stage, the upper part shows the case where it is assumed that a virtual sample with 1/4 pixel accuracy is used for all these modes, and K is used for virtual sample generation. In the case of using the tap filter, that is, it is indicated that it is necessary to read integer pixel data for K pixels (K ≧ 2) from the end points of the motion vector detection unit area from the memory. In the conventional example, an example in which half of K pixels are created by folding is assumed, but here it is assumed that natural filtering is performed by using all consecutive K pixels without performing folding.

In contrast, as in the first embodiment, in each of these 8 × 4, 4 × 8, and 4 × 4 MC modes, it is determined in advance that only half-pixel precision virtual samples are used. For example, the data that needs to be read from the memory for generating the virtual sample may be only one pixel around the motion vector detection unit region according to the conventional half-pixel accuracy sample generation procedure. In a small-sized motion vector detection unit region, this is significant because each detection unit region is spatially discontinuous.
[3] -1-2 Calculation of prediction error amount (steps S2 and S3)
According to the virtual sample generation rule determined in step S1, a prediction image is generated for each motion vector candidate for each motion vector detection unit region in each mode, and the difference from the motion vector detection unit region to be predicted The prediction error amount is calculated by taking Here, with respect to the virtual sample, generation of a half pixel accuracy sample as shown in FIG. 17 of the conventional example and generation of a 1/4 pixel accuracy sample as shown in FIG. 18 are performed. However, in the case of the first embodiment, pixel value folding at the end points in FIG. 18 is not used, and the number of filter taps is hereinafter referred to as K taps in order to have generality. Therefore, in the case of a mode smaller than 8 × 8 MC using a half-pixel precision virtual sample, for example, 8 × 4, 4 × 8, or 4 × 4 MC or less, pixel data used for virtual sample generation is shown in the lower part of FIG. As shown in the figure, only one pixel around the 8 × 4, 4 × 8, 4 × 4 motion vector detection unit area is read from the memory (step S2).

The prediction error amount is generally calculated by adding the error amount of each pixel unit based on the block matching method (step S3), and the error amount is mainly a square error (p p ') ² or absolute difference | pp '| Here, p is a pixel value to be predicted, and p ′ is a pixel value at a corresponding position in the predicted image. In the following, the error amount is assumed to be the latter absolute difference value, and the term SAD (Sum of Absolute Difference) is used for each motion vector detection unit area or the sum in the macroblock.

[3] -1-3 Calculation of motion vector code amount (step S4)
Next, the code amount of the motion vector is calculated (step S4). Since the motion vector usually has a high correlation with the surrounding area, the motion vector of the surrounding area is used as a predicted value, and the predicted difference value (MVD) between the motion vector predicted value of the surrounding area and the calculated motion vector is calculated. Variable length coding. There are various methods for setting the predicted value. Here, the predicted value is determined by a predetermined rule, and a motion vector predicted difference value (MVD) is obtained, and details thereof are omitted. .

  And in this Embodiment 1, when calculating | requiring the code amount of a prediction difference value (MVD), the virtual sample precision defined by [3] -1-1 is considered.

  A method of obtaining the predicted difference value (MVD) in step S4 will be described with reference to FIG. This operation is executed in the motion detection unit 2, but the same rule is applied when the motion vector finally determined in step S9 is encoded by the variable length encoding unit 6.

  In FIG. 6, the motion vectors to be encoded are MV1 to MV5, the prediction vectors defined for MV1 and MV3 according to a predetermined prediction value setting rule are PMV1, and the prediction vector obtained for MV5 is PMV2. And Assume that MV2 is MV1 and MV4 is MV3. Since PMV1 and PMV2 are already encoded values, they may be appropriately cached.

  PMV1 is a motion vector based on 16 × 8 MC, and MV5 is a motion vector based on 8 × 8 MC. Therefore, according to the rule defined in [3] -1-1, it is determined using a virtual sample with 1/4 pixel accuracy. Motion vector. On the other hand, since MV1 to MV4 and PMV2 are motion vectors based on 4 × 4 MC, according to the rule defined in [3] -1-1, the motion vectors are determined using a half-pixel precision virtual sample. That is, there is a difference in the accuracy of virtual samples among PMV1, MV5, MV1 to MV4, and PMV2. On the other hand, at the time of motion vector coding, the value of a prediction vector and its virtual sample accuracy are already known. Utilizing this fact, the first embodiment adaptively adjusts the accuracy of motion vectors in order to obtain a prediction difference value (MVD). That is, a predicted difference value (MVD) is obtained under the following conditions.

(1) Condition 1: When the self (MV) is a motion vector obtained by prediction using a virtual sample with 1/2 pixel accuracy, it is divided into two as follows according to the accuracy of PMV.
Condition 1-1: When PMV is a motion vector using virtual samples with the same accuracy
MVD = MV − PMV

Condition 1-2: When PMV is a motion vector using a virtual sample with 1/4 pixel accuracy
MVD = MV − (PMV >> 1)

(2) Condition 2: When the self (MV) is a motion vector obtained by prediction using a virtual sample with 1/4 pixel accuracy, it is divided into two as follows according to the accuracy of PMV.
Condition 2-1: When PMV is a motion vector using virtual samples with the same accuracy
MVD = MV − PMV

Condition 2-2: When PMV is a motion vector using a virtual sample with 1/2 pixel accuracy
MVD = MV − (PMV << 1)

  However, x << y represents a y-bit shift operation in the left direction with respect to x, and x >> y represents a y-bit shift operation in the right direction with respect to x.

  As a rule between PMV1 and MV1, MV3, the above condition 1-2 is applied, and as a rule between MV1, MV3 and MV2, MV4, the above condition 1-1 is applied, and PMV2 and MV5 Condition 2-2 is applied as a rule between.

  With this procedure, it is possible to calculate MVD with half-pixel accuracy for half-pixel precision motion vectors, and it is possible to reduce the amount of code compared to using MVD with 1 / 4-pixel accuracy at all times. is there.

[3] -1-4 Cost calculation / minimum cost update (steps S5, S6, S7)
A code amount R _MVD is obtained by encoding the prediction difference value (MVD) obtained as a result of the above. Using this and SAD in step S2, cost C is obtained for each motion vector candidate by the following equation (step S5).

C = SAD _MV + λR _MVD
(λ is a positive constant)

  Each time the motion compensation unit 7 calculates the cost as described above, the motion compensation unit 7 determines whether or not the calculated cost is the minimum (step S6), and a value smaller than the cost of the mode calculated before then is calculated. If it appears (step S6 “Y”), the minimum cost is updated and the corresponding prediction mode and motion vector data are held (step S7).

  Steps S1 to S7 are executed for all division modes of 16 × 16 MC to 8 × 8 MC and lower, and steps S2 to S5 are set in advance in the encoding device for each motion vector detection unit region. It is executed for all motion vector candidates within a predetermined motion vector search range, that is, within a window that defines the upper limit of the parallel movement amount in the horizontal and vertical directions.

[3] -1-5 Final mode / motion vector determination (steps S8 and S9)
When the cost calculation / minimum cost update process (steps S5, S6, and S7) described above in [3] -1-4 is completed, it is subsequently determined whether or not the cost has been calculated in all prediction modes (step S8) If the cost calculation is not performed in all prediction modes (step S8 “N”), the processes (steps S1 to S7) described in [3] -1-4 described above are performed, while in all prediction modes. When the cost is calculated (step S8 “Y”), the prediction mode with the lowest cost among the cost of the unit of the macroblock obtained in [3] -1-4 is set as the prediction mode for actually encoding. Determine (step S9). Simultaneously with the determination of the prediction mode, a motion vector corresponding to the prediction mode is determined (step S9).

As the prediction mode determined by the motion compensation prediction process described above, an optimum mode is finally determined by comparison with the intra mode, and video is recorded as coding mode information 13 in units of macroblocks through the variable length coding unit 6. The compressed data 26 is multiplexed. Also, the determined motion vector data 5 is converted into MVD data by the procedure [3] -1-3, and is multiplexed into the video compression data 26 in units of macroblocks through the variable length encoding unit 6.
[3] -2 Motion Compensation Processing in Decoding Device FIG. 7 shows a flowchart of motion compensation processing on the decoding device side. Hereinafter, the motion compensation processing on the decoding device side will be described in detail with reference to the flowchart.

[3] -2-1 Prediction mode, decoding of motion vector data (Step S10)
As shown in FIG. 2, on the decoding device side, the variable length decoding unit 27 decodes the coding mode information 13 in units of macroblocks from the compressed video data 26 output from the coding device shown in FIG. When this indicates the inter (interframe prediction) mode, the variable length decoding unit 27 subsequently decodes the motion vector data 5 encoded in the format of the prediction difference value (MVD) (step S10).

[3] -2-2 Determination of virtual sample accuracy (step S11)
When the coding mode information 13 represents the inter (interframe prediction) mode, that is, in the case of the first embodiment, for example, one of the motion compensation prediction modes shown in FIG. Similar to the case of the described procedure [3] -1-1 (step S1), the virtual sample accuracy is determined. That is, as described in the operation on the encoding apparatus side, the shape information indicating the shape and size of the region of the motion compensation prediction unit, that is, the motion vector detection unit is the motion prediction mode in the encoding mode information 13. Is encoded by the variable-length encoding unit 6 as a part of the image, so that the decoding device side uses the shape information included as a part of the motion prediction mode in the decoded encoding mode information 13 to perform motion compensation prediction. It is possible to determine the shape and size of the motion vector detection unit area that is a unit, and the accuracy of the virtual sample that is uniquely determined from the shape and size.

[3] -2-3 Motion vector decoding (step S12)
Next, the motion vector decoded in the form of a prediction difference value (MVD) is actually used for each motion vector application unit region, i.e., the motion vector data used for each motion vector detection unit region in the description of the encoding device ( MV) (step S12). In the first embodiment, this procedure is performed in the variable length decoding unit 27 and the like, and the procedure reverse to [3] -1-3 described as the motion compensation prediction processing procedure in the encoding device may be taken. That is, in the case of the first embodiment, the motion vector prediction / restoration method is uniquely defined from the shape information included as part of the motion prediction mode in the coding mode information 13 as in the case of determining the accuracy of the virtual sample. Therefore, the motion vector is decoded by switching the motion vector prediction / restoration method based on the shape information. This will be described in comparison with the procedure [3] -1-3 with reference to FIG.

  [3] Similar to 1-3, here, a common prediction value setting method decided in advance between the encoding device and the decoding device is used. First, for MV1 and MV3, using PMV1,

MV1 = MVD1 + (PMV1 >> 1)
MV3 = MVD3 + (PMV1 >> 1)

Is decrypted as Here, MVD1 is a prediction difference value (MVD) corresponding to MV1, and MVD3 is a prediction difference value (MVD) corresponding to MV3.

For MV2 and MV4,
MV2 = MVD2 + MV1
MV4 = MVD4 + MV3

For MV5,
MV5 = MVD5 + (PMV2 << 1)
Is decrypted as follows.

That is, the following conditional expression is followed.
(1) Condition 1: When the self (MV) is a motion vector obtained by prediction using a virtual sample with 1/2 pixel accuracy, it is divided into two as follows according to the accuracy of PMV.
Condition 1-1: When PMV is a motion vector using virtual samples with the same accuracy
MV = MVD + PMV

Condition 1-2: When PMV is a motion vector using a virtual sample with 1/4 pixel accuracy
MV = MVD + (PMV >> 1)

(2) Condition 2: When the self (MV) is a motion vector obtained by prediction using a virtual sample with 1/4 pixel accuracy, it is divided into two as follows according to the accuracy of PMV.
Condition 2-1: When PMV is a motion vector using virtual samples with the same accuracy
MV = MVD + PMV

Condition 2-2: When PMV is a motion vector using a virtual sample with 1/2 pixel accuracy
MV = MVD + (PMV << 1)
The motion vector is decoded by using the following rule.

[3] -2-4 Prediction image generation (steps S13 and S14)
In accordance with the virtual sample generation rule determined in [3] -2-2, a predicted image is generated for each individual motion vector application unit region using the motion vector data decoded in [3] -2-3. As for the virtual sample, a half-pixel accuracy sample as shown in FIG. 17 of the conventional example and a 1 / 4-pixel accuracy sample as shown in FIG. 18 are generated. However, pixel value folding at the end points in FIG. 18 is not used, and the number of filter taps is hereinafter referred to as K taps in order to have generality. Therefore, in the case of the mode of 8 × 4, 4 × 8, 4 × 4 MC or less, for example, smaller than 8 × 8 MC using a half-pixel precision virtual sample, in the case of step S2 of the motion compensation prediction process in the encoding device Similarly to the above, pixel data used for virtual sample generation is read from the memory as shown in the lower part of FIG. 5 to generate a predicted image.

  Therefore, by using the video encoding apparatus or decoding apparatus according to the first embodiment having the above configuration, the motion can be adapted to the local situation of motion according to the size of the block serving as a motion compensation prediction unit. Since the accuracy of the virtual sample at the time of compensation prediction is switched and the calculation method of the motion vector is also switched, it is possible to perform compression coding while maintaining the image quality while suppressing the memory bandwidth. In particular, the reduction in memory bandwidth has a significant effect on simplification of video decoding processing and power consumption, especially when a video playback player is mounted on a mobile phone, a personal digital assistant, etc. .

  In the description of the first embodiment, the accuracy of virtual samples when performing motion compensation prediction is switched according to the size of a block serving as a motion compensation prediction unit, and the motion vector calculation method is also switched. In the present invention, the present invention is not limited to this, and only the accuracy of virtual samples when performing motion compensation prediction is switched according to the size of a block serving as a motion compensation prediction unit, and the motion vector calculation method is not switched. Of course. However, in this case, the encoding efficiency can be improved while suppressing the memory bandwidth, but the image quality is lowered by the amount of lowering the accuracy of motion compensation prediction. This also applies to all the following embodiments.

  In Embodiment 1, after determining the virtual sample accuracy in [3] -1-1 in the encoding device and [3] -2-2 in the decoding device, the virtual samples are matched to the virtual sample accuracy to be used. It was configured to change the filtering process for generation. In the case of 1/4 pixel accuracy, first, a half pixel accuracy virtual sample is generated by a K (= 8) tap filter using integer pixel data as shown in FIG. 18, and the generated half pixel accuracy virtual sample is further generated. A 1/4 pixel precision sample was generated by linear interpolation. In the case of half pixel accuracy, a half pixel accuracy sample is generated by linear interpolation of integer pixel data. In this case, it suffices to read out from the memory by the motion compensation prediction target block size + one peripheral pixel. Due to the difference in filter processing, the key point was to reduce the amount of data read from memory in motion compensation prediction with a small block size, but this filter processing itself is configured to be uniquely determined without depending on the virtual sample accuracy. May be. That is, even in the case of a small block size that uses only half-pixel accuracy samples, the half-pixel accuracy samples may be configured with a K tap filter. This fixed filter processing does not reduce the memory bandwidth with respect to the amount of data read from the memory, but on the other hand, it is necessary to create a sample with 1/4 pixel accuracy from the half-pixel sample generated by the K-tap filter. In addition, as in [3] -1-3 and [3] -2-3, the motion vector representation accuracy can still be limited, and the motion vector encoding can be made more efficient.

  Furthermore, in the first embodiment, the unit of video input is always described as a frame. However, when an interlaced video input such as an odd field and an even field is assumed, strictly speaking, a frame is composed of two field image data. Defined by a combination of In this case, it is obvious that the video encoding apparatus and video decoding apparatus according to the first embodiment can be applied to a case where a macroblock is configured for each field to perform encoding / decoding. This also applies to all the following embodiments.

  Further, in the first embodiment, it has been described that half-pixel precision virtual samples are used in motion vector detection unit regions of 8 × 4, 4 × 8, and 4 × 4 sizes as sizes smaller than 8 × 8. In the present invention, the size is not limited to this, and the size smaller than 8 × 8 may be other than 4 × 2, 8 × 4 such as 2 × 4, 4 × 8, 4 × 4, or 8 × 8 as a standard. Instead, the accuracy of the virtual sample may be changed depending on the size based on another size such as 8 × 16 or 16 × 8. Furthermore, instead of using a virtual sample with half-pixel accuracy in a motion vector detection unit region of a size smaller than a predetermined size such as 8 × 8, motion compensation is performed with integer pixel accuracy in a motion vector detection unit region of a size smaller than the predetermined size. Of course, the prediction may be performed. In this way, although the image quality is somewhat lowered, the memory bandwidth can be greatly reduced. In short, in the motion vector detection unit region that performs motion compensation prediction by further dividing a macroblock that is a coding unit, the motion vector search accuracy is lowered based on a predetermined block size in which the memory bandwidth is a problem, It is only necessary to reduce the memory bandwidth. This also applies to all the following embodiments.

Embodiment 2. FIG. (Example of B frame and multiple reference frame prediction)
In the second embodiment, in addition to the video encoding device and the video decoding device described in the first embodiment, a frame memory group including a plurality of frame memories is prepared and a macro block or a macro block is divided into motion compensations. An apparatus capable of performing motion compensation prediction using a plurality of frame memories in units of prediction blocks will be described.

  FIG. 8 shows the configuration of the video encoding apparatus according to the second embodiment. In the figure, the difference from the encoding apparatus of the first embodiment shown in FIG. 1 is that the frame memory 3 is replaced with a frame memory group 28, and the motion detection unit 2 and motion compensation unit 7 use the frame memory group 28. Thus, an optimum predicted image and motion vector are obtained from a plurality of frame memories. The motion detection unit 2 and the motion compensation unit 7 are different in operation details from the encoding device of FIG.

[1] Outline of Operation of Encoding Device The input video signal 1 is input in units in which individual video frames are divided into macro blocks, and is first stored in the frame memory group 28 in the motion detector 2. A motion vector 5 is detected for each macroblock using a plurality of reference images 4.

  As a method of motion vector detection using a plurality of reference images, there is a bidirectional prediction disclosed in, for example, ISO / IEC13818-2 (MPEG-2 video standard).

FIG. 9 shows a method of performing bidirectional prediction disclosed in ISO / IEC13818-2 (MPEG-2 video standard). In the figure, F (t) is an input video frame to be encoded at present, and a reference image stored in the frame memory is distinguished as F ′ (). Let B (x, y, t) be a block of a motion compensated prediction unit in F (t). In bi-directional prediction, a block image in the past reference image F ′ (t−1) that has been moved by the motion vector MV _f (B (x, y, t)) from the position of B (x, y, t) is moved forward. A future reference image obtained by moving the direction prediction image P _f (B (x, y, t)) by the motion vector MV _b (B (x, y, t)) from the position of B (x, y, t). A block image in F ′ (t + 1) is a backward predicted image P _b (B (x, y, t)), and P _f (B (x, y, t)) and P _b (B (x, The predicted image P _i (B (x, y, t)) of B (x, y, t) is generated by the addition average with y, t)). MV _f (B (x, y, t)) and MV _b (B (x, y, t)) are detected by B (x (x, y, t)) within the given search range on the corresponding reference image in the motion detector 2. , y, t) corresponds to a value obtained by searching for a block image having a high degree of similarity of the pattern with respect to the image, or having the smallest pixel difference, and detecting the shift.

FIG. 10 shows an example of unidirectional prediction in which a motion vector is detected using a plurality of reference images. Another example shown in FIG. 10 is an encoding device configured to be able to store a plurality of past reference images disclosed in, for example, Japanese Patent Laid-Open No. Hei 4 (1990) -27689 in the frame memory group 28. Even if a block image similar to the block B (x, y, t) of the prediction unit is not found in the immediately preceding reference image F (t-1) and can be found in the preceding reference image F (t-2) Since motion compensated prediction can be performed using the motion vector MV _t-2 (B (x, y, t)), motion compensated prediction adapted to the local nature of the video can be performed.

  Since the second embodiment includes the frame memory group 28 including a plurality of frame memories, the motion vector is detected using a plurality of reference images stored in the plurality of frame memories, respectively. The present invention can be applied to any of the ten configurations of the encoding apparatus.

  In the second embodiment, the motion vector 5 detected as in the case of FIG. 9 or FIG. 10 described above indicates which frame memory in the frame memory group 28 the motion vector 5 referred to. The information to be directed is clearly shown as a set.

  Therefore, in the motion compensation unit 7 according to the second embodiment, the predicted image 8 is obtained by referring to an appropriate frame memory in the frame memory group 28 according to the information. Furthermore, the prediction residual signal 9 is obtained by taking the difference from the input signal 1. The information indicating which frame memory in the frame memory group 28 the motion vector 5 has referred to is expressed not in the motion vector 5 itself but in the form of encoded data for notifying the decoder as separate information. May be.

In addition, the coding mode determination unit 12 according to the second embodiment has a plurality of macroblock coding methods such as a motion prediction mode for coding the prediction residual signal 9 and an intra mode for performing intraframe coding. From the modes, a mode capable of encoding the macroblock most efficiently is selected and output as encoding mode information 13. In the motion prediction mode, whether the prediction is performed using the shape of the intra-macroblock division as shown in FIG. 4 described in the first embodiment or only P _f (B (x, y, t)) in FIG. , P _b (B (x, y, t)), information that identifies whether prediction is performed or an addition average of them is taken. The encoding mode information 13 is transferred to the variable length encoding unit 6 as encoding target information. When the motion prediction mode is selected as the encoding mode information 13, the motion vector 5 is transferred to the variable length encoding unit 6 as encoding target information and is variable length encoded.

  In addition, since the process after the orthogonal transformation part which is the encoding of the encoding object signal 11 selected in the encoding mode determination part 12 is the same as Embodiment 1, description is abbreviate | omitted.

Next, the video decoding apparatus according to the second embodiment will be described.
FIG. 11 shows the configuration of the video decoding apparatus according to the second embodiment. In the figure, the difference from the decoding apparatus of the first embodiment shown in FIG. 2 is that the frame memory 3 is replaced with the frame memory group 28 and the motion compensation unit 7 is decoded by the variable length decoding unit 27. And the encoding mode information 13, the prediction image is obtained from the designated frame memory in the frame memory group 28. The details of the operation of the motion compensation unit 7 are different from those of the decoding device of FIG.

[2] Outline of Operation of Decoding Device When receiving the compressed video data 26, the decoding device first performs an entropy decoding process, which will be described later, in the variable length decoding unit 27 to obtain a motion vector 5, coding mode information 13, orthogonal transform coefficients. Data 17, quantization step parameter 23, etc. are restored. Note that the reconstruction process of the prediction residual signal using the orthogonal transform coefficient data 17 and the quantization step parameter 23 is the same as that of the decoding apparatus of the first embodiment, and thus the description thereof is omitted.

  Then, as in the case of the decoding apparatus of the first embodiment, the motion compensation unit 7 is based on the motion vector 5 decoded by the variable length decoding unit 27 and the encoding mode information 13 in the frame memory group 28. The predicted image 8 is restored using the reference image stored in the predetermined frame memory.

  Based on the coding mode information 13, the switch 54 outputs the predicted image 8 from the motion compensation unit 7 to the adder 55 in the motion prediction mode, and outputs 0 to the adder 55 in the intra mode. To do. In the adder 55, the decoded image 21 is obtained by adding the output from the switch 54 to the decoded signal that is the output of the inverse orthogonal transform unit 19. The decoded image 21 is stored in the frame memory group 28 because it is used to generate a predicted image of a subsequent frame.

[3] Motion compensation prediction processing performed using the motion detection unit 2, the motion compensation unit 7 and the frame memory group 28 in the motion encoding device for motion compensation prediction, and the motion compensation unit 7 and the frame memory group 28 in the decoding device As is apparent from FIGS. 9 and 10, the motion compensation processing performed using these can be considered separately for each processing unit for each frame memory. The processing for obtaining the motion vector 5 and the predicted image 8 using the frame memory in which individual reference images (F ′ (t−1), etc.) are stored on the encoding device side has been described in the first embodiment [3. ] Motion compensation prediction processing in the encoding device [3] -1 consisting of [1-1] to [3] -1-5, and individual reference images (F ′ (t−1), etc.) on the decoding device side ) Stored in the decoding device of [3] -2 consisting of [3] -2-1 to [3] -2-4 of the first embodiment, using the frame memory storing The procedure described in the first embodiment can be applied as it is.

  Therefore, according to the video encoding device or decoding device of the second embodiment having the above-described configuration, as in the case of the video encoding device or decoding device of the first embodiment, the local situation of motion can be obtained. In addition to switching the accuracy of the virtual samples described in the conventional example and switching the motion vector encoding / decoding method according to the size of the block that is the motion compensation prediction unit, the memory bandwidth is suppressed. However, it is possible to perform compression encoding while maintaining image quality.

  In particular, when a predicted image needs to be generated from a plurality of frame memories in both directions including the forward direction and the backward direction as shown in FIG. 9, or a predicted image is generated from a plurality of frame memories in one direction as shown in FIG. It is clear that the number of pixels read out from the frame memory increases when it is necessary, but according to the second embodiment, the effect of precise motion compensation prediction is diminished due to complicated movements, etc. In other words, by limiting the virtual sample accuracy to half pixels or integer pixels and reducing the motion vector representation accuracy, it is possible to reduce the memory bandwidth when accessing the frame memory while maintaining the coding efficiency. As a result, according to the second embodiment, particularly in the case of bidirectional prediction or the like in which motion compensation prediction is performed with reference to a plurality of reference images stored in a plurality of frame memories, motion vector detection processing in the encoding device In addition, a significant amount of calculation reduction effect is expected for the predicted image generation processing of the decoding device.

In addition, since the second embodiment has a plurality of frame memories, not only the normal bidirectional prediction shown in FIG. 9 and the unidirectional prediction from a plurality of past reference images shown in FIG. Ordinarily, the contents of the plurality of frame memories are sequentially updated with the decoded image, while if it is instructed not to update only the contents of one frame memory among the plurality of frame memories, the plurality of frame memories Among them, the content of one frame memory is not updated, and a short term frame memory that sequentially updates a plurality of frame memories and a long term frame memory that does not update the reference image until the next event occurs are used. , Motion compensation prediction is performed using reference images of short term frame memory and long term frame memory. Also it does not matter of course. In this way, a plurality of frame memories can be used flexibly according to the temporal local characteristics of the video signal, and the encoding sequence is affected while using the frame memories efficiently. Thus, encoding can be performed while maintaining high prediction efficiency.
In particular, by adapting to the local situation of motion, simply switching the accuracy of virtual samples when performing motion compensation prediction according to the size of the block that is the motion compensation prediction unit, or switching the motion vector calculation method Alternatively, the bidirectional prediction shown in FIG. 9 or the unidirectional prediction shown in FIG. 10 is adaptively switched in units of regions for which motion compensation prediction is performed, or the contents of the plurality of frame memories are usually decoded images in units of the regions. When the update is performed sequentially, when it is instructed not to update only the contents of one of the plurality of frame memories, only the contents of one of the plurality of frame memories are not updated. If you switch between them in an adaptive manner, even if various features such as local movement in time as well as space appear, Without increasing the bandwidth, thereby improving the coding efficiency, image quality becomes possible to improve.

Embodiment 3 FIG. (Virtual pixel calculation means switching flag)
In the third embodiment, virtual sample calculation method switching that further improves the degree of freedom of adaptive switching of the virtual sample calculation method with respect to the video encoding device and video decoding device described in the first and second embodiments. A video encoding device and a video decoding device in which a flag is introduced will be described.

  In the video encoding device and the video decoding device shown in the first and second embodiments, the motion compensation prediction in the size of 8 × 4, 4 × 8, 4 × 4, for example, which is smaller than 8 × 8 MC in FIG. Although it is configured to perform prediction only for virtual samples with pixel accuracy, depending on the video, even for motion compensated prediction in units smaller than smaller than 8 × 8 MC, 1/4 is used to improve prediction efficiency. There may be cases where motion compensated prediction with pixel accuracy is required. For example, there may be a case in which a subtle noise component included in the input signal causes variation in motion vectors while the motion compensation prediction target image sufficiently preserves the texture. In such a case, the virtual sample accuracy is not simply fixed assuming only the complexity of the movement, but is adapted to the local signal condition of the video and the optimum virtual sample accuracy is selected. Thus, the apparatus configuration can be configured to add the minimum amount of computation only when truly fine virtual sample accuracy is required, and it is possible to further improve the video coding quality. .

FIG. 12 shows the configuration of the video encoding apparatus according to the third embodiment.
FIG. 13 shows the configuration of the video decoding apparatus according to the third embodiment. 12 and 13, the virtual sample accuracy switching flag 29 that is an accuracy switching signal for motion compensation prediction plays a role for adaptation of the motion compensation prediction accuracy described above. This is the same as the video encoding device or video decoding device of the first embodiment shown in FIG. 1 or FIG.

Next, the operation will be described.
On the encoding device side of the third embodiment shown in FIG. 12, the virtual sample accuracy switching flag 29 is used for pre-analysis of input video data in the encoding device or an external factor in the system including the encoding device, such as a transmission buffer. Based on the transmission environment such as the 24 remaining buffer capacity and the encoding bit rate, the flag value is determined in units of predetermined video data and input to the motion detection unit 2 and the motion compensation unit 7. In the motion detection unit 2 and the motion compensation unit 7, based on the input virtual sample accuracy switching flag 29, as shown below, while changing the switching unit adaptively, virtual sample accuracy in motion compensation prediction, A motion vector and a prediction image are generated by changing a motion vector calculation method.

  Regarding the value of the virtual sample accuracy switching flag 29, for example, the following meanings can be considered. Specifically, if the virtual sample accuracy switching flag 29 is ON or 1, the motion compensated prediction with all block sizes of less than 8 × 8 MC is performed within 1/4 pixel accuracy within the video data unit as the switching unit. This is instructed to be performed by motion compensation prediction. That is, in this case, the prediction efficiency is prioritized even if a large amount of pixel data is allowed to be read from the frame memory on the assumption that a multi-tap filter as shown in FIG. At this time, since all the motion vectors have the same accuracy, the motion vector prediction and the code amount calculation accompanied by the motion vector accuracy conversion described in [3] -1-3 are not performed.

  On the other hand, if the virtual sample precision switching flag 29 is OFF or 0, all blocks of less than 8 × 8MC in the case of FIG. 4 are 8 × 4, 4 × 8, 4 × 4 size blocks. It is instructed that the motion compensated prediction with the size should be performed with the half-pixel motion compensated prediction. This designation is used in a case where necessary and sufficient prediction efficiency can be obtained even with a virtual sample with half-pixel accuracy as shown in FIG. In this case, since the accuracy of the motion vector is different with a block size of less than 8 × 8 MC, the motion vector prediction and the code amount calculation with the motion vector accuracy conversion described in [3] -1-3 are performed as appropriate. It will be. It should be noted that the relationship between ON / OFF and 1/0 is not fixed to this, and of course, reverse association may be performed (ON = 0, OFF = 1).

  The video data unit as the switching unit of the virtual sample accuracy switching flag 29 is, for example, a macro block, a slice (a unit area of a video frame composed of a plurality of macro blocks), a frame or a field (in the case of an interlace signal), a sequence ( A time series unit composed of a plurality of video frames).

  The virtual sample accuracy switching flag 29 set on the encoding side in this way is multiplexed into the bit stream in the variable length encoding unit 6 in a predetermined video data unit.

  In the decoding apparatus, as shown in FIG. 13, the variable length decoding unit 27 decodes the value of the virtual sample precision switching flag 29, and variable length decoding is performed on the basis of the above rule for each video data unit to which the flag 29 is assigned. The unit 27 performs the decoding process of the motion vector 5 with the accuracy adaptation described in [3] -2-3 as necessary, and the motion compensation unit 7 generates the virtual sample with the accuracy specified by the virtual sample accuracy switching flag 29. Based on the processing and the motion vector 5, the predicted image 8 is generated.

  Therefore, according to the video encoding device and the video decoding device of the third embodiment having the above configuration, it is possible to freely control the trade-off between the prediction efficiency and the calculation load, and the video code having a high degree of freedom. Can be made.

  Although the third embodiment has been described based on the case of the first embodiment, the virtual sample accuracy switching flag 29 can be applied to the video encoding device and the video decoding device according to the second embodiment. Needless to say.

Embodiment 4 FIG. (With spatial prediction)
In the image coding apparatus and the image decoding apparatus according to the first to third embodiments described above, in the intra mode, the normal intra-frame coding without performing spatial prediction has been described. However, in the fourth embodiment, as the intra mode, An example in which a spatial prediction unit 10a that performs encoding in an intra prediction mode in which a difference signal between a moving image signal and a prediction signal is encoded using spatial prediction in a frame will be described.

  FIG. 14 shows the configuration of the coding apparatus according to the fourth embodiment in which a spatial prediction unit 10a is added to the coding apparatus according to the first embodiment shown in FIG. As shown in FIG. 14, in the coding apparatus according to the fourth embodiment, since the spatial prediction unit 10a is added, the intra prediction mode 14 is output from the spatial prediction unit 10a to the variable length coding unit 14, and the switching unit. In 52, the spatial prediction image 20 is output from the spatial prediction unit 10a instead of 0 input of the intra mode in the case of the first to third embodiments. Others are the same as in the first embodiment.

  FIG. 15 shows the configuration of the decoding apparatus according to the fourth embodiment in which a spatial prediction unit 10a is added to the decoding apparatus according to the first embodiment shown in FIG. As shown in FIG. 15, since the spatial prediction unit 10a is added in the decoding apparatus of the fourth embodiment, that is, the spatial prediction unit 10a is added on the encoding device side as shown in FIG. The intra prediction mode 14 is output from the unit 27 to the spatial prediction unit 10a, and the spatial prediction image 20 is output from the spatial prediction unit 10a to the switch 54 instead of the 0 input of the intra mode in the first to third embodiments. Will be output. Others are the same as in the first embodiment.

  The operation will be briefly described. In the encoding apparatus according to the fourth embodiment shown in FIG. 14, when intra prediction mode encoding is performed by the spatial prediction unit 10a, the spatial prediction unit 10a to the variable length encoding unit 14 are used. Intra prediction mode 14 is output, and variable length coding unit 6 performs entropy coding in intra prediction mode 14 together with motion vector 5, quantization step parameter 23, coding mode information 13, and orthogonal transform coefficient data 17, The compressed video data 26 is transmitted via the transmission buffer 24. Then, in the switch 52 of the fourth embodiment, when the coding mode information 13 indicates a motion prediction mode that is prediction in the time direction according to the coding mode information 13, inverse quantization and inverse orthogonality are performed. The transformed orthogonal transform coefficient data 17 and the predicted image 8 from the motion compensation unit 7 are added and output to the frame memory 3 as a local decoded image 21 or the coding mode information 13 is prediction in the spatial direction. In the case of indicating the intra prediction mode, the orthogonal transform coefficient data 17 subjected to inverse quantization and inverse orthogonal transform and the spatial prediction image 20 are added and output as a locally decoded image 21, and the motion prediction of the downstream frame is performed. Therefore, it is stored in the frame memory 3 as reference image data.

  On the other hand, in the decoding apparatus according to the fourth embodiment shown in FIG. 15, the variable length decoding unit 27 decodes the intra prediction mode 14 and outputs the decoded image to the spatial prediction unit 10a, so that the spatial prediction image 20 is restored. In the switch 54 according to the fourth embodiment, based on the coding mode information 13 from the variable length decoding unit 27, the predicted image 8 from the motion compensation unit 7 is a motion prediction mode that is prediction in the time direction. Is output to the adder 55, while the spatial prediction image 20 is output to the adder 55 when the intra prediction mode which is prediction in the spatial direction is indicated. In the adder 55, the decoded image 21 is obtained by adding the output from the switch 54 to the decoded signal that is the output of the inverse orthogonal transform unit 19, and is used to generate a predicted image of the subsequent frame. Stored in the memory 3.

  Therefore, according to the image coding apparatus and the image decoding apparatus according to the fourth embodiment, spatial prediction is used in a frame instead of the intra mode in the image coding apparatus and the image decoding apparatus according to the first embodiment. Since the intra prediction mode for encoding the difference signal between the moving image signal and the prediction signal is employed, the same effects as those of the first embodiment can be obtained, and the normal intra performed in the first embodiment can be obtained. The compression efficiency can be improved as compared with the mode.

  In addition, in this Embodiment 4, what added the space prediction part 10a with respect to the encoding apparatus of Embodiment 1 shown in FIG. 1 and the decoding apparatus of Embodiment 1 shown in FIG. 2 is shown as an embodiment. Although shown and described in the present invention, the present invention is not limited to this, it is also possible to add a spatial prediction unit 10a to the encoding device and the decoding device of Embodiment 2 shown in FIGS. It is also possible to add a spatial prediction unit 10a to the encoding apparatus and decoding apparatus of Embodiment 3 shown in FIG. 13, and these cases can also be described in the same manner as described above.

Embodiment 5 FIG. (Applied products)
In Embodiments 1 to 4 described above as element products such as an image encoding device or an image decoding device, in Embodiment 5, such as the image encoding device or the image decoding device of Embodiments 1 to 4 are described. Briefly describe the final product on which the component product is mounted.

  FIG. 16 shows the configuration of a mobile phone according to a sixth embodiment in which element products such as the image encoding device and the image decoding device according to the first to fourth embodiments are mounted. As shown in FIG. 16, this mobile phone includes a transmission / reception unit 71, a baseband processing unit 72, a control unit 73, an audio codec 74, a video codec 75, an interface 76, a camera 77, a display 78, a microphone / speaker 79, and an antenna 80. The video codec 75 is mounted with the image encoding device and the image decoding device according to any of the first to fourth embodiments.

  Therefore, according to the mobile phone of the sixth embodiment, by implementing the element products such as the image encoding device and the image decoding device of the first to fourth embodiments, it can be adapted to the local situation of motion, It becomes possible to perform compression encoding while maintaining the image quality while suppressing the memory bandwidth, and by reducing the memory bandwidth, the video decoding processing implementation can be simplified and the power consumption can be significantly reduced.

  In the fifth embodiment, both the image encoding device and the image decoding device are used as a video codec 75 as a final product on which the component products such as the image encoding device or the image decoding device of the first to fourth embodiments are mounted. In the present invention, the present invention is not limited to this, but is not limited to the illustration. However, the present invention is not limited to this. Of course, the present invention can also be applied to a DVD player or the like equipped with only one to four image decoding devices. When hardware is mounted on these players mainly for video playback, mobile phones, personal digital assistants, etc., a significant effect is achieved in simplifying video decoding processing and reducing power consumption by reducing the memory bandwidth.

1 is a diagram showing a configuration of a video encoding device in Embodiment 1. FIG. FIG. 3 shows a configuration of a video decoding apparatus in the first embodiment. The flowchart which shows the motion compensation prediction process in an encoding apparatus. 2 shows a configuration of a motion vector detection unit region in the first embodiment. In order to explain that the local setting of the virtual sample accuracy according to the first embodiment can reduce the memory bandwidth required for virtual sample generation for each of the 8 × 4, 4 × 8, and 4 × 4 MC modes. Illustration. The figure for demonstrating how to obtain | require the prediction difference value (MVD) in step S4. The flowchart which shows the motion compensation process in the decoding apparatus side. FIG. 10 shows a configuration of a video encoding apparatus in Embodiment 2. The figure which shows the method of execution of bidirectional | two-way prediction. The figure which shows an example different from the bidirectional | two-way prediction which detects a motion vector using a some reference image. FIG. 10 shows a configuration of a video encoding apparatus in Embodiment 2. FIG. 10 shows a configuration of a video encoding apparatus in Embodiment 3. FIG. 10 shows a configuration of a video decoding apparatus in Embodiment 3. FIG. 10 shows a configuration of a video encoding apparatus in Embodiment 4. FIG. 10 shows a configuration of a video decoding apparatus in a fourth embodiment. The figure which shows the structure of the mobile telephone of Embodiment 5 by which element products, such as the image coding apparatus of Embodiment 1-4 and an image decoding apparatus, were mounted. The figure which shows the mode of the production | generation of the sample of 1/2 pixel precision. Diagram showing the positional relationship between half-pixel-accurate virtual sample a created for 1 / 4-pixel-accuracy virtual sample generation only in the case of horizontal processing and X components X _{-4 to} X ₄ of integer pixels in the following formula .

  2 motion detection unit, 3 frame memory, 6 variable length encoder, 7 motion compensation unit, 10 subtractor, 10a spatial prediction unit 12 encoding mode determination unit, 16 quantization unit, 18 inverse quantization unit, 19 inverse orthogonal Conversion unit, 22 encoding control unit, 24 transmission buffer, 27 variable length decoding unit, 28 frame memory group, 52 switch, 53 adder, 54 switch, 55 adder.

Claims (2)

A moving image that generates a coded bitstream obtained by compressing and encoding a difference signal between a predicted image generated by performing motion compensation prediction in units of regions obtained by dividing each frame of the moving image signal by a predetermined method and the moving image signal An encoding device comprising:
A frame memory group for storing a reference image used to generate a predicted image;
A motion compensation unit that detects a motion vector between an encoding target frame and a reference image stored in the frame memory group and generates a motion compensated prediction image;
The difference signal between the encoding target frame and the prediction image generated by the motion compensation unit is subjected to orthogonal transform / quantization to perform variable length encoding, and the encoding parameter including the motion vector and variable length encoding of the difference signal An encoding unit that multiplexes data into a bitstream,
The motion compensation unit switches between virtual pixel accuracy in which a motion vector to be detected can be described and an interpolation filtering method used when generating a predicted image including virtual pixel accuracy according to a shape of a region serving as a unit of motion compensated prediction. equipped with a obtain mechanism,
A unit of areas obtained by dividing the frame, with reference to the plurality of reference images on the frame memory group, performs detection of the motion vector follow based Ku virtual pixel accuracy and the interpolation filtering method to the mechanism, the Generate multiple predicted images corresponding to the reference image,
The encoding unit is information indicating a based rather the virtual pixel accuracy and the interpolation filtering method to the mechanism, multiplexes the shape information indicating the shape of the region which is a unit of motion compensation prediction in the bit stream,
Performs coding of the motion vector detected in accordance with the interpolation filtering process and the virtual pixel accuracy rather based on the mechanism, the macro blocks from among a plurality of information indicating whether to generate a predictive image using any of the reference image A video encoding apparatus characterized in that the information that can be encoded most efficiently is encoded and multiplexed into a bitstream. A moving image decoding apparatus that restores a moving image signal based on a coded bitstream that is compression-coded by performing motion compensation prediction in units of regions obtained by dividing each frame of the moving image signal by a predetermined method,
A frame memory group for storing a plurality of reference images used for generating a predicted image;
A differential signal based on the encoded bit stream, and a motion vector, and information indicating a reference image motion vector is referred to when the prediction image generation, and decoded in the unit of dividing the frame region, and the virtual pixel accuracy A decoding unit that decodes shape information indicating a shape of a region serving as a unit of motion compensation prediction, which is instruction information indicating an interpolation filtering method used when generating a predicted image including virtual pixels;
Based on the motion vector decoded by the decoding unit and information indicating a reference image that the motion vector refers to when generating a predicted image, the predicted image is generated with reference to a reference image stored in the frame memory group And a motion compensation unit that
It said decoding unit is an instruction information representing the interpolation filtering method used when generating the predicted image including the virtual pixel as the virtual pixel accuracy, based on the shape information indicating the shape of the region which is a unit of motion compensation prediction Te, before Symbol switching the accuracy of the virtual pixel which is a component of the predicted image, decodes the motion vector based on the cut-conversion that was precision,
The motion compensation unit generates a predicted image using an interpolation filter selected from a plurality of the interpolation filtering methods based on the shape information when the virtual pixel accuracy is switched,
A moving picture decoding apparatus that restores a moving picture signal by adding the difference signal decoded by the decoding section and the predicted image generated by the motion compensation section.

Images