JP2002209213A - Motion vector detection method and device, and image coder - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce an arithmetic quantity required for motion vector detection. SOLUTION: A macro block motion vector detector 11 detects a motion vector of a macro block of 16×16 pixels and a block motion vector detector 15 detects a motion vector of a macro block of 8×8 pixels respectively. A block motion vector detection discriminator 13 discriminates whether or not a motion vector is to be detected from 4 blocks in the macro block from which the motion vector is detected. A block motion vector detector 15 detects the block motion vector only from a required block.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method and an apparatus for detecting a motion vector and an image coding apparatus, and more particularly to obtaining a predicted image from a locally decoded image of a previously coded frame using a motion vector. The present invention relates to a motion vector detecting method and apparatus for detecting a motion vector used in image coding that performs so-called motion compensation, and an image coding apparatus.

[0002]

2. Description of the Related Art For example, in a system for transmitting moving image data to a remote place, such as a video conference system or a video telephone system, image data is converted into a line correlation or a frame in order to use a transmission path efficiently. The compression encoding is performed using the inter-correlation.

As a typical high-efficiency coding method for moving images, MPEG (Moving Picture Image Coding Exper
ts Group) method is known. This is ISO-IE
C / JTC1 / SC2 / WG11, is a video coding scheme for storage proposed as a standard proposal,
Motion Compensated Prediction Coding and DCT (Discrete Cosine Transform: Disc
rete Cosine Transform) A hybrid method combining coding is adopted.

FIG. 10 is a block diagram showing a general configuration example of an MPEG encoder.

In FIG. 10, input image data to be encoded is input to a frame memory 101 and is temporarily stored.

The motion vector detector 102 reads out the image data stored in the frame memory 101 in units of macroblocks composed of, for example, 16 × 16 pixels, and detects the motion vector.

Here, the motion vector detector 102 converts the image data of each frame into an I picture (intra-frame coding), a P picture (forward prediction coding), or a B picture (bidirectional prediction coding). Is processed as any one of It should be noted that it is predetermined, for example, as to which of I, P, and B pictures the image of each frame input sequentially is processed (for example, I, B, P, B, P,...). -Processed as B, P).

That is, the motion vector detector 102
In the image data stored in the frame memory 101,
A predetermined reference frame is referred to, and the reference frame and a small block (macro block) of 16 pixels × 16 lines of the frame to be currently encoded are referred to.
Are subjected to pattern matching (block matching) to detect a motion vector of the macroblock.

[0009] Here, in MPEG, there are four types of image prediction modes: intra coding (intra-frame coding), forward prediction coding, backward prediction coding, and bidirectional prediction coding. Coded, P pictures are coded by either intra coding or forward prediction coding, and B pictures are coded by intra coding, forward prediction coding, backward prediction coding, or both method prediction coding. Encoded.

For this reason, the motion vector detector 102
For the I picture, the intra coding mode is set as the prediction mode. In this case, the motion vector detector 10
No. 2 does not detect a motion vector and outputs only information on its prediction mode (intra prediction mode) to a VLC (variable length coding) unit 106 and a motion compensator 112.

The motion vector detector 102 performs forward prediction on a P picture and detects the motion vector. Further, the motion vector detector 102 compares a prediction error caused by performing forward prediction with, for example, a variance of a coding-target macroblock (P-picture macroblock). As a result of the comparison, when the variance of the macroblock is smaller than the prediction error, the motion vector detector 102 sets the intra coding mode as the prediction mode, and transmits the information of the mode together with the detected motion vector to the VLC unit 106 and Output to the motion compensator 112. If the prediction error caused by performing forward prediction is smaller, the motion vector detector 102 sets the forward prediction coding mode as the prediction mode, and outputs the information of the mode together with the detected motion vector to the VLC unit 106.
And to the motion compensator 112.

Further, the motion vector detector 102
For a picture, forward prediction, backward prediction, and bidirectional prediction are performed, and respective motion vectors are detected. Then, the motion vector detector 102 detects a minimum prediction error (hereinafter, appropriately referred to as a minimum prediction error) among forward prediction, backward prediction, and bidirectional prediction, and calculates the minimum prediction error and For example, the variance of the target macroblock (macroblock of the B picture) is compared. As a result of the comparison, when the variance of the macroblock is smaller than the minimum prediction error, the motion vector detector 102
Sets the intra coding mode as the prediction mode,
The information of the detected motion vector and its mode is stored in VL
Output to the C unit 106 and the motion compensator 112. Also,
If the minimum prediction error is smaller, the motion vector detector 102 sets the prediction mode in which the minimum prediction error was obtained as the prediction mode, and outputs the detected motion vector and information on the mode to the VLC unit 106 and Motion compensator 1
12 is output.

When the motion compensator 112 receives both the prediction mode and the motion vector from the motion vector detector 102, the motion compensator 112 calculates the motion vector in accordance with the prediction mode and the motion vector.
The coded and already locally decoded image data stored in the frame memory 111 is read, and the read image data is used as predicted image data by the arithmetic unit 10.
3 and 110.

The arithmetic unit 103 includes the motion vector detector 10
2 reads the same macroblock as the image data read from the frame memory 101 from the frame memory 101, and calculates the difference between the macroblock and the predicted image from the motion compensator 112. This difference value is obtained by the DCT unit 1
04.

On the other hand, when only the prediction mode is received from the motion vector detector 102, that is, when the prediction mode is the intra coding mode, the motion compensator 112 does not output a predicted image. In this case, the arithmetic unit 103
The same applies to the arithmetic unit 110, and the macroblock read from the frame memory 101 is output to the DCT unit 104 without any processing.

The DCT unit 104 performs a DCT process on the output data of the arithmetic unit 103, and obtains the resulting D
The CT coefficient is supplied to the quantizer 105. Quantizer 10
5, the data storage amount of the buffer 107 (the buffer 10
A quantization step (quantization scale) is set in accordance with the amount of data stored in (7) (buffer feedback), and the DCT coefficient from the DCT unit 104 is quantized in the quantization step. The quantized DCT coefficients (hereinafter, appropriately referred to as quantization coefficients) are supplied to the VLC unit 106 together with the set quantization steps.

The VLC unit 106 converts the quantized coefficient supplied from the quantizer 105 into a variable length code such as a Huffman code, and outputs it to the buffer 107. Further, the VLC unit 106 receives a quantization step from the quantizer 105, a prediction mode (intra-coding (intra-picture prediction coding), forward prediction coding, backward prediction coding, or bidirectional coding) from the motion vector detector 102. The motion vector is also variable-length coded, and the coded data obtained as a result is output to the buffer 107.

The buffer 107 temporarily stores the encoded data from the VLC unit 106, smoothes the data amount, and outputs the encoded data as an encoded bit stream to, for example, a transmission path or records it on a recording medium.

The buffer 107 outputs the data storage amount to the quantizer 105.
Sets the quantization step according to the amount of data stored from the buffer 107. That is, the quantizer 10
5 is when buffer 107 is about to overflow,
The quantization step is increased, thereby reducing the data amount of the quantization coefficient. Further, when the buffer 107 is about to underflow, the quantizer 105 reduces the quantization step, thereby increasing the data amount of the quantization coefficient. In this way, overflow and underflow of the buffer 107 are prevented.

The quantization coefficients and the quantization steps output from the quantizer 105 are supplied not only to the VLC unit 106 but also to an inverse quantizer 108. The inverse quantizer 108 inversely quantizes the quantized coefficient from the quantizer 105 in accordance with a quantization step from the quantizer 105. As a result, the quantized coefficients are converted into DCT coefficients. This DCT coefficient is calculated by an IDCT unit (inverse DCT).
Device 109). In the IDCT unit 109, DC
The T coefficient is subjected to inverse DCT processing, and data obtained as a result of the processing is supplied to the arithmetic unit 110.

The arithmetic unit 110 is supplied with the same data as the predicted image supplied from the motion compensator 112 to the arithmetic unit 103, as described above, in addition to the output data of the IDCT unit 109. The arithmetic unit 110 locally decodes the original image data by adding the output data (prediction residual (difference data)) of the IDCT unit 109 and the predicted image data from the motion compensator 112, and locally decodes the original image data. It outputs the obtained image data (locally decoded image data). However, when the prediction mode is the intra coding, the output data of the IDCT unit 109 passes through the arithmetic unit 110 and is supplied as it is to the frame memory 111 as locally decoded image data. The decoded image data is the same as the decoded image data obtained on the receiving side.

The decoded image data (local decoded image data) obtained by the arithmetic unit 110 is stored in a frame memory 111.
, Is stored, and then used as reference image data (reference frame) for an image to be inter-coded (forward predictive coding, backward predictive coding, bidirectional predictive coding).

Here, as the above-mentioned MPEG system, M
PEG1 and MPEG2 are widely used, and MPEG2 is
It mainly targets high-quality encoding suitable for broadcasting, but does not support an encoding method with a lower code amount (bit rate) than MPEG1, that is, a higher compression rate. On the other hand, with the spread of mobile terminals in recent years, it is expected that the need for such a high compression rate encoding system will increase in the future,
In response to this, standardization of the MPEG4 encoding system has been performed. Regarding the image coding method of MPEG4, 1998
In December 2000, the standard was approved as an international standard as ISO / IEC 14496-2.

In this MPEG4, other MPEG
Unlike the scheme, a new predictive coding scheme called INTER4V has been added. That is, MPEG1, MPEG1
In contrast to the previous MPEG system such as 2 or the like, as shown in FIG. 11, an image is divided into 16 × 16 pixel areas called macroblocks, and one motion vector is used for each macroblock. In MPEG4, 16 ×
Different vectors can be used for an area called an 8 × 8 pixel block obtained by further dividing the area of 16 pixels.

Therefore, in MPEG4, other MPs
It is possible to select a prediction method of a macroblock of one motion vector for a macroblock similar to EG4, and a prediction mode of a macroblock having four motion vectors allocated to each block in the macroblock. It is possible.

ISO-IEC / JTC1 / SC2 / WG
VM4 related to MPEG4Video issued from No.11
(Verification Model) In 15.0, the following method is recommended as a means for obtaining a macroblock motion vector and a block motion vector.

Hereinafter, the operation of the motion vector detector 102 recommended in VM 15.0 will be described in more detail with reference to FIGS.

First, FIG. 12 is a diagram schematically showing the configuration of the motion vector detector 102 when describing a method of detecting a motion vector of a macroblock or a block in the VM system. ,
It comprises a macroblock motion vector detector 116 and a block motion vector detector 117.

Macroblock motion vector detector 116
, The input image stored in the frame memory 101 of FIG. 10 and the locally decoded image stored in the frame memory 111 of FIG. 10 are input. The macroblock motion vector detector 116 detects a motion vector for each macroblock using the input image and the locally decoded image.

The operation of the macroblock motion vector detector 116 will now be described with reference to FIG. This FIG.
FIG. 9 is a schematic diagram for explaining the operation of the macroblock motion vector detector 116.

FIG. 13A shows a locally decoded image input from the frame memory 111, and shows a prediction reference image 121 used for prediction. FIG. 13B shows an input image input from the frame memory 101. An encoding target image 122 which is an image (original image) and is an encoding target is shown. In FIG. 13, it is assumed that the motion vector of the macro block 123 indicated by the bold line in FIG.

In order to detect a macroblock motion vector, as shown in FIG. 13 (C), a locally decoded image (prediction reference image 121) has a size equal to that of a macroblock.
An area 124 of 6 × 16 pixels is extracted. The pixel value of the image of the macro block 123 indicated by the bold line in FIG. 13B is compared with the region 124 extracted from the locally decoded image, and the absolute value sum of the difference between the pixel values is calculated. This operation is repeated while moving the acquisition position of the image from the local image, and a position at which the sum of absolute values of the differences is minimized is obtained.
The detection of the position where the absolute value with respect to the local image is the smallest is usually performed centering on the position of the current macroblock to be coded, and its range is determined according to the characteristics of the image to be coded.

Here, it is assumed that the block at the position indicated by the bold line in FIG. 13D has been obtained as a so-called matching block 126 having the minimum absolute value sum, and the rectangle indicated by the broken line in FIG. , And the current position of the macroblock 123 shown in FIG. The macroblock motion vector 125 is calculated as the difference between the position of the minimum absolute value sum shown in FIG. 13D and the macroblock position of the original image. As described above, the motion vector 125 of the macroblock is detected.

Returning to FIG. 12, the motion vector of the macroblock output from the macroblock motion vector detector 116 is
Is input to The block motion vector detector 117
The motion vector of the macroblock output from the macroblock motion vector detector 116, the input image (original image) input from the frame memory 101 in FIG. 10, and the locally decoded image input from the frame memory 111 are input. .

Next, the block motion vector detector 117
Will be described with reference to FIG. (A) of FIG.
FIG. 1 shows a prediction reference image 121 which is a locally decoded image input from the frame memory 111 and used for prediction.
4B is an original image input from the frame memory 101 and an encoding target image 12 to be encoded.
2 is shown. Also, in FIG. 14, the macro block 12 indicated by the broken-line rectangle shown in FIG.
3, a block motion vector of a block 131 indicated by a bold small rectangle is obtained. The macro block 123 has 16 × 16 pixels, and the block 131 has 8 × 8 pixels. Each block obtained by dividing one macroblock into four is represented by B0, B1, and B, respectively.
2, B3.

In the block motion vector detection method, similarly to the above-described macro block motion vector detection method, the pixel value of the original image (encoding target image 122) of the block 131 for which the current motion vector is to be obtained and the locally decoded image ( Detection is performed by obtaining a point at which the sum of absolute values of the differences from the pixel values of the 8 × 8 pixel area (the block 132 in FIG. 14C) of the prediction reference image 121) is minimized.

At this time, the search range of the absolute value sum minimum point of the difference is different from the macroblock motion vector detection method.
A range 133 schematically shown in FIG. 14C is selected. That is, in the VM method, the motion vector 125 of the macroblock 123 to which the block currently being detected belongs is set as the center of the search range, and the range of two pixels above, below, left, and right is set as the search range 133. When the block at the position where the sum of the absolute values of the differences is minimum in the search range 133 is the matching block 134 in FIG. 14D, the difference between the position of the matching block 134 and the current block 131 is the block. Motion vector 13
5 is output.

By the method described above, the motion vector detector 102 shown in FIG. 10 detects the motion vector 125 of the macro block and the motion vector 135 of the block. The block motion vector is, for example, four blocks B0, B1, B2, and B in the macroblock.
3 are detected.

[0039]

By the way, the above-mentioned conventional motion vector detecting apparatus performs a block motion vector search for all blocks in a macroblock after a macroblock motion vector is detected. .
The search for the motion vector is performed as described in the above conventional example.
The calculation is performed by calculating the sum of the absolute values of the differences between the locally decoded image (prediction reference image) and the original image (encoding target image) (hereinafter, this process is referred to as block matching). Since block matching is used heavily in motion vector detection, searching for a motion vector for all blocks at all times increases the frequency of block matching calculations, resulting in extremely heavy processing. It also causes the execution time to be delayed.

The present invention has been made in view of the above-described circumstances, and provides a motion vector detecting method and apparatus, and an image encoding apparatus which can reduce the amount of calculation involved in detecting a motion vector. The purpose is to provide.

[0041]

SUMMARY OF THE INVENTION The present invention detects a motion vector for obtaining a predicted image from a temporally different reference image from an encoding target image to be subjected to image encoding with motion compensation. In doing so, as a detection unit area serving as a unit of the motion vector detection, at least a first detection unit area and a second detection unit area obtained by dividing the first detection unit area, A first motion vector for the first detection unit area is detected, and a second motion vector for the second detection unit area in the first detection unit area is detected based on the detected first motion vector. By determining whether or not to detect a motion vector, and by detecting a second motion vector for the second detection unit area determined to perform the detection of the second motion vector,
The above-mentioned problem is solved.

Further, the image encoding apparatus of the present invention obtains a predicted image from a reference image temporally different from the image to be encoded using a motion vector, and performs image encoding with motion compensation. In the encoding device, the detection unit area serving as a unit for detecting the motion vector includes at least a first detection unit area and a second detection unit area obtained by dividing the first detection unit area. A first motion vector for detecting a first motion vector for the first detection unit area;
Determining whether or not to detect a second motion vector for the second detection unit area in the first detection unit area based on the detected first motion vector. Determining means for determining whether the second motion vector is to be detected by the determining means.
The above-mentioned problem is solved by having a second motion vector detecting means for detecting a second motion vector for the detection unit region of (1).

Here, the reference image may be a locally decoded image of a frame encoded in the past, and the first detection unit area may be a macroblock of 16 × 16 pixels. The second detection unit area is a block of 8 × 8 pixels.

[0044]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Preferred embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 is a block diagram showing a schematic configuration of a motion vector detecting device which is an embodiment of a motion vector detecting method and device according to the present invention.

The motion vector detecting device shown in FIG.
For example, it can be used for an image coding apparatus as shown in FIG. 10 and is used in the same manner as the motion vector detector 102 in FIG.

The motion vector detecting device shown in FIG.
A macroblock motion vector detector 11 that detects a motion vector of the macroblock (a detection unit area of a first motion vector, for example, 16 × 16 pixels); and a block (a detection unit area of a second motion vector, for example, 8 × 16 pixels). And a block motion vector detector 15 for detecting a motion vector of (8 pixels). This motion vector detecting device is, for example, a frame memory 101 shown in FIG.
An encoding target image output from the frame memory 111 and a predicted reference image output from the frame memory 111 as a local decoded image are input.

Here, for convenience of explanation, FIG.
It is described that the locally decoded image output from the frame memory 111 is input. However, even if the image of the corresponding frame of the original image used to encode this locally decoded image is input instead of the locally decoded image, no problem. In addition, in bidirectional prediction or the like, it is needless to say that the present invention can be easily applied to a case where an image of a frame in the future than the encoding target image is used as a prediction reference image.

The input original image and the locally decoded image are supplied to a macroblock motion vector detector 11. The macroblock motion vector detector 11 performs motion vector detection by the above-described block matching for each macroblock of the input image.

In the block matching at this time,
Rather than directly calculating the absolute value sum of the differences (hereinafter referred to as AD values) for each pixel of the entire macroblock for all block matching or partial block matching, each block within the macroblock (for example, four The AD value is calculated for each of the blocks B0, B1, B2, and B3), and the sum of the AD values of the four blocks is set as the AD value of the macro block. Further, a part or all of the AD value at the position of each motion vector calculated for each block, the AD value of a macroblock, and the like are supplied to the motion vector information buffer 12. The macroblock motion vector obtained by the block matching as described above is output from the macroblock motion vector detector 11 and sent to the changeover switch 14.

The motion vector information buffer 12 stores a part or all of the AD value of each block, the AD value of the macro book, and the like for each macro block motion vector output from the macro block motion vector detector 11. These stored data can be freely extracted from the block motion vector detection / determination unit 13 and the block motion vector detection unit 15.

The block motion vector detection judging unit 13 judges whether a block motion vector is necessary for each macroblock for which the above macroblock motion vector detection has been performed.

That is, the block motion vector detection / judgment unit 13 calculates the above-mentioned values of each block obtained from the reference image at the macroblock motion vector position of the macroblock.
AD value information is obtained from the motion vector information buffer 12. The block motion vector detection determiner 13 predicts the state of each block at the macroblock motion vector position based on each AD value in the macroblock obtained from the motion vector information buffer 12. In other words, when the difference between the AD values of the blocks at the position indicated by the macroblock motion vector is small, Inter encoding using only the macroblock motion vector is applied as the prediction mode of the macroblock selected at the later stage of the encoding. There are many cases. Therefore, the prediction mode of a macroblock is
If the encoding is predicted, the block motion vector detection routine is set to “0FF”, and the cost required for block motion vector detection can be reduced.

Hereinafter, a determination method in motion vector detection according to the first embodiment of the present invention will be described.

Here, the AD value of each block at the position indicated by the macroblock motion vector is AD0, AD1, AD
2, AD3. The average value of AD0, AD1, AD2, and AD3 is set to AD_ave. Here, based on AD0, AD1, AD2, AD3, and AD_ave, MB_dist indicating the state of the macroblock is displayed.
Is calculated. MB_dist indicating the state of this macroblock
Is defined as follows:

[0056]

(Equation 1)

MB_dist obtained by equation (1)
Is greater than or equal to a certain threshold value Th determined by the properties of the input image, encoding conditions, and the like, so that the block motion vector detection / determination unit 1 performs encoding of the block motion vector.
3 outputs a Block ME (block motion prediction) execution flag “ON” to the changeover switch 14. Also, MB_dist
Is smaller than or equal to the threshold value Th, it is not necessary to detect the block motion vector.
Outputs ME execution flag “0FF”.

When the block ME execution flag "ON" is input to the changeover switch 14, the block motion vector detector 15 detects a block motion vector. The block motion vector detector 15 includes an original image, a locally decoded image, and a macroblock motion vector detector 11.
Using the macroblock motion vector output from the input as an input, the same operation as that of the block motion vector detector 117 shown in FIG. However, if some or all of the AD values of each block for which block matching is to be obtained are already stored in the motion vector information buffer 12, the block motion vector detector 11 can use the information for encoding. Differences. When it is desired to simplify the configuration of the apparatus, the operation described above may not be used and the operation may be completely the same as that of the block motion vector detector 117.

When the block ME execution flag “OFF” is input to the changeover switch 14 from the motion vector coast information buffer 12, the block motion vector detector 15 does not operate and the macroblock motion vector detector 11 detects the block ME execution flag. A macroblock motion vector is output as each of the four block motion vectors.

The above-described motion vector detection operation will be described with reference to the flowchart of FIG.

In FIG. 2, first step S21
Then, a macroblock motion vector is detected. At that time, as described above, the four blocks B0, B1, B2,
The sum AD0, AD1, of the difference absolute values for each pixel for B3
Find AD2 and AD3.

In the next step S22, the average AD_ave of the sum of absolute differences of the four blocks is obtained, and in step S23
Then, the absolute value of each difference between the sum of absolute differences AD of the four blocks and the average AD_ave of the sum of absolute differences is obtained, and it is determined whether or not the total sum is smaller than a threshold Th. If YES is determined in this step S23, the process proceeds to step S2.
Proceeding to step S4, the macroblock motion vector is output, and the process ends.

When NO is determined in the step S23, the block number i is initialized (i =
0), and in step S27, for the i-th block,
For example, the block motion vector MVi is detected by the method described with reference to FIG. Next step S28
To determine whether the block is the last block (i = 3) or not, and repeats the block motion vector detection process in step S27 while incrementing the block number i (i = i + 1) until the last block is reached. i =
After reaching 3), the process ends. The threshold value Th
Substitutes an appropriate value according to the properties of the image, the encoding conditions, and the like.

As described above, in the first embodiment, a macroblock motion vector and a motion vector are detected and output.

By the way, temporarily in the motion vector detector,
Make a prediction mode decision for the corresponding macroblock,
Alternatively, when the prediction mode of the macroblock is input from outside, the motion vector detector does not need to output both the macroblock motion vector and the block motion vector, and outputs only the motion vector according to the macroblock prediction mode. It is also possible to adopt such a configuration.

Next, the block motion vector detection decision unit 1
A modified example of the method for selecting the Block ME (block motion prediction) execution flag in No. 3 will be described.

The block motion vector detection / determination unit 1 shown in FIG.
3, for each block at the position indicated by the macroblock motion vector,
AD0, AD1, AD2, AD3, and their maximum value is ADmax
And the minimum value is ADmin. The difference between ADmax and ADmim is calculated. If this value is less than a certain threshold value Th, the Block ME execution flag output from the block motion vector detection / determination unit 13 is set to “OFF”. It outputs the Block ME execution flag “ON” as the detection of the motion vector is necessary.

A modification of such a method of selecting the Block ME execution flag will be described with reference to the flowchart of FIG. The processing content 35 in FIG. 3 is different from the processing content 25 in the flowchart in FIG.
Other parts are the same.

In FIG. 3, first step S31
Then, a macroblock motion vector is detected. At that time, the sums of the absolute differences of the four blocks AD0, AD1, AD2,
AD3 is determined in advance.

In the next step S32, the maximum value ADmax and the minimum value ADmin of the absolute difference sums AD0, AD1, AD2, and AD3 of these blocks are obtained, and in step S33, the maximum value ADmax and the minimum value ADmin are calculated. It is determined whether or not the absolute value of the difference is smaller than a predetermined threshold Th. This step S33
If the answer is YES, the process proceeds to step S34,
The macroblock motion vector is output, and the process ends.

When NO is determined in the step S33, the block number i is initialized (i =
0), and in step S37, the block motion vector MVi
Is detected. In step S38, is the last block (i =
3) It is determined whether or not the step S37 is repeated while incrementing the block number i (i = i + 1) until the last block (i = i + 1) is reached.
After reaching 3), the process ends. The threshold value Th
, An appropriate value may be substituted according to the properties of the image, the encoding conditions, and the like.

According to the first embodiment of the present invention, the second motion vector can be detected only for the necessary second detection unit area (block).
It is possible to reduce the amount of calculation when detecting a motion vector,
Processing can be speeded up.

In the first embodiment described above, the conditions for determining the execution of the two types of block motion vectors have been described. However, the present invention is not limited to the two methods of executing the block motion vectors. When obtaining a motion vector of a minimum unit (corresponding to a block in the present embodiment) to be performed, a motion vector is detected once by an aggregate (corresponding to a macroblock in the present embodiment) in which some of the minimum units are collected, The present invention can be widely applied to all motion vector detection methods in which the determination of a lower-order motion vector detection method is determined based on the motion vector and data generated at the time of detecting the aggregate motion vector.

Next, a second embodiment according to the present invention will be described below with reference to FIG.

FIG. 4 is a block diagram showing a schematic configuration of a motion vector detecting device according to a second embodiment of the present invention. Similar to the motion vector detector 102 shown in FIG. 10, image compression using motion compensation is performed. This is used when encoding the system.

The motion vector detecting device shown in FIG. 4 receives, as input images, an original image of an image to be coded and a locally decoded image or an original image of a predicted frame used at the time of encoding. Output a motion vector.

The original image, the locally decoded image and the like input from the outside are input to the macroblock motion vector detector 43. The macroblock motion vector detector 43 is configured as shown in FIG.
Performs the same operation as that of the macroblock detector 11 to generate a motion vector of the macroblock from the input original image, the locally decoded image, and the like. Also, the block AD value and the like created by the calculation at the time of detecting the macroblock motion vector are:
It is output to the motion vector information buffer 42 as in the first embodiment. The macroblock motion vector generated by the macroblock detector 43 is sent to the changeover switch 44.

The motion vector information buffer 42 performs the same operation as the motion vector information buffer 12 of FIG.
Values, etc., and accumulate these values as required in the block motion vector detection / determination unit 41, the block B0 determination unit 45a,
Block B1 determiner 45b, Block B2 determiner 45
c, Output necessary data to the block B3 determiner 45d.

The block motion vector detection / judgment unit 41 performs the same operation as the block motion vector detection / judgment unit 13 shown in FIG.
Value information is obtained, and the average AD_ave is obtained to obtain a value MB_dist indicating the state of the above-described macroblock.
By comparing MB_dist with a predetermined threshold value Tha determined by the properties of the input image, coding conditions, and the like, the block
The ME execution flag is output to the changeover switch 44. That is, the value MB_dist of the state of the macroblock is equal to the threshold Th
At this time, the Block ME execution flag is set to “ON” (Block ME execution).

The changeover switch 44 is set to “ON” (Bl) in accordance with the Block ME execution flag from the block motion vector detection / determination unit 41, similarly to the changeover switch 14 in FIG.
When ockME is executed), switching to the selected terminal a is performed, and when “OFF”, switching to the selected terminal b is controlled. The macroblock motion vector from the selected terminal b of the changeover switch 44 is output to the outside, and the macroblock motion vector from the selected terminal a is sent to the changeover switches 46a, 46b, 46c, 46d. These changeover switches 46a, 46b, 46
c and 46d are the block B0 determiner 45a and the block B
The switching is controlled by the 1 determiner 45b, the block B2 determiner 45c, and the block B3 determiner 45d, respectively.

Block B0 determiner 45a, block B1
Judge 45b, Block B2 Judge 45c, Block B
The 3 determiner 45d is an AD value AD0, AD1, AD2, AD3 for each of the blocks B0, B1, B2, B3 from the motion vector information buffer 42, and an average value thereof.
AD_ave, and when the absolute value of the difference is smaller than a predetermined threshold value (for example, Thb), it is determined that the accuracy is sufficient even if the macroblock motion vector is substituted for the block motion vector of the block, and the flag is set. “OF
F ”is output. The absolute value of the difference is equal to a predetermined threshold (Thb).
For the blocks described above, a flag “ON” is output to detect a block motion vector. The changeover switches 46a, 46b, 46c, 46d are provided with the corresponding block B0 determiner 45a and block B1 determiner 45.
b, block B2 determiner 45c, block B3 determiner 4
When the output from 5d is the above flag "ON", the selected terminal a
In addition, when the flag is OFF, switching to the selected terminal b is controlled.

Changeover switches 46a, 46b, 46c, 4
The output from each selected terminal 6d is sent to a block B0 motion vector detector 47a, a block B1 motion vector detector 47b, a block B2 motion vector detector 47c, and a block B3 motion vector detector 47d, respectively. Outputs from the selected terminals b of 46a, 46b, 46c and 46d are taken out. The block B0 motion vector detector 47a, the block B1 motion vector detector 47b, the block B2 motion vector detector 47c, and the block B3 motion vector detector 47d
The same operation as that of the block motion vector detector 15 of FIG. 1 is performed, but the blocks B0, B1, and B
2, in that block motion vector detection is performed only for B3. These block B0 motion vector detector 47a and block B1 motion vector detector 4
7b, the output from the block B2 motion vector detector 47c, the output from the block B3 motion vector detector 47d, or the output from each of the selected terminals b of the change-over switches 46a, 46b, 46c, 46d is the respective block B0, B
It is output to the outside as a motion vector for each of 1, B2, and B3.

The operation of the second embodiment of the present invention having the configuration shown in FIG. 4 will be described with reference to the flowchart of FIG. The processing content 50 in FIG. 5 is a unique part of the second embodiment.

In the first step S51 of FIG. 5, a macroblock motion vector is detected. At that time, as described above, the sums AD0, AD1, AD2, and AD3 of the absolute values of the differences for each pixel for the four blocks B0, B1, B2, and B3
Ask for.

In the next step S 52, the average AD_ave of the sum of absolute differences of the four blocks is obtained, and in step S 53
Then, the absolute value of the difference between the sum of absolute differences AD of the four blocks and the average AD_ave of the sum of absolute differences is obtained, and it is determined whether or not the sum is smaller than a first threshold value Th. When YES is determined in the step S53, the process proceeds to a step S54 to output a macroblock motion vector and end the process.

When NO is determined in the step S53, the block number i is initialized (i =
0), and then proceed to step S56. In step S56,
The absolute value of the difference between the sum ADi of the difference absolute values for the i-th block and the average AD_ave is obtained, and this is the second threshold value Thb
It is determined whether or not it is smaller.

If YES is determined in the step S56, the process proceeds to a step S57, and the accuracy is sufficient even if the macro block motion vector MV is substituted for the block motion vector MVi of the block.
The process proceeds to step S59 as MV.

If NO is determined in the step S56, the process proceeds to a step S58, and for the i-th block,
For example, the block motion vector MVi is detected by the method described with reference to FIG. Next step S59
It is determined whether or not the block is the last block (i = 3), and the block number i is incremented (i = i + 1) until the last block is reached, and the process returns to step S56. The process ends after YES (the last block) is determined in step S59. Note that the first and second threshold values Th,
For Thb, an appropriate value is substituted according to the properties of the image, the encoding conditions, and the like.

In such a second embodiment,
A block motion vector detection / determination unit 41 in FIG.
The determination units 45a to 45d for each block determine whether a macroblock motion vector can be used as a block motion vector for each macroblock and each block. That is, for each block, the entire macroblock and each block are determined twice.

On the other hand, by omitting the determination for the entire macroblock, the apparatus can be simplified as shown in FIG.

FIG. 6 is a block diagram showing a configuration example of a motion vector detecting device as a modification of the second embodiment.

The motion vector detecting device shown in FIG.
Block motion vector detection / determination unit 4 in configuration of FIG.
1 and the changeover switch 44 are omitted, but the other configuration is the same as that of FIG. 4 and performs substantially the same operation as that of FIG. That is, the motion vector detection device shown in FIG. 6 is a modified example in which the detection determination of the block motion vector for the entire macroblock is omitted in the device of FIG.
In FIG. 4, the same operation is performed as when the output of the block motion vector detection / determination unit 41 is always "ON".

The operation of the modification of the second embodiment as shown in FIG. 6 will be described with reference to the flowchart shown in FIG. Processing content 7 in the flowchart of FIG.
5 is the processing content 50 of the flowchart of FIG.
And the other parts are the same.

In FIG. 7, the first step S71
Now, we will detect the macroblock motion vector,
The sums AD0 to AD3 of the difference absolute values for each pixel for the four blocks B0, B1, B2, and B3 are obtained.

In the next step S72, the block number i
Is initialized (i = 0), and the process proceeds to step S73. In step S73, the absolute value of the difference between the sum ADi of the difference absolute values for the i-th block and the above average AD_ave is determined, and it is determined whether or not this is smaller than a predetermined threshold Th.

If YES is determined in the step S73, the process proceeds to a step S74, in which the macro block motion vector MV is substituted for the block motion vector MVi of the block (MVi = MV), and the process proceeds to a step S77. If NO is determined in the step S73, the process proceeds to a step S76 to detect a block motion vector MVi for the i-th block. In the next step S77, it is determined whether or not the block is the last block (i = 3), and the block number i is incremented (i
= I + 1) and returns to step S73. Step S
After it is determined 77 as YES (the last block), the processing is terminated. It should be noted that an appropriate value is substituted for the threshold value Th according to the characteristics of the image, the encoding conditions, and the like.

Next, a third embodiment of the present invention will be described with reference to FIGS. The third embodiment is an apparatus for performing motion vector detection, which is necessary when creating a bit stream of an image coding method using motion compensation, as in the first embodiment.

The motion vector detecting device according to the third embodiment of the present invention shown in FIG. 8 operates substantially the same as the motion vector detecting device shown in FIG. The difference is that an Intra decision unit 81 based on AD_intra of the original image, which determines whether to select the Intra encoding mode by using the properties of the original image, is added before the above.

Intra decision unit 81 based on AD_intra of original image
From the original image input from the outside, select the image area of the macro block currently being determined, find the AD_intra of the macro block, compare with a certain threshold, whether to select the Intra encoding mode judge.

To find the AD_intra of a macroblock,
First, let the pixel value of the original image of this macroblock be pixel [n]
[m] (n = 0... 16, m = 0... 16), and the average is defined as pixel_ave. Here, the AD value AD_intra for the Intra encoding mode is determined by the following equation (2).

[0101]

(Equation 2)

If the value of AD_intra is equal to or smaller than a predetermined threshold Th, it is determined that the image distribution is flat.
a Select the encoding mode. This is because, when the distribution of the image is flat, a malfunction in the detection of the motion vector is likely to occur, and the noise is easily noticeable. This is because the image quality is improved by selecting the Intra encoding mode.

If the value of AD_intra is smaller than the threshold value Th, the macroblock for which the motion vector is to be detected is sent to the macroblock motion vector detector 82, and is used in the first and second embodiments. The motion vector of the macroblock and the motion vector of the block are detected by the above-described motion vector detection method.

Macro block motion vector detector 82
Performs an operation similar to that of the macroblock motion vector detector 11 shown in FIG. 1, receives an externally input original image and a locally coded image, inputs a macroblock motion vector to a switch 85, and generates a macroblock motion vector when detecting a motion vector. The information such as the adjusted AD value is sent to the motion vector information buffer 83.

The motion vector information buffer 83 performs the same operation as the motion vector information buffer 12 of FIG.
AD described above from the macroblock motion vector detector 82
Accumulates data generated when detecting motion vectors such as values,
It outputs to the motion vector detection decision unit 84 and the block motion vector detector 86 as needed.

The changeover switch 85 of FIG. 8 performs the same operation as the changeover switch 14 of FIG. 1, receives the Block ME execution flag from the block motion vector detector decision unit 84, and sets the Block ME execution flag to “ON”. In this case, the macro block motion vector is output to the block motion vector detector 86, and the block motion vector is detected. Bloc
If the kME execution flag is “OFF”, the macro block motion vector is output to the outside as a block motion vector.

The block motion vector detector 86 performs the same operation as the block motion vector detector 15 shown in FIG. 1, and outputs the detected block motion vector to the outside.

FIG. 9 is a flowchart for explaining the operation of the third embodiment of the present invention.

In the first step S91 of FIG. 9, AD_intra of the original image is calculated. In the next step S92,
It is determined whether or not the obtained AD_intra is smaller than the predetermined threshold Th. When YES is determined in the step S92, the process proceeds to a step S93, and the macro block type Mb Type is set to the Intra encoding mode. Step S9
If NO is determined in step S2, the process proceeds to step S94, in which a macroblock motion vector is detected and each block B is detected, as in the first and second embodiments.
The sum AD0, AD1, AD2, AD3 of the difference absolute values of each pixel for 0, B1, B2, and B3 is obtained, and the next step S9
At 5, a motion vector is detected for each block.

The motion vector detecting device as described above is used as the motion vector detector 102 of the image coding device (MPEG encoder) shown in FIG. The device can be configured.

Also in the case of the image encoding apparatus according to the embodiment of the present invention, as in the case of the above-described embodiment of the motion vector detection apparatus, necessary By performing block motion vector detection only on a block having the characteristic (second detection unit area), it is possible to reduce the amount of calculation and increase the processing speed.

The present invention is not limited to the above embodiment. For example, in the above embodiment, a macro block of 16 × 16 pixels and a second motion vector are used as a first motion vector detection unit. Although a block of 8 × 8 pixels is used as a vector detection unit, each detection unit area is not limited to these numerical values. In addition, the present invention relates to a minimum unit of motion vector detection (corresponding to a block in the above embodiment. )), A motion vector is detected once by combining several minimum units (corresponding to a macroblock in the above embodiment), and the motion vector and the motion vector of the aggregate are detected at the time of detection. The present invention can be widely applied to all motion vector detection methods that determine the determination of a lower-order motion vector detection method based on generated data. In addition, it goes without saying that various changes can be made without departing from the spirit of the present invention.

[0113]

As is apparent from the above description, according to the present invention, a predicted image is obtained from a temporally different reference image from an encoding target image to be subjected to image encoding with motion compensation. When detecting a motion vector for detecting a motion vector, at least a first detection unit area (for example, a macro block) is used as a detection unit area serving as a unit for detecting the motion vector.
And a second detection unit area (for example, a block) obtained by dividing the first detection unit area. A first motion vector for the first detection unit area is detected and detected. It is determined whether to detect the second motion vector for the second detection unit area in the first detection unit area based on the first motion vector obtained, By performing the detection of the second motion vector for the second detection unit area determined to be detected, the second motion vector detection is performed only for the necessary second detection unit area (block). By doing so, the amount of calculation at the time of motion vector detection can be reduced, and the execution time can be shortened.

Therefore, according to the present invention, the processing required for encoding can be reduced by adaptively changing the motion vector detection method using the properties of the image and known information.
It is possible to improve the image quality by allocating the processing to places where image quality degradation is likely to occur due to image compression in the frame. The present invention relates to software and the like.
This is particularly effective when performing image coding.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a schematic configuration of a motion vector detection device according to a first embodiment of the present invention.

FIG. 2 is a flowchart illustrating an operation of the motion vector detection device according to the first embodiment.

FIG. 3 is a flowchart illustrating an operation of a motion vector detection device according to a modification of the first embodiment.

FIG. 4 is a block diagram illustrating a schematic configuration of a motion vector detection device according to a second embodiment of the present invention.

FIG. 5 is a flowchart illustrating an operation of the motion vector detection device according to the second embodiment.

FIG. 6 is a block diagram illustrating a schematic configuration of a motion vector detection device according to a modification of the second embodiment.

FIG. 7 is a flowchart illustrating an operation of a motion vector detection device according to a modification of the second embodiment.

FIG. 8 is a block diagram illustrating a schematic configuration of a motion vector detection device according to a third embodiment of the present invention.

FIG. 9 is a flowchart illustrating an operation of the motion vector detection device according to the third embodiment.

FIG. 10 is a block diagram illustrating a schematic configuration of an MPEG encoder as an example of an image encoding device.

FIG. 11 is a diagram for explaining macro blocks and blocks.

FIG. 12 is a block diagram illustrating a configuration example used for detecting a motion vector of a macroblock and a block.

FIG. 13 is a diagram for explaining a macroblock motion vector detection operation.

FIG. 14 is a diagram for explaining a block motion vector detection operation.

[Explanation of symbols]

11, 43, 62, 82 macroblock motion vector detector, 12, 42, 61, 83 motion vector information buffer, 13, 41, 84 block motion vector detection determiner, 14, 44, 46a to 46d, 64a to
64d, 85 changeover switch, 15, 47a-47d,
65a-64d, 86 block motion vector detector,
45a to 45d, 63a to 63d block determiners,
102 Motion vector detector

 ────────────────────────────────────────────────── ─── Continuing on the front page (72) Inventor Yoichi Yagasaki 6-35, Kita-Shinagawa, Shinagawa-ku, Tokyo Inside Sony Corporation (72) Inventor Takefumi Nagumo 7-35, Kita-Shinagawa, Shinagawa-ku, Tokyo No. Sony Corporation F term (reference) 5C059 KK15 MA00 MA23 MC11 MC38 ME01 NN02 NN28 PP04 SS07 TA62 TB07 TB08 TC12 TD03 TD05 TD06 TD12 UA02 UA33

Claims (14)

[Claims] 1. A motion vector detecting method for detecting a motion vector for obtaining a predicted image from a temporally different reference image from an encoding target image to be subjected to image encoding with motion compensation, The first detection unit area includes at least a first detection unit area and a second detection unit area obtained by dividing the first detection unit area as a detection unit area serving as a unit of motion vector detection. A first motion vector detecting step of detecting a first motion vector of the unit area, and a step of detecting the second motion unit area in the first detection unit area based on the detected first motion vector. Second
A determination step of determining whether or not to detect the motion vector of the second detection unit; and a second motion vector of the second detection unit area determined to perform the detection of the second motion vector in the determination step. And a second motion vector detecting step of detecting a motion vector. 2. The method according to claim 1, wherein the reference image is a locally decoded image of a previously encoded frame.
The described motion vector detection method. 3. The apparatus according to claim 1, wherein the first detection unit area is a macro block of 16 × 16 pixels, and the second detection unit area is a block of 8 × 8 pixels.
The described motion vector detection method. 4. In the first motion vector detecting step,
2. The motion vector detection method according to claim 1, wherein an error between the encoding target image and the reference image is obtained for each of the second detection unit areas, and the obtained error is used for the determination in the determination step. 5. The method according to claim 1, wherein the determining step determines whether to perform a second motion vector detection for the first detection unit area based on an error between an encoding target image of the first detection unit area and a reference image. When it is determined that the second motion vector detection is to be performed, it is determined whether to perform the second motion vector detection for each second detection unit area, and the second motion vector detection is performed. Performing a second motion vector detection for the second detection unit area determined to be performed, and using the first motion vector for the other second detection unit areas as a motion vector of the second detection unit area The method according to claim 1, wherein: 6. The motion vector detecting method according to claim 1, wherein the determining step determines whether or not to detect a second motion vector using the encoding target image. 7. The determining step determines whether the prediction method of the first detection unit area is intra-coding or inter-coding by using the encoding target image, and uses a result of the determination to perform a second 7. The motion vector detecting method according to claim 6, wherein it is determined whether or not to detect a motion vector. 8. A motion vector detecting apparatus for detecting a motion vector for obtaining a predicted image from a temporally different reference image from an encoding target image to be subjected to image encoding with motion compensation, The first detection unit area includes at least a first detection unit area and a second detection unit area obtained by dividing the first detection unit area as a detection unit area serving as a unit of motion vector detection. First motion vector detection means for detecting a first motion vector for the unit area; and a first motion vector detection means for detecting the second detection unit area in the first detection unit area based on the detected first motion vector. Second
Determining means for determining whether or not to perform the detection of the motion vector, and a second motion vector for the second detection unit area determined to perform the detection of the second motion vector by the determining means. And a second motion vector detecting means for detecting a motion vector. 9. The image processing method according to claim 8, wherein the reference image is a locally decoded image of a previously encoded frame.
The motion vector detecting device as described in the above. 10. The first detection unit area is 16 × 16.
9. The motion vector detecting device according to claim 8, wherein the motion vector detecting device is a macro block of pixels, and the second detection unit area is a block of 8 × 8 pixels. 11. The first motion vector detecting means,
9. The motion vector detecting device according to claim 8, wherein an error between the encoding target image and the reference image is obtained for each of the second detection unit areas, and the obtained error is used for the judgment by the judging means. 12. An image coding apparatus which obtains a predicted image using a motion vector from a reference image temporally different from an encoding target image and performs image coding with motion compensation. Has at least a first detection unit area and a second detection unit area obtained by dividing the first detection unit area. The first detection unit area A first motion vector detecting means for detecting a first motion vector of the second detection unit area based on the detected first motion vector.
Determining means for determining whether or not to perform the detection of the motion vector, and a second motion vector for the second detection unit area determined to perform the detection of the second motion vector by the determining means. And a second motion vector detecting means for detecting a motion vector. 13. The image encoding apparatus according to claim 12, wherein the reference image is a locally decoded image of a previously encoded frame. 14. The first detection unit area is 16 × 16.
13. The image coding apparatus according to claim 12, wherein the image detection apparatus is a macro block of pixels, and the second detection unit area is a block of 8 × 8 pixels.