JP3309519B2 - Motion vector detection device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a motion vector detecting apparatus suitable for use in predictive encoding of an image, such as the MPEG system, and more particularly to detecting a motion vector with an accuracy of one pixel or less. The present invention relates to a motion vector detection device capable of reducing the number of calculation processes.

[0002]

2. Description of the Related Art The MPEG (Moving Picture Coding Experts Group) system is known as an international standard system for efficient coding of images. The MPEG system uses DCT (Discrete
Cosine Transform) is a high-efficiency compression technique for images that combines transform and motion compensation.

FIG. 11 shows an example of a predictive encoding apparatus that encodes an image using a correlation with another frame, as in the MPEG system. In FIG. 11, image data is supplied to an input terminal 101. This image data is supplied to the motion vector detection circuit 102 and the subtraction circuit 10
3 is supplied. The motion vector detection circuit 102 calculates a motion vector between the current frame and the reference frame.
This motion vector is supplied to the motion compensation circuit 104.

On the other hand, image data of a reference frame is stored in a frame memory 105. Frame memory 1
The output of 05 is supplied to the motion compensation circuit 104. In the motion compensation circuit 104, the image data of the reference frame from the frame memory 105 is motion-compensated based on the motion vector obtained by the motion vector detection circuit 102.
The motion-compensated image data is supplied to a subtraction circuit 103 and an addition circuit 106.

The subtraction circuit 103 is supplied with the image data of the current frame from the input terminal 101 and the image data of the reference frame which has been motion-compensated by the motion compensation circuit 104. In the subtraction circuit 103, the image data of the current frame and the image data of the motion-compensated reference frame are subtracted to obtain difference data between the current frame and the reference frame. This difference data is supplied to the DCT circuit 107. This difference data is DCT-transformed by the DCT circuit 107. The output of the DCT circuit 107 is supplied to a quantizer 108. The quantizer 108 quantizes the output of the DCT circuit 107. The output of the quantizer 108 is output from an output terminal 109.

The DCT-transformed and quantized difference data is supplied to an inverse quantizer 110 and an inverse DCT circuit 11.
1 and returned to the original difference data,
6. The addition circuit 106 includes a motion compensation circuit 1
04 supplies the image data of the reference frame. The addition circuit 106 adds the difference data between the reference frame and the current frame to the image data of the reference frame to obtain the image data of the current frame. The obtained image data of the current frame is stored in the frame memory 105 as the next reference frame.

[0007] As described above, in the inter-frame predictive coding process, the difference between the reference frame motion-compensated based on the motion vector and the current frame is coded. As a method of detecting a motion vector used in such an inter-frame predictive encoding process, a check block of a reference frame is moved within a predetermined vector search range, and a block that best matches a reference block of the current frame is determined. There is known a block matching method for obtaining a motion vector by detecting.

FIG. 12 illustrates such a block matching method. In FIG.
1 indicates a current frame, and 122 indicates a reference frame. A reference block 123 is set in the current frame 121, and an inspection block 124 is set in the reference frame 122. The inspection block of the reference frame 122 is moved around within a predetermined vector search range. Then, the reference block 123 of the current frame 121 and the reference frame 12
The degree to which the second inspection block 124 matches is detected. The inspection block 124 that best matches the reference block 123 is determined as a matching block. A motion vector is obtained from the matching block.

FIG. 13 shows an example of a conventional motion vector detecting apparatus using full search. In the full search, the inspection block of the reference frame is moved over the entire area within a predetermined vector search range, and a matching block is detected.

In FIG. 13, reference numeral 131 denotes a current frame memory for storing image data of the current frame. The current frame memory 131 is supplied with the image data of the current frame from the input terminal 133.

Reference numeral 132 denotes a reference frame memory. The reference frame memory 132 is supplied with image data of the reference frame from the input terminal 134. Current frame memory 1
The reading / writing of the reference frame memory 132 and the reference frame memory 132 is controlled by the system controller 135.
From the current frame memory 131, the image data of the reference block of the current frame is read. From the reference frame memory 132, the image data of the inspection block of the reference frame is read. This reference frame is
It is moved within a predetermined vector search range.

The outputs of the current frame memory 131 and the reference frame memory 132 are supplied to a difference circuit 137. The difference circuit 137 determines that the pixel data of the reference block of the current frame and the pixel data of the inspection block of the reference frame are 1
It is subtracted for each pixel. The output of the difference circuit 137 is supplied to the absolute value sum circuit 138. The absolute value sum circuit 138 calculates the absolute value sum of the difference between the pixel data of the reference block of the current frame and the pixel data of the inspection block of the reference frame. The output of the absolute value sum circuit 138 is used as a comparison and
9.

The inspection block 141 for the reference frame is 1
A predetermined vector search range is moved for each pixel. The comparison and determination circuit 139 determines the test block closest to the reference block from the sum of the absolute values of the differences between the pixel data of the reference block of the current frame and the pixel data of the test block of the reference frame. This test block is used as a matching block, and a motion vector is determined from the matching block. The obtained motion vector is output terminal 1
It is output from 40.

In the above example, since the inspection block is moved one pixel at a time, the motion vector detection accuracy is one pixel accuracy. By inserting interpolated pixel data obtained as described below between each pixel, the inspection block is reduced to 1 /
Processing equivalent to moving two pixels at a time can be performed, and a motion vector can be detected with half-pixel accuracy.

[0015] That is, in FIG. 14, the pixel data H _{-1 / 2, 0} between the pixel data D _{-1, 0} and the pixel data D _0,0 arranged in the horizontal _direction, H _{-1 / 2,0} = (D- _1,0 + _D0,0 ) / 2 (1) The pixel data D arranged in the vertical direction
Pixel data H between _0,1 and pixel data D _{0,0
0,1 / 2} is obtained as H _{0,1 / 2} = (D _0,1 + D _0,0 ) / 2 (2). Further, the pixels D _-1,1 , D _0,1 ,
D _0,0, pixel data H _{-1 / 2,1 / 2} in D _{-1, 0 is, H -1 / 2,1 / 2 =} (D -1,1 + D 0,1 + D 0,0 + D- _1,0 ) / 4 (3)

Therefore, as shown in FIG. 15, an interpolation circuit 141 for two pixels, an interpolation circuit 142 for four pixels, and a switch circuit 143 are provided. The two-pixel interpolation circuit 141
Based on Equations (1) and (2), two pixels arranged in the vertical and horizontal directions are used to obtain a pixel therebetween by interpolation. The four-pixel interpolation circuit 142 obtains pixels between the four pixels by interpolation based on equation (3).

As described above, when the two-pixel interpolation circuit 141 and the four-pixel interpolation circuit 142 are provided, the number of inspection blocks is one.
It is possible to move every half pixel, and a motion vector can be obtained with half-pixel accuracy.

[0018]

As described above, conventionally, interpolation data is formed between two pixel data arranged in the vertical and horizontal directions as shown in equations (1) and (2). (Hereinafter referred to as 2-pixel interpolation),
As shown in equation (3), interpolation data is formed using the four pixels (hereinafter referred to as four-pixel interpolation), and a half-precision motion vector is detected. Conventionally, two-pixel interpolation and four-pixel interpolation are required to detect a motion vector with half-pixel accuracy, which causes a problem that the amount of calculation increases and the circuit scale increases.

Accordingly, it is an object of the present invention to provide a vector detecting device capable of reducing the amount of calculation without lowering the accuracy of detecting a motion vector and reducing the circuit scale.

[0020]

According to the present invention, there is provided reference block reading means for extracting image data of a reference block from image data of a current frame, and inspection block reading means for extracting image data of an inspection block from image data of a reference block. , an interpolation pixel forming means for forming an interpolation pixel between a pixel of the image data of the test block, the interpolation pixel forming means is for forming an interpolation pixel in the horizontal and vertical directions with respect to the current pixel, the current Pixel-centered region and interpolation
Error detection means for detecting an error between a reference block and an inspection block in a region centered on a pixel, and detecting an inspection block most matching the reference block with an accuracy of less than one pixel from an output of the error detection means This is a motion vector detecting device.

[0021]

Operation Only interpolation of two pixels in the horizontal and vertical directions is performed, and interpolation of four pixels in the peripheral direction is not performed. This allows
The amount of calculation can be greatly reduced, and the circuit scale can be reduced. That is, since interpolation of four pixels is not performed, the search point is 50%. Therefore, by itself, the circuit scale can be reduced to 50%. Further, compared to the interpolation of four pixels, the interpolation of two pixels requires half the amount of calculation.

[0022]

An embodiment of the present invention will be described below with reference to the drawings. FIG. 1 shows an embodiment of the present invention. In FIG. 1, reference numeral 1 denotes a current frame memory for storing image data of the current frame. Current frame memory 1
Is supplied with the image data of the current frame from the input terminal 3.

Reference numeral 2 denotes a reference frame memory. The reference frame memory 2 is supplied with the image data of the reference frame from the input terminal 4. Reading / writing of the current frame memory 1 and the reference frame memory 2 is controlled by the system controller 5. From the current frame memory 1, the image data of the reference block of the current frame is read. From the reference frame memory 2, the image data of the inspection block of the reference frame is read. The inspection block of this reference frame is moved within a predetermined vector search range as shown in FIG.

The pixel data of the reference block is read from the current frame memory 1, and the pixel data of the reference block is supplied to the difference circuit 7. The pixel data of the test block is read from the reference frame memory 2, and the pixel data of the test block is supplied to the interpolation circuit 8 of two pixels and to the terminal 9 A of the switch circuit 9.

The two-pixel interpolation circuit 8 uses the two pixels arranged in the horizontal direction and the vertical direction, and uses the above-described equations (1) and (2).
As shown by the equation, interpolation of two pixels is performed. When the switch circuit 9 is set to the terminal 9A side, actual pixel data read from the reference frame memory 2 is output via the switch circuit 9. Switch circuit 9 to terminal 9
When set to the B side, the interpolated pixel data output from the two-pixel interpolation circuit 8 is output via the switch circuit 9.
Therefore, the inspection block of the reference frame can perform a process equivalent to moving within the vector search range for each half pixel.

In the embodiment of the present invention, the two-pixel interpolation circuit 8 performs only interpolation of two pixels arranged in the horizontal and vertical directions, and does not perform interpolation of four pixels. this is,
This is because, as will be described in detail later, it has been confirmed that the motion vector detection accuracy can be sufficiently maintained only by interpolation of two pixels. Conventionally, when detecting a motion vector with half-pixel accuracy, pixel data between two pixels arranged in the horizontal and vertical directions is formed by interpolation, and pixel data is formed by interpolation using four pixels. I was For this reason, the amount of calculation has increased and the circuit scale has increased. In the embodiment of the present invention, since only interpolation of two pixels is performed, the amount of calculation can be reduced, and the circuit scale can be reduced.

In FIG. 1, the output of the switch circuit 9 is supplied to a difference circuit 7. The difference circuit 7 subtracts the pixel data of the reference block of the current frame from the pixel data of the inspection block of the reference frame. The output of the difference circuit 7 is supplied to the absolute value sum circuit 10. The absolute value sum circuit 10 calculates the absolute value sum of the difference between the pixel data of the reference block of the current frame and the pixel data of the inspection block of the reference frame. The output of the absolute value sum circuit 10 is used as a comparison and judgment circuit 11
Supplied to The comparison and determination circuit 11 determines the test block closest to the reference block. Thus, a motion vector is obtained. The determined motion vector is
Output from the output terminal 12.

The detection of a motion vector of 1/2 pixel accuracy is performed by moving a test block within a predetermined vector search range and detecting a test block closest to a reference block, as well as a motion vector of 1 pixel accuracy. Is calculated in the vicinity of the obtained motion vector with 1-pixel accuracy.
A step search for detecting a motion vector with two-pixel accuracy may be performed.

FIG. 3 shows that a two-step search has performed
The motion vector of the pixel is obtained. In FIG. 3, reference numeral 21 denotes a current frame memory for storing image data of the current frame. The current frame memory 21 has an input terminal 2
3 supplies the image data of the current frame.

Reference numeral 22 denotes a reference frame memory. The reference frame memory 22 is supplied with the image data of the reference frame from the input terminal 24. Reading / writing of the current frame memory 21 and the reference frame memory 22 is controlled by the system controller 25. From the current frame memory 21, the image data of the reference block of the current frame is read. From the reference frame memory 22, the image data of the inspection block of the reference frame is read. This reference frame is moved within a predetermined vector search range.

26 is a one-pixel precision motion vector detecting circuit for detecting a motion vector with one-pixel precision, and 27 is
This is a half-pixel accuracy motion vector detection circuit that detects a motion vector with half-pixel accuracy using interpolation pixels in the vicinity of the motion vector obtained by the one-pixel accuracy motion vector detection circuit 26. 1-pixel precision motion vector detection circuit 26
And a half-pixel precision motion vector detecting circuit 27 calculates an absolute value sum (or square sum) of a difference between each pixel of the test block and each pixel of the reference block, and obtains a test block closest to the reference block. It is. Further, the half-pixel accuracy motion vector detection circuit 27 includes a two-pixel interpolation circuit that uses two pixels arranged in the horizontal direction and the vertical direction, and forms data between them by interpolation.

First, the pixel data of the reference block from the current frame memory 21 is transferred to the one-pixel precision motion vector detecting circuit 2.
6 is read. 1-pixel precision motion vector detection circuit 2
At 6, the test block is moved within a predetermined vector search range for each pixel, and the test block that best matches the reference block is detected. As a result, a motion vector with one-pixel accuracy is obtained. When a one-pixel accuracy motion vector is obtained, the one-pixel accuracy motion vector detection circuit 2
6 is supplied to a 1/2 pixel precision motion vector detection circuit 27.

[0033] 1/2 pixel precision motion vector detection circuit 27
Is provided with a two-pixel interpolation circuit. Using the two-pixel interpolation circuit, the inspection block is moved every half pixel in the vicinity of the motion vector obtained with one-pixel accuracy in the half-pixel accuracy motion vector detection circuit 27. Then, the inspection block that best matches the reference block is detected. As a result, a motion vector with 1/2 pixel precision is obtained. The half-pixel motion vector is output from the output terminal 28.

As described above, in the embodiment of the present invention, interpolation data is formed using only two pixels arranged in the horizontal and vertical directions. Even if interpolation data is formed using only two pixels arranged in the horizontal and vertical directions, the detection accuracy of the motion vector is sufficiently maintained.

FIG. 4 illustrates this. In FIG. 4, pixels P1, P2, P3,.
Then, it is assumed that the inspection block is moved for each pixel, a matching block is detected, and a motion vector is obtained.

Now, it is assumed that as a result of detecting such a motion vector with one-pixel accuracy, a test block centered on the pixel P5 is determined as a matching block, and the vector V5 is detected as a motion vector. In this case, the correct motion vector V is in any of the regions A5.

When an interpolation pixel is used, a motion vector with higher accuracy is detected. In FIG. 5, a “black circle” mark is an actual pixel, a “x” pixel is a pixel interpolated from two pixels arranged in the horizontal or vertical direction, and a “white circle” pixel is Pixels interpolated from four pixels. That is, for example, the interpolation pixel Q2 of the "X" mark is obtained as Q2 = (P2 + P5) / 2. Further, the interpolation pixel Q1 marked with a “white circle”
Is obtained as Q1 = (P1 + P2 + P5 + P4) / 4.

Assuming that a vector V5 is detected as a motion vector as a result of detecting a motion vector with one pixel accuracy,
Further, the interpolation pixels Q1, Q2, Q3, Q4, Q6, Q7,
The inspection block is moved every 画素 pixel around Q8 and Q9, and a matching block is detected. As a result of the detection of such a 1/2 pixel motion vector, if the correct motion vector is in the area B8, for example, the inspection block centered on the interpolation pixel Q8 becomes the matching block.

As described above, when a motion vector with 1/2 pixel precision is detected, the area A5 is further divided into nine areas B1 to B9.
Is divided into Correct motion vectors exist in these nine areas B1 to B9. As can be seen from the figure, of these areas B1 to B9, the area of the area B5 including the actual pixel P5 is the largest. The next largest area is a region B2, B4, B6, B8 including two interpolated pixels Q2, Q4, Q6, Q8. Regions B1, B3, B including four interpolation pixels Q1, Q3, Q7, Q9
7, the area of B9 is the smallest. A region B5 including an actual pixel P5 and two interpolated pixels Q2, Q4, Q6, and Q8
When combined with the regions B2, B4, B6, and B8 including Therefore, the area B5 including the actual pixel P5 and the two interpolated pixels Q2,
If a motion vector is detected in a region including regions B2, B4, B6, and B8 including Q4, Q6, and Q8, a correct motion vector can be detected with high probability. By doing so, the four interpolation pixels Q1, Q3, Q7, Q
Since the regions B1, B3, B7, and B9 including the region No. 9 need not be searched, the amount of calculation can be reduced by that amount, and the accuracy of vector detection does not deteriorate much.

That is, for example, the size of a block is set to (1
(6 × 16), (16 × 16) calculations are required to determine the error between one test blot and the reference block for each pixel. Therefore, interpolation of four pixels and interpolation of two pixels are all performed, and eight interpolation pixels Q1, Q2, Q
When the error between the test block and the reference block centered at 3, Q4, Q6, Q7, Q8, and Q9 is obtained, ((16 ×
16) × 8) operations are required. On the other hand, in the present invention, since only interpolation of two pixels is performed, an error between the inspection block and the reference block centered on the interpolation pixels Q2, Q4, Q6, and Q8 can be obtained.
× 16) × 4) operations are sufficient.

In order to verify the above, the present inventor performed a simulation. 6 to 8 show simulation results. The simulation is used for motion compensation in MPEG2, and verifies the detection accuracy of the motion vector from the S / N ratio. The coding conditions are as follows. Method: MPEG2, M = 3 N = 15 Rate: 8 Mbps Measurement: S / N ratio of P picture Sequence: Picture 1 (Fig. 6 (mobile calendar1)) Picture 2 (Fig. 8 (mobile calendar2)) Picture 3 (Fig. 9 ( flower garden))

FIGS. 6 to 8 show the case where the motion vector detection with 1/2 pixel precision is not performed and the case where the motion vector detection is performed with only 2 pixels.
What is the improvement amount of the S / N ratio between when the motion vector detection with pixel accuracy is performed and when the motion vector detection with 1/2 pixel accuracy is performed by 2-pixel interpolation and 4-pixel interpolation? It is shown. In the figure, white circles indicate the amount of improvement in the frame, and black circles indicate the amount of improvement in the field. As can be seen from FIGS. 6 to 8, in any sequence and frame / field, a significant improvement in the S / N ratio can be seen only by interpolation of two pixels in the horizontal and vertical directions. Even if you interpolate 4 pixels,
The S / N ratio does not improve so much. From this, it can be seen that by interpolation of two pixels in the horizontal direction and the vertical direction, a motion vector with half-pixel accuracy can be detected with sufficient accuracy.

In the above embodiment, the error between the reference block and the test block is detected by using the sum of absolute values or the sum of squares of the differences between pixels. May be divided into small blocks, feature extraction is performed on each small block, and the matching block may be detected using the sum of absolute values or the sum of squares of the feature values. That is, the reference block and the inspection block are divided into a plurality of small blocks. The characteristic value of each small block is obtained from the sum, average, and the like of the pixel data of each small block. This feature value is calculated using the sum of absolute values or the sum of squares of the differences.
Detect matching blocks. In this case, the amount of calculation can be significantly reduced as compared with the case where a matching block is detected using the sum of absolute values or the sum of squares of the differences for each pixel.

In the above-described embodiment, the configuration is such that a motion vector with a precision of 1/2 pixel is detected. However, the present invention is also applicable to a case where a motion vector with a precision of 1/2 pixel or less is detected. And can be similarly applied.

That is, as shown in FIG. 9, if an interpolation pixel is further formed between 1/2 pixels, a motion vector can be detected with 1/4 pixel accuracy. In the figure, the “black circle” mark is an actual pixel, the “x” pixel is an interpolation pixel that requires interpolation from at least two pixels, and the “white circle” and “triangle” pixels are , Pixels for which interpolation from at least four pixels is required.

In this case, as shown in FIG. 10A, when a motion vector is detected using actual pixels and pixels interpolated from two pixels, the ratio of the number of search points is about 33% of the whole, and Point area ratio is 4
4 percent. As shown in FIG. 10B, when a motion vector is detected using an actual pixel, a pixel interpolated from two pixels indicated by a cross, and a pixel interpolated from four pixels indicated by a white circle, a search point is determined. Is about 5
At 0%, the area ratio of search points is about 6%
9 percent. Excluding only the interpolated pixels from the four pixels marked with “white circles” at the four corners, as shown in FIG. 10C, the ratio of the number of search points is about 83% of the whole, and the area ratio of the search points is the whole. About 94 percent.

As described above, by removing the actual pixels from the pixels interpolated from the four pixels at the four corners, the amount of calculation can be reduced without lowering the accuracy.

[0048]

According to the present invention, only interpolation of two pixels in the horizontal and vertical directions is performed, and interpolation of four pixels is not performed. As a result, the amount of calculation can be significantly reduced, and the circuit scale can be reduced. That is, since interpolation of four pixels is not performed,
The search point is 50%. Therefore,
This alone can reduce the circuit size to 50%.
Further, compared to the interpolation of four pixels, the interpolation of two pixels requires half the amount of calculation.

[Brief description of the drawings]

FIG. 1 is a block diagram of one embodiment of the present invention.

FIG. 2 is a schematic diagram used for describing one embodiment of the present invention.

FIG. 3 is a block diagram of another embodiment of the present invention.

FIG. 4 is a schematic diagram used for describing an embodiment of the present invention.

FIG. 5 is a schematic diagram used for describing an embodiment of the present invention.

FIG. 6 is a graph used to explain the effect of the embodiment of the present invention.

FIG. 7 is a graph used to explain the effect of the embodiment of the present invention.

FIG. 8 is a graph used to explain the effect of the embodiment of the present invention.

FIG. 9 is a schematic diagram used for describing an application example of the present invention.

FIG. 10 is a schematic diagram used for describing an application example of the present invention.

FIG. 11 is a block diagram illustrating an example of a predictive encoding device.

FIG. 12 is a schematic diagram used for describing a conventional motion vector detection device.

FIG. 13 is a block diagram of an example of a conventional motion vector detection device.

FIG. 14 is a schematic diagram used to explain a conventional 1 / 2-precision motion vector detecting device.

FIG. 15 is a block diagram of an example of a conventional half-precision motion vector detecting device.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Current frame memory 2 Reference frame memory 7 Difference circuit 8 Interpolation circuit of 2 pixels 10 Absolute value sum circuit

Claims (1)

(57) [Claims] A reference block reading unit for extracting image data of a reference block from image data of a current frame; an inspection block reading unit for extracting image data of an inspection block from image data of a reference block; Interpolated pixel forming means for forming an interpolated pixel between one pixel, and the interpolated pixel forming means for forming interpolated pixels in the horizontal direction and the vertical direction with respect to the current pixel, and an area centered on the current pixel And the area centered on the interpolation pixel
Error detecting means for detecting an error between the reference block and the inspection block in the area, and detecting an inspection block most matching the reference block with an accuracy of less than one pixel from an output of the error detecting means. Motion vector detection device.