JPWO1998042135A1 - Image encoding device, image decoding device, image encoding method, image decoding method, and image encoding/decoding system - Google Patents

Abstract

(57) [Abstract] An image encoding device and a decoding device are provided that can predict image movements other than translational movements on a relatively small scale with reduced processing time and high accuracy. To this end, the device divides an input image into predetermined blocks and includes a motion compensation prediction means for detecting motion between blocks, compressing and encoding the input image. The device includes a motion detection unit that transforms and extracts only integer pixels, which are actual sample points in a corresponding partial region of a reference image for motion detection, into a predetermined format, and compares them with the integer pixels of the block in the input image, outputting a motion vector that provides the minimum error extracted by specifying the coordinates. The device also includes a corresponding point determination unit that transforms and determines corresponding coordinates of blocks in the reference image according to motion parameters obtained from the comparison output including the transformed block matching unit, outputting a predicted partial image. The image decoding device also has a configuration corresponding to the encoding device. Furthermore, the image decoding device further includes a motion compensation means that calculates coordinates using multiple motion vectors, also using coordinate values of half pixels of the reference image, and transforms the obtained pixel values into a predetermined format using a transformation pattern.

Description

[Detailed Description of the Invention] Image Encoding Apparatus, Image Decoding Apparatus, Image Encoding Method, Image Decoding Method, and Image Encoding/Decoding System Technical Field This invention relates to an apparatus and system for highly efficient image encoding or decoding, which performs motion-compensated prediction of an image to be encoded or decoded from an existing image, and encodes the prediction error or decodes by adding the prediction error to reference image data.

Background Art This paper describes the prior art of motion compensation prediction methods for highly efficient image coding.

A first example of a conventional motion compensation prediction method is one that uses block matching based on translation. For example, ISO/IEC 11172-2 (MPEG-1 Video Standard) describes forward, backward, and interpolation motion compensation prediction methods using block matching. A second example of a conventional motion compensation prediction method is motion compensation using affine transformation. For example, "Study on Motion Compensation Prediction Using Affine Transformation" (IEICE Technical Report IE94-36) describes a method that models the amount of motion in an arbitrarily shaped region of an image using affine motion parameters and performs motion compensation prediction by detecting these parameters.

Based on these explanations, the conventional motion compensation method using translation and the motion compensation method using affine transformation will be explained below.

The concept of motion compensation prediction using block matching is shown in FIG.

In the figure, i is the on-screen position of the block that serves as the unit of motion compensation prediction, fi(x,y,t) is the pixel value at the block position (x,y) at on-screen position i and time t, R is the motion vector search range, v is the motion vector (∈R). Block matching corresponds to the process of finding the block closest to the pixel value fi(x,y,t) of the block i to be predicted in the predicted image 202 within the search range R of the reference image 201, as shown in the figure, i.e., the block fi+v(x,y,t-1) that minimizes the error power Dv shown in equation (1).

The value of v that minimizes Dv becomes the motion vector. In the figure, a method of searching for a matching block using only real pixels, which are real sample points in the reference image, is called integer-pixel precision search, while a search method that uses half-pixels, which are intermediate between integer pixels, in addition to integer pixels, is called half-pixel precision search. Generally, under the same search range conditions, half-pixel precision search results in more search points than integer-pixel precision search, resulting in higher prediction efficiency.

Figure 43 shows an example of the configuration of a motion compensation prediction unit (block matching unit) in an image coding device that uses a motion compensation prediction method, such as that adopted in the MPEG-1 video standard.

In the figure, 207 is a horizontal displacement counter, 208 is a vertical translation displacement counter, 211 is a memory read address generator, 213 is a pattern matching unit, and 216 is a minimum prediction error power determiner. Also, 203 is a horizontal translation displacement search range indication signal, 204 is a vertical translation displacement search range indication signal, 205 is predicted block data, 206 is a predicted block intra-image position signal, 209 is horizontal translation displacement search point data, 210 is vertical translation displacement search point data, 212 is a read address, 214 is read image data, 215 is a prediction error power signal, 217 is a motion vector, and 218 is a minimum prediction error power signal. 219 is a frame memory for storing a reference image.

On the other hand, FIG. 44 is an operational flowchart showing the operation of the motion compensation prediction unit having the configuration shown in FIG.

In the figure, dx is the horizontal translation amount search point, dy is the vertical translation amount search point, range h min is the lower limit of the horizontal translation amount search range, h max is the upper limit of the horizontal translation amount search range, v min is the lower limit of the vertical translation amount search range, v max is the upper limit of the vertical translation search range, D min is the minimum prediction error power, (x, y) are coordinates representing the pixel position in the macroblock, D(dx, dy) is the prediction error power during dx, dy search, f(x, y) is the value of pixel (x, y) in the predicted image, fr(x, y) is the value of pixel (x, y) in the reference image, D(x, y) is the prediction error at (x, y) during dx, dy search, MV h is the horizontal component of the motion vector (parallel movement amount), MV v is the vertical component of the motion vector (translation amount).

The operation of block matching will be described in detail below with reference to FIGS.

1) Setting of motion vector search range The horizontal translation amount search range instruction signal 203 and the vertical translation amount search range instruction signal 204 are used to set the range of the horizontal translation amount counter 207. h min /range h max is set to the vertical translation amount counter 208 v min/range v Also, set the counter initial values as dx = range h min, dy=range v The minimum prediction error power determination unit 216 sets the minimum prediction error power D min is set to the maximum value MAXINT (for example, 0xFFFFFFFF). This corresponds to S201 in FIG.

2) Reading the Candidate Predicted Image The pixel at position (x + dx, y + dy) in the reference image, which is a distance (dx, dy) from the pixel position (x, y) of the predicted macroblock, is retrieved from the frame memory. The memory read address generator 211 in Figure 43 receives the value dx from the horizontal translation counter 207 and the value dy from the vertical translation counter 208, and generates an address in the frame memory.

3) Calculating Prediction Error Power First, the prediction error power D(dx, dy) for a motion vector (dx, dy) is initialized to zero. This corresponds to S202 in Figure 44. The difference between the pixel value read in 2) and the pixel value at position (x, y) within the predicted macroblock is calculated, and the absolute value is accumulated as D(dx, dy). This process is repeated until x = y = 16 to obtain the prediction error power D(dx, dy) at (dx, dy), i.e., Dv in equation (1). This process is performed by the pattern matching unit 213 in Figure 43, which then passes D(dx, dy) to the minimum prediction error power determination unit 216 via the prediction error power signal 215. This process corresponds to S203 to S209 in Figure 44.

4) Updating the minimum prediction error power value It is determined whether D(dx, dy) obtained as a result of 3) provides the minimum error power among the search results up to that point. This determination is made by the minimum prediction error power determination unit 216 in FIG. 43. S210 in FIG. 44 corresponds to this determination process. The minimum prediction error power determination unit 216 updates the minimum prediction error power D The value of min is compared with the magnitude of D(dx, dy) delivered by the prediction error power signal 215, and only when D(dx, dy) is smaller, D The value of min is updated with D(dx, dy). The value of (dx, dy) at that time is used as the motion vector candidate (MV h, MV v) These update processes correspond to S211 in FIG.

5) Determination of the motion vector value The above steps 2) to 4) are repeated for all (dx, dy) in the motion vector search range R (S212 to S215 in FIG. 44), and the calculated value (MV) stored in the minimum prediction error power determination unit 216 is finally determined. h, MV v) is output as a motion vector 217.

Figure 45 shows an overview of the motion compensation prediction method adopted in the MPEG-1 video standard.

In the MPEG1 video standard, each frame of a moving image is called a picture, and the picture is divided into blocks of 16x16 pixels (8x8 pixels for color difference signals) called macroblocks. Motion compensation prediction is performed on each macroblock using block matching, and the resulting motion vector and prediction error signal are then coded.

The MPEG-1 video standard allows different motion compensation methods to be used for different pictures. In the diagram, I-pictures perform closed coding within the picture without motion compensation prediction. P-pictures perform forward motion compensation, which predicts from a temporally earlier image. B-pictures allow not only forward motion compensation prediction, but also backward motion compensation, which predicts from a temporally later image, and interpolated motion compensation, which predicts by averaging two predicted images obtained from forward and backward motion compensation predictions. However, forward, backward, and interpolated motion compensation are all essentially block-matching motion compensation predictions; the only difference is the reference image used for prediction.

As mentioned above, block matching has been established as the primary method for implementing motion-compensated prediction in current video coding systems. However, the block matching process is based on the isoluminance assumption that "areas of equal luminance represent the same object," and is equivalent to calculating the translational displacement of an object in square block units such as macroblocks. Therefore, it is theoretically impossible to detect movement other than in the direction of the square block shape. Prediction accuracy drops in areas where movement that cannot be adequately explained by translation, such as rotation, enlargement, reduction, camera zooming, and three-dimensional object movement, occurs.

Affine transformation-based motion compensation prediction aims to resolve these problems with block-matching-based motion estimation and achieve more accurate motion estimation by incorporating not only translation but also rotation and scaling motions. This method estimates the parameters of the affine transformation as motion parameters, based on the assumption that the pixel value (x, y) to be predicted is transformed to the pixel value (x', y') in the reference image by the affine transformation shown in Equation (2) below. "Study on Motion Compensation Prediction Using Affine Transformation" (IEICE Technical Report IE94-36) proposes a method for detecting affine motion parameters for a prediction image region of arbitrary shape and performing motion compensation prediction.

Here, the definitions of θ, (Cx, Cy), and (tx, ty) are shown below.

Figure 46 shows the concept of motion compensation prediction processing using affine transformation.

In the figure, i is the on-screen position of the region that serves as the unit of motion compensation prediction, fi(x,y,t) is the pixel value at the region position (x,y) at on-screen position i and time t, Rv is the translation search range, Rrot,scale is the rotation/scale search range, v is the motion vector including the translation parameter (=(tx,ty)), rot is the rotation parameter (=rotation angle θ), scale is the scale parameter (=(Cx,Cy)). Affine motion compensation prediction requires the detection of affine motion parameters, consisting of five parameters: the translation parameter (tx,ty) equivalent to the motion vector, the rotation angle θ, and the scale (Cx,Cy). The optimal solution would be found by exhaustively searching all parameters, which would require an enormous amount of computation. Therefore, a two-stage search algorithm is adopted here, based on the assumption that translation is dominant. In the first step, the translation amount (tx, ty) of the region is searched for. In the second step, the rotation angle θ and scale (Cx, Cy) are searched for near the (tx, ty) determined in the first step, and the translation amount is further fine-tuned. The difference between the current region and the prediction region that yields the smallest prediction error power is calculated and the prediction error is coded. The prediction error power for the affine transformation method is expressed by the following equation (3):

FIG. 47 is a diagram showing an example of the configuration of a motion compensation prediction unit using affine transformation.

In the figure, 220 denotes a translation fine-adjustment amount search range indication signal, 221 denotes a rotation amount search range indication signal, 222 denotes a scale amount search range indication signal, 223 denotes a translation amount search range indication signal, 224 denotes a predicted region intra-screen position signal, 225 denotes predicted region data, 226 denotes a horizontal translation amount counter, 227 denotes a vertical translation amount counter, 228 denotes a translation amount adder, 229 denotes a first-stage minimum prediction error power determiner, 230 denotes a memory read address generator, 231 denotes an interpolation calculation unit, 232 denotes a half-pixel generator, 233 denotes a rotation amount counter, 234 denotes a scale amount counter, 235 denotes a translation/rotation/scale amount adder, 236 denotes a second-stage minimum prediction error power determiner, 237 denotes a translation fine-adjustment amount counter, 238 denotes a translation fine-adjustment amount adder, and 239 denotes a final minimum prediction error power determiner.

FIG. 48 is a flowchart showing the operation of the conventional device, and FIG. 49 is a flowchart showing the details of the affine motion parameter detection step shown in S224 in FIG.

In these figures, MV h[4] is the horizontal component of the motion vector (4 candidates), MV v[4] is the vertical component of the motion vector (4 candidates), D min is the minimum prediction error power, θ is the rotation amount [radian], Cx, Cy are the scale amounts, tx, ty are the motion vector fine adjustment amounts, and D(θ[i], Cx[i], Cy[i], tx[i], ty[i]) is the MV h[i],MV The minimum prediction error power obtained as a result of affine motion parameter detection when v[i] is selected, dθ is a rotation amount search point, dCx is a horizontal scale amount search point, dCy is a vertical scale amount search point, dtx is a horizontal translation fine adjustment amount search point, dty is a vertical translation fine adjustment amount search point, range radian min is the lower limit of the rotation amount search range, radian max is the upper limit of the rotation amount search range, scale min is the lower limit of the scale amount search range, scale max is the upper limit of the scale amount search range, t h min is the lower limit of the horizontal translation fine adjustment amount search range, t h max is the upper limit of the horizontal translation fine adjustment amount search range, t v min is the lower limit of the vertical translation fine adjustment amount search range, t v max is the upper limit of the vertical translation fine adjustment amount search range, D min is the minimum prediction error power, (x, y) is the pixel position within the predicted region, f(x, y) is the value of pixel (x, y) in the predicted image, fr(x, y) is the value of pixel (x, y) in the reference image, ax is the horizontal affine transformation value, ay is the vertical affine transformation value, D(ax, ay) is the prediction error power when searching ax, ay, and D(x, y) is the prediction error at (x, y) when searching ax, ay.

The operation of the motion compensation prediction process using affine transformation will be described in detail below with reference to FIGS.

In these figures, elements or steps with the same reference numerals as those in the previous figures perform the same operations or processes.

1) First Stage In the first stage, conventional methods first use a process similar to the block matching described above to detect translation parameters (i.e., motion vectors) within a given search range for each region.

In Fig. 47, a translation search range instruction signal 223 sets a search range in a horizontal translation counter 226 and a vertical translation counter 227, and changes the search point. A translation amount adder 228 adds the current area position in the predicted image area to this count value, and the result is passed to a memory read address generator 230, which reads out pixel values of predicted image candidates from the frame memory 219. The read pixel values are passed to a pattern matching unit 213, which performs an error calculation similar to block matching. This matching result is sent to a first-stage minimum prediction error power determiner 229, which obtains translation parameters for four candidates with the smallest prediction errors. These are then used to calculate the MV h[4] (horizontal component) and MV The operation of the first stage minimum prediction error power determination unit 229 is the same as that of the minimum prediction error power determination unit 216. This processing step corresponds to S221 and S222 in FIG.

2) Second stage 2-1) Preparation (setting the search range, initializing the minimum prediction error power value) h[i]/MV For v[i] (0≦i≦3), the rotation amount/scale amount is searched for in the small space nearby. This corresponds to S224 in Fig. 48, and the detailed processing steps are shown in Fig. 49. The operation will be explained in relation to the operation of the device in Fig. 47.

First, search ranges are set in the rotation amount counter 233 and the scale amount counter 234 by the rotation amount search range instruction signal 221 and the scale amount search range instruction signal 222. Also, a search range is set in the translation fine adjustment amount counter 237 by the translation fine adjustment amount search range instruction signal 220. The second-stage minimum prediction error power determination unit 236 determines the minimum prediction error power D The value of min is set to MAX_INT. This corresponds to S229 in FIG.

2-2) Search for the amount of rotation For each MV, h[i]/MV Repeat the same process for v[i] (0≦i≦3), and then use MV h[0]/MV We will only explain the case of v[0]. The values of the scale amounts Cx and Cy and the translation fine-tuning amounts tx and ty are fixed, and the value of the rotation amount θ is varied within the search range to obtain the following affine transformation values ax and ay.

The absolute difference between pixel values fr(ax, ay) at (ax, ay) in the reference image and f(x, y) is calculated and accumulated in D(ax, ay).

The above processing is performed by fixing the count values of the scale amount counter 234 and the translation fine adjustment amount counter 237 in Fig. 47, determining ax and ay in equation (4) in the translation/rotation/scaling amount adder 235 according to the count value of the rotation amount counter 233, reading the pixels necessary to calculate fr(ax, ay) from the frame memory 219 via the memory read address generator 230, then calculating fr(ax, ay) from these pixels in the interpolation calculation unit 231, and calculating the absolute difference between the calculated value and the predicted pixel value f(x, y) in the pattern matching unit 213. This corresponds to steps S231 to S234 in Fig. 49.

The above process is performed over the entire rotation search range, and the second-stage minimum prediction error power determination unit 236 determines the rotation θ that results in the minimum prediction error within the rotation search range.

2-3) Searching for the Scale Amount As with the search for the rotation amount, the count value of the translation fine adjustment amount counter 237 is fixed, and the rotation amount θ determined in 2-2) is substituted into equation (4) as the rotation amount. The values of the scale amounts Cx and Cy are varied within the search range to obtain the affine transformation values ax and ay of equation (4).

Thereafter, a process similar to that for searching for the rotation amount is performed to obtain the scale amounts Cx and Cy that minimize D(ax, ay). The scale amount search points are counted by the scale amount counter 234.

2-4) Search for the Fine Translation Adjustment Amount Using the rotation amount θ and scale amounts Cx and Cy determined in 2-2) and 2-3), the values of the fine translation adjustment amounts tx and ty are changed within the search range to obtain the affine transformation values ax and ay of equation (4).

The following process is similar to the rotation/scale search. The translation fine adjustment amount search Point counting is performed by the translation fine adjustment amount counter 237. However, since tx and ty are searched with half-pixel accuracy, half-pixel values are calculated as necessary in the half-pixel generation unit 232 before being sent to the pattern matching unit 213. The half-pixel values are calculated using the following equation (5) based on the spatial relationship with integer pixels, as shown in Figure 50. However, both x and y are counted from 0, and the integer pixel positions are both even.

This completes the processing flow of FIG.

2-5) Final affine motion parameter determination All MVs h[i]/MV For v[i], the prediction error between v[i] and the predicted image obtained using θ[i], Cx[i], Cy[i], tx[i], and ty[i] obtained as a result of the parameter search of 2-2) to 2-4) above is calculated, and the region position i that gives the smallest error value and its parameter set are determined as the final search results. This corresponds to S225 to S228 in Figure 48.

As mentioned above, affine motion parameter search not only requires a large number of processing steps, but also imposes a large computational load during the search.

Figure 51 shows how to calculate non-integer pixel values that arise during the process of searching for rotation and scale amounts, i.e., how the interpolation calculation unit 231 calculates fr(ax, ay).

In the figure, circles represent actual sample points of the image, and black circles represent pixel values generated by calculation.

fr(ax, ay) is expressed as the value of I(x, y) (where x = ax, y = ay) in the following equation (6), calculated on the reference image.

That is, in the affine motion parameter search, pixel-to-pixel matching is performed to select the one with the smallest error power, so that the image region serving as the predicted image candidate must be generated again each time the above five parameters are changed. Because rotation and scaling generate non-integer pixel values, the calculation of equation (6) is repeated many times during the search process. This makes the affine motion parameter search process very time-consuming and burdensome.

Japanese Patent Application Laid-Open No. 6-153185 discloses a motion compensation device and an encoding device using the same, as a method of performing matching on a simple enlarged or reduced image to obtain motion compensation. This device uses a reference image stored in a frame memory, which is then enlarged or reduced using a thinning or interpolation circuit, and motion vectors are then detected. This configuration does not perform complex operations such as affine transformations, but instead extracts fixed blocks from the reference image and performs interpolation or thinning operations. In other words, a fixed screen area is extracted, subjected to predetermined processing, and then compared with the input image. Therefore, the processing is fixed and effectively limited to simple enlargement or reduction.

The motion prediction method of the conventional image coding device is configured and operates as described above.

Therefore, in the first conventional example, the predicted image area is formed by translating the extracted area of the reference image, so only simple translational movements can be predicted. There is a problem in that performance deteriorates significantly when movements other than translational movements, such as rotation, enlargement, reduction, and camera zooming, occur.

On the other hand, in the second example, the predicted image is generated using an affine transformation, which increases the number of predictable objects, such as rotation, but also increases the complexity of the calculation process and the size of the device.

As mentioned above, simplifying the process often results in incomplete predictions, while using affine transformations increases the number of predictable cases, making the process more difficult.

Regarding the decoding process, there were no concrete proposals for performing complex processing while maintaining the configuration of the conventional device.

Disclosure of the Invention This invention was conceived to address the above-mentioned problems. It aims to provide a coding device using a motion compensation prediction method that can accommodate various types of motion and temporal changes with relatively simple processing by using existing pixels on a reference screen or pixels obtained through simple filtering to form a prediction image area of a different shape or size from the image area to be predicted.

Another object of the present invention is to provide a decoding device that performs decoding corresponding to a relatively simple encoding process, and an image decoding device with a similar configuration that reproduces precise and smoother motion.

The image coding device according to the present invention is configured to divide an input image into predetermined blocks, and includes motion compensation prediction means for detecting inter-frame motion between these blocks, and compression-encodes the input image. The motion detection unit includes a modified block matching unit that transforms only integer pixels, which are actual sample points present in a corresponding partial region of a reference image for motion detection, into a predetermined format, specifies coordinates, extracts them, and compares them with the integer pixels of the block in the input image, outputting the motion vector extracted using the specified coordinates that provides the minimum error; and a motion compensation unit that includes a corresponding point determination unit that transforms corresponding blocks of the reference image using the specified coordinates in accordance with motion parameters obtained from the comparison output including the modified block matching unit, and outputs a predicted partial image.

Furthermore, when transforming a partial region of the reference image in a predetermined format, the transformation block matching unit performs the transformation using integer pixels and half pixels that are the midpoints of the integer pixels.

Furthermore, a pre-processing unit is added that separates the input image into subregions of image objects as target regions for encoding, and each of these separated image objects is divided into blocks for motion detection and motion compensation.

Furthermore, the modified block matching unit and corresponding point determination unit are configured such that, when specifying the coordinates of integer pixels or half pixels, the modified block matching unit specifies the coordinates of adjacent points or adjacent points that are a predetermined multiple of the coordinates, extracts them, and compares them, and the corresponding point determination unit similarly processes and outputs a reference image.

Furthermore, the modified block matching unit and corresponding point determination unit are configured as follows: the modified block matching unit extracts and compares coordinates designated by rotating integer pixels or half pixels in a predetermined angle direction, and the corresponding point determination unit similarly processes and outputs a reference image.

The rotation in the predetermined angular direction was plus or minus 45 degrees, 90 degrees, 135 degrees, or 180 degrees.

Furthermore, the deformation block matching unit and the corresponding point determination unit are configured as follows: The deformation block matching unit searches for the area indicated by the partial area of the reference image after translation, and then enlarges or reduces this search area, or moves it by combining rotations in a predetermined angular direction, and compares it; The corresponding point determination unit similarly processes and outputs the reference image.

Furthermore, the modified block matching unit is provided with a deformation pattern table for modifying and processing a partial region of the reference image for comparison, and the image of the partial region based on the transformation values extracted from this deformation pattern table is compared with integer pixels or half pixels of the block of the input image. Similarly, the corresponding point determination unit is a corresponding point determination unit that processes the reference image and outputs the result.

Furthermore, the modified block matching unit selectively filters specific pixels of the reference image extracted for correspondence evaluation and performs comparison.

The frame for motion detection is the previous or next frame in time, and the reference image is the previous or next frame stored and compared with the input image.

An image decoding device according to the present invention is configured to decompress and reproduce image compression codes of input information and includes a motion compensation means. The image decoding device includes an entropy decoding unit that extracts motion parameters from the input information to obtain motion vectors representing the direction and amount of motion and transformation pattern information representing instructions for transformation processing. The motion compensation means uses the motion parameters output by the entropy decoding unit to transform integer pixel coordinate values of a subregion of a reference image stored corresponding to a frame into a predetermined format based on the transformation pattern information in the input information, thereby generating an image to be added to the input predicted image using the resulting pixel values.

Furthermore, the motion compensation means calculates coordinates using coordinate values of half pixels of the reference image, and transforms the obtained pixel values into a predetermined format.

The image coding method according to the present invention includes a motion compensation prediction means for storing a reference image for compression coding of an input digital image, dividing the reference image into predetermined blocks, and detecting motion between frames. The method also includes a modified block matching step for transforming integer pixels of a partial region of the reference image into a predetermined format, extracting the transformed integer pixels by specifying coordinates, generating a predicted partial image, and comparing the resulting predicted partial image with the block of the input image. The method also includes a corresponding point determination step for determining corresponding points of the partial region by specifying the coordinates from a motion vector that provides the smallest error and is selected using the modified block matching, thereby producing a motion-compensated output.

Furthermore, the modified block matching step is a step in which a partial region of the reference image is transformed into a predetermined format using not only integer pixels but also half pixels at its midpoint as a reference standard, and then the coordinates are specified and extracted to generate a predicted partial image for comparison.

Furthermore, a deformation pattern table is provided for deforming a partial region of the reference image, and during deformation block matching, the deformation pattern table is referenced, and the image of the partial region based on the transformation value read from the corresponding address is compared with the input image.

The image decoding method according to the present invention includes an entropy decoding step for extracting motion parameters from input information to obtain motion vectors representing the direction and amount of motion and transformation pattern information representing transformation processing instructions, in order to perform motion compensation and decompress and reproduce the image compression code of input information; and a motion compensation step for generating an image to be added to the input predicted image using pixel values obtained by transforming integer pixel coordinate values of a partial region of a reference image stored corresponding to a frame into a predetermined format based on the transformation pattern information in the input information, using the motion parameters obtained in the entropy decoding step.

Furthermore, the motion compensation step also uses coordinate values of half pixels of the reference image to perform coordinate calculations, and transforms the obtained pixel values into a predetermined format.

An image coding/decoding system according to the present invention comprises an image coding device that divides an input image into predetermined blocks and, in order to compress and encode the input image, includes a motion estimation unit that includes a modified block matching unit that transforms only integer pixels, which are actual sample points present in corresponding partial regions of a reference image for motion estimation, into a predetermined format, specifies coordinates, and extracts them, and compares them with the integer pixels of the blocks of the input image, outputting a motion vector that provides the minimum error extracted using the specified coordinates; a motion compensation unit that includes a corresponding point determination unit that transforms corresponding blocks of the reference image in accordance with motion parameters obtained from the comparison output including the modified block matching unit, and determines the corresponding points using specified coordinates, and outputs a predicted partial image; and an image decoding device that includes motion compensation prediction means that detects motion between frames and, in order to decompress and reproduce image compression codes of input information, the motion compensation prediction means includes a mechanism that specifies coordinates and extracts pre-prepared integer pixels of corresponding partial regions based on the motion parameters in the input information, and outputs and adds the image signals of the partial regions processed in the predetermined format.

BRIEF DESCRIPTION OF THE DRAWINGS Figure 1 shows the basic configuration of the image encoding device of the present invention.

FIG. 2 is a diagram showing the internal configuration of the motion detection unit 8 in FIG.

FIG. 3 is a flowchart showing the operation of the motion detection unit 8 in the configuration of FIG. 2.

FIG. 4 is a diagram illustrating an overview of the operation of the modified block matching unit 21 in embodiment 1.

FIG. 5 is a diagram showing the internal configuration of the modified block matching unit 21. As shown in FIG.

FIG. 6 is a flowchart showing the operation of the modified block matching unit 21.

FIG. 7 is a diagram showing the internal configuration of the motion compensation unit 9 in FIG.

FIG. 8 is a flowchart showing the operation of the motion compensation unit 9.

FIG. 9 is a diagram illustrating the image object separation operation performed by the preprocessing unit 2. In FIG.

FIG. 10 is another internal configuration diagram of the motion detection unit 8b according to the second embodiment.

FIG. 11 is a diagram showing the internal configuration of the motion detection unit 8c according to the third embodiment.

FIG. 12 is a diagram illustrating an overview of the operation of the modified block matching unit 42.

FIG. 13 is a diagram showing the internal configuration of the modified block matching unit 42.

FIG. 14 is a flowchart illustrating the operation of the modified block matching unit 42.

FIG. 15 is a diagram illustrating an overview of the operation of the modified block matching unit 42b in embodiment 4.

FIG. 16 shows the internal configuration of the modified block matching unit 42b in the fourth embodiment.

FIG. 17 is a flowchart illustrating the operation of the modified block matching unit 42b.

FIG. 18 illustrates another modified block matching method according to the fourth embodiment.

FIG. 19 illustrates another modified block matching method according to the fourth embodiment.

FIG. 20 is another internal configuration diagram of the corresponding point determination unit 34 according to the fifth embodiment.

FIG. 21 illustrates modified block matching in embodiment 6.

FIG. 22 illustrates an example of filtering applied to integer pixels constituting a predicted image in the sixth embodiment.

Figure 23 is a diagram illustrating the operation of the modified block matching unit 42c.

FIG. 24 is a diagram showing the internal configuration of the modified block matching unit 42c.

FIG. 25 is a flowchart illustrating the operation of the modified block matching unit 42c.

FIG. 26 is a diagram showing the internal configuration of the motion compensation unit 9b according to the sixth embodiment.

FIG. 27 is a flowchart showing the operation of the motion compensation unit 9b in the sixth embodiment.

FIG. 28 is a diagram showing a configuration of an image decoding apparatus according to the seventh embodiment.

FIG. 29 is a diagram showing the internal configuration of the motion compensation unit 9 according to the seventh embodiment.

FIG. 30 is a flowchart showing the operation of the motion compensation unit 9 in FIG.

FIG. 31 is a diagram for explaining the coordinate point moving operation performed by the motion compensation unit 9 in FIG.

FIG. 32 is a diagram illustrating an example of the transformation process performed by the motion compensation unit 9 in FIG.

FIG. 33 is a diagram showing a coordinate point calculation for obtaining a half pixel.

Figure 34 illustrates the operation when the transformation process is rotation and enlargement.

FIG. 35 is a diagram showing a configuration of an image decoding apparatus according to the eighth embodiment.

FIG. 36 is a diagram showing the internal configuration of the motion compensation unit 90 according to the eighth embodiment.

FIG. 37 is a flowchart showing the operation of the motion compensation unit 90 in FIG.

FIG. 38 is a diagram illustrating an example of the transformation process performed by the motion compensation unit 90 in FIG.

FIG. 39 is a diagram showing an example of coordinate point calculation performed by the motion compensation unit 90 in FIG.

FIG. 40 is a flowchart illustrating the operation of the corresponding point determination unit 37c in the motion compensation unit according to the ninth embodiment.

FIG. 41 illustrates an example of the transformation process performed by the motion compensation unit in the ninth embodiment.

FIG. 42 is a diagram illustrating the concept of motion compensation prediction using block matching according to Conventional Example 1.

FIG. 43 shows the configuration of the motion compensation prediction unit (block matching unit) of the image coding device of Prior Art 1.

FIG. 44 is a flowchart showing the operation of the motion compensation prediction unit of the first conventional example.

Figure 45 shows an overview of the motion compensation prediction method adopted in the MPEG-1 video standard.

FIG. 46 illustrates the concept of motion compensation prediction using affine transformation in Conventional Example 2.

FIG. 47 shows the configuration of a motion compensation prediction unit using affine transformation in Conventional Example 2.

FIG. 48 is a flowchart showing the operation of the motion compensation prediction unit of the second conventional example.

FIG. 49 is a flowchart showing details of the affine motion parameter detection step of FIG.

FIG. 50 is a diagram illustrating a method for calculating half-pixel values in the half-pixel generation unit 232.

FIG. 51 is a diagram illustrating a method for calculating non-integer pixel values that occur in the rotation/scale amount search step in the interpolation calculation unit 231.

BEST MODE FOR CARRYING OUT THE INVENTION Embodiment 1. The encoding and decoding devices of the present invention are specifically used in digital image transmission systems via satellite, terrestrial, or wired communication networks, digital image recording devices, digital image storage databases, and search/viewing systems.

FIG. 1 is a diagram showing the basic configuration of an image encoding device.

In the figure, reference numeral 1 denotes an input digital image signal, 2 denotes a preprocessing unit, 3 and 13 denote intra (within-frame)/inter (between-frame) coding selection units, 4 denotes an orthogonal transform unit, 5 denotes a quantization unit, 6 denotes an inverse quantization unit, 7 denotes an inverse orthogonal transform unit, 8 denotes a motion estimation unit, 9 denotes a motion compensation unit, 10 denotes a frame memory (reference image), 11 denotes motion parameters including motion vectors, 12 denotes predicted image data, 14 denotes a coding control unit, 15 denotes a forced mode indication flag, 16 denotes an intra/inter coding indication flag, 17 denotes a quantization step parameter, 18 denotes an entropy coding unit, and 19 denotes compressed image data. The motion estimation unit 8 and motion compensation unit 9 are key elements of the present invention.

The operation of the image coding device according to the first embodiment of the invention will now be described.

This device receives as input image signals 1 for each frame, which are components of a color video sequence. The input image signals 1 are digitized, and then preprocessed, formatted, and segmented into block data in a preprocessing unit 2. In this embodiment, the segmented block data consists of a pair of a luminance signal component and its spatially corresponding color difference signal component. Hereinafter, the luminance component will be referred to as a luminance block, and the color difference component will be referred to as a color difference block.

Next, the intra/inter coding selection unit 3 determines whether each block of data is intraframe coded or interframe coded. If intraframe coding is selected, block data composed of the original image data output from the preprocessing unit 2 is input to the orthogonal transform unit 4. If interframe coding is selected, prediction error block data composed of the difference between the original image data output from the preprocessing unit 2 and the predicted image data 12 output from the motion compensation unit 9 is input to the orthogonal transform unit 4. This selection of intraframe/interframe coding may be forced by a forced mode instruction flag 15 from the coding control unit 14. The selected coding mode is sent to the entropy coding unit 18 as the intraframe/interframe coding instruction flag 16 and multiplexed into the coded bitstream 19.

The orthogonal transform unit 4 uses, for example, a discrete cosine transform (DCT).

The orthogonal transform coefficients are quantized in the quantization unit 5 using the quantization step parameter 17 calculated by the encoding control unit 14. The quantized orthogonal transform coefficients are then multiplexed into an encoded bitstream 19 after redundancy is reduced in the entropy encoding unit 18. At the same time, the coefficients are inversely quantized in the inverse quantization unit 6 and then inversely orthogonally transformed in the inverse orthogonal transform unit 7 to restore a prediction error signal. A locally decoded image is generated by adding predicted image data 12 output by the motion compensation unit 9 to this locally decoded image. However, if the intra/inter encoding instruction flag 16 indicates intra mode, the encoding selection unit 13 selects a 0 signal, and no prediction error signal is added. The locally decoded image is used as a reference image for motion-compensated prediction of the next frame and thereafter, and its contents are written to the frame memory 10.

Below we will explain the operation of motion compensation prediction, which is one of the most important elements of the device of this embodiment.

In this embodiment, the block extracted by the preprocessing unit 2 is used as the block to be predicted in motion-compensated prediction. Motion-compensated prediction processing is performed by a motion estimation unit 8 and a motion compensation unit 9. The motion estimation unit 8 estimates motion parameters 11, including a motion vector for the block to be predicted, and the motion compensation unit 9 uses the motion parameters 11 to extract predicted image data 12 from a frame memory 10. The motion estimation processing is performed using the luminance block, and motion-compensated prediction of the chrominance block utilizes the motion estimation results for the luminance block. The following explanation is limited to the operation of motion-compensated prediction of the luminance block.

First, the motion detection process will be described.

The motion detection process is performed by motion detection unit 8. Motion detection unit 8 searches within a specified range of the reference image for an area that is most similar to the luminance block of the block to be predicted, and detects parameters that represent the change in the block from its on-screen position. In the conventional block matching, a block that is most similar to the luminance block of the block to be predicted is searched for, and the amount of translation from the on-screen position of the block to be predicted is detected as a motion vector.

The motion estimation unit 8 of this embodiment is configured to perform both conventional block matching based on square blocks and block matching using modified blocks, which will be described later, and select the one with the higher prediction accuracy.

The operation of the motion detection unit 8 in this embodiment will now be described.

FIG. 2 is a detailed block diagram of the motion detection unit 8 in FIG. 1, and FIG. 3 is a flowchart showing its operation.

In FIG. 2, 20 denotes a block matching unit, 21 denotes a modified block matching unit, 22 denotes a motion compensation prediction mode determination unit, 23 denotes a motion vector obtained by modified block matching, 24 denotes a minimum prediction error value obtained by modified block matching, 25 denotes a final motion vector, and 26 denotes a motion compensation prediction mode signal. The final motion vector 25 and the motion compensation prediction mode signal 26 are collectively represented as motion parameters 11.

The internal configuration and operation flowchart of the block matching unit 20 are the same as those shown in FIGS. 43 and 44 in the conventional example. BM is the minimum prediction error power value by block matching, D DEF represents the value of the minimum prediction error power by modified block matching.

Figure 4 is a diagram outlining the operation of the modified block matching unit 21, which is the most important part of this invention. Figure 5 is a detailed diagram of the internal configuration of the modified block matching unit 21. Figure 6 is a flowchart showing the operation of the modified block matching unit 21.

In Figure 5, 29 denotes a horizontal translation amount search range indication signal, 30 denotes a vertical translation amount search range indication signal, 31 denotes a horizontal translation amount counter, 32 denotes a vertical translation amount counter, 33 denotes a rotation amount counter (a new element), 34 denotes a corresponding point determination unit (also a new element), and 35 denotes a memory read address generation unit. The pattern matching unit 213 and minimum prediction error power determination unit 216 perform the same operations as their corresponding elements in the configuration shown in Figure 47.

In addition, in FIG. 6, dx is a horizontal translation amount search point, dy is a vertical translation amount search point, and range h min is the lower limit of the horizontal search range, h max is the upper limit of the horizontal search range, v min is the lower limit of the vertical search range, v max is the upper limit of the vertical search range, min is the minimum prediction error power, D(dx, dy) is the prediction error power during dx, dy search, (x, y) is the pixel position in the block to be predicted, (rx, ry) is the corresponding point in the reference image for (x, y), (rdx, rdy) is the rotation parameter, D(dx, dy) is the prediction error at (x, y) during dx, dy search, f(x, y) is the value of pixel (x, y) in the image to be predicted, fr(x, y) is the value of pixel (x, y) in the reference image, MV h is the horizontal component of the motion vector, v is the vertical component of the motion vector, ix is the horizontal offset value (constant), iy is the vertical offset value (constant), block size is the predicted block size.

1) Detection of motion vector by block matching In the block matching unit 20, a motion vector for a block to be predicted is calculated by the procedure and operation shown in the conventional example. As a result, the motion vector 217 and the minimum prediction error power D BM218 is obtained, which corresponds to S1 in FIG.

2) Motion Vector Detection by Modified Block Matching Next, the modified block matching unit 21 performs modified block matching (S2 in Figure 3).

This operation is explained in more detail below. Note that in the following explanation, the prediction block will be an 8x8 pixel block in integer pixel units.

2-1) Processing Overview Figure 4 shows an overview of the processing performed by the modified block matching unit 21.

In the figure, the predicted image 27 is coded using motion-compensated prediction. For example, the frame (picture) in the preprocessing unit 2, the reference image 28, is a locally decoded frame (picture) image coded before the predicted image 27 and stored in the frame memory 10. The circles in each image represent integer pixels, which are actual sample points of the luminance signal that actually exist within the frame, and the crosses represent half-pixels, which are midpoints between the actual sample points. A subregion of 8x8 (integer pixels) in the predicted image 27 is considered the predicted block (the luminance block portion), and a group of square pixels in the reference image 28 constitutes the modified block of the predicted image candidate. That is, portions of the frame memory 10 output and the preprocessing unit 2 output, shown in parentheses in Figures 1 and 2, are extracted and compared in the modified block matching unit 21 in the motion estimation unit 8.

In this embodiment, the luminance block of the reference image is rotated 45 degrees right or left, and each side is scaled by √2. In other words, when comparing the size of the reference image with the predicted image (input image), the distance is set to 1/√2. The region that matches the horizontal and vertical sampling point distance of the input digital image 28 is defined as a modified block. This region is characterized by being composed only of pixel points in the reference image 28 spaced at integer pixel intervals. In other words, modified block matching in this embodiment corresponds to the process of finding, within a given search range, the modified block region shown in Figure 4 that most closely resembles the luminance block of the predicted block, which consists of 8x8 integer pixels.

2-2) Initial Settings (Search Range and Initial Value Settings) Because the shapes of the predicted block and the candidate predicted image area are different, it is necessary to determine the origin of the detected motion vector during the search. That is, each component point of the luminance block of the predicted block is first matched one-to-one with each component point of the transformation block of the candidate predicted image area.

In the following, as shown by the dotted arrow in Figure 4, we assume that the pixel position of the upper left corner of the predicted block corresponds to the left vertex of the transformed block. In other words, the candidate predicted image is a partial image obtained by rotating the transformed block in reference image 28 45 degrees to the right and modifying each side to a length of 1/√2. Changing this correspondence changes the direction of rotation. By defining it in this way, each of the other construction points can be uniquely matched. Because each construction point of the predicted block corresponds one-to-one to each construction point of the candidate predicted image, motion estimation can be performed in the same way as block matching.

4 is patterned and specified by addressing (coordinates), and in this case, integer pixels are specified to be selected, and the error between the pixel specified by this addressing and the corresponding pixel in the original image data, which is the predicted image 27, is accumulated to determine the minimum error power. Therefore, since only addressing instructions are required and no calculation is involved, high-speed comparison is possible, and the addressing (coordinate specification) method allows flexible extraction instructions, such as not only simple enlargement and reduction but also rotation, or simultaneous processing of rotation and enlargement and reduction, etc. Specifically, the horizontal translation amount search range instruction signal 29 and the vertical translation amount search range instruction signal 30 set the search ranges for modified block matching for the horizontal translation amount counter 31 and the vertical translation amount counter 32. The minimum prediction error power determination unit 216 calculates the minimum prediction error power D min is set to the maximum value MAX INT (for example, 0xFFFFFFFF). This corresponds to S4 in FIG.

2-3) Setting Block Transformation Parameters In this embodiment, the rdx and rdy parameters shown in S6 and S8 of Figure 6 are used as block transformation parameters. These parameters are set by the rotation amount counter 33. In other words, they define the relationship for rotating the reference image in Figure 4 by 45 degrees to the right. The initial value of these is the y value, and rdx is incremented and rdy is decremented each time x is incremented. These processes correspond to S6 through S8 in Figure 6. Note that this setting is for right rotation; setting rdy = -y in S6 and ry = iy + (rdy++) in S8 implies a left rotation transformation. Note that S8 can also be expressed as rx = ix + (rdx + 1), ry = iy + (rdy - 1).

That is, S8 specifies the addressing of the pixel to be extracted from the reference image 28, and the address of the next pixel, rx and ry, specifies the next integer pixel 45 degrees downward to the right. This is repeated in S12 up to the block size x, and in S14 up to the block size y. In this way, the extracted pixel addressing in S8 undergoes error comparison in S9 and accumulation in S10, so the operational flow of FIG. 6 does not require any calculations, enabling high-speed operation. Note that S10 can also be expressed as D(dx, dy) = D(dx, dy) + D(x, y). Similarly, S11, S13, S17, and S19 can also be expressed as x = x + 1 and y = y + 1. This also applies to subsequent flowcharts.

2-4) Reading Candidate Predicted Images First, determine the corresponding points rx and ry in the reference image that correspond to the position (x, y) in the luminance block of the predicted block. In other words, the initial position matching shown in Figure 4 is performed. This is performed by the corresponding point determination unit 34. As shown in S8 of Figure 6, rx and ry are obtained by adding the rdx and rdy obtained in 2-3) to the pre-specified offset values ix and iy. Next, the pixel in the reference image that is located a distance (rx + dx, ry + dy) from the reference image is retrieved from the frame memory. The memory read address generation unit 35 in Figure 5 receives the value dx from the horizontal movement amount counter 31, the value dy from the vertical movement amount counter 32, and rx and ry from the corresponding point determination unit 34, and generates an address in the frame memory.

2-5) Calculation of prediction error power First, the prediction error power D(dx, dy) when the motion vector is (dx, dy) is initialized to zero. This corresponds to S5 in Fig. 6. The difference between the pixel value read in 2-4) and the pixel value at the corresponding position in the luminance block of the predicted block is calculated, and the absolute value is accumulated in D(dx, dy). This process is repeated for x = y = block size (here, block This process is repeated until the minimum prediction error power D(dx, dy) is obtained at the time of (dx, dy). This process is performed by the pattern matching unit 213 in FIG. 5, and the pattern matching unit 213 passes D(dx, dy) to the minimum prediction error power determination unit 216 via a prediction error power signal 215.

The above processing corresponds to the processing in steps S9 to S14 in FIG.

2-6) Update of Minimum Prediction Error Power Value 2-5) determines whether D(dx, dy) obtained as a result of the search provides the minimum power error among the search results up to that point. This determination is made by the minimum prediction error power determination unit 216 in Figure 5. S15 in Figure 6 corresponds to this determination process.

The minimum prediction error power determination unit 216 determines the minimum prediction error power D The value of min is compared with the magnitude of D(dx, dy) delivered by the prediction error power signal 215, and only when D(dx, dy) is smaller, D The value of min is updated with D(dx, dy). The value of (dx, dy) at that time is used as the motion vector candidate (MV h, MV v) These update processes correspond to S16 in FIG.

2-7) Determination of Motion Vector Values The above steps 2-2) to 2-6) are repeated for all (dx, dy) in the search range (S17 to S20 in FIG. 6), and finally the (MV h, MV v) is output as a motion vector 23.

In this way, a predicted image that is most similar to the predicted block in the sense of having the smallest error power is found. As a result of the search, the displacement of the selected predicted image from the starting point is obtained as a motion vector 23 as a result of modified block matching, and the prediction error power D DEF24 is also retained.

3) Determination of Final Motion Compensation Prediction Mode Next, the motion compensation prediction mode determination unit 22 determines the minimum prediction error power D obtained by the block matching unit 20. BM218 and the minimum prediction error power D obtained by the modified block matching unit 21 DEF24 and the smaller of either block matching or modified block matching is selected as the final motion compensation mode. This corresponds to S3 in FIG.

The motion compensation prediction mode determination unit 22 sends the finally selected motion compensation prediction mode signal 26 and the final motion vector 25 as motion parameters 11 to the motion compensation unit 9 and the entropy coding unit 18.

Next, the motion compensation process will be described.

Motion compensation processing is performed by the motion compensation unit 9. The motion compensation unit 9 extracts a predicted image from a reference image based on the motion parameters 11 obtained by the motion estimation unit 8. The motion compensation unit 9 of this embodiment supports both conventional block matching based on square blocks and block matching using specific distorted blocks, and is configured to switch between these types of motion compensation processing depending on the motion compensation prediction mode in the motion parameters 11.

The operation of the motion compensation unit 9 in this embodiment will now be described.

FIG. 7 is a block diagram of the motion compensation unit 9 in FIG. 1, and FIG. 8 is a flowchart showing its operation.

In FIG. 7, reference numeral 37 denotes a corresponding point determination unit, which is a new element, and 38 denotes a memory read address generation unit.

1) Determining Corresponding Points This process corresponds to S21 in Figure 8. A sample point in the reference image 28 corresponding to the predicted image is determined based on the intra-screen position indication signal 206 of the predicted block and the motion parameters 11 sent from the motion estimation unit 8. This process is performed by the corresponding point determination unit 37 in Figure 7. When the motion compensation prediction mode included in the motion parameters 11 indicates block matching, the corresponding point is a sample point included in an area translated by the amount indicated by the motion vector from the intra-screen position signal 206 of the predicted block. This process corresponds to S204 in Figure 44, which determines the position (x + dx, y + dy) in the reference image 28 when (dx, dy) is the motion vector. When the motion compensation prediction mode included in the motion parameters 11 indicates modified block matching, as described in 2-4) of the description of the motion detection unit 8, the in-screen position signal 206 of the predicted block is added with a rotation amount corresponding to each pixel position, and then the sample point is translated by the amount indicated by the motion vector. This process corresponds to the operation of S9 in Figure 6, in which the position (rx + dx, ry + dy) in the reference image 28 is determined when (dx, dy) is the motion vector.

2) Reading Predicted Image Data In the process corresponding to steps S22 through S25 in Figure 8, the memory read address generator 38 receives the results of the corresponding point determiner 34, generates a memory address that identifies the position of the predicted image in the reference image 28 stored in the frame memory 10, and reads the predicted image.

In this case, if the predicted image contains pixels with half-pel accuracy, half-pel values are generated by the half-pel generator 232 before it is output from the motion compensation unit 9. This process corresponds to steps S23 and S24 in FIG. 8. The correspondence point determiner 37 determines whether the predicted image contains pixels with half-pel accuracy based on the motion vector values in the motion parameters 11, and notifies the selection switch 36.

In the configuration of the modified block matching unit 21 in FIG. 5, corresponding points are generated only for actual sample points, as explained in FIG. 4. However, in the case of a configuration in which half pixels are present, the modified block matching unit 42 has a half pixel generation unit 232 as shown in FIG. 13, as will be explained later.

After the above processing steps, the final predicted image data 12 is output. Note that while the rotation for modified block matching in the above embodiment is an example of 45 degrees, other rotations such as 90 degrees, 135 degrees, and 180 degrees can also be achieved by changing the settings of dx and dy.

Furthermore, although this embodiment describes an image coding device that uses image frames as units, this invention can also be applied to devices that code image objects as units by configuring the preprocessing unit 2 to separate the input digital image sequence into image objects (subregions with similar characteristics, such as movement or pattern, or a single subject), and defining each image object as a group of blocks that contain it.

For example, in a scene with a human figure in front of a stationary background, as shown in Figure 9, the human figure may be treated as an image object, the area within the circumscribing rectangle surrounding the figure may be divided into small blocks, and the block containing the image object may be coded as a valid block. In this case, the same processing as for the modified book matching and motion compensation described in the above embodiment is applied to these valid blocks. This also applies to the following embodiments.

In this embodiment, a coding device using orthogonal transform coding has been described, but it goes without saying that the present invention can also be applied to a device that codes a motion compensation prediction error signal using a different coding method. This also applies to the following embodiments.

Embodiment 2. The value of the translational motion vector allows for a rough understanding of the amount of movement of the subregion to be subjected to the modified block matching process. By receiving the region information indicated by the motion vector 217, which is the search result of the block matching unit 20, and limiting the destination of the modified block matching subregion to this vicinity for comparison, the number of processing steps and processing time can be reduced. This configuration will be described in this embodiment. Note that this also applies to the other embodiments described below.

This embodiment shows another embodiment of the motion detection unit 8.

FIG. 10 shows the internal configuration of the motion detection unit 8b in this embodiment, with reference numeral 39 representing a modified block matching unit, 40 representing an adder, and 41 representing a search initial position indication signal. Note that the modified block matching unit 39 operates in exactly the same way as the modified block matching unit 21 in embodiment 1, except that it uses the search initial position indication signal 41 instead of input 206.

A specific circuit for obtaining a rough value is shown in FIG.

In FIG. 10 , instead of the intra-screen position signal 206 of the predicted block, an adder 40 adds a motion vector 217 obtained as a result of the block matching unit 20 to the predicted block data 205, and the result of this addition is input to the modified block matching unit 39 as a search initial position indication signal 41. Furthermore, the search range set by the horizontal translation amount search range indication signal 29 and the vertical translation amount search range indication signal 30 is set smaller than in the first embodiment. This allows the iterative processing from S17 to S20 in FIG. 6 to be shortened.

Embodiment 3. In the previous embodiment, we described a case where the deformation block region is composed only of pixel points at integer pixel intervals in the reference image 28. In this embodiment, we describe a case where the deformation block region is composed also of pixel points at half-pixel intervals in the reference image 28.

In this embodiment, the internal configurations of the motion estimation unit 8 and the motion compensation unit 9 shown in FIG. 1 differ from those of the first embodiment. Furthermore, the only differences in operation are the distorted block matching unit in the motion estimation unit and the corresponding point determination unit in the motion compensation unit; the other components and operations are identical to those of the first embodiment. Therefore, the following will only describe in detail the operation of the distorted block matching unit and the corresponding operation of the motion compensation unit. As in the first embodiment, the operations of the motion estimation unit 8c and the motion compensation unit 9 will be described separately.

Figure 11 shows the internal configuration of the motion detection unit 8c in this embodiment. Figure 12 shows an overview of the operation of the modified block matching unit 42, one of the most important components of this invention. Figure 13 shows a detailed internal configuration of the modified block matching unit 42. Figure 14 is a flowchart showing the operation of the modified block matching unit 42.

In these figures, elements and steps with the same numbers as in the previous figures represent the same elements and operations.

First, the operation of the modified block matching unit 42 will be described.

1) Processing Overview Figure 12 shows an overview of the processing performed by the modified block matching unit 42.

In the figure, the predicted image 27 and reference image 28 are as defined in embodiment 1. In each image, the circles represent actual sample points (integer pixels) of the luminance signal of the frame, and the crosses represent midpoint pixels (half pixels) between the actual sample points. A subregion consisting of 8x8 (integer pixels) in the predicted image 27 is defined as the predicted block (its luminance block portion), and a group of square pixels in the reference image 28 constitutes the transformed block of the predicted image candidate.

In this embodiment, the luminance block is rotated 45 degrees to the right or left and each side scaled by 1/√2. In other words, the size of the reference image is a distance of √2, and the region that matches the horizontal and vertical sample point distances of the input digital image 1 of the frame is defined as the deformation block. This region is characterized by including pixel points at half-pixel intervals in the reference image 28. In other words, deformation block matching in this embodiment corresponds to the process of finding, within a given search range, the deformation block region shown in Figure 12 in the reference image 28 that most closely resembles the luminance block of the prediction block, which consists of 8 x 8 samples (hereinafter, "sample" refers to either integer pixels or half pixels).

2) Initial Settings (Search Range Setting, Initial Value Setting) Using the same logic as in Embodiment 1, a one-to-one correspondence is established between each component point of the luminance block of the predicted block and each component point of the transformation block of the candidate predicted image region.

In the following, we assume that the pixel position of the upper left corner of the predicted block corresponds to the left vertex of the transformation block, as indicated by the dotted arrow in Figure 12. In this figure, the left vertex of the transformation block is located at a half-pixel position, indicating that the motion vector includes a half-pixel component. In other words, the candidate predicted image is a partial image obtained by rotating the transformation block in reference image 28 by 45 degrees clockwise and correcting each side to a length that is √2 times longer. Changing this correspondence changes the direction of rotation. This arrangement ensures that each of the other composition points corresponds uniquely. Because each composition point of the predicted block corresponds one-to-one to each composition point of the candidate predicted image, motion estimation can be performed in a similar manner to block matching. The search range setting operation in the actual device is the same as in embodiment 1, and is set using the necessary elements in Figure 13. This operation corresponds to step S26 in Figure 14.

3) Setting Block Transformation Parameters In this embodiment, as in embodiment 1, the rdx and rdy shown in Figure 14 are used as block transformation parameters. These parameters are set by the rotation amount counter 45. The value of y is given as the initial value, and rdx is incremented by 0.5 and rdy is decremented by 0.5 each time x is incremented by 1. These processes correspond to steps S28-S30 in Figure 14.

This setting is a right rotation setting, and setting rdy = -y in S28 and ry = iy + (rdy + = 0.5) in S30 means a left rotation deformation.

4) Reading the Candidate Predicted Image First, determine the corresponding points rx and ry in the reference image that correspond to the position (x, y) in the luminance block of the predicted block. This is performed by the corresponding point determination unit 46. As shown in S30 of Figure 14, rx and ry are obtained by adding the rdx and rdy obtained in 3) to the pre-specified offset values ix and iy.

Next, a pixel in the reference image located a distance (rx + dx, ry + dy) from the reference image is retrieved from the frame memory. The memory read address generator 47 in FIG. 13 receives the value of dx from the horizontal displacement counter 31, the value of dy from the vertical translation counter 32, and rx and ry from the corresponding point determiner 46, and generates an address in the frame memory. The data read in S31 in FIG. 14 is also used by the half-pixel generator 232 to generate half-pixel values as needed.

5) Calculating Prediction Error Power First, the prediction error power D(dx, dy) for the motion vector (dx, dy) is initialized to zero. This corresponds to S27 in Figure 14. The difference between the pixel value read in 4) and the pixel value at the corresponding position in the luminance block of the predicted block is calculated, and the absolute value is accumulated as D(dx, dy). This process is repeated until x = y = block_size (here, block_size = 8) to obtain the prediction error power D(dx, dy) for (dx, dy). This process is performed by the pattern matching unit 213 in Figure 13, which then passes D(dx, dy) to the minimum prediction error power determination unit 216 via the prediction error power signal 215. This process corresponds to S32 through S37 in Figure 14.

6) Update of Minimum Prediction Error Power Value It is determined whether D(dx, dy) obtained as a result of 5) provides the smallest error power value among the search results up to that point. This determination is performed by the minimum prediction error power determination unit 216 in Figure 13. S38 in Figure 14 corresponds to this determination process. The determination process is exactly the same as in Embodiment 1, and the (dx, dy) value at that time is stored as the motion vector candidate (MV_h, MV_v). This update process corresponds to S39 in Figure 14.

7) Determining the Motion Vector Value Steps 2) through 6) above are repeated for all (dx, dy) values in the search range (S40 through S43 in Figure 14), and the final (MV_h, MV_v) value stored in the minimum prediction error power determination unit 216 is output as the motion vector 43.

In this way, the predicted image that is most similar to the predicted block in terms of minimizing error power is found. As a result of the search, the displacement of the selected predicted image from the origin is obtained as a motion vector 43 resulting from modified block matching, and the prediction error power D_DEF 44 at that time is also retained.

The motion vector 43 and prediction error power D_DEF 44 are used for the final motion compensation mode determination, which is exactly the same as in the first embodiment.

Next, the motion compensation process will be described.

The motion compensation process is performed by the motion compensation unit 9. In this embodiment, only the operation of the corresponding point determination unit 37 differs from that of the first embodiment, and only this part will be described. The overall flowchart of the motion compensation process is as shown in Figure 8.

In this embodiment, the corresponding points are determined as follows.

When the motion compensation prediction mode included in the motion parameters 11 indicates block matching, the corresponding point is a sample point included in an area translated by the amount indicated by the motion vector from the intra-screen position signal 206 of the predicted block. This process corresponds to the operation of determining the position (x + dx, y + dy) in the reference image 28 when (dx, dy) is the motion vector in S204 in Figure 44.

When the motion compensation prediction mode included in the motion parameters 11 indicates modified block matching, as described in section 4) of the motion estimation unit 8, the in-screen position signal 206 of the predicted block is added with a rotation amount corresponding to each pixel position, and then the resulting sample point is located within an area translated by the amount indicated by the motion vector. This process corresponds to the operation of determining the position (rx + dx, ry + dy) in the reference image 28 when (dx, dy) is the motion vector in step S32 of FIG. 14.

The following reading of predicted image data and generation of predicted images conform to the first embodiment.

Embodiment 4. This embodiment describes the use of a modified block, in which the area of the predicted block is simply reduced. Although not explained here, the same applies to simple enlargement.

Thus, we describe simpler modified block matching and motion compensation.

The following describes in detail, with reference to FIG. 16, only the operation of the distorted block matching unit 42b in the motion estimation unit and the corresponding point determiner in the motion compensation unit, which operate differently from the above-described embodiment. To avoid confusion, the distorted block matching unit 42b is assumed to be a variation of the distorted block matching unit 42 in FIG. 13, with the same inputs and variations of the motion vector 43 and prediction error power 44 as its outputs. The corresponding point determiner in the motion compensation unit 9 is also assumed to be a variation of the corresponding point determiner 37 in FIG. 7. Therefore, the following description will be given assuming that the distorted block matching unit in this embodiment is numbered 42b and the corresponding point determiner is numbered 37.

FIG. 15 is a diagram outlining the operation of the modified block matching unit 42b in this embodiment, FIG. 16 is a detailed diagram of the internal configuration of the modified block matching unit 42b, and FIG. 17 is a flowchart illustrating the operation of the modified block matching unit 42b.

In these figures, elements and steps with the same numbers as in the previous figures represent the same elements and operations.

First, the operation of the modified block matching unit 42b will be described.

1) Processing Overview Figure 15 shows an overview of the processing performed by the modified block matching unit 42b. The predicted image 27, reference image 28, and the marks within each image are as described above. In this embodiment, a modified block is defined as a reduced area obtained by simply halving each side of a luminance block. Modified block matching in this embodiment corresponds to the process of finding, within a given search range, the modified block area shown in Figure 15 in the reference image 28 that is most similar to the luminance block of the predicted block consisting of 8x8 samples.

2) Initial Settings (Search Range Setting, Initial Value Setting) Using the same logic as in Embodiment 1, a one-to-one correspondence is established between each component point of the luminance block of the predicted block and each component point of the transformation block of the candidate predicted image region.

In this embodiment, as shown by the dotted arrow in Figure 15, the pixel position of the upper left corner of the predicted block is previously associated with the pixel position of the upper left corner of the transformed block. Because there is a one-to-one correspondence between each component point of the predicted block and each component point of the candidate predicted image, motion estimation can be performed in the same manner as block matching. The search range setting operation in the actual device is the same as in embodiment 1, and is set using the necessary elements in Figure 16. This operation corresponds to step S44 in Figure 17.

3) Reading Candidate Predicted Images In this embodiment, no specific block transformation parameters are used. Instead, as shown in S47 of FIG. 17, the corresponding points sx and sy of x and y are obtained by adding the values x/2 and y/2 to the offset values ix and iy of the horizontal and vertical components. This corresponding point is determined by the corresponding point determination unit 48. Next, a pixel in the reference image located a distance (sx + dx, sy + dy) from the reference image is retrieved from the frame memory. The memory read address generation unit 49 in FIG. 16 receives the value dx from the horizontal translation amount counter 31, the value dy from the vertical translation amount counter 32, and sx and sy from the corresponding point determination unit 48, and generates an address in the frame memory. Furthermore, the data read in S48 of FIG. 17 is used as needed to generate half-pixel values in the half-pixel generation unit 232.

4) Calculating Prediction Error Power First, the prediction error power D(dx, dy) for the motion vector (dx, dy) is initialized to zero. This corresponds to S45 in Figure 17. The difference between the pixel value read in 3) and the pixel value at the corresponding position in the luminance block of the predicted block is calculated, and the absolute value is accumulated as D(dx, dy) in S50. This process is repeated in S52 and S54 until x = y = block_size (here, block_size = 8), thereby obtaining the prediction error power D(dx, dy) for (dx, dy).

This process is performed by the pattern matching unit 213 in FIG. 16 , which passes D(dx, dy) to the minimum prediction error power determination unit 216 via the prediction error power signal 215. This process corresponds to steps S49 to S54 in FIG. 17 .

5) Update of minimum prediction error power value It is determined whether D(dx, dy) obtained as a result of step 4) provides the minimum power error among the search results up to that point. This determination is made by the minimum prediction error power determination unit 216 in Figure 16. This determination process corresponds to S55 in Figure 17.

The determination process is exactly the same as in embodiment 1, and the (dx, dy) values at that time are stored as motion vector candidates. This update process corresponds to S56 in Figure 17.

6) Determining the Motion Vector Value Steps 2) through 5) above are repeated for all (dx, dy) values in the search range in steps S57 through S60 of Figure 17, and the final (dx, dy) value stored in the minimum prediction error power determination unit 216 is output as the motion vector 43.

In this way, the predicted image that is most similar to the predicted block in terms of minimizing error power is found. As a result of the search, the displacement of the selected predicted image from the origin is obtained as a motion vector 43 resulting from modified block matching, and the prediction error power D_DEF 44 at that time is also retained.

The motion vector 43 and prediction error power D_DEF 44 are used for the final motion compensation mode determination, which is exactly the same as in the first embodiment.

Next, the motion compensation process will be described.

Motion compensation processing is performed by the motion compensation unit 9. In this embodiment, only the operation of the corresponding point determination unit 37 differs from that of embodiment 1, so only that part will be described. The overall flowchart for motion compensation is as shown in Figure 8.

In this embodiment, the corresponding points are determined as follows.

When the motion compensation prediction mode included in the motion parameters 11 indicates block matching, the corresponding point is a sample point included in an area translated by the amount indicated by the motion vector from the intra-screen position signal 206 of the predicted block. This process corresponds to the operation of determining the position (x + dx, y + dy) in the reference image 28 when (dx, dy) is the motion vector in S204 in Figure 44.

When the motion compensation prediction mode included in the motion parameters 11 indicates modified block matching, the in-screen position signal 206 of the predicted block is added with a displacement amount corresponding to each pixel position, and then the sample point is included in the area translated by the amount indicated by the motion vector. This process corresponds to the operation of determining the position (sx + dx, sy + dy) in the reference image 28 when (dx, dy) is the motion vector in S47 of Figure 17. The following reading of predicted image data and generation of predicted images are similar to those in embodiment 1.

In each of the above embodiments, the deformed blocks can have any shape, provided that two assumptions are met: 1) there is a one-to-one correspondence between the pixel positions of the predicted block and the predicted image, and 2) corresponding pixels in the reference image are spaced at integer pixel intervals. For example, shapes such as those shown in Figures 18 and 19 are possible. Furthermore, block matching can be achieved by deforming blocks into various shapes by independently scaling each side by any ratio, rather than simply shrinking only one side by half. By defining various shapes in this way, it is possible to select the deformed block that provides the best prediction results. In this case, the type of selected deformed block is simply included in the motion parameters 11 and sent to the entropy coding unit 18.

According to the above embodiments, motion compensation, including rotation and downscaling, can be performed simply by generating interpolated pixel values with half-pel accuracy, without generating interpolated pixel values through complex calculations such as affine transformation. This allows for accurate prediction even for partial images where prediction error cannot be minimized using motion vectors alone, which are translational translations.

In the above embodiments, the fixed points prepared in advance are described as integer pixels or half pixels. However, intermediate pixels with other ratios, such as 1:3, may also be prepared for comparison. Even in this case, unlike conventional affine transformations, no interpolation is required during the comparison process, thereby reducing the processing scale and enabling faster processing.

Embodiment 5. In the above embodiments, the block transformation parameter counting process or the corresponding coordinate transformation process is performed for each pixel. However, it is also possible to prepare these coordinate transformation values for each pixel in advance as a transformation pattern table in ROM or other storage, and determine corresponding points based on the transformation values retrieved from the table for each pixel position in the predicted block. This enables effective transformation block matching and motion compensation for arbitrary correspondences that are difficult to express using mathematical formulas.

For example, take the first embodiment as an example.

FIG. 20 is another internal block diagram of the corresponding point determiner 34 in FIG. 5, showing a configuration (corresponding point determiner 34b) for implementing this embodiment. Instead of incrementing or decrementing the values of the parameters rdx and rdy in S8 in FIG. 6, which shows the specific operation of embodiment 1, the values of rdx and rdy corresponding to x and y can be stored in ROM, and the corresponding points rx and ry can be obtained by retrieving them from there according to the values of x and y. In this case, the rotation amount counter 33 in FIG. 5 is unnecessary, and as shown in FIG. 20, this can be implemented by providing a ROM table (deformation pattern table 100) within the corresponding point determiner 34b. The corresponding point determiner 34b retrieves the values of the deformation parameters rdx and rdy from the deformation pattern table 100 according to each pixel position (x, y) of the predicted block, and determines the corresponding points by adding them in the adder 110. The result is then output to the memory read address generator 35. This is also true for the other embodiments described above. Thus, by simply adding a small amount of ROM memory (deformation pattern table 100), it is possible to eliminate elements that perform corresponding point calculations, simplifying the circuit and reducing the amount of corresponding point calculation processing. It also becomes possible to support deformations that cannot be expressed with simple mathematical formulas such as those shown in Figure 21, allowing for the creation of a richer deformation pattern library.

Embodiment 6. This embodiment describes a coding device that equalizes the frequency characteristics in a predicted image extracted as a modified block using the methods described in the above embodiments, thereby reducing mismatches when predicting a predicted block.

When a predicted image is composed of pixel values existing in integer pixel space and half pixel space, the spatial frequency characteristics differ between integer pixels and half pixels. On the other hand, since the predicted block is composed entirely of pixels in integer pixel space, this difference in characteristics can potentially cause mismatches during prediction. Therefore, in this embodiment, after defining a distorted block in the same way as in the previous embodiments, filtering is performed on pixels in integer pixel space.

Pixels in the half-pel space are generated by filtering the surrounding integer pixels by [1/2, 1/2]. In other words, a low-pass filter with cos(ωt/2) characteristics is applied. The predicted image defined in each of the above embodiments contains a mixture of unfiltered integer pixels and half-pel-accurate pixels generated by the filtering, resulting in variations in the spatial frequency characteristics within the predicted image. If this variation reduces prediction accuracy, it is effective to apply a filter with equivalent characteristics to the integer pixels, as described below.

Figure 22 shows an example of filtering, where a low-pass filter F of [1/8, 6/8, 1/8] is applied to integer pixels as shown in Equation (7).

The filter characteristic is {cos(ωt/2)}2, which reduces variations in spatial frequency characteristics within the predicted image. After this filtering process, similar to the above embodiments, one-to-one correspondence is established between each point of the predicted block and each point of the predicted image, search is performed, motion vectors are determined, and mode determination is performed.

The specific configuration and operation of the device will be described below.

In this embodiment, the modified block matching unit and motion compensation unit differ from those of the previous embodiments. In the following, the modified block is defined as a simple downsized pattern based on the fourth embodiment, the internal configuration of the modified block matching unit is considered to be a variation of the modified block matching unit 42 in the motion estimation unit 8c, and the motion compensation unit is considered to be a variation of the motion compensation unit 9. Therefore, in the following explanation, the modified block matching unit will be numbered 42c and the motion compensation unit will be numbered 9b.

Figure 23 is a diagram outlining the operation of the modified block matching unit 42c in this embodiment, Figure 24 is a detailed internal configuration diagram of the modified block matching unit 42c, and Figure 25 is a flowchart illustrating the operation of the modified block matching unit 42c in this embodiment.

In these figures, elements and steps with the same numbers as in the previous figures represent the same elements and operations.

First, the operation of the modified block matching unit 42c will be described. Description of operations that are the same as those in the fourth embodiment will be omitted.

1) Processing Overview The definition of a transformation block is exactly the same as in embodiment 4, but this embodiment differs in that filtering is performed on pixels at integer pixel positions. That is, as shown in Figure 23, pixels marked with a triangle are defined in the reference image as pixels to be filtered, and the transformation block is composed of the pixels marked with a triangle and a square.

2) Initial settings (search range setting, initial value setting) Identical to embodiment 4.

3) Reading the Candidate Predicted Image The method for obtaining the corresponding points sx and sy corresponding to the pixel at position x and y in the predicted block is exactly the same as in embodiment 4. Next, a pixel in the reference image located a distance (sx + dx, sy + dy) from the reference image is retrieved from the frame memory. At this time, it is determined whether the pixel position corresponding to sx + dx and sy + dy is in integer pixel space or half pixel space. This can be determined simply by checking whether sx + dx and sy + dy each have half pixel components. This determination is made by the corresponding point determination unit 48 in Figure 24. In Figure 25, this corresponds to step S61. If it is determined to be in half pixel space, the half pixel generation unit 232 generates half pixel values. If it is determined to be in integer pixel space, the filter unit 50 performs the filtering shown in Figure 22. This corresponds to step S62 in Figure 25.

4) Calculating the prediction error power 5) Updating the minimum prediction error power value 6) Determining the motion vector value This is exactly the same as in embodiment 4.

Next, the motion compensation process will be described.

The motion compensation process is carried out by the motion compensation unit 9b.

FIG. 26 is a diagram showing the internal configuration of the motion compensation unit 9b in this embodiment, and FIG. 27 is a flowchart showing the operation of the motion compensation unit 9b in this embodiment.

This embodiment is characterized by the addition of a filter unit 50 compared to the motion compensation unit 9 shown in FIG. 7. The corresponding point determination unit 37 operates in exactly the same way as in embodiment 4. When the motion compensation prediction mode included in the motion parameters 11 indicates block matching, the corresponding point is determined to be a sample point included in an area translated by the amount indicated by the motion vector from the intra-screen position signal 206 of the predicted block. This process corresponds to the operation of S204 in FIG. 44, which determines the position (x + dx, y + dy) in the reference image 28 when (dx, dy) is the motion vector.

When the motion compensation prediction mode included in the motion parameters 11 indicates modified block matching, the in-screen position signal 206 of the predicted block is added with a displacement amount corresponding to each pixel position, and then the resulting sample point is located in a region translated by the amount indicated by the motion vector. This process corresponds to the operation of determining the position (sx + dx, sy + dy) in the reference image 28 when (dx, dy) is the motion vector in S47 of FIG. 17. In either case, each pixel is determined to be in half-pixel space, and pixels in integer pixel space are subjected to filtering shown in FIG. 22, exactly as in the predicted image generation process of the modified block matching unit described above. Filtering is performed by the filter unit. The following reading of predicted image data and generation of predicted images conform to those in embodiment 1.

The modified block matching unit 42c in this embodiment may perform a search independently for the predicted image without filter application and the predicted image with filter F applied, and send the search results to the motion compensation prediction mode determination unit 22, or may perform a search only for the case without filter F application, and apply filter F only to the search results to select the best result.

In this way, when a mechanism for adaptively turning the filter F on and off is provided, the motion parameters 11 also include information on whether the filter is on or off.

According to this embodiment, it is possible to eliminate spatial frequency variations within the predicted image simply by filtering integer pixel values, and it is possible to perform good predictions even for subimages where prediction error cannot be minimized solely by using motion vectors (translation amounts), i.e., where prediction accuracy is poor.

Embodiment 7. Figure 28 shows the configuration of an image decoding device that decompresses and plays back digital images that have been compression-encoded using the image prediction method of this embodiment. This image decoding device receives and decompresses the compression-encoded data (hereinafter, "bitstream") 19 generated by the image encoding device shown in Embodiment 1.

In FIG. 28, reference numeral 51 denotes an entropy decoding unit, 6 denotes an inverse quantization unit, 7 denotes an inverse orthogonal transform unit, 53 denotes a decoding and adding unit, 54 denotes a frame memory, and 56 denotes a display control unit.

The decoding device of the present invention is characterized by the configuration and operation of the motion compensation unit 9. Since the configuration and operation of each of the above-mentioned elements other than the motion compensation unit 9 are already known, detailed description will be omitted. The motion compensation unit 9 is identical to the motion compensation unit 9 in FIG. 1. In other words, its internal configuration diagram is identical to the internal configuration diagram shown in FIG. 7, and its operational flowchart is identical to the operational flowchart shown in FIG. 8.

The operation of the device having the above configuration will now be described.

First, the bitstream is analyzed in the entropy decoding unit 51 and separated into individual pieces of coded data. The quantized orthogonal transform coefficients 52 are sent to the inverse quantization unit 6, where they are inversely quantized using the inverse quantization step parameter 17. The result is then inversely orthogonally transformed in the inverse orthogonal transform unit 7 and sent to the decoding and adding unit 53. The inverse orthogonal transform unit uses the same techniques as those used in the coding device, such as DCT.

The motion compensation unit 9 receives the following three types of information as motion parameters 11:

That is, the entropy decoding unit 51 inputs the motion vector 25, transformation pattern information 26a, and information 27a indicating the on-screen position of the predicted image region (in this embodiment, a fixed-size block) decoded from the bitstream. The motion vector 25 and the on-screen position 27a of the predicted image region are unique to each predicted image region. However, the transformation pattern information 26a may be unique to each predicted image region, or may be coded for a larger image (e.g., an image frame or a VOP as disclosed in ISO/IEC JTC1/SC29/WG11) that combines multiple predicted image regions, with the same transformation pattern information being used for all predicted image regions included in that unit. The motion compensation unit 9 extracts predicted image data 12 from the reference image in the frame memory 54 based on these three types of information. The process of generating a predicted image will be described in the section explaining the operation of the motion compensation unit 9.

The motion compensation unit 9 receives the motion parameters 11 decoded by the entropy decoding unit 51.

The motion compensation unit 9 extracts predicted image data 12 from the reference image in the frame memory 54 in accordance with these motion parameters 11. In the image prediction method of this invention, the pixels constituting the predicted block correspond one-to-one to the pixels constituting the predicted image, so the predicted image area is uniquely determined by the motion parameters 11, just like in conventional block matching motion compensation.

Based on the value of the intra/inter coding indication flag 16, the decoding and adding unit 53 outputs the output of the inverse orthogonal transform unit as a decoded image 55 if the block is intra-coded, or adds the predicted image data 12 to the output of the inverse orthogonal transform unit and outputs the result as a decoded image 55 if the block is inter-coded. The decoded image 55 is sent to the display control unit 56 and output to a display device (not shown), and is also written to the frame memory 54 for use as a reference image in the decoding process of subsequent frames.

Next, the predicted image generation process in the motion compensation unit 9 will be described.

In this embodiment, the image prediction method uses transformation pattern information 26a to predetermine the positional correspondence between pixels constituting the predicted image area and pixels constituting the predicted image. Therefore, a predicted image is generated by simple address calculation and interpolation based on the displacement caused by motion vector 25 and the position correction caused by transformation pattern information 26a.

The internal configuration of the motion compensation unit 9 is shown in FIG.

In the figure, 37 is a corresponding point determination unit, and 38 is a memory read address generation unit.

FIG. 30 is a flowchart showing the operation.

FIG. 31 illustrates the movement of a reference image extracted by a motion vector and moved to a coordinate position in a predicted image by a specified amount, and FIG. 32 illustrates the operation of addressing the destination image using a further specified transformation pattern.

In all the figures, ◯ indicates an integer pixel position, and × indicates a half pixel position.

The operation of the motion compensation unit 9 in this embodiment will be explained below with reference to Figures 29 and 30.

1) Determining Corresponding Points First, the corresponding point determination unit 37 calculates the sample position of the predicted image corresponding to each pixel in the predicted image's increased area based on the input motion vector 25 and transformation pattern information 26a. First, the reference position of the predicted image relative to the current position of the predicted image is determined based on the motion vector 25. As shown in Figure 31, this process corresponds to determining (i', j') = (i + dx, j + dy) when the in-screen position 27a of the predicted image is (i, j) and the motion vector 25 is (dx, dy) (S71 in Figure 30).

Next, the coordinate point (i', j') is corrected based on the deformation pattern information 26a, to determine the final sample position of the predicted image. Figure 32 shows an example where the deformation pattern information 26a indicates "1/2 reduction vertically and horizontally." According to this deformation pattern, the effective area of the predicted image is 1/4 of the effective area of the image region to be predicted on the screen. In other words, the predicted image is reduced relative to the image region to be predicted, which improves the efficiency of prediction of movements that involve enlargement on the screen. Specifically, the position correction process determines the corrected position (i", J") corresponding to the position (i', j') in the reference image. This can be achieved by the following calculation (S72 in Figure 30):

Note that Figure 32 shows the case where block_width = block_height = 4, that is, where one block has 4 x 4 pixels, but these can be any positive integers, that is, blocks with height and width of any number of pixels.

The coordinate point (i", j") obtained in this way is output as the predicted image sample position corresponding to (i, j).

2) Reading Data for Predicted Image Generation Based on the predicted image sample positions output by the corresponding point determination unit 37, the memory read address generation unit 38 generates a memory address that identifies the location of the image data required to generate the predicted image within the reference image stored in the frame memory 54, and reads the data for generating the predicted image.

3) Generation of Predicted Image When addressing only integer-pel coordinate values of pixels for generating a predicted image, the data for generating the predicted image directly becomes the pixels that make up the predicted image. On the other hand, when addressing coordinate values of positions with half-pel accuracy, the half-pel generator 232 interpolates the data for generating the predicted image to generate half-pel values. Specifically, half-pel values are generated as shown in Figure 33. The method shown in Figure 33 is a simple addition/division operation, and is a re-explanation of S24 in Figure 8, the flow diagram for the half-pel generator 232 described in the first embodiment of the encoding device.

Although the operation of the motion compensation unit 9 has been described above using FIG. 32, if the transformation pattern information contains content different from that shown in FIG. 32, the transformation process will be different.

As an example of another transformation pattern, the case of Figure 34 will be explained. In this case, (i", j") after transformation can be calculated as follows:

In this way, if the transformation pattern information determines how to extract the transformation blocks, it is possible to perform the transformation-processed motion-compensated decoding using simple addressing based on the transformation pattern information.

As described above, the image decoding device of this embodiment can efficiently predict complex motion that cannot be tracked by translation alone, and generate a reconstructed image from the coded bitstream, simply by preparing transformation patterns and performing simple calculations of sample positions according to the corresponding mode information.

In this embodiment, even if the bitstream is one in which the prediction error signal is coded using a coding method other than orthogonal transform coding, the same effect can be achieved by changing elements for the prediction error signal decoding process other than the motion compensation unit 9.

Furthermore, while this embodiment describes an example of decoding processing in units of fixed-size blocks, this is applicable not only to decoding devices that process frames of regular television signals as units, but also to decoding devices that process arbitrary-shaped image objects composed of fixed-size blocks (e.g., the Video Object Plane disclosed in ISO/IEC JTCI/SC29/WG11/N1796). For example, in a scene with a human figure in front of a stationary background, as shown in Figure 9 described in embodiment 1, the human figure may be treated as a single image object, the area within the circumscribing rectangle surrounding the figure may be divided into small blocks, and the block containing the image object may be treated as a valid block, and the encoded bitstream may be decoded. In this case, similar processing may be applied to these valid blocks.

Embodiment 8. The image decoding device of Embodiment 7 corresponds to the image decoding device of Embodiments 1 through 6. It performs motion compensation by performing predetermined transformations using only integer or half-pixel addressing (coordinate specification). In this embodiment, we describe an image decoding device that performs more precise motion compensation by performing calculations other than half-pixel generation during addressing.

FIG. 35 shows the configuration of an image decoding device for decompressing and reproducing compressed and encoded digital images in this embodiment.

In the figure, 90 denotes a motion compensation unit, 25b denotes zero to four motion vectors, and 60 denotes interpolation processing precision instruction information.

FIG. 36 is a diagram showing the internal configuration of the motion compensation unit 90.

In the figure, reference numeral 37b denotes a corresponding point determination unit that determines corresponding points using motion parameters, such as the motion vector 25b shown in FIG. 35, deformation pattern information 26a, the on-screen position 27a of the predicted image region, and interpolation processing accuracy instruction information 60, as input. Reference numeral 232b denotes an interpolation processing unit that calculates the interpolated coordinate position. In this case, the on-screen position 27a of the predicted image region is a unique value for each predicted image region. However, the motion vector 25b and deformation pattern information 26a may be unique values for each predicted image region. Alternatively, they may be coded for a larger image (e.g., an image frame or a VOP as disclosed in ISO/IEC JTC1/SC29/WG11) that combines multiple predicted image regions, and the same motion vector and deformation pattern information may be used for all predicted image regions included in that unit.

FIG. 37 is a flowchart illustrating the operation of the motion compensation unit of FIG. 36, and FIG. 38 is a diagram illustrating the same operation.

The operation of the device having the above configuration will now be described.

In this embodiment, whereas conventionally only one motion vector was used to represent the block, up to four vertices of the rectangle of the reference image block are input, and corresponding coordinate positions are first determined by a calculation described later in the operation for determining corresponding points. The determined coordinate positions are then rounded off using interpolation processing instruction information to determine the coordinate positions.

The operation of the components other than the motion compensation unit 90 is the same as in the device of embodiment 7. That is, the entropy decoding unit 51 analyzes the bitstream and separates it into individual coded data. The quantized orthogonal transform coefficients 52 are decoded by the inverse quantization unit 6 and the inverse orthogonal transform unit 7 using the quantization step parameter 17 and sent to the decoding and summing unit 53. The decoding and summing unit 53 outputs the predicted image data 12 as is or adds it to the predicted image data 12 depending on whether it is an intra-coded block or an inter-coded block, based on the value of the intra/inter coding indication flag 16, as a decoded image 55. The decoded image 55 is sent to the display control unit 56, output to a display device, and written to the frame memory 54 as a reference image.

The predicted image generation process in the motion compensation unit 90 will be described below.

In this embodiment, the transformation formula required for transformation is obtained using the required number of motion vectors 25b in accordance with the transformation pattern information 26a, and the sample positions of the predicted image constituent pixels corresponding to each pixel in the predicted image region are determined using the transformation formula. After that, the predicted image is generated by a simple interpolation process in accordance with the pixel precision determined by the interpolation process precision instruction information.

The operation of the motion compensation unit 90 in this embodiment will be described below with reference to Figures 36 to 38.

1) Determining Corresponding Points The corresponding point determiner 37b calculates the coordinate positions at which to sample the predicted image corresponding to each pixel in the predicted image area based on the input motion vector 25b and transformation pattern information 26a. As shown in Figure 38, the motion vectors 25b are the four motion vectors at the vertices of the circumscribing rectangle of the predicted image area. First, the transformation formula required for transformation is obtained in accordance with the transformation pattern information 26a. For example, the following transformation formula is used:

1-1) No motion, stationary state (number of required motion vectors: 0) (i', j') = (i, j) (9) 1-2) Translation (number of required motion vectors: 1) (i', j') = (i + dx0, j + dy0) (10) 1-3) Isotropic transformation (number of required motion vectors: 2) where (x0, y0): coordinates of the upper left vertex of the circumscribing rectangle of the image area to be predicted (x1, y1): coordinates of the upper right vertex of the circumscribing rectangle of the image area to be predicted (x0', y0'): coordinates obtained by displacing (x0, y0) by the first motion vector (dx0, dy0) (x1', y1'): coordinates obtained by displacing (x1, y1) by the second motion vector (dx1, dy1) W: x1 - x0 1-4) Affine transformation (number of motion vectors required: 3) where (x0, y0): coordinates of the upper left vertex of the circumscribing rectangle of the image area to be predicted (x1, y1): coordinates of the upper right vertex of the circumscribing rectangle of the image area to be predicted (x2, y2): coordinates of the lower left vertex of the circumscribing rectangle of the image area to be predicted (x0', y0'): coordinates obtained by displacing (x0, y0) by the first motion vector (dx0, dy0) (x1', y1'): coordinates obtained by displacing (x1, y1) by the second motion vector (dx1, dy1) (x2', y2'): coordinates obtained by displacing (x2, y2) by the third motion vector (dx2, dy2) W: x1 - x0 H: y2 - y0 1-5) Perspective transformation (number of vectors required: 4) however, (x0, y0): Coordinates of the upper left vertex of the circumscribing rectangle of the image area to be predicted (x1, y1): Coordinates of the upper right vertex of the circumscribing rectangle of the image area to be predicted (x2, y2): Coordinates of the lower left vertex of the circumscribing rectangle of the image area to be predicted (x3, y3): Coordinates of the lower right vertex of the circumscribing rectangle of the image area to be predicted (x0', y0'): Coordinates obtained by displacing (x0, y0) by the first motion vector (dx0, dy0) (x1', y1'): Coordinates obtained by displacing (x1, y1) by the second motion vector (dx1, dy1) (x2', y2'): Coordinates obtained by displacing (x2, y2) by the third motion vector (dx2, dy2) (x3', y3'): Coordinates obtained by displacing (x3, y3) by the fourth motion vector (dx3, dy3) W: x1-x0 H: y2 - y0 The format of the transformation pattern information 26a may be bits that directly identify the above transformation formulas (9) to (13), or bits that represent the number of motion vectors, since each transformation corresponds to the number of motion vectors. The above transformation formulas associate point (i, j) in the predicted image area with point (i', j') in the reference image. Furthermore, when calculating the corresponding point position, the sample position of the predicted image can take on values up to the accuracy specified by the interpolation processing accuracy instruction information 60. For example, if rounding is to be performed to half-pixel accuracy, (i', j') obtained by the above transformation formulas is rounded to a value with half-pixel accuracy. If quarter-pixel information is used, (i', j') is rounded to a value with quarter-pixel accuracy. This information representing the sample position accuracy is extracted from the bitstream.

As described above, in this embodiment, a corresponding point determination rule is determined directly from the motion vector 25b, and the sample positions of the predicted image are determined based on this rule.

2) Reading Data for Predicted Image Generation Based on the predicted image sample positions output by the corresponding point determination unit 37b, the memory read address generation unit 38b generates a memory address that identifies the location of the image data required for predictive image generation within the reference images stored in the frame memory 54, and then reads the predicted image generation data.

3) Generation of Predicted Image When addressing only the coordinate values of integer pixel positions of the pixels that make up the predicted image, the data used to generate the predicted image becomes the pixels that make up the predicted image. In this embodiment, the positions at which the predicted image is addressed and sampled can have a predetermined precision, such as half-pixel or quarter-pixel values, as described above. For pixels located at real-number precision, the interpolation processing unit 232b generates integer pixel values of the predicted image based on the integer precision specified in the interpolation processing precision instruction information 60.

In this embodiment, the corresponding point determination unit already rounds the final sample positions to the precision specified by the interpolation processing precision instruction information 60, and the interpolation processing is performed using the following equation (15), as shown in Figure 39. Note that if the position is at half-pixel precision, the processing is exactly the same as that performed by the half-pixel generation unit 232 described in embodiment 1.

As described above, according to the image decoding device of this embodiment, by performing simple sample position calculations using zero or multiple motion vectors, it is possible to efficiently predict motion of different degrees of complexity and obtain a reconstructed image from an encoded bitstream.

The image coding apparatuses according to the first through sixth embodiments and the image decoding apparatus according to the seventh embodiment perform high-speed, complex image coding and decoding using motion compensation with modified processing using only integer-pixel and half-pixel addressing.

In contrast, the image decoding device in this embodiment uses a similar configuration, but the calculation for determining corresponding points has been strengthened to ensure a better match between the reference image and the current block in the predicted image, thereby obtaining more appropriate motion. This allows for smoother motion.

In this embodiment, as in the seventh embodiment, even if the bitstream is one in which the prediction error signal is coded by a coding method other than orthogonal transform coding, the same effect can be obtained by changing elements for the prediction error signal decoding process other than the motion compensation unit 90.

Furthermore, while this embodiment describes an example of decoding processing performed in units of fixed-size blocks, this is applicable not only to decoding devices that use frames of regular television signals as units, but also to decoding devices that use arbitrary-shape image objects (e.g., video object planes) composed of fixed-size blocks as units, as in embodiment 7.

Embodiment 9. In the above embodiments, no mention was made of the number of pixels constituting one block of a predicted image for which motion is to be detected. In other words, we considered pixels of any height (H) and width (W). In this embodiment, we explain a case in which the number of pixels in H and W is limited to a power of two, thereby simplifying coordinate calculations. This reduces the load on the corresponding point determination unit and speeds up calculations.

In this embodiment, only the operation of the corresponding point determination unit 37c of the motion compensation unit 90 in the eighth embodiment shown in FIG. 36 is different, and therefore only the operation of the corresponding point determination unit will be described.

FIG. 40 is a flowchart showing the operation of the corresponding point determining unit 37c.

FIG. 41 is a diagram for explaining the operation of the corresponding point determining unit 37c.

The operation of the corresponding point determination unit 37c in this embodiment will be described below with reference to FIG.

In this embodiment, the corresponding point determination unit 37c receives the motion vector 25b, transformation pattern information 26a, interpolation processing accuracy instruction information 91, and the on-screen position 27a of the predicted image area as input, and calculates and outputs the sample position of the predicted image corresponding to each pixel in the predicted image area based on the following formula. In this case, the on-screen position 27a of the predicted image area is a unique value for each predicted image area. However, the motion vector 25b and transformation pattern information 26a may be unique values for each predicted image area. Alternatively, they may be coded for a larger image (e.g., an image frame or a VOP as disclosed in ISO/IEC JTC1/SC29/WG11) that combines multiple predicted image areas, and the same motion vector and transformation pattern information may be used for all predicted image areas included in that unit. The following describes an example in which up to three motion vectors are used.

Motion vector 25b is defined as the motion vectors of (x0, y0) and the points (x0+W', y0) (W'≧W, W'=2 ^m ) and (x0, y0+H') (H'≧H, H'=2 ⁿ ) obtained by extending the vertices of the upper left and lower right corners of the circumscribing rectangle of the predicted image area from the upper left vertex to a distance that can be expressed as a power of 2, as shown in Figure 41.

Based on these motion vectors, the transformation formulas (16) through (19) necessary for the following transformation are obtained in accordance with the transformation pattern information 26a.

1-1) No movement (number of vectors required: 0) (i', j') = (i, j) (16) 1-2) Translation (number of vectors required: 1) (i', j') = (i + dx0, j + dy0) (17) 1-3) Isotropic transformation (number of vectors required: 2) where (x0, y0): coordinates of the upper left vertex of the circumscribing rectangle of the image area to be predicted; (x1, y1): coordinates of the upper right vertex of the circumscribing rectangle of the image area to be predicted; (x0', y0'): coordinates obtained by displacing (x0, y0) by the first motion vector (dx0, dy0); (x1', y1'): coordinates obtained by displacing (x0 + W', y0) by the second motion vector (dx1, dy1); 1-4) Affine transformation (number of motion vectors required: 3) where (x0, y0): coordinates of the upper left vertex of the circumscribing rectangle of the image area to be predicted (x1, y1): coordinates of the upper right vertex of the circumscribing rectangle of the image area to be predicted (x2, y2): coordinates of the lower left vertex of the circumscribing rectangle of the image area to be predicted (x0', y0'): coordinates obtained by displacing (x0, y0) by the first motion vector (dx0, dy0) (x1", y1"): coordinates obtained by displacing (x0 + W', y0) by the second motion vector (dx1, dy1) (x2", y2"): coordinates obtained by displacing (x0, y0 + H') by the third motion vector (dx2, dy2) The format of the transformation pattern information 26a may be a bit string composed of a plurality of bits expressed in order to directly identify the above transformation formulas (16) to (19), or may be bits representing the number of motion vectors, since each transformation corresponds to the number of motion vectors.

The above conversion formula associates point (i,j) in the predicted image region with point (i',j') in the reference image. Furthermore, when calculating the corresponding point position, the sample position of the predicted image can take on values up to a certain precision. For example, if rounding is performed to half-pixel precision, (i',j') obtained by the above conversion formula will have half-pixel precision; if rounding is performed to quarter-pixel precision, (i',j') will have quarter-pixel precision. Information indicating this sample position precision is extracted from the bitstream.

As described above, in this embodiment, a corresponding point determination rule is determined directly from the motion vector 25b, and the sample positions of the predicted image are determined based on this rule.

2) Reading data for generating a predicted image 3) Generating a predicted image The operations are exactly the same as in embodiment 8, so detailed description is omitted.

As described above, the image decoding device of this embodiment can replace all division operations by W' or H' with bit shift operations when calculating sample positions using zero or multiple motion vectors. This allows for faster determination of sample positions and efficient prediction of motion with different degrees of complexity to produce a reconstructed image from the coded bitstream.

When the motion compensation of this embodiment is applied to an image decoding device based on another encoding method, similar effects can be achieved by changing the corresponding elements. Also, as with the seventh embodiment, it can be applied to a decoding device that uses an arbitrary-shape image object (such as a video object plane) composed of fixed-size blocks as a unit.

The image encoding device and image decoding device of the present invention are combined to form a distinctive image encoding/decoding system.

Furthermore, by performing the operations shown in each operational flowchart, i.e., by including a modified block matching step, a corresponding point determination step, a motion-compensated image generation step, and a decoding and addition step, a distinctive image encoding method and an image decoding method can be obtained.

INDUSTRIAL APPLICABILITY As described above, this invention predicts image motion using integer pixels or intermediate half-pixels of actual sample points, using deformation blocks obtained by coordinate specification alone. This effectively enables efficient prediction without complex calculations such as affine transformations, even for partial images where prediction is difficult using only translational displacements such as motion vectors. Furthermore, this invention can handle not only transformations that can be described mathematically, such as rotation and scaling, but also transformations that cannot be easily described mathematically, i.e., transformations that are difficult to achieve through calculations.

This has the effect of enabling efficient reproduction of high-quality images even on compatible decoding devices.

Furthermore, it has the advantage that it can effectively predict movements that combine rotation and shrinking or enlarging by simply specifying the coordinates of corresponding points, without the need for complex pixel interpolation operations such as affine transformation.

Furthermore, by using motion vectors from translational block matching, the search range for modified block matching can be effectively reduced, thereby reducing the overall computational complexity of motion compensated prediction.

Furthermore, there is an advantage that the motion due to simple scaling can be efficiently predicted by specifying coordinates only, without performing complex pixel interpolation operations such as affine transformation.

Furthermore, since corresponding points can be determined simply by referencing a deformation pattern table, it has the effect of being able to effectively predict movements associated with arbitrary deformations that cannot be expressed by simple mathematical formulas such as affine transformations.

Furthermore, the spatial frequency characteristics within the transformed block can be flattened using a filter, which has the effect of reducing prediction mismatch.

The decoder is designed to support the modified block matching and motion prediction of the image coding device, enabling high-speed decoding and playback of image data with optimal motion prediction.

Furthermore, the addressing of the image decoding device allows for highly flexible motion prediction, which has the effect of enabling smooth image reproduction.

───────────────────────────────────────────────────── Continued from the front page (72) Inventor: Hirofumi Nishikawa 2-2-3 Marunouchi, Chiyoda-ku, Tokyo Ryo Electric Co., Ltd. (72) Inventor: Shinichi Kuroda 2-2-3 Marunouchi, Chiyoda-ku, Tokyo Ryo Electric Co., Ltd. (72) Inventor: Yoshimi Isu 2-2-3 Marunouchi, Chiyoda-ku, Tokyo Ryo Electric Co., Ltd. (72) Inventor: Yuri Hasegawa 2-2-3 Marunouchi, Chiyoda-ku, Tokyo Ryo Electric Co., Ltd. (Note) This publication is based on the publication published internationally by the International Bureau of International Patent Publication (WIPO). Please note that the effect of the international publication of the Japanese-language patent application (Japanese-language utility model registration application) related to this publication arises pursuant to Article 184-10, Paragraph 1 of the Patent Act (Article 48-13, Paragraph 2 of the Utility Model Act), and is unrelated to this publication.

Claims (18)

[Claims] 1. An image coding device that divides an input image into predetermined blocks, includes a motion compensation prediction means for detecting inter-frame motion between the blocks, and performs compression coding of the input image. The image coding device includes: a motion detection unit that transforms only integer pixels, which are actual sample points present in corresponding partial regions of a reference image for motion detection, into a predetermined format, extracts them by specifying coordinates, and compares them with the integer pixels of the blocks of the input image, outputting a motion vector that provides the minimum error extracted by specifying coordinates; and a motion compensation unit that transforms the blocks of the reference image correspondingly according to motion parameters obtained from the comparison output including the transformed block matching unit, and determines the corresponding points by specifying coordinates, outputting a predicted partial image. 2. The image coding device according to claim 1, wherein the modified block matching unit, when performing a predetermined format of modification on a partial region of the reference image, performs the modification using integer pixels and half pixels that are midpoints of the integer pixels. 3. The image coding device according to claim 1 or 2, further comprising a preprocessing unit that separates an input image into subregions of image objects as target regions for coding, and divides each of the separated image objects into blocks for motion estimation and motion compensation. 4. The image coding device according to claim 1 or 2, wherein the modified block matching unit and the motion compensation unit are configured as follows: when specifying integer pixel or half pixel coordinates, the modified block matching unit specifies the coordinates of adjacent points or adjacent points that are a predetermined integer multiple of the coordinates, extracts them, and compares them; and the motion compensation unit similarly processes and outputs a reference image. 5. The image coding device according to claim 1 or 2, wherein the modified block matching unit and the motion compensation unit are configured as a modified block matching unit that extracts and compares integer pixels or half pixels by specifying coordinates rotated in a predetermined angle direction, and a motion compensation unit that similarly processes and outputs a reference image. 6. The image encoding device according to claim 5, wherein the rotation in the predetermined angular direction is positive or negative 45 degrees, 90 degrees, 135 degrees, or 180 degrees. 7. The image coding device according to claim 1, wherein the modified block matching unit and the motion compensation unit are a modified block matching unit that searches for an area indicated by a partial area of the reference image after translation, and moves and compares the search area by enlarging or reducing it or by rotating it in a predetermined direction, and a motion compensation unit that similarly processes and outputs the reference image. 8. The image coding device according to claim 1 or 2, wherein the modified block matching unit includes a transformation pattern table for transforming and comparing a partial region of a reference image, and the modified block matching unit compares an image of the partial region based on transformation values extracted from the transformation pattern table with integer pixels or half pixels of the block of the input image, and the motion compensation unit similarly processes and outputs the reference image. 9. The image coding device according to claim 1 or 2, wherein the modified block matching unit selectively filters and compares specific pixels of the reference image extracted for correspondence evaluation. 10. The image coding device according to claim 1 or 2, characterized in that the frame used for motion detection is a temporally preceding or succeeding frame, and the reference image is a stored temporally preceding or succeeding frame that is compared with the input image. 11. An image decoding device that decompresses and reproduces image compression codes of input information and is equipped with a motion compensation prediction means that detects motion between frames, wherein the motion compensation prediction means includes a mechanism for manually extracting pre-prepared integer pixels of corresponding partial regions based on motion parameters in the input information by specifying coordinates in a predetermined format, and the image signals of the partial regions that have been transformed into the predetermined format are output and added. 12. The image decoding device according to claim 11, wherein the motion compensation prediction means also has a mechanism for specifying and extracting coordinates for half pixels, and performs processing corresponding to the enlargement, reduction, or rotation of the modified block matching of the motion estimation means of the corresponding image encoding device. 13. An image coding method comprising: a motion compensation prediction means for storing a reference image for compression coding of an input digital image, dividing the reference image into predetermined blocks, and detecting motion between frames; a modified block matching step for transforming integer pixels of a partial region of the reference image into a predetermined format, extracting the transformed pixels by specifying coordinates, generating a predicted partial image, and comparing it with the block of the input image; and a corresponding point determination step for determining corresponding points of the partial region by specifying coordinates from a motion vector that provides the smallest error, selected using the modified block matching, to produce a motion-compensated output. 14. The image coding method according to claim 13, wherein the modified block matching step extracts a reference block by specifying coordinates, transforms the reference image into a predetermined format by adding half pixels at its midpoint in addition to the integer pixels of the partial region of the reference image, extracts the reference image by specifying coordinates, and generates a predicted partial image for comparison. 15. The image coding method according to claim 13 or 14, further comprising a transformation pattern table for transforming a partial region of a reference image, and a transformation block matching step for comparing an image of a partial region based on transformation values read from corresponding addresses by referencing the transformation pattern table with the input image during the transformation block matching. 16. An image decoding method comprising: a motion compensation prediction means for storing a reference image and dividing it into predetermined blocks to perform inter-frame motion compensation in order to decompress and reproduce an input image compression code; a motion compensation image generation step for transforming pre-prepared integer pixels of a partial region of the reference image corresponding to a partial region based on reference parameters of the input image code into a predetermined format corresponding to the image encoding method of the transmitting side, extracting the pixels by specifying coordinates, and generating a predicted partial image; and a decoding and addition step for adding the predicted partial images to obtain a reproduced image. 17. The image decoding method according to claim 16, wherein the motion-compensated image generating step is a step of transforming a partial region of the reference image into a predetermined format by adding a half pixel at its midpoint in addition to integer pixels of the partial region to generate a predicted image, specifying and extracting the coordinates, and generating a predicted partial image. 18. An image encoding/decoding system comprising: an image encoding device for dividing an input image into predetermined blocks and compressing and encoding the input image; a motion estimation unit including a modified block matching unit that transforms only integer pixels, which are actual sample points present in corresponding partial regions of a reference image for motion estimation, into a predetermined format, extracts them by specifying their coordinates, and compares them with the integer pixels of the blocks of the input image, outputting a motion vector that minimizes the error extracted by specifying the coordinates; a motion compensation unit including a corresponding point determination unit that transforms corresponding blocks of the reference image in accordance with motion parameters obtained from the comparison output including the modified block matching unit, and determines the corresponding coordinates, outputting a predicted partial image; and an image decoding device including motion compensation prediction means for detecting motion between frames, for decompressing and reproducing image compression codes of input information, wherein the motion compensation prediction means includes a mechanism for extracting pre-prepared integer pixels of corresponding partial regions by specifying their coordinates in a predetermined format based on the motion parameters in the input information, and outputs and adds the image signals of the partial regions processed in the predetermined format.