CN101431681B - Image decoding method and image decoding device - Google Patents

Abstract

A coding control unit ( 110 ) and a mode selection unit ( 109 ) are included. The coding control unit ( 110 ) determines the coding order for a plurality of consecutive B-pictures located between I-pictures and P-pictures so that the B-picture whose temporal distance from two previously coded pictures is farthest in display order is coded by priority, so as to reorder the B-pictures in coding order. When a current block is coded in direct mode, the mode selection unit 109 scales a forward motion vector of a block which is included in a backward reference picture of a current picture and co-located with the current block, so as to generate motion vectors of the current block, if the forward motion vector has been used for coding the co-located block.

Description

Image decoding method and image decoding device

The application of the present invention was submitted by the applicant on February 26, 2003, the application number is 03805346.2, and the title of the invention is a divisional application of "dynamic image coding method and dynamic image decoding method". the

technical field

The present invention relates to an image decoding method and an image decoding device, in particular to a method for performing inter-frame predictive encoding processing and inter-frame predictive decoding processing on a processed frame as a reference frame. the

Background technique

In video coding processing, the amount of information is usually compressed by utilizing redundancy in the space direction and time direction that video has. Here, transformation to the frequency domain is generally used as a method of utilizing redundancy in the spatial direction, and inter-frame (image) predictive coding processing is used as a method of utilizing redundancy in the temporal direction. In the interframe predictive coding process, when a certain frame is coded, the coded frame that is before or after the coding target frame in order of display time is used as a reference frame corresponding to the coding target frame. In addition, detect the motion amount of the encoding target frame relative to the reference frame, and perform motion compensation processing according to the motion amount, and remove the redundancy in the time direction by taking the difference between the image data obtained after the motion compensation processing and the image data of the encoding target frame. And by removing redundancy in the spatial direction from the difference value, the amount of information relative to the encoding target frame is compressed. the

In the dynamic image coding method called H.264 in the current standard, inter-frame predictive coding processing is not performed, and the frame that is about to undergo intra-frame coding processing is called an I frame (image). In addition, a frame that refers to a frame that has been processed before or after the encoding target frame in display time order and is subjected to inter-frame predictive encoding is called a P frame, and refers to a frame that is before or after the encoding target frame in display time order. The two frames that have been processed and inter-frame predictive coding are called B frames (for example, refer to ISO/IEC 14496-2[Information technology-Coding of audio-visual objects-Part2: Visual]pp.218-219). the

Fig. 1(a) is a diagram showing the relationship between each frame and the corresponding reference frame in the above-mentioned dynamic image coding method, and Fig. 1(b) is a diagram showing the sequence of code sequences generated by coding. the

Frame I1 is an I frame, frames P5, P9, and P13 are P frames, and frames B2, B3, B4, B6, B7, B8, B10, B11, and B12 are B frames. That is, P frames P5 , P9 , and P13 are interframe predictively encoded, respectively, using I frame I1 , P frame P5 , and P frame P9 as reference frames, as indicated by arrows. the

In addition, the B frames B2, B3, and B4 are respectively shown by the arrows, and the I frame I1 and the P frame P5 are used as reference frames to perform inter-frame predictive coding. The B frames B6, B7, and B8 are respectively shown by the arrows, and the P frame P5 and P frame B9 are used as reference frames for interframe predictive coding, and B frames B10, B11, and B12 are respectively shown by arrows, and P frame P9 and P frame P13 are used as reference frames for interframe predictive coding. the

In the above encoding, the frame used as the reference frame is encoded before the frame that refers to it. Therefore, the code sequence generated by the above encoding becomes the frame order shown in FIG. 1( b ). the

However, in the video coding method of H.264, a coding mode called direct mode (direct mode) can be selected for coding B frames. The inter prediction method in the direct mode is explained using FIG. 2 . FIG. 2 is an explanatory diagram showing motion vectors in direct mode, and shows a case where block a of frame B6 is coded in direct mode. At this time, the motion vector c used for encoding the block b located at the same position as the block a in the frame P9 which is a reference frame after the frame B6 is used. Motion vector c is a motion vector used when encoding block b, and refers to frame P5. Block a uses a motion vector parallel to motion vector c to obtain a reference block (block) from frame P5 as a forward reference frame and frame P9 as a backward reference frame, and performs bidirectional prediction to encode. That is, the motion vector used for encoding block a becomes motion vector d for frame P5 and motion vector e for frame P9. the

However, as described above, when interframe predictive coding is performed on a B frame with reference to an I frame or a P frame, the temporal distance between the coding target frame and the reference frame may become longer, and in this case, the coding efficiency is reduced. In particular, when the number of inserted B frames, that is, the number of B frames placed between adjacent I frames and P frames, or between two P frames at the closest position increases, the coding efficiency significantly decreases.

Therefore, in order to solve the above problems, the present invention is proposed, and its purpose is to provide a dynamic image encoding method and a dynamic image decoding method, even if the number of B frames inserted between I frames and P frames or inserted between P frames changes In the case where there are too many frames, it is also possible to avoid deterioration of the coding efficiency of B frames. In addition, an object thereof is to provide a video encoding method and a video decoding method capable of improving encoding efficiency in direct mode. the

Contents of the invention

In order to achieve the above object, the present invention takes the following technical solutions:

An image decoding method, which decodes an encoded image, is characterized in that it includes: a decoding step, when decoding a target block (a), the motion vector (C1) of the same position block (b) is used to determine the motion vectors (E1, E2) of the decoding target block (a), the same position block (b) is a block in the decoded frame (B7), and is a block at the same position as the decoding target block, And using the motion vector (E1, E2) of the decoding target block and the reference frame corresponding to the motion vector (E1, E2) of the decoding target block, the decoding target block (a) is subjected to motion compensation in direct mode and decoding, in the decoding step, when the same position block (b) is decoded using a motion vector (C1) and a backward reference frame corresponding to the motion vector (C1), the decoded The one motion vector (C1) used for the same position block (b) is converted using the difference of information indicating the display order of frames, thereby generating a Two motion vectors (E1, E2) for which the target block is subjected to motion compensation and decoded in direct mode, using the generated two motion vectors (E1, E2) and the two generated motion vectors corresponding to each Referring to a frame, the decoding target block (a) is subjected to motion compensation and decoded in direct mode. the

The image decoding method is characterized in that: among the two reference frames respectively corresponding to the two motion vectors of the block to be decoded, the first reference frame is a frame including the block at the same position, and the second reference frame is a frame containing the same position block. The reference frame is one subsequent reference frame used when decoding the block at the same position, and is a reference frame corresponding to a motion vector to be converted when generating two motion vectors of the block to be decoded. the

The image decoding method described above is characterized in that: the information indicating the display order of frames is: first information indicating the display order of frames including the decoding target block, and second information indicating the decoding target block The display order of the second reference frame and the third information indicate a display order of a frame that includes the block at the same position and is the first reference frame of the block to be decoded, and the information includes The difference is a difference between the first information and the second information, a difference between the first information and the third information, and a difference between the second information and the third information. the

An image decoding device, which decodes an encoded image, is characterized by comprising: a decoding unit, when decoding the decoding target block, determining the motion vector of the decoding target block according to the motion vector of the same position block, the The block at the same position is a block in a frame that has been decoded and is at the same position as the block to be decoded, and the motion vector of the block to be decoded and the reference corresponding to the motion vector of the block to be decoded are used frame, performing motion compensation and decoding the decoding target block in a direct mode, and the decoding unit decoding the same position block using one motion vector and one rear reference frame corresponding to the one motion vector , by converting the one motion vector used when decoding the block at the same position by using the difference of the information indicating the display order of frames, thereby generating the performing motion compensation and decoding two motion vectors in the direct mode, and using the two generated motion vectors and the two reference frames corresponding to the two generated motion vectors respectively, the decoding target block is directly mode for motion compensation and decoding. the

In addition, in the case of encoding the encoding target block of the encoding target B frame as the direct mode, the first block referenced based on the encoding of the block in the next P frame at the same position as the encoding target block is used. 1 Referring to the frame and converting the first motion vector using the information difference indicating the display order of the frame, the motion vector for motion compensation of the coding target block can also be obtained. the

Accordingly, when the direct mode is selected, the first motion vector of the subsequent P frame is converted, so that it is not necessary to add motion vector information to the code string, and prediction efficiency can be improved. the

In addition, when coding the coding target block of the coding target B frame as the direct mode, at least the first motion vector based on the first reference frame is used to code the second motion vector located at the same position as the coding target block. When referring to a block in a frame, the first motion vector is converted using the information difference indicating the display order of the frame, and the second motion vector at the same position is encoded using only the second motion vector based on the second reference frame. 2. When referring to a block in a frame, the second motion vector is converted by using the information difference indicating the frame display order, thereby obtaining a motion vector for motion compensation of the coding target block. the

Thus, when the direct mode is selected, if the second reference frame has the first motion vector, the first motion vector is converted, and if the second reference frame does not have the first motion vector but only the second If there is no motion vector, the second motion vector is converted, so it is not necessary to add motion vector information to the code string, and the prediction efficiency can be improved. the

According to the dynamic image decoding method of the present invention, the code sequence generated after encoding the image data corresponding to each frame of the dynamic image is decoded, and it is characterized in that: a decoding step is included, and the decoded frame is used as a reference frame for decoding The target frame of the target is subjected to inter-frame predictive decoding. In the decoding step, when inter-frame predictive decoding is performed based on a bidirectional reference using a decoded frame as a first reference frame and a second reference frame, the decoding includes at least The code sequences of the frames that are closest in display time order are used as the first reference frame and the second reference frame. the

Thus, at the time of decoding, it is possible to decode the code sequence generated by encoding the adjacent frames in the order of display time as the first reference frame and the second reference frame when performing interframe predictive encoding processing based on bidirectional reference. . the

In addition, according to the moving image decoding method of the present invention, the code sequence generated after decoding the image data corresponding to each frame of the moving image is characterized in that: a decoding step is included, and the decoded frame is used as a reference frame, The target frame (target picture) to be decoded is subjected to inter-frame predictive decoding. In the decoding step, the decoded target frame is a frame having a block for performing inter-frame predictive decoding based on a bidirectional reference to a decoded frame. It is used as the first reference frame and the second reference frame, and is used as a direct mode in which the decoding target block is decoded based on the motion vector of the decoded block and motion compensation is performed by the motion vector of the decoding target block , for the first reference frame that is referred to when decoding the block in the second reference frame that is at the same position as the decoding target block, the information difference indicating the frame display order is used to compare the first motion vector Scaling is performed to obtain a motion vector for motion compensation of the decoding target block. the

As a result, when the direct mode is selected, the first motion vector of the second reference frame is scaled, so decoding can be correctly performed at the time of decoding. the

The frame to be decoded is a frame having a block to be decoded by inter-frame prediction based on bidirectional reference, and when the block to be decoded is decoded as a direct mode, the first 2. The second reference frame that the block refers to when referring to the block in the frame converts the second motion vector using the information difference indicating the display order of the frames to obtain the motion vector for motion compensation of the block to be decoded. the

As a result, when the direct mode is selected, the second motion vector of the second reference frame (reference picture) is scaled, so decoding can be correctly performed at the time of decoding. the

In addition, the frame to be decoded is a frame having a block to be decoded with inter-frame prediction based on bidirectional reference, and when the block to be decoded is decoded as a direct mode, when decoding in the direct mode is at the same position as the block to be decoded When decoding the block in the second reference frame above, according to the first reference frame that the block actually refers to when decoding the block in the second reference frame, use the information difference indicating the display order of the reference frame to compare the first reference frame The motion vector is converted to obtain the motion vector when motion compensating the decoding target block. the

As a result, when the direct mode is selected, the first motion vector substantially used in the second reference frame is converted, so that decoding can be accurately performed at the time of decoding.

Also, the frame to be decoded is a frame having a block to be decoded by inter-frame prediction based on bidirectional reference, and when the block to be decoded is decoded as a direct mode, a block located at the same position as the block to be decoded is decoded. The first reference frame used by the block to refer to, and the first motion vector is converted by using the information difference representing the display order of the frame, and the motion vector of the decoding object block can also be obtained when the motion compensation is performed, wherein the block is at the display time It is located later in order, and is within a frame for which inter-frame predictive decoding is performed based on a unidirectional reference using a decoded frame as the first reference frame. the

As a result, when the direct mode is selected, the first motion vector of the frame whose inter-frame predictive decoding is performed based on the unidirectional reference is scaled, so that the decoding process can be accurately performed at the time of decoding. the

A dynamic image decoding method, which decodes encoded images, is characterized in that:

Including a decoding step, when decoding the decoding object block, the motion vector of the decoding object block is determined according to the motion vector of the block at the same position, the block at the same position is a block in the frame that has been decoded, and is the same as the The block whose decoding target block is at the same position,

And using the motion vector of the decoding target block and the reference frame corresponding to the motion vector of the decoding target block, the decoding target block is motion compensated and decoded in direct mode, and in the decoding step, at the same position When a block is decoded using two motion vectors and two reference frames corresponding to the two motion vectors,

One of the two motion vectors used for decoding the block at the same position is converted by using information difference indicating the display order of frames, thereby generating the motion vector for the block to be decoded. Two motion vectors decoded by motion compensation in direct mode for the decoding target block,

Using the two generated motion vectors and the two reference frames respectively corresponding to the two generated motion vectors, the block to be decoded is subjected to motion compensation and decoded in a direct mode. the

The dynamic image decoding method is characterized in that: among the two reference frames respectively corresponding to the two motion vectors of the decoding target block, the first reference frame is a frame including the same position block, and the second 2. The reference frame is one of two reference frames used when decoding the block at the same position, and is a reference frame corresponding to a motion vector to be converted when generating two motion vectors of the block to be decoded. the

The moving image decoding method described above is characterized in that: when the block at the same position is decoded in direct mode, one of the two motion vectors used when decoding the block at the same position is used to generate the block to be decoded The two motion vectors of . the

The dynamic image decoding method is characterized in that: the information representing the display order of the frames is: first information indicating the display order of the frames including the decoding target block;

The second information indicates the display order of the second reference frame of the decoding target block; and the third information indicates the first reference frame of the decoding target block in a frame including the same position block The frame display order of the frame,

The information difference is a difference between the first information and the second information, a difference between the first information and the third information, and a difference between the second information and the third information. the

A dynamic image decoding device, which decodes an encoded image, is characterized in that: it includes a decoding unit that determines the motion vector of the decoding target block according to the motion vector of the same position block when decoding the decoding target block, the The block at the same position is a block in the frame that has been decoded, and is a block at the same position as the decoding target block,

and using the motion vector of the decoding target block and the reference frame corresponding to the motion vector of the decoding target block to perform motion compensation and decoding on the decoding target block in direct mode, the decoding unit uses the When two motion vectors and two reference frames corresponding to the two motion vectors are decoded,

One of the two motion vectors used for decoding the block at the same position is converted by using information difference indicating the display order of frames, thereby generating the motion vector for the block to be decoded. performing motion compensation and decoding two motion vectors of the decoding object block in direct mode, using the generated two motion vectors and the two reference frames respectively corresponding to the generated two motion vectors, and converting the decoding object block to Motion compensated and decoded in direct mode.

In addition, the present invention can be realized not only as such a moving picture coding method and a moving picture decoding method, but also as a moving picture coding device and a moving picture having the characteristic steps included in the moving picture coding method and the moving picture decoding method as components. Image decoding device. Also, it can be implemented as a code string encoded by a video encoding method, and can be distributed via a recording medium such as a CD-ROM or a transmission medium such as the Internet. the

Description of drawings

Fig. 1 is a schematic diagram showing the relationship and order of frame prediction in the conventional dynamic image coding method, (a) is a diagram showing the relationship between each frame and the corresponding reference frame, (b) is a diagram showing the sequence of code sequences generated by coding . the

FIG. 2 is a schematic diagram showing motion vectors in a direct mode in a conventional video encoding method. the

Fig. 3 is a block diagram showing an embodiment of a video coding apparatus using the video coding method of the present invention. the

FIG. 4 is an explanatory diagram of a frame number and a relative index (index) in an embodiment of the present invention. the

Fig. 5 is a schematic diagram of an image encoding signal format of a dynamic image encoding device according to an embodiment of the present invention. the

6 is a schematic view showing the order of frames in the replacement memory according to the embodiment of the present invention, (a) is a diagram showing the input order, and (b) is a diagram showing the replacement order. the

7 is a schematic diagram showing motion vectors in the direct mode in the embodiment of the present invention, (a) shows the case where the target block is frame B7, and (b) shows the first and second motion vectors when the target block a is frame B6. In the diagrams of the two examples, (c) is a diagram showing the third example when the target block a is the frame B6, and (d) is a diagram showing the fourth example when the target block a is the frame B6. the

8 is a schematic diagram showing motion vectors in the direct mode in an embodiment of the present invention, (a) is a diagram showing a fifth example in the case where the target block a is frame B6, and (b) is a diagram showing that the target block a is frame B6 In the figure of the sixth example of the case, (c) is a figure showing the seventh example when the target block a is the frame B6, and (d) is a figure showing the case where the target block a is the frame B8. the

Fig. 9 is a model diagram showing the frame prediction relationship and order in the embodiment of the present invention, (a) is a diagram showing the prediction relationship of each frame represented in the order of display time, (b) is a representation of a replacement code sequence (code sequence order) ) of the frame sequence diagram. the

Fig. 10 is a model diagram showing the frame prediction relationship and order in the embodiment of the present invention, (a) is a diagram showing the prediction relationship of each frame shown in display time order, (b) is a representation of a replacement code sequence (code sequence order) ) of the frame sequence diagram. the

Fig. 11 is a model diagram showing the frame prediction relationship and order in the embodiment of the present invention, (a) is a diagram showing the prediction relationship of each frame shown in display time order, (b) is a representation of a replacement code sequence (code sequence order) ) of the frame sequence diagram. the

FIG. 12 is a schematic diagram hierarchically representing the frame prediction structure shown in FIG. 6 according to an embodiment of the present invention. the

FIG. 13 is a schematic diagram hierarchically representing the frame prediction structure shown in FIG. 9 according to an embodiment of the present invention. the

Fig. 14 is a schematic diagram showing the frame prediction structure shown in Fig. 10 hierarchically according to the embodiment of the present invention. the

Fig. 15 is a schematic diagram showing the frame prediction structure shown in Fig. 11 hierarchically according to the embodiment of the present invention. the

Fig. 16 is a block diagram showing an embodiment of a video decoding apparatus using the video decoding method of the present invention. the

17 is an explanatory diagram of a recording medium for storing a program for realizing the video coding method and the video decoding method of the embodiment by a computer system, (a) is an explanatory diagram showing an example of the physical format of a floppy disk as the main body of the recording medium, ( b) is an explanatory diagram showing the appearance, cross-sectional structure, and floppy disk seen from the front of the floppy disk, and (c) is an explanatory diagram showing the structure for recording and reproducing the above-mentioned program on the floppy disk FD. the

Fig. 18 is a block diagram showing the overall configuration of a content providing system for realizing a content delivery service. the

Fig. 19 is a schematic diagram showing an example of a mobile phone. the

Fig. 20 is a block diagram showing the internal structure of the mobile phone. the

Fig. 21 is a block diagram showing the overall configuration of a digital broadcasting system. the

Detailed ways

Embodiments of the present invention are described with reference to the drawings. the

(Example 1)

Fig. 3 is a block diagram showing an embodiment of a video coding apparatus using the video coding method of the present invention. the

As shown in FIG. 3 , the moving picture encoding device includes: a replacement memory 101, a difference operation unit 102, a prediction error encoding unit 103, a code sequence generation unit 104, a prediction error decoding unit 105, an addition unit 106, a reference frame (reference image ) memory 107, motion vector detection section 108, mode selection section 109, encoding control section 110, switches 111-115 and motion vector storage section 116. the

The replacement memory 101 stores moving images input in frame units in order of display time. The coding control unit 110 replaces each frame stored in the replacement memory 101 in coding order. In addition, the encoding control unit 110 controls the storage operation of the motion vector in the motion vector storage unit 116 . the

The motion vector detection unit 108 uses the coded decoded image data as a reference frame, and detects a motion vector indicating a predicted best position in a search area within the frame. The mode selection unit 109 uses the motion vector detected by the motion vector detection unit 108 to determine the encoding mode of the macroblock, and generates predictive image data based on the encoding mode. The difference calculation unit 102 calculates the difference between the image data read from the replacement memory 101 and the predicted image data input from the mode selection unit 109 to generate prediction error image data. the

The prediction error coding unit 103 performs coding processing such as frequency conversion and quantization on the input prediction error image data to generate coded data. The code sequence generator 104 performs variable-length coding on the input coded data, and generates a code sequence by adding motion vector information, coding mode information, and other related information input from the mode selector 109 . the

The prediction error decoding unit 105 performs decoding processing such as dequantization and inverse frequency transformation on the input encoded data to generate decoded difference image data. The addition unit 106 adds the decoded difference image data input from the prediction error decoding unit 105 and the predicted image data input from the mode selection unit 109 to generate decoded image data. The reference frame memory 107 stores the generated decoded image data. the

Fig. 4 is an explanatory diagram of a frame and a relative index (pointer). The relative index is used to uniquely identify the reference frame stored in the reference frame memory 107, and as shown in FIG. 4, it is a serial number corresponding to each frame. In addition, the relative index is used to indicate a reference frame used when encoding a block by inter prediction. the

Fig. 5 is a schematic diagram of a format of a video coding signal by a video coding device. The coded signal Picture in one frame unit is composed of a coded header signal Header included in the head of a frame, a coded signal Block1 of a block based on a direct mode, a coded signal Block2 of a block based on an inter-frame prediction other than the direct mode, and the like. Also, the coded signal Block2 of a block based on inter prediction other than the direct mode sequentially includes a first relative index Rldx1 and a second relative index Rldx2 for indicating two reference frames used in inter prediction, a first motion vector MV1, a second relative index 2 Motion Vector MV2. On the other hand, the encoded signal Block1 based on the direct mode does not have the first relative index Rldx1, the second relative index Rldx2, the first motion vector MV1, and the second motion vector MV2. Which of the first relative index Rldx1 and the second relative index Rldx2 is used can be determined by the prediction type PredType. In addition, the first relative index Rldx1 indicates the first reference frame, and the second relative index Rldx2 indicates the second reference frame. That is, whether it is the first reference frame or the second reference frame is determined by the data position in the code sequence. the

A frame for inter-frame predictive encoding based on using the encoded frame that is located in the front or back in the display time sequence as the first reference frame is a P frame, and based on the display time sequence that is located in the front or back First, the encoded frame is used as a bidirectional reference between the first reference frame and the second reference frame. The frame for inter-frame predictive coding is a B frame, but in this embodiment, the first reference frame is used as a forward reference frame, The second reference frame will be described as a backward reference frame. In addition, a first motion vector and a second motion vector, which are motion vectors with respect to the first reference frame and the second reference frame, respectively, will be described as a forward motion vector and a backward motion vector, respectively. the

Next, the method of adding the first relative index and the second relative index will be described with reference to FIG. 4(a). the

In the first relative index, first, as for the information indicating the display order, values starting from 0 are assigned to the reference frames before the encoding target block in the order closer to the encoding target frame. Values starting from 0 are assigned to all reference frames before the encoding target block, and next values are assigned in the order closer to the encoding target frame for the reference frames subsequent to the encoding target block. the

In the second relative index, first, as information indicating the display order, values starting from 0 are assigned to reference frames subsequent to the current block to be coded in order closer to the current frame to be coded. Values starting from 0 are assigned to all reference frames after the encoding target, and next values are assigned in the order closer to the encoding target frame (target picture) for the reference frames preceding the encoding target block. the

For example, in Figure 4(a), when the first relative index (pointer) Rldx1 is 0 and the second relative index Rldx2 is 1, the forward reference frame is the B frame whose frame number is 6, and the backward reference frame is P frame with frame number 9. Here, the frame number is a number indicating the order of display. the

The relative index in the block is represented by a variable-length code word, and a code with a shorter code length is assigned as the value is smaller. Usually, the closest frame to the encoding target frame is selected as a reference frame (reference picture) for inter prediction, so if the relative index values are assigned in order from the encoding target frame (target picture) as described above, the encoding efficiency will improve. the

On the other hand, the assignment of relative indexes to reference frames can be changed arbitrarily by explicitly instructing them with a buffer control signal (RPSL in the Header shown in FIG. 5 ) in the coded signal. By changing the assignment, the reference frame whose second relative index becomes 0 can be changed to any reference frame in the reference frame memory 107, for example, as shown in FIG. 4(b), the assignment of the relative index to the frame can be changed. the

Next, the operation of the video encoding device configured as described above will be described. the

6 is an explanatory diagram showing the sequence of frames in the replacement memory 101, (a) is an explanatory diagram showing the input sequence, and (b) is an explanatory diagram showing the replacement sequence. Among them, the vertical line represents the frame, and among the symbols shown at the lower right of each frame, the α header of the first character represents the frame type (I, P or B), and the numbers after the second character represent the frame number, and the frame number represents the display order. the

The input images are input into the replacement memory 101 in units of frames in order of display time, for example, as shown in FIG. 6( a ). When frames are input to the replacement memory 101 , the encoding control unit 110 replaces the frames input into the replacement memory 101 with the order of encoding. The replacement to the coding order is performed according to the reference relationship in the interframe predictive coding, and the frame used as the reference frame is replaced so that it is coded before the frame used as the reference frame. the

Here, it is assumed that a P frame refers to a coded I or P frame near the front or rear in the display time order. In addition, it is assumed that the B frame refers to two frames that have already been encoded in the vicinity of the front or rear in the order of display time. the

The encoding order of the frames is that among the B-frames located between two P-frames (three in the example in Fig. . For example, for frames B6-P9, encoding is performed in the order of frames P9, B7, B6, and B8. the

At this time, among the frames B6-P9, the frame at the end of the arrow shown in FIG. 6( a ) refers to the frame at the start of the arrow. That is, frame B7 refers to frames P5 and P9, frame B6 refers to frames P5 and B7, and frame B8 refers to frames B7 and P9. In addition, at this time, the encoding control unit 110 replaces each frame with the order of encoding shown in FIG. 6( b ). the

Next, each frame after replacement in the replacement memory 101 is read out in units of motion compensation. Here, the motion compensation unit is called a macroblock, and the macroblock is set to have a size of 16 horizontal x 16 vertical pixels. Next, encoding processing of frames P9, B7, B6, and B8 shown in FIG. 6(a) will be described sequentially. the

(Encoding processing of frame P9)

Since the frame P9 is a P frame, inter-frame predictive coding is performed with reference to a previously processed frame in the order of display time. In encoding of frame P9, as described above, the reference frame becomes frame P5. After the encoding of frame P5 is terminated, the decoded image is stored in the reference frame memory 107 . In encoding of a P frame, the encoding control unit 110 controls the switches so that the switches 113 , 114 , and 115 are turned on. Accordingly, the macroblock of the frame P9 read from the replacement memory 101 is first input to the motion vector detection unit 108 , the mode selection unit 109 , and the difference calculation unit 102 . the

The motion vector detection unit 108 uses the decoded image data of the frame P5 stored in the reference frame memory 107 as a reference frame, and detects a motion vector for the macroblock of the frame P9. Also, the motion vector detection unit 108 outputs the detected motion vector to the mode selection unit 109 .

The mode selection unit 109 uses the motion vector detected by the motion vector detection unit 108 to determine the coding mode of the macroblock in the frame P9. Here, the so-called encoding mode indicates which method is used to encode the macroblock. In the case of a P frame, it is determined to perform encoding by a certain method, for example, from among intra-frame encoding, inter-frame predictive encoding using motion vectors, and inter-frame predictive encoding not using motion vectors (moving is treated as 0). In the determination of the coding mode, the method of further reducing the coding error by a small amount of bits is generally selected. the

The mode selection unit 109 outputs the determined encoding mode to the code sequence generation unit 104 . At this time, when the encoding mode determined by the mode selection unit 109 is inter predictive encoding, the motion vector used for the inter predictive encoding is output to the code sequence generation unit 104 and stored in the motion vector storage unit 116 . the

The mode selection unit 109 generates predicted image data based on the determined encoding mode, and outputs the predicted image data to the difference calculation unit 102 and the addition calculation unit 106 . However, when the mode selection unit 109 selects intra coding, no predictive image data is output. In addition, when the mode selection unit 109 selects intra-frame encoding, the control switch 111 is connected to the a side, and the control switch 112 is connected to the c-side, and when the inter-frame predictive encoding is selected, the control switch 111 is connected to the b side, and the control switch 112 is connected to side d. Next, a case where inter prediction coding is selected by the mode selection unit 109 will be described. the

The image data of the macroblock of the frame P9 read from the replacement memory 101 and the predicted image data output from the mode selection unit 109 are input to the difference calculation unit 102 . The difference calculation unit 102 calculates the difference between the image data of the macroblock of the frame P9 and the predicted image data to generate predicted image data, and outputs it to the prediction error coding unit 103 . the

The prediction error encoding unit 103 performs encoding processing such as frequency conversion and quantization on the input prediction error image data to generate encoded data, and outputs the coded data to the code sequence generation unit 104 and the prediction error decoding unit 105 . Here, it is assumed that frequency conversion or quantization processing is performed in units of, for example, 8 horizontal x 8 vertical pixels, or 4 horizontal x 4 vertical pixels. the

The code sequence generator 104 performs variable-length coding on the input coded data, and adds information such as motion vectors and coding modes, header information, etc., to generate a code sequence and output it.

On the other hand, the prediction error decoding unit 105 performs decoding processing such as dequantization and inverse frequency transformation on the input encoded data, generates decoded difference image data, and outputs it to the addition unit 106 . The addition unit 106 generates decoded image data by adding the decoded difference image data to the predicted image data input from the mode selection unit 109 , and stores the decoded image data in the reference frame memory 107 . the

Through the above processing, the processing of one macroblock of the frame P9 is completed. Through the same processing, encoding processing is also performed on the remaining macroblocks in the frame P9. In addition, when the processing is completed for all the macroblocks in the frame P9, the encoding processing of the frame B7 is performed next. the

(Encoding processing of frame B7)

Among the reference frames of frame B7, the forward reference frame is frame P5, and the backward reference frame is P9. Since the frame B7 is used as a reference frame when encoding other frames, the encoding control unit 110 controls the switches so that the switches 113, 114, and 115 are turned on. Accordingly, the macroblock of the frame B7 read from the replacement memory 101 is input to the motion vector detection unit 108 , the mode selection unit 109 , and the difference calculation unit 102 . the

The motion vector detection unit 108 uses the decoded image data of the frame P5 stored in the reference frame memory 107 as a forward reference frame, and uses the decoded image data of the frame P9 as a backward reference frame, and detects the forward direction for the macroblock of the frame B7. motion vector and backward motion vector. Also, the motion vector detection unit 108 outputs the detected motion vector to the mode selection unit 109 . the

The mode selection unit 109 uses the motion vector detected by the motion vector detection unit 108 to determine the macroblock coding mode of the frame B7. Here, the coding mode of the B frame can be selected from, for example, intra-frame coding, inter-frame predictive coding using forward motion vectors, inter-frame predictive coding using backward motion vectors, inter-frame predictive coding using bidirectional motion vectors, direct mode choose. the

The operation when encoding is performed in the direct mode will be described with reference to FIG. 7(a). FIG. 7( a ) is an explanatory diagram showing motion vectors in the direct mode, and shows a case where block a of frame B7 is coded in the direct mode. In this case, the frame P9 is used as a reference frame after the frame B7 using the motion vector c used for encoding the block b at the same position as the block a in the frame P9. The motion vector c is stored in the motion vector storage unit 116 . The block a is bidirectionally predicted using the motion vector obtained from the motion vector c, based on the frame P5 as the forward reference frame and the frame P9 as the backward reference frame. For example, as a method of using the motion vector c, there is a method of generating a vector parallel to the motion vector c. The motion vector used for encoding the block a at this time becomes the motion vector d for the frame P5 and the motion vector e for the frame P9. the

At this time, if the size of the motion vector d as the forward motion vector is MVF, the size of the motion vector e as the backward motion vector is MVB, and the size of the motion vector c is MV, the current frame (frame B7) after The time distance to the frame (frame P5) referred to by the block of the reference frame (frame P9) and its backward reference frame is TRD, and the time distance between the current frame (frame B7) and the forward reference frame (frame P5) is TRF, then The magnitude MVF of the motion vector d and the magnitude MVB of the motion vector e are obtained by (Equation 1) and (Equation 2), respectively. In addition, the temporal distance between frames can be specified, for example, based on information indicating the display order (position) added to each frame or the information difference. the

MVF＝MV×TRF/TRD Formula 1

MVB＝(TRF-TRD)×MV/TRD Formula 2

Among them, MVF and MVB represent the horizontal component and vertical component of the motion vector respectively, and the positive and negative symbols represent the direction of the motion vector. the

In the selection of the coding mode, the method of further reducing the coding error by a small amount of bits is usually selected. The mode selection unit 109 outputs the determined encoding mode to the code sequence generation unit 104 . At this time, when the encoding mode determined by the mode selection unit 109 is inter predictive encoding, the motion vector used for the inter predictive encoding is output to the code sequence generation unit 104 and stored in the motion vector storage unit 116 . Also, when the direct mode is selected, the motion vector used in the direct mode calculated by (Equation 1) and (Equation 2) is stored in the motion vector storage unit 116 . the

The mode selection unit 109 generates predicted image data based on the determined encoding mode, and outputs the predicted image data to the difference calculation unit 102 and the addition calculation unit 106 . However, when the mode selection unit 109 selects intra coding, no predictive image data is output. In addition, when the mode selection unit 109 selects intra-frame coding, the control switch 111 is connected to the a side, and the control switch 112 is connected to the c side, and when inter-frame predictive coding or direct mode is selected, the control switch 111 is connected to the b side , the control switch 112 is connected to side d. Next, the case where the mode selection unit 109 selects the inter prediction coding or the direct mode will be described. the

The image data of the macroblock of the frame B7 read from the replacement memory 101 and the predicted image data output from the mode selection unit 109 are input to the difference calculation unit 102 . The difference calculation unit 102 calculates the difference between the image data of the macroblock of the frame B7 and the predicted image data to generate predicted image data, and outputs it to the prediction error coding unit 103 . the

The prediction error encoding unit 103 performs encoding processing such as frequency conversion and quantization on the input prediction error image data to generate encoded data, and outputs the coded data to the code sequence generation unit 104 and the prediction error decoding unit 105 . the

The code sequence generator 104 performs variable-length coding or the like on the input coded data, and adds information such as a motion vector and a coding mode to generate a code sequence and output it. the

On the other hand, the prediction error decoding unit 105 performs decoding processing such as dequantization and inverse frequency transformation on the input encoded data, generates decoded difference image data, and outputs it to the addition unit 106 . The addition unit 106 generates decoded image data by adding the decoded difference image data to the predicted image data input from the mode selection unit 109 , and stores the decoded image data in the reference frame memory 107 . the

Through the above processing, the processing of one macroblock of the frame B7 is completed. Through the same processing, encoding processing is also performed on the remaining macroblocks in the frame B7. In addition, when all the macroblocks in the frame B7 are processed, the encoding process of the frame B6 is performed next. the

(Encoding processing of frame B6)

Since the frame B6 is a B frame, inter-frame predictive coding is performed with reference to two processed frames located in the front or back in the order of display time. As described above, among the reference images of frame B6, the forward reference frame is frame P5, and the backward reference frame is B7. Frame B6 is not used as a reference frame when encoding other frames. Accordingly, the encoding control unit 110 controls the switches so that the switch 113 is turned on and the switches 114 and 115 are turned off. Thus, the macroblock of the frame B6 read from the replacement memory 101 is input to the motion vector detection unit 108 , the mode selection unit 109 , and the difference calculation unit 102 .

The motion vector detection unit 108 uses the decoded image data of the frame P5 stored in the reference frame memory 107 as a forward reference frame, and uses the decoded image data of the frame B7 as a backward reference frame, and detects the forward direction for the macroblock of the frame B6. motion vector and backward motion vector. Also, the motion vector detection unit 108 outputs the detected motion vector to the mode selection unit 109 . the

The mode selection unit 109 uses the motion vector detected by the motion vector detection unit 108 to determine the macroblock coding mode of the frame B6. the

Here, a first example of the operation when the direct mode is used for the macroblock of the frame B6 will be described with reference to FIG. 7(b). Fig. 7(b) is an explanatory diagram showing motion vectors in the direct mode, and shows a case where block a of the frame B6 is coded in the direct mode. At this time, frame B7 is used as a reference frame after frame B6 by using the motion vector c used when encoding block b at the same position as block a in frame B7. Assume that block b is coded only by forward reference or by bidirectional reference, and this forward motion vector is assumed to be motion vector c. It is assumed that the motion vector c is stored in the motion vector storage unit 116 . The block a is bidirectionally predicted using the motion vector generated using the motion vector c, based on the frame P5 as the forward reference frame and the frame B7 as the backward reference frame. For example, if a method of generating a motion vector parallel to motion vector c is used as in the case of frame B7 above, the motion vector used for encoding block a becomes motion vector d for frame P5 and motion vector e for frame B7 . the

At this time, if the size of the motion vector d as the forward motion vector is MVF, the size of the motion vector e as the backward motion vector is MVB, and the size of the motion vector c is MV, the current frame (frame B6) after The time distance to the frame (frame P5) referred to by the block B of the reference frame (frame B7) and its backward reference frame is TRD, and the time distance between the current frame (frame B6) and the forward reference frame (frame P5) is TRF, Then, the magnitude MVF of the motion vector d and the magnitude MVB of the motion vector e are obtained by the above-mentioned (Equation 1) and (Equation 2), respectively. In addition, the temporal distance between frames can be specified, for example, based on information indicating the order of display added to the frames or the information difference. the

In this way, in the direct mode, by converting the forward motion vector of the B frame as the backward reference frame, it is not necessary to transmit the information of the motion vector, and the motion prediction efficiency can be improved. Thus, encoding efficiency can be improved. Also, encoding efficiency can be improved by using the latest available reference frame in display time order as the forward reference frame and the backward reference frame. the

Next, a second example of using the direct mode will be described with reference to FIG. 7(b). At this time, frame B7 is used as a reference frame after frame B6 by using the motion vector used for encoding block b at the same position as block a in frame B7. Here, block b is coded in the direct mode, and the forward motion vector substantially used at this time is assumed to be motion vector c. That is, the motion vector c is a motion vector obtained by scaling the motion vector used when encoding the block i located at the same position as the block b in the frame P9 back-referenced by the frame B7. The motion vector c is calculated by using the motion vector stored in the motion vector storage unit 116 or by reading from the motion vector storage unit 116 the motion vector of the block i in the frame P9 used when encoding the block b in the direct mode. The mode selection unit 109 may store only the forward motion vector when the motion vector obtained by the scaling process when encoding the block b of frame B7 in the direct mode is stored in the motion vector storage unit 116 . The block a is bidirectionally predicted using the motion vector generated using the motion vector c, based on the frame P5 as the forward reference frame and the frame B7 as the backward reference frame. For example, if a method of generating a motion vector parallel to the motion vector c is used as in the case of the first example above, the motion vector used for encoding block a becomes motion vector d for frame P5 and motion vector d for frame B7 e. the

At this time, the magnitude MVF of the motion vector d as the forward motion vector for block a and the magnitude MVB of the motion vector e as the backward motion vector are the same as in the first example of the direct mode, and (Equation 1), ( Formula 2) can be obtained. the

In this way, in the direct mode, the forward motion vector substantially used in the direct mode of the B frame as the backward reference frame is converted, so there is no need to transmit the information of the motion vector, and even if the backward reference frame is coded in the direct mode The motion prediction efficiency can also be improved when there are blocks at the same position in the frame. Thus, encoding efficiency can be improved. Also, encoding efficiency can be improved by using the latest reference frame available in display time order as the forward and backward reference frames. the

Next, the third example when using the direct mode will be described with reference to FIG. 7(c). FIG. 7(c) is an explanatory diagram showing motion vectors in the direct mode, and shows a case where block a of the frame B6 is coded in the direct mode. In this case, frame B7 is used as a reference frame after frame B6 by using motion vector c used when encoding block b at the same position as block a in frame B7. Wherein, it is assumed that only the backward motion vector is used to encode the block b, and this backward motion vector is assumed to be the motion vector f. It is assumed that the motion vector f is stored in the motion vector storage unit 116 . The block a is bidirectionally predicted using the motion vector generated using the motion vector f, based on the frame P5 as the forward reference frame and the frame B7 as the backward reference frame. For example, if the method of generating a motion vector parallel to the motion vector f is used as in the case of the first example above, the motion vector used for encoding block a becomes motion vector g for frame P5 and motion vector h for frame B7 . the

At this time, if the size of the motion vector g as the forward motion vector is MVF, the size of the motion vector h as the backward motion vector is MVB, and the size of the motion vector f is MV, the current frame (frame B6) after The temporal distance to the reference frame (frame B7) and the frame (frame P9) referred to by the blocks of the backward reference frame is TRD, the temporal distance between the current frame (frame B6) and the forward reference frame (frame P5) is TRF, and the current The temporal distance between the frame (frame B6) and the backward reference frame (frame B7) is TRB, and the magnitude MVF of the motion vector g and the magnitude MVB of the motion vector h are obtained by (Equation 3) and (Equation 4), respectively. the

MVF＝-TRF×MV/TRD Formula 3

MVB＝TRB×MV/TRD Formula 4

In this way, in the direct mode, the backward motion vector used when encoding the block at the same position in the B frame as the backward reference frame is converted, so there is no need to transmit the information of the motion vector, and even in the backward reference frame The prediction efficiency can also be improved when the block at the same position of the α has only backward motion vectors. Thus, encoding efficiency can be improved. Also, encoding efficiency can be improved by using the latest reference frame available in display time order as the forward and backward reference frames. the

Next, a fourth example of using the direct mode will be described with reference to FIG. 7(d). FIG. 7( d ) is an explanatory diagram showing motion vectors in the direct mode, and shows a case where block a of the frame B6 is coded in the direct mode. At this time, frame B7 is used as a reference frame after frame B6 by using the motion vector used for encoding block b at the same position as block a in frame B7. Assume that, as in the third example, only the backward motion vector is used to encode the block b, and this backward motion vector is assumed to be the motion vector f. It is assumed that the motion vector f is stored in the motion vector storage unit 116 . The block a uses the motion vector generated by the motion vector f, and performs bidirectional prediction based on the frame P9 as the reference frame of the motion vector f and the frame B7 as the backward reference frame. For example, if a method of generating a motion vector parallel to the motion vector f is used as in the case of the first example above, the motion vector used when encoding block a becomes motion vector g for frame P9 and motion vector g for frame B7 h. the

At this time, if the size of the motion vector g as the forward motion vector is MVF, the size of the motion vector h as the backward motion vector is MVB, and the size of the motion vector f is MV, the current frame (frame B6) after The time distance to the frame (frame P9) referred to by the block of the reference frame (frame B7) and its backward reference frame is TRD, the current frame (frame B6) and the frame referred to by the block of the backward reference frame (frame B7) ( When the temporal distance of frame P9) is TRF, the magnitude MVF of the motion vector g and the magnitude MVB of the motion vector h are obtained from (Equation 1) and (Equation 2), respectively. the

In this way, in the direct mode, the backward motion vector used when encoding the block at the same position in the B frame as the backward reference frame is converted, so that there is no need to send the information of the motion vector, and even if the same block in the backward reference frame Even when the block at the position has only backward motion vectors, the prediction efficiency can be improved. Thus, encoding efficiency can be improved. In addition, by using the frame referred to by the backward motion vector as the forward reference frame and using the latest reference frame available in display time order as the backward reference frame, coding efficiency can be improved. the

Next, a fifth example when the direct mode is used will be described with reference to FIG. 8(a). FIG. 8( a ) is an explanatory diagram showing motion vectors in the direct mode, and shows a case where block a of the frame B6 is coded in the direct mode. In this case, the magnitude of the motion vector is set to 0, the frame P5 is used as a forward reference frame, and the frame B7 is used as a backward reference frame, and motion compensation is performed by performing bidirectional reference. the

In this way, in the direct mode, by forcibly setting the motion vector to 0, when the direct mode is selected, it is not necessary to transmit the information of the motion vector, and it is not necessary to perform conversion processing on the motion vector, thereby reducing the amount of processing. the

Next, a sixth example when the direct mode is used will be described with reference to FIG. 8(b). FIG. 8(b) is an explanatory diagram showing motion vectors in the direct mode, and shows a case where block a of the frame B6 is coded in the direct mode. At this time, using the motion vector g used when encoding the block f in the same position as the block a in the frame P9, the frame P9 is regarded as a P frame located after the frame B6. The motion vector g is stored in the motion vector storage unit 116 . The block a is bidirectionally predicted from the frame P5 as the forward reference frame and the frame B7 as the backward reference frame using the motion vector generated by the motion vector g. For example, if the method of generating a motion vector parallel to the motion vector g is used as in the first example above, the motion vector used for encoding block a becomes motion vector h for frame P5 and motion vector i for frame B7 . the

At this time, if the size of the motion vector h as the forward motion vector is MVF, the size of the motion vector i as the backward motion vector is MVB, and the size of the motion vector g is MV, the display time sequence is located in the current frame ( The time distance between the P frame (frame P9) after frame B6) and the frame (frame P5) referred to by the block f of the P frame is TRD, and the time distance between the current frame (frame B6) and the forward reference frame (frame P5) is TRF, and the temporal distance between the current frame (frame B6) and the backward reference frame (frame B7) is TRB, then the size MVF of the motion vector h and the size MVB of the motion vector i are calculated by (Formula 1) and (Formula 5) respectively out. the

MVB＝-TRB×MV/TRD Formula 5

In this way, in the direct mode, the motion vector of the P frame located behind in the display time order is converted, and when the backward reference frame is a B frame, it is not necessary to store the motion vector of the B frame, and it is not necessary to transmit the motion vector information. Also, encoding efficiency can be improved by using the latest reference frame in display time order as the forward and backward reference frames. the

Next, a seventh example of using the direct mode will be described with reference to FIG. 8(c). FIG. 8(c) is an explanatory diagram showing motion vectors in the direct mode, and shows a case where block a of the frame B6 is coded in the direct mode. In this example, the assignment of the relative index is changed (remapped) to the frame number described above, and the backward reference frame is changed to the frame P9. At this time, frame P9 is used as a backward reference frame of frame B7 by using the motion vector g used when encoding block f in the same position as block a in frame P9. The motion vector g is stored in the motion vector storage unit 116 . The block a is bidirectionally predicted using the motion vector generated using the motion vector g, based on the frame P5 as the forward reference frame and the frame P9 as the backward reference frame. For example, if a method of generating a motion vector parallel to the motion vector g is used as in the case of the first example above, the motion vector used for encoding block a becomes motion vector h for frame P5 and motion vector h for frame P9. Vector i. the

At this time, if the size of the motion vector h as the forward motion vector is MVF, the size of the motion vector i as the backward motion vector is MVB, and the size of the motion vector g is MV, the current frame (frame B6) after The time distance to the frame (frame P5) referred to by the block of the reference frame (frame P9) and its backward reference frame is TRD, and the time distance between the current frame (frame B6) and the forward reference frame (frame P5) is TRF, then The magnitude MVF of the motion vector h and the magnitude MVB of the motion vector i are obtained by (Equation 1) and (Equation 2), respectively. the

In this way, in the direct mode, even when the assignment of the relative index to the frame number is changed, the motion vector of the coded frame can be converted, and when the direct mode is selected, the motion vector information does not need to be transmitted. the

In addition, when encoding block a of frame B6 by direct mode, only the block in the backward reference frame of frame B6 with the same position as block a is encoded by forward reference, bidirectional reference or direct mode, and the forward motion is used in encoding In the case of a vector, the forward motion vector is scaled, and the block a is coded in the direct mode as in the above-mentioned first example, second example, or seventh example. On the other hand, when a block at the same position as block a is coded only by backward reference, and when a backward motion vector is used for coding, the backward motion vector is converted, as in the third or fourth example above. , block a is coded by direct mode. the

The direct mode described above is applicable not only when the time interval between frames is constant, but also when the interval between frames is variable. the

The mode selection unit 109 outputs the determined encoding mode to the code sequence generation unit 104 . Also, the mode selection unit 109 generates predicted image data based on the determined encoding mode, and outputs the predicted image data to the difference calculation unit 102 . However, the mode selection unit 109 does not output predictive image data when intra coding is selected. In addition, the mode selection unit 109 controls the switch 111 to be connected to the a side and the control switch 112 to the c side when intra-frame coding is selected, and controls the switch 111 to be connected to the b side when inter-frame predictive coding or direct mode is selected. , the control switch 112 is connected to side d. Also, when the identified encoding mode is inter predictive encoding, the mode selection unit 109 outputs the motion vector used for the inter predictive encoding to the code sequence generation unit 104. Here, since the frame B6 is not used as a reference frame when encoding other frames, motion vectors used in inter-frame predictive encoding do not need to be stored in the motion vector storage unit 116 . Next, the case where the mode selection unit 109 selects the inter prediction coding or the direct mode will be described. the

The image data of the macroblock of the frame B6 read from the replacement memory 101 and the predicted image data output from the mode selection unit 109 are input to the difference calculation unit 102 . The difference calculation unit 102 calculates the difference between the image data of the macroblock of the frame B6 and the predicted image data to generate prediction error image data, and outputs it to the prediction error coding unit 103 . The prediction error coding unit 103 generates coded data by performing coding processing such as frequency conversion or quantization on the input prediction error image data, and outputs it to the code sequence generation unit 104 . the

The code sequence generator 104 performs variable-length coding or the like on the input coded data, and adds information such as a motion vector and a coding mode to generate a code sequence and output it. the

Through the above processing, the coding processing of one macroblock of the frame B6 is completed. By performing the same processing on the remaining macroblocks of the frame B6, once the processing is completed, the encoding processing of the frame B8 is performed. the

(encoding processing of frame B8)

Since the frame B8 is a B frame, inter-frame predictive coding is performed with reference to two processed frames located in the front or back in the order of display time. As described above, among the reference images of frame B8, the forward reference frame is frame B7, and the backward reference frame is P9. When encoding other frames, the frame B8 is not used as a reference frame, so the encoding control unit 110 controls the switches so that the switch 113 is turned on and the switches 114 and 115 are turned off. Thus, eight macroblocks in a frame read from the replacement memory 101 are input to the motion vector detection unit 108 , the mode selection unit 109 , and the difference calculation unit 102 . the

The motion vector detection unit 108 uses the decoded image data of the frame B7 stored in the reference frame memory 107 as a forward reference frame, and uses the decoded image data of the frame P9 as a backward reference frame, and detects the forward direction for the macroblock of the frame B8. motion vector and backward motion vector. In addition, the motion vector detection unit 108 outputs the detected motion vector to the mode selection unit 109. the

The mode selection unit 109 uses the motion vector detected by the motion vector detection unit 108 to determine the macroblock coding mode of the frame B8. the

Here, the operation when the direct mode is used for the macroblock of the frame B8 will be described with reference to FIG. 8( d ). FIG. 8( d ) is an explanatory diagram showing motion vectors in the direct mode, and shows a case where block a of the frame B8 is coded in the direct mode. At this time, frame P9 is used as a reference frame after frame B8 using the motion vector used for encoding block b at the same position as block a in frame P9. Let block b be coded by forward reference only, and let this forward motion vector be motion vector c. It is assumed that the motion vector c is stored in the motion vector storage unit 116 . The block a is bidirectionally predicted using the motion vector generated using the motion vector c, based on the frame B7 as the forward reference frame and the frame P9 as the backward reference frame. For example, if a method of generating a motion vector parallel to motion vector c is used as in the case of frame B7 above, the motion vector used for encoding block a becomes motion vector d for frame B7 and motion vector e for frame P9 . the

At this time, if the size of the motion vector d as the forward motion vector is MVF, the size of the motion vector e as the backward motion vector is MVB, and the size of the motion vector c is MV, the current frame (frame B8) after The time distance to the frame (frame P5) referred to by the block b of the reference frame (frame P9) and its backward reference frame is TRD, and the time distance between the current frame (frame B8) and the forward reference frame (frame B7) is TRF, The temporal distance between the current frame (frame B8) and the backward reference frame (frame P9) is TRB, then the magnitude MVF of the motion vector d and the magnitude MVB of the motion vector e are obtained by the above (Equation 1) and (Equation 5), respectively. the

In this way, in the direct mode, by converting the forward motion vector of the backward reference frame, it is not necessary to transmit the information of the motion vector, and the prediction efficiency can be improved. Thus, encoding efficiency can be improved. Furthermore, encoding efficiency can be improved by using the latest available reference frame in display time order as a forward reference frame and a backward reference frame. the

The direct mode described above is applicable not only when the time interval between frames is constant, but also when the interval between frames is variable. the

The mode selection unit 109 outputs the determined encoding mode to the code sequence generation unit 104 . In addition, the mode selection unit 109 generates predicted image data based on the determined encoding mode, and outputs the predicted image data to the difference calculation unit 102. However, the mode selection unit 109 does not output predictive image data when intra coding is selected. In addition, the mode selection unit 109 controls the switch 111 to be connected to the a side and the control switch 112 to the c side when intra-frame coding is selected, and controls the switch 111 to be connected to the b side when inter-frame predictive coding or direct mode is selected. , the control switch 112 is connected to side d. In addition, the mode selection unit 109 outputs, to the code sequence generation unit 104 , the motion vector used for the inter prediction encoding when the identified encoding mode is the inter prediction encoding. Here, since the frame B8 is not used as a reference frame when encoding other frames, motion vectors used in inter-frame predictive encoding need not be stored in the motion vector storage unit 116 . Next, the case where the mode selection unit 109 selects the inter prediction coding or the direct mode will be described. the

The image data of the macroblock of the frame B8 read from the replacement memory 101 and the predicted image data output from the mode selection unit 109 are input to the difference calculation unit 102 . The difference calculation unit 102 calculates the difference between the image data of the macroblock of the frame B8 and the predicted image data to generate prediction error image data, and outputs it to the prediction error coding unit 103 . The prediction error coding unit 103 generates coded data by performing coding processing such as frequency conversion or quantization on the input prediction error image data, and outputs it to the code sequence generation unit 104 . the

The code sequence generator 104 performs variable-length coding or the like on the input coded data, and adds information such as a motion vector and a coding mode to generate a code sequence and output it. the

Through the above processing, the encoding processing for one macroblock of the frame B8 is completed. Do the same for the rest of the macroblocks in frame B8. the

Next, encoding processing of each frame is performed by the same method as that of frames P9, B7, B6, and B8 with an encoding method corresponding to the frame type and frame display time order position of each frame. the

In the above embodiments, the operation of the video coding method of the present invention is described by taking the case of using the frame prediction structure shown in FIG. 6( a ) as an example. FIG. 12 is an explanatory diagram hierarchically showing the frame prediction structure at this time. In FIG. 12 , the arrows represent prediction relationships, and the frame at the end point of the arrow refers to the frame at the start point. In the frame prediction structure shown in Figure 6(a), in the case of display time order considerations, as shown in Figure 12, the encoding order is determined by giving priority to the frame farthest from the encoded frame. For example, the frame farthest from an I or P frame is the frame in the center of consecutive B frames. Therefore, for example, in a state where the encoding of the frames P5 and P9 has been completed, the frame B7 becomes the next encoding target frame. In the state where the frames P5, B7, and P9 have already been coded, the frames B6 and B8 become the next frames to be coded. the

In addition, even in the case of having different frame prediction structures as shown in FIG. 6 and FIG. 12, the same method as the video encoding method of the present invention can be used, and the effect of the present invention can be realized. Figures 9-11 illustrate other frame prediction structure examples. the

FIG. 9 shows a case where the number of B frames interposed between I or P frames is three, and the encoding order of the B frames is selected from the frame closest to the already encoded frame. FIG. 9( a ) is a diagram showing the prediction relationship of each frame shown in order of display time, and FIG. 9( b ) is a diagram showing the frame order replaced by the encoding order (code sequence order). Fig. 13 is a hierarchical diagram corresponding to the frame prediction structure of Fig. 9(a). In the frame prediction structure shown in FIG. 9( a ), in the case of display time order, as shown in FIG. 13 , coding starts from the frame closest to the coded frame. For example, in the state where the frames P5 and P9 have already been coded, the frames B6 and B8 become the next frames to be coded. In the state where the encoding of the frames P5, B6, B8, and P9 has been completed, the frame B7 becomes the next encoding target frame. the

FIG. 10 shows that the number of B frames sandwiched between I or P frames is five, and the frame farthest from the encoded frame among the B frames is preferentially encoded. FIG. 10( a ) is a diagram showing the prediction relationship of each frame shown in order of display time, and FIG. 10( b ) is a diagram showing the frame order replaced by the encoding order (code sequence order). Fig. 14 is a hierarchical diagram corresponding to the frame prediction structure of Fig. 10(a). In the frame prediction structure shown in FIG. 10( a ), considering the order of display time, as shown in FIG. 14 , the encoding sequence is determined by giving priority to the frame farthest from the encoded frame. For example, the frame farthest from an I or P frame is the frame located in the center of consecutive B frames. Therefore, for example, the frame B10 becomes the next encoding target frame in the state where the encoding of the frames P7 and P13 is completed. In the state where the encoding of frames P7, B10, and P13 is completed, frames B8, B9, B11, and B12 become the next encoding target frames. the

Fig. 11 shows that the number of B frames sandwiched between I or P frames is 5, and the situation of the frame closest to the frame that has been encoded in the preferentially encoded B frames is shown. FIG. 11( a ) is a diagram showing the prediction relationship of each frame shown in order of display time, and FIG. 11( b ) is a diagram showing the frame order replaced by the encoding order (code sequence order). Fig. 15 is a hierarchical diagram corresponding to the frame prediction structure of Fig. 11(a). In the frame prediction structure shown in FIG. 11( a ), considering the order of display time, as shown in FIG. 15 , coding starts from the frame closest to the coded frame. For example, in the state where the frames P5 and P9 have already been coded, the frames B8 and B12 become the next frames to be coded. In the state where the frames P5, B8, B12, and P9 have already been coded, the frames B9 and B11 become the next frames to be coded. Then, in the state where the encoding of the frames P5, B8, B9, B11, B12, and P9 has been completed, the frame B10 becomes the next encoding target frame. the

As described above, in the moving image coding method of the present invention, when B frames subjected to inter-frame predictive coding processing are coded using bidirectional prediction, multiple frames sandwiched between I or P frames are coded in an order different from the display time order. B-frames. In this case, the closest frame in the display time order is used as the forward and backward reference frame. When B-frames are available, B-frames may also be used as the reference frame. Also, when encoding a plurality of B frames sandwiched between I or P frames in an order different from the display time order, encoding is performed sequentially from the frame farthest from the already encoded frame. Also, when encoding a plurality of B frames interposed between I or P frames in an order different from the display time order, encoding is performed sequentially from the frame closest to the already encoded frame. the

Through this operation, using the moving picture encoding method of the present invention, when encoding a B frame, a frame that is closer in display time order can be used as a reference frame, and thus the prediction efficiency at the time of motion compensation can be improved, so Encoding efficiency can be improved. the

In addition, in the dynamic image coding method of the present invention, refer to the frame coded as a B frame as a backward reference frame, and encode the blocks in the B frame by direct mode. At this time, when coding by forward reference or bidirectional reference When referring backward to a block at the same position within a frame, a motion vector obtained by converting the forward motion vector is used as a motion vector in the direct mode. the

In this way, in the direct mode, by converting the forward motion vector of the B frame as the backward reference frame, it is not necessary to transmit the information of the motion vector, and the prediction efficiency can be improved. Furthermore, encoding efficiency can be improved by using the temporally closest reference frame as a forward reference frame.

Also, when encoding a co-located block in a B frame that is a backward reference frame in the direct mode, a motion vector obtained by converting a forward motion vector substantially used in the direct mode is used as a motion vector in the direct mode. the

In this way, in the direct mode, by converting the forward motion vector substantially used in the direct mode by the B frame as the backward reference frame, there is no need to transmit the information of the motion vector, and even if the backward reference frame is coded in the direct mode It can also improve the prediction efficiency when the blocks in the same position are included. Furthermore, encoding efficiency can be improved by using the temporally closest reference frame as a forward reference frame. the

Also, when encoding a co-located block in a B frame as a backward reference frame by backward reference, a motion vector obtained by scaling the backward motion vector is used as a motion vector in the direct mode. the

In this way, in the direct mode, by converting the backward motion vector used when coding the block at the same position in the B frame as the backward reference frame, it is not necessary to send the information of the motion vector, and even if the same block in the backward reference frame Even when the block at the position has only backward motion vectors, the prediction efficiency can be improved. Furthermore, encoding efficiency can be improved by using the temporally closest reference frame as a forward reference frame. the

In addition, when encoding a block at the same position in a B frame as a backward reference frame by backward reference, the backward motion vector used at this time is divided between the frame referred to by the backward motion vector and the backward reference frame The motion vector converted to the reference frame is used as the motion vector in the direct mode. the

In this way, in the direct mode, by converting the backward motion vector used when coding the block at the same position in the B frame as the backward reference frame, it is not necessary to send the information of the motion vector, and even if the same block in the backward reference frame Even when the block at the position has only backward motion vectors, the prediction efficiency can be improved. Thus, encoding efficiency can be improved. Furthermore, by using the frame referred to by the backward motion vector as the forward reference frame and using the latest reference frame available in the display time order as the backward reference frame, coding efficiency can be improved. the

Also, in direct mode, motion vectors whose size is forced to 0 are used.

In this way, by forcibly setting the motion vectors in the direct mode to 0, when the direct mode is selected, it is not necessary to transmit the information of the motion vectors, and the conversion processing of the motion vectors is unnecessary, thereby reducing the amount of processing. the

In addition, in the dynamic image coding method of the present invention, when referring to a frame coded as a B frame as a backward reference frame, and using the direct mode to code a block in a B frame, the conversion coded at the same position in the subsequent P frame The motion vectors obtained from the forward motion vectors used in the direct mode are used as the motion vectors in direct mode. the

In this way, in the direct mode, by converting the motion vector of the subsequent P frame, when the backward reference frame is a B frame, it is not necessary to store the motion vector of the B frame, and it is not necessary to transmit the information of the motion vector. Improve forecasting efficiency. Furthermore, encoding efficiency can be improved by using the temporally closest reference frame as a forward reference frame. the

Also, when the allocation of the relative index is changed for the frame number, and when the block at the same position in the backward reference frame is coded by the forward reference, the motion vector obtained by scaling the forward motion vector is used as the motion vector in the direct mode. the

In this way, in the direct mode, even when the assignment of the relative index to the frame number is changed, the motion vector of the coded frame can be converted, and there is no need to transmit the information of the motion vector. the

In this embodiment, the case where motion compensation is handled in units of horizontal 16×vertical 16 pixels, and prediction error image coding is handled in units of horizontal 8×vertical 8 pixels or horizontal 4×vertical 4 pixels is described, but these units also Other pixel numbers are possible. the

In addition, in this embodiment, the case where the number of consecutive B frames is 3 or 5 is described as an example, but the number of B frames may also be other numbers. the

In this embodiment, it is illustrated that the encoding mode of a P frame is selected from intra-frame encoding, inter-frame predictive encoding using motion vectors, and inter-frame predictive encoding without using motion vectors, and the encoding mode of B frames is selected from intra-frame encoding Encoding, interframe predictive encoding using forward motion vectors, interframe predictive encoding using backward motion vectors, interframe predictive encoding using bidirectional motion vectors, direct mode, but these encoding modes can also be other method. the

In addition, in this embodiment, seven examples have been described for the direct mode, but one method uniquely determined for each macroblock or block may be used, or one method may be selected for each block or macroblock from a plurality of methods. method. In the case of using a plurality of methods, information indicating which direct mode is used is recorded in the code string. the

In addition, in this embodiment, it is described that a P frame is coded by referring to a coded I or P frame that is located before or after the display time sequence, and a B frame is coded by referring to two nearby coded frames that are located before or after the display time sequence. When the coded frames are coded, but when these frames are P frames, a plurality of coded I frames or P frames located before or after the display time sequence are used as reference frame substitutes, and each block is referred to In the case of a B frame, multiple coded frames near the front or back in the display time sequence are used as reference frame substitutes, and the two largest frames in each block are referred to to encode. the

In addition, when the mode selection unit 109 stores the motion vector in the motion vector storage unit 116, when the target block is coded by bidirectional prediction or direct mode, both forward and backward motion vectors may be stored, or only Forward motion vector. If only forward motion vectors are stored, the amount of memory in the motion vector storage unit 116 can be reduced. the

(Example 2)

Fig. 16 is a block diagram showing an embodiment of a video decoding device using the video coding method of the present invention. the

As shown in FIG. 16, the moving image decoding device includes: a code sequence analysis unit 1401, a prediction error decoding unit 1402, a mode decoding unit 1403, a frame memory control unit 1404, a motion compensation decoding unit 1405, a motion vector storage unit 1406, and a frame memory 1407. , an addition unit 1408, and switches 1409 and 1410. the

The code sequence analysis unit 1401 extracts various data such as encoding mode information and motion vector information from the input code sequence. The prediction error decoding unit 1402 decodes the prediction error coded data input from the code sequence analysis unit 1401 to generate prediction error image data. The mode decoding unit 1403 controls the switches 1409 and 1410 with reference to the coding mode information extracted from the code string.

The frame memory control unit 1404 outputs the decoded image data stored in the frame memory 1407 as an output image based on the information indicating the frame display order input from the code sequence analysis unit 1401 . the

The motion compensation decoding unit 1405 decodes the reference frame number and motion vector information, and acquires motion compensation image data from the frame memory 1407 based on the decoded reference frame number and motion vector. The motion vector storage unit 1406 stores motion vectors. the

The addition unit 1408 adds the prediction error coded data input from the prediction error decoding unit 1402 to the motion compensation image data input from the motion compensation decoding unit 1405 to generate decoded image data. The frame memory 1407 stores the generated decoded image data. the

Next, the operation of the video decoding device configured as described above will be described. Here, it is assumed that the code sequence generated by the above-mentioned video coding device is input to the video decoding device. That is, here, it is assumed that a P frame refers to a coded I or P frame near the front or rear in the display time order. In addition, it is assumed that the B frame refers to two coded nearby frames located in the front or rear in the display time order. the

The frames in the code sequence at this time are in the order shown in FIG. 6( b ). Next, decoding processing of frames P9, B7, B6, and B8 will be described in order. the

(Decoding processing of frame P9)

The code sequence of the frame P9 is input to the code sequence analysis unit 1401 . The code sequence analysis unit 1401 extracts various data from the input code sequence. Here, various types of data refer to mode selection information, motion vector information, and the like. The extracted mode selection information is output to the mode decoding unit 1403 . In addition, the extracted motion vector information is output to the motion compensation decoding unit 1405 . Then, the prediction error coded data is output to the prediction error decoding unit 1402 . the

The mode decoding unit 1403 controls the switches 1409 and 1410 with reference to the encoding mode selection information extracted from the code string. When the encoding mode is selected as intra encoding, the mode decoding unit 1403 controls the switch 1409 to be connected to the a side, and the control switch 1410 to be connected to the c side. Also, when the encoding mode is selected as inter-frame predictive encoding, the mode decoding unit 1403 controls the switch 1409 to be connected to the b side, and the control switch 1410 to be connected to the d side.

The mode decoding unit 1403 also outputs encoding mode selection information to the motion compensation decoding unit 1405 . Next, a case where the encoding mode is selected as inter-frame predictive encoding will be described. The prediction error decoding unit 1402 decodes the input prediction error coded data to generate prediction error image data. The prediction error decoding unit 1402 outputs the generated prediction error image data to the switch 1409 . Here, since the switch 1409 is connected to the side b, the prediction error image data is output to the adder 1408 . the

The motion compensation decoding unit 1405 acquires motion compensation image data from the frame memory 1407 based on input motion vector information and the like. The frame P9 is coded with reference to the frame P5, and the frame P5 is decoded and held in the frame memory 1407 . Therefore, the motion compensation decoding unit 1405 acquires motion compensation image data from the image data of the frame P5 held in the frame memory 1407 based on the motion vector information. The motion compensation image data generated in this way is output to the addition unit 1408 . the

When decoding a P frame, the motion compensation decoding unit 1405 stores motion vector information in the motion vector storage unit 1406 . the

The addition unit 1408 adds the input prediction error image data and motion compensation image data to generate decoded image data. The generated decoded image data is output to frame memory 1407 via switch 1410 . the

As described above, the processing of one macroblock of frame P9 is completed. Through the same process, the remaining macroblocks are sequentially decoded. If all the macroblocks of frame P9 are decoded, frame B7 is decoded. the

(Decoding processing of frame B7)

Operations up to generation of prediction error image data in the code sequence analysis unit 1401 , mode decoding unit 1403 , and prediction error decoding unit 1402 are the same as those in the decoding process of frame P9 , and thus description thereof will be omitted. the

The motion compensation decoding unit 1405 generates motion compensation (motion compensation) image data based on input motion vector information and the like. Frame B7 is encoded by referring to frame P5 as a forward reference frame and referring to frame P9 as a backward reference frame, and these frames are decoded and held in the frame memory 1407 .

When the mode selection is bidirectional predictive interframe predictive encoding, the motion compensation decoding unit 1405 acquires forward reference image data from the frame memory 1407 based on the forward motion vector information. In addition, backward reference image data is acquired from the frame memory 1407 based on the backward motion vector information. The motion compensation decoding unit 1405 adds and averages forward reference image data and backward reference image data to generate motion compensation image data. the

When the mode selection is the direct mode, the motion compensation decoding unit 1405 acquires the motion vector of the frame P9 stored in the motion vector storage unit 1406 . Also, the motion compensation decoding unit 1405 acquires forward reference image data and backward reference image data from the frame memory 1407 using the motion vector. The motion compensation decoding unit 1405 adds and averages forward reference image data and backward reference image data to generate motion compensation image data. the

The case where the mode selection is the direct mode will also be described using FIG. 7( a ). Here, block a of frame B7 is decoded, and a block of frame P9 located at the same position as block a is block b. In addition, the motion vector of the block b is the motion vector c, and this motion vector c refers to the frame P5. At this time, the motion vector d referring to the frame P5 obtained by the motion vector c is used as the forward motion vector, and the motion vector e referring to the frame P9 obtained by the motion vector c is used as the backward motion vector. For example, as a method of using the motion vector c, there is a method of generating a motion vector parallel to the motion vector c. The image data after adding and averaging the forward-reference data and the backward-reference data obtained from these motion vectors is assumed to be motion-compensated image data. the

At this time, if the size of the motion vector d as the forward motion vector is MVF, the size of the motion vector e as the backward motion vector is MVB, and the size of the motion vector c is MV, the current frame (frame B7) after The time distance to the frame (frame P5) referred to by the block b of the reference frame (frame P9) and its backward reference frame is TRD, and the time distance between the current frame (frame B7) and the forward reference frame (frame P5) is TRF, Then, the magnitude MVF of the motion vector d and the magnitude MVB of the motion vector e are obtained by (Equation 1) and (Equation 2), respectively. Among them, MVF and MVB represent the horizontal component and vertical component of the motion vector, respectively. Also, for example, the temporal distance between frames can be specified based on information indicating the display order (position) added to the frames or information differences thereof. the

The motion compensation image data generated in this way is output to the addition unit 1408 . Also, the motion compensation (motion compensation) decoding unit 1405 stores motion vector information in the motion vector storage unit 1406. the

The addition unit 1408 adds the input prediction error image data and motion compensation image data to generate decoded image data. The generated decoded image data is output to frame memory 1407 via switch 1410 . the

As described above, the processing of one macroblock of frame B7 is completed. Through the same process, the remaining macroblocks are sequentially decoded. If all macroblocks of frame B7 are decoded, frame B6 is decoded. the

(Decoding processing of frame B6)

Operations up to generation of prediction error image data in the code sequence analysis unit 1401 , mode decoding unit 1403 , and prediction error decoding unit 1402 are the same as those in the decoding process of frame P9 , and thus description thereof will be omitted. the

The motion compensation decoding unit 1405 generates motion compensation image data based on input motion vector information and the like. Frame B6 refers to frame P5 as a forward reference frame, and refers to frame B7 as a backward reference frame, and is coded, and these frames are decoded and held in the frame memory 1407 . the

When the mode selection is bidirectional predictive interframe predictive encoding, the motion compensation decoding unit 1405 acquires forward reference image data from the frame memory 1407 based on the forward motion vector information. In addition, backward reference image data is acquired from the frame memory 1407 based on the backward motion vector information. The motion compensation decoding unit 1405 adds and averages forward reference image data and backward reference image data to generate motion compensation image data. the

When the mode selection is the direct mode, the motion compensation decoding unit 1405 acquires the motion vector of frame B7 stored in the motion vector storage unit 1406 . The motion compensation decoding unit 1405 acquires forward reference image data and backward reference image data from the frame memory 1407 using the motion vector. The motion compensation decoding unit 1405 adds and averages forward reference image data and backward reference image data to generate motion compensation image data. the

A first example when the mode selection is the direct mode will be described using FIG. 7(b). Here, block a of the frame B6 is decoded, and a block of the frame B7 located at the same position as the block a is block b. In addition, assume inter-frame predictive coding based on forward reference or inter-frame predictive coding based on bidirectional reference, block b, and set the forward motion vector of block b as motion vector c. This motion vector c refers to the frame P5. At this time, the motion vector d referring to the frame P5 generated using the motion vector c is used as the forward motion vector for the block a, and the motion vector e referring to the frame B7 generated using the motion vector c is used as the backward motion vector. For example, as a method of using the motion vector c, there is a method of generating a motion vector parallel to the motion vector c. The image data obtained by adding and averaging the forward-reference image data and the backward-reference image data obtained from these motion vectors is motion-compensated image data. the

At this time, if the size of the motion vector d as the forward motion vector is MVF, the size of the motion vector e as the backward motion vector is MVB, and the size of the motion vector c is MV, the current frame (frame B6) after The time distance to the frame (frame P5) referred to by the block b of the reference frame (frame B7) and its backward reference frame is TRD, and the time distance between the current frame (frame B6) and the forward reference frame (frame P5) is TRF, Then, the magnitude MVF of the motion vector d and the magnitude MVB of the motion vector e are obtained by (Equation 1) and (Equation 2), respectively. Also, for example, the temporal distance between frames can be specified based on information indicating the display order (position) added to the frames or information differences thereof. In addition, as the values of TRD and TRF, predetermined values determined for each frame may be used. This predetermined value may also be recorded in the code string as header information. the

In addition, a second example in the case where the mode selection is the direct mode will be described using FIG. 7( b ). the

At this time, the frame B7 is a reference frame after the frame B6 using the motion vector used for decoding the block b at the same position as the block a in the frame B7. Here, it is assumed that the block b is encoded using the direct mode, and the forward motion vector substantially used at this time is assumed to be the motion vector c. The motion vector c is obtained by using the motion vector stored in the motion vector storage unit 1406, or by reading the motion vector of the frame P9 used when encoding the block b in the direct mode from the motion vector storage unit 1406, and performing conversion calculation. The motion compensation decoding unit 1405 may store only the forward motion vector when storing in the motion vector storage unit 1406 the motion vector obtained by the scaling process when decoding the block b of the frame B7 in the direct mode. the

At this time, the motion vector d referring to the frame P5 generated using the motion vector c is used as the forward motion vector for the block a, and the motion vector e referring to the frame B7 generated using the motion vector c is used as the backward motion vector. For example, as a method of using the motion vector c, there is a method of generating a motion vector parallel to the motion vector c. The image data obtained by adding and averaging the forward-reference image data and the backward-reference image data obtained from these motion vectors is motion-compensated image data. the

At this time, the magnitude MVF of the motion vector d as the forward motion vector and the magnitude MVB of the motion vector e as the backward motion vector are the same as in the first example of the direct mode, and can be obtained by (Equation 1) and (Equation 2). out. the

Next, a third example in the case where the mode selection is the direct mode will be described with reference to FIG. 7(c). the

Here, block a of frame B6 is decoded, and a block of frame B7 whose position is the same as block a is block b. Let the backward reference predictive coding block b be used, and let the backward motion vector of block b be the motion vector f. This motion vector f refers to frame P9. At this time, the motion vector g referring to the frame P5 obtained by the motion vector f is used as the forward motion vector for the block a, and the motion vector h referring to the frame B7 obtained by the motion vector f is used as the backward motion vector. For example, as a method of using the motion vector f, there is a method of generating a motion vector parallel to the motion vector f. The image data obtained by adding and averaging the forward-reference image data and the backward-reference image data obtained from these motion vectors is motion-compensated image data. the

At this time, if the size of the motion vector g as the forward motion vector is MVF, the size of the motion vector h as the backward motion vector is MVB, and the size of the motion vector f is MV, the current frame (frame B6) after The temporal distance to the reference frame (frame B7) and the frame (frame P9) referred to by the blocks of the backward reference frame is TRD, the temporal distance between the current frame (frame B6) and the forward reference frame (frame P5) is TRF, and the current The temporal distance between the frame (frame B6) and the backward reference frame (frame B7) is TRB, and the magnitude MVF of the motion vector g and the magnitude MVB of the motion vector h are obtained by (Equation 3) and (Equation 4), respectively. the

Next, a fourth example in the case where the mode selection is the direct mode will be described using FIG. 7( d ). the

Here, block a of frame B6 is decoded, and a block of frame B7 whose position is the same as block a is block b. As in the third example, it is assumed that block b is backwardly referred to for predictive coding, and the backward motion vector of block b is set as motion vector f. This motion vector f refers to frame P9. At this time, the motion vector g of frame P9 obtained by referring to the motion vector f is used as the forward motion vector for block a, and the motion vector h of frame B7 obtained by referring to the motion vector f is used as the backward motion vector . For example, as a method of using the motion vector f, there is a method of generating a motion vector parallel to the motion vector f. The image data obtained by adding and averaging the forward-reference image data and the backward-reference image data obtained from these motion vectors is motion-compensated image data. the

At this time, if the size of the motion vector g as the forward motion vector is MVF, the size of the motion vector h as the backward motion vector is MVB, and the size of the motion vector f is MV, the current frame (frame B6) after The time distance to the frame (frame P9) referred to by the block of the reference frame (frame B7) and its backward reference frame is TRD, and the frame (frame P9) referred to by the block of the current frame (frame B6) and the backward reference frame (frame B7) When the temporal distance of frame P9) is TRF, the magnitude MVF of the motion vector g and the magnitude MVB of the motion vector h are obtained from (Equation 1) and (Equation 2), respectively. the

In addition, a fifth example in the case where the mode selection is the direct mode will be described using FIG. 8( a ). Here, it is assumed that block a of frame B6 is decoded by direct mode. At this time, the size of the motion vector is assumed to be 0, the frame P5 is used as a forward reference frame, and the frame B7 is used as a backward reference frame, and bidirectional reference is performed to perform motion compensation. the

Next, a sixth example in which the mode selection is the direct mode will be described with reference to FIG. 8(b). Here, it is assumed that block a of frame B6 is decoded by direct mode. Here, using the motion vector g used when decoding the block f at the same position as the block a in the frame P9, the frame P9 is a P frame located after the frame B6. The motion vector g is stored in the motion vector storage unit 1406 . The block a is bidirectionally predicted from the frame P5 as the forward reference frame and the frame B7 as the backward reference frame using the motion vector obtained from the motion vector g. For example, if the method of generating a motion vector parallel to the motion vector g is used as in the case of the first example above, the motion vector used to obtain the motion-compensated image data of block a becomes motion vector h for frame P5 and motion vector h for frame B7 becomes the motion vector i. the

At this time, if the size of the motion vector h as the forward motion vector is MVF, the size of the motion vector I as the backward motion vector is MVB, and the size of the motion vector g is MV, which is located after the current frame (frame B6) The time distance between the P frame (frame P9) of the frame and the frame (frame P5) referred to by the block f of the following frame is TRD, and the time distance between the current frame (frame B6) and the forward reference frame (frame P5) is TRF , the temporal distance between the current frame (frame B6) and the backward reference frame (frame B7) is TRB, then the motion vector MVF and the motion vector MVB are obtained by (Equation 1) and (Equation 5), respectively. the

Next, a seventh example in the case where the mode selection is the direct mode will be described with reference to FIG. 8(c). Here, it is assumed that block a of frame B6 is decoded by direct mode. In this example, the assignment of the relative index is changed (remapped) to the frame number described above, and the backward reference frame becomes the frame P9. At this time, frame P9 is used as a backward reference frame of frame B6 by using the motion vector g used when encoding the block f at the same position as block a in frame P9. The motion vector g is stored in the motion vector storage unit 1406 . The block a is bidirectionally predicted using the motion vector generated using the motion vector g, based on the frame P5 as the forward reference frame and the frame P9 as the backward reference frame. For example, if the method of generating a motion vector parallel to the motion vector g is used as in the case of the first example above, the motion vector used to obtain the motion-compensated image data of block a becomes motion vector h for frame P5, and motion vector h for frame P5. P9 becomes motion vector i. the

At this time, if the size of the motion vector h as the forward motion vector is MVF, the size of the motion vector i as the backward motion vector is MVB, and the size of the motion vector g is MV, the current frame (frame B6) after The time distance to the frame (frame P5) referred to by the block of the reference frame (frame P9) and its backward reference frame is TRD, and the time distance between the current frame (frame B6) and the forward reference frame (frame P5) is TRF, then The magnitude MVF of the motion vector h and the magnitude MVB of the motion vector i are obtained by (Equation 1) and (Equation 2), respectively. the

The motion compensation image data generated in this way is output to the addition unit 1408 . The addition unit 1408 adds the input prediction error image data and motion compensation image data to generate decoded image data. The generated decoded image data is output to frame memory 1407 via switch 1410 . the

As described above, the processing of one macroblock of frame B6 is completed. Through the same process, the remaining macroblocks are sequentially decoded. If all the macroblocks of frame B6 are decoded, then frame B8 is decoded. the

(Decoding processing of frame B8)

Operations up to generation of prediction error image data in the code sequence analysis unit 1401 , mode decoding unit 1403 , and prediction error decoding unit 1402 are the same as those in the decoding process of frame P9 , and thus description thereof will be omitted. the

The motion compensation decoding unit 1405 generates motion compensation image data based on input motion vector information and the like. Frame B8 is coded by referring to frame B7 as a forward reference frame and referring to P9 as a backward reference frame, and these frames are decoded and held in the frame memory 1407 . the

When the mode selection is bidirectional predictive interframe predictive coding, the motion compensation decoding unit 1405 acquires forward reference image data from the frame memory 1407 based on the forward motion vector information. In addition, backward reference image data is acquired from the frame memory 1407 based on the backward motion vector information. The motion compensation decoding unit 1405 adds and averages forward reference image data and backward reference image data to generate motion compensation image data. the

When the mode selection is the direct mode, the motion compensation decoding unit 1405 acquires the motion vector of the frame P9 stored in the motion vector storage unit 1406 . The motion compensation decoding unit 1405 acquires forward reference image data and backward reference image data from the frame memory 1407 using the motion vector. The motion compensation decoding unit 1405 adds and averages forward reference image data and backward reference image data to generate motion compensation image data. the

An example when the mode selection is the direct mode will be described with reference to FIG. 8( d ). Here, block a of frame B8 is decoded, and block b located at the same position as block a in frame P9 which is a backward reference frame is assumed. Also, let the forward motion vector of block b be motion vector c. This motion vector c refers to the frame P5. At this time, a motion vector d referring to frame B7 generated using motion vector c is used as a forward motion vector for block a, and a motion vector e referring to frame P9 generated using motion vector c is used as a backward motion vector. For example, as a method of using the motion vector c, there is a method of generating a motion vector parallel to the motion vector c. The image data obtained by adding and averaging the forward-reference image data and the backward-reference image data obtained from these motion vectors is motion-compensated image data. the

At this time, if the size of the motion vector d as the forward motion vector is MVF, the size of the motion vector e as the backward motion vector is MVB, and the size of the motion vector c is MV, the current frame (frame B8) after The time distance to the frame (frame P5) referred to by the block b of the reference frame (frame P9) and its backward reference frame is TRD, and the time distance between the current frame (frame B8) and the forward reference frame (frame B7) is TRF, The temporal distance between the current frame (frame B8) and the backward reference frame (frame P9) is TRB, then the magnitude MVF of the motion vector d and the magnitude MVB of the motion vector e are obtained by (Equation 1) and (Equation 5), respectively. the

The motion compensation image data generated in this way is output to the addition unit 1408 . The addition unit 1408 adds the input prediction error image data and motion compensation image data to generate decoded image data. The generated decoded image data is output to frame memory 1407 via switch 1410 . the

As described above, the processing of one macroblock of frame B8 is completed. Through the same process, the remaining macroblocks are sequentially decoded. Next, each frame is decoded by the same process corresponding to the frame type. the

Next, the frame memory control unit 1404 replaces the image data of each frame held in the frame memory 1407 in chronological order as described above, as shown in FIG. 6( a ), and outputs it as an output image. the

As described above, in the moving image decoding method of the present invention, when decoding a B frame subjected to inter-frame predictive coding processing using bidirectional prediction, as frames used as a forward reference frame and a backward reference frame, decoded frames are used. The nearby frames in the display time order are decoded. the

In addition, when the direct mode is selected as the encoding mode for the B frame, the reference image data is obtained from the decoded image data by referring to the motion vector of the decoded backward reference frame held in the motion vector storage unit 1406. , and get motion compensated image data. the

With this operation, when encoding a B frame subjected to an inter-frame predictive encoding process using bidirectional prediction, frames located nearby in the order of display time are used as frames serving as forward reference frames and backward reference frames for decoding, and When the code sequence generated after encoding can be correctly decoded. the

In addition, in this embodiment, seven examples have been described for the direct mode, but a method uniquely determined for each macroblock or block may be used, for example, by decoding a block at the same position in a backward reference frame. , and multiple methods can be switched and used in units of blocks or macroblocks. When a plurality of methods are used, decoding is performed using information indicating which direct mode is used recorded in the code string. At this time, the operation of the motion compensation decoding unit 1405 changes according to this information. For example, when adding this information in units of motion-compensated blocks, the mode decoding unit specifies which direct mode to use for encoding, and sends it to the motion-compensated decoding unit 1405 . In addition, the motion compensation decoding unit 1405 performs decoding processing using the decoding method described in this embodiment according to which direct mode is used. the

In addition, in this embodiment, a case of frame structure in which three B frames are sandwiched between I or P frames is described, but the number of B frames may be other values, such as four or five. the

In addition, in this embodiment, it is explained that the decoded P frame is coded by referring to one coded I or P frame that is located before or after the display time sequence, and the B frame is coded by referring to two nearby locations that are located before or after the display time sequence. In the case of the encoded code sequence of the encoded frame, but in the case where these frames are P frames, multiple encoded I frames or P frames located before or after the display time sequence can be used as reference frames Substitute and refer to the code sequence encoded by the largest frame in each block. In the case of a B frame, multiple encoded frames near the front or back in the display time sequence can be used as reference frame substitutes , and refer to the code sequence encoded by the largest two frames in each block. the

When storing the motion vector in the motion vector storage unit 1406, the motion compensation decoding unit 1405 may store both forward and backward motion vectors or only Store the forward motion vector. If only forward motion vectors are stored, the amount of memory in the motion vector storage unit 1406 can be reduced. the

(Example 3)

By recording a program for realizing the structure of the moving picture encoding method or moving picture decoding method shown in each of the above embodiments on a storage medium such as a floppy disk, the processing shown in each of the above embodiments can be easily implemented in an independent computer system. the

Fig. 17 is an explanatory diagram in the case of implementing a computer system using a floppy disk storing the moving picture coding method and the moving picture decoding method of each of the above-mentioned embodiments. the

Fig. 17(b) shows the appearance, cross-sectional structure and floppy disk seen from the front of the floppy disk, and Fig. 17(a) shows an example of the physical format of the floppy disk which is the main body of the recording medium. The floppy disk FD is housed in the case F, and on the surface of the disk, a plurality of tracks Tr are formed concentrically from the outside to the inside, and each track is divided into 16 sectors Se along the angular direction. Therefore, on the floppy disk storing the above-mentioned program, the video encoding method as the above-mentioned program is recorded in an area allocated on the floppy disk FD. the

17(c) shows a configuration for recording and reproducing the above-mentioned program on the floppy disk FD. When the above-mentioned program is recorded on the floppy disk FD, the video encoding method or the video decoding method as the above-mentioned program is written from the computer system Cs via the floppy disk drive. In addition, when implementing the above-mentioned video coding method in a computer system with a program in a floppy disk, the program is read from the floppy disk with a floppy disk drive and transferred to the computer system. the

In the above description, a floppy disk is used as the recording medium, but the same can be done even if an optical disk is used. In addition, the recording medium is not limited thereto, and any program-recordable medium such as an IC card or a ROM cassette can be implemented in the same manner. the

Here, an application example of the moving picture encoding method or moving picture decoding method shown in the above-mentioned embodiments and a system using the application example will also be described. the

Fig. 18 is a block diagram showing the overall configuration of a content providing system ex100 for realizing a content distribution service. The communication service provision area is divided into desired sizes, and base stations ex107-ex110 serving as fixed wireless stations are installed in each cell. the

The content providing system ex100 connects various devices such as a computer ex111, a PDA (personal digital assistant) ex112, a camera ex113, a mobile phone ex114, and a mobile phone with a camera ex115 via the Internet service provider ex102, the telephone network ex104, and the base stations ex107-ex110. On the Internet ex101. the

However, the content providing system ex100 is not limited to the combination shown in FIG. 18, and may be connected in any combination. In addition, each device may be directly connected to the telephone network ex104 without passing through the base stations ex107-ex110 which are fixed base stations. the

The camera ex113 is a device capable of shooting moving images such as a digital video camera. In addition, the mobile phone is PDC (Personal Digital Communications: Personal Digital Communications) method, CDMA (Code Division Multiple Access: Code Division Multiple Access) method, W-CDMA (Wideband-Code Division Multiple Access: Wideband Code Division Multiple Access) method, or GSM (Global System for Mobile Communications: Global System for Mobile Communications) mobile phone, or PHS (Personal Handyphone System: Personal Handyphone System), etc., it does not matter. the

In addition, the streaming server ex103 is connected to the camera ex113 through the base station ex109 and the telephone network ex104, and the video camera ex113 can be used for on-site distribution and the like based on the coded data sent by the user. The encoding processing of the captured data may be performed by the camera ex113, or may be performed by a server or the like that performs data transmission processing. In addition, video data captured by the camera ex116 can also be sent to the streaming server ex103 via the computer ex111. The camera ex116 is a device capable of shooting still images and moving images, such as a digital video camera. Thus, it does not matter whether the encoding of the motion data is carried out by the camera ex116 or by the computer ex111. In addition, encoding processing is performed in the LSI ex117 included in the computer ex111 or the camera ex116. The software for video encoding and decoding can also be loaded on any storage medium (CD-ROM, floppy disk, hard disk, etc.) that can be read by the computer ex111. In addition, moving image data can also be sent from the mobile phone ex115 with a camera. The video data at this time is encoded by the LSI included in the mobile phone ex115. the

In this content providing system ex100, the user encodes and processes the content captured by the camera ex113, the camera ex116, etc. (for example, a picture of a music scene, etc.), and sends it to the streaming server ex103. On the other hand, the streaming server ex103 The content data described above is streamed to the requesting client. As clients, there are computers ex111, PDA ex112, video cameras ex113, mobile phones ex114, etc., which can decode the encoded data. Therefore, the content providing system ex100 can receive and reproduce coded data by the client, and by receiving, decoding and reproducing the data in real time by the client, personal broadcasting can also be realized.

The video encoding device or the video decoding device shown in the above-mentioned embodiments may also be used for encoding and decoding of each device constituting the system. the

As an example, a mobile phone will be described. the

Fig. 19 is a diagram showing a mobile phone ex115 using the video encoding method and video decoding method described in the above-mentioned embodiments. The mobile phone ex115 has: an antenna ex201, which transmits and receives radio waves with the base station ex110; an imaging unit ex203 such as a CCD camera, which can take photos and still images; a display unit ex202 such as a liquid crystal display, which displays and decodes photos taken by the imaging unit ex203 and received by the antenna ex201 Data such as photos; the main body is composed of a group of operation keys ex204; the sound output part ex208 such as a speaker is used for sound output; the sound input part ex205 such as a microphone is used for sound input; the recording medium ex207 is used for storing captured moving images or still images Encoded data or decoded data such as data, received mail data, moving image data, or still image data; the slotting part ex206 enables the recording medium ex207 to be mounted on the mobile phone ex115. The recording medium ex207 stores a flash memory element as one of EEPROM (Electrically Erasable and Programmable Read Only Memory: Electrically Erasable Read Only Memory) which is electrically rewritable or erasable nonvolatile storage in a plastic case such as an SD card. the

Use Figure 20 to illustrate the mobile phone ex115. The mobile phone ex115 is connected to the main control unit ex311 via the synchronous bus ex313. The unit ex309, the demultiplexing unit ex308, the recording and reproducing unit ex307, the modem circuit unit ex306, the audio processing unit ex305, and the main control unit ex311 collectively control each unit of the main unit including the display unit ex202 and the operation keys ex204. the

The power supply circuit unit ex310, when the call end and the power button are turned on by the user operation, supplies power from the battery pack to each unit, thereby activating the digital mobile phone with camera ex115 to an operable state. the

The mobile phone ex115 converts the sound signal collected by the sound input part ex205 into a digital sound signal through the sound processing part ex305 through the sound processing part ex305 according to the control of the main control part ex311 composed of CPU, ROM and RAM, etc. The portion ex306 performs spread spectrum processing on the signal, and after the digital-to-analog conversion processing and the frequency conversion processing are performed by the transceiver circuit portion ex301, the signal is transmitted through the antenna ex201. In addition, the mobile phone ex115 amplifies the reception data received by the antenna ex201 in the voice call mode, performs frequency conversion processing and inverse spread spectrum processing, converts it into analog voice data by the voice processing unit ex305, and outputs it through the voice output unit ex208. the

When sending an e-mail in the data communication mode, the text data of the e-mail input by operating the operation keys ex204 of the main body is sent to the main control unit ex311 via the operation input control unit ex304. The main control unit ex311 spreads the text data through the modulation and demodulation circuit ex306, and after the digital-to-analog conversion and frequency conversion are performed by the transceiver circuit ex301, the text data is sent to the base station ex110 through the antenna ex201. the

In the case of transmitting image data in the data communication mode, the image data captured by the imaging unit ex203 is supplied to the image encoding unit ex312 via the camera interface unit ex303. In addition, when the image data is not transmitted, the image data captured by the imaging unit ex203 can be directly displayed on the display unit ex202 via the camera interface unit ex303 and the LCD control unit ex302. the

The image encoding unit ex312 has the moving image encoding device described in the present invention, compresses and encodes the image data supplied from the imaging unit ex203 by the encoding method used in the moving image encoding device shown in the above embodiments, converts it into encoded image data, and sends it to Multiplexing and separating part ex308. At the same time, the mobile phone ex115 sends the voice collected by the voice input unit ex205 during shooting by the camera unit ex203 as digital voice data to the multiplexing and demultiplexing unit ex308 via the voice processing unit ex305. the

The demultiplexing unit ex308 multiplexes the encoded image data supplied from the image encoding unit ex312 and the audio data supplied from the audio processing unit ex305 in a predetermined manner, and the multiplexed data obtained by the modulation and demodulation circuit ex306 spreads the result of the processing, and is processed by the modem circuit ex306. The transmission and reception circuit unit ex301 performs digital-to-analog conversion processing and frequency conversion processing, and then transmits through the antenna ex201.

In the case of receiving moving image file data linked to a homepage, etc. in the data communication mode, the modem circuit ex306 inversely spreads the received data received from the base station ex110 via the antenna ex201, and transmits the resulting multiplexed data to the multiplexing and separating unit ex308. the

In decoding the multiplexed data received via the antenna ex201, the demultiplexing unit ex308 separates the multiplexed data into a bit stream of image data and a bit stream of audio data, and supplies the encoded image data to image decoding via a synchronous bus ex313. part ex309, and at the same time, provide the sound data to the sound processing part ex305. the

The image decoding unit ex309 has the moving image decoding device described in the present invention, generates and reproduces moving image data by decoding the bit stream of the image data by a decoding method corresponding to the encoding method shown in the above-mentioned embodiments, and transmits the data through the LCD control unit ex302. It is provided to the display unit ex202 to display the moving image data included in the moving image file linked to, for example, the homepage. At the same time, the audio processing unit ex305 converts the audio data into analog audio data and supplies it to the audio output unit ex208, thereby reproducing audio data included in a video file linked to, for example, a homepage. the

Not limited to the example of the above-mentioned system, recently, digital broadcasting based on satellite and terrestrial waves has become a topic. As shown in FIG. . Specifically, the broadcast station ex409 transmits the bit stream of photo information to the communication or broadcast satellite ex410 via radio waves. The broadcast satellite ex410 receiving the bit stream emits radio waves for broadcasting, and the home antenna ex406 equipped with satellite broadcast receiving equipment receives the radio waves, and the bit stream is decoded and reproduced by devices such as a television (receiver) ex401 or a set-top box (STB) ex407. In addition, the video decoding device shown in the above-mentioned embodiments may also be installed in the playback device ex403 that reads and decodes the bit stream recorded on the storage medium ex402 such as CD or DVD as the recording medium. At this time, the reproduced photo signal is displayed on the monitor ex404. In addition, a video decoding device may be installed in a set-top box ex407 connected to a cable ex405 for cable TV or an antenna ex406 for satellite/terrestrial broadcasting, and the video may be reproduced by a TV monitor ex408. In this case, instead of the set-top box, the moving image decoding device may be installed in the television. Also, a car ex412 equipped with an antenna ex411 can receive signals from a satellite ex410 or a base station ex107, etc., and reproduce moving images on a display device such as a car navigation system ex413 included in the car ex412. the

In addition, the video signal may be coded by the video coding apparatus shown in the above-mentioned embodiments, and recorded on a recording medium. As a specific example, there is a recorder ex420 such as a DVD recorder for recording image signals on a DVD disk ex412 or a disk recorder for recording on a hard disk. In addition, it can also be recorded in the SD card ex422. If the recorder ex420 is equipped with the video decoding device shown in the above-mentioned embodiment, the video signal recorded on the DVD disc ex421 or SD card ex422 can be reproduced and displayed on the monitor ex408. the

The structure of the car navigation device ex413 is, for example, the structure shown in FIG. 20 excluding the camera unit ex203, the camera interface unit ex303, and the image encoding unit ex312. The computer ex111 or the television (receiver) ex401 can also be considered in the same way. the

For terminals such as the above-mentioned mobile phone ex114, in addition to transmitting and receiving terminals having both encoders and decoders, three types of installation types are conceivable: transmitting terminals having only encoders and receiving terminals only having decoders. the

In this way, the dynamic image encoding method or the dynamic image decoding method shown in the above-mentioned embodiments can be used in any of the above-mentioned devices and systems, thereby obtaining the effects described in the above-mentioned embodiments. the

In addition, this invention is not limited to the said Example, Various deformation|transformation and correction are possible in the range which does not deviate from this invention. the

As described above, according to the moving image coding method of the present invention, when coding B frames, frames closer in display time order can be used as reference frames, thereby improving prediction efficiency during motion compensation, thereby improving coding efficiency. . the

In addition, in the direct mode, by converting the first motion vector of the second reference frame, it is not necessary to send motion vector information, and the prediction efficiency is improved. the

In addition, in the direct mode, by converting the first motion vector substantially used in the direct mode of the second reference frame, it is not necessary to send the motion vector information, and even if the same position in the second reference frame is coded in the direct mode The prediction efficiency can also be improved when there are blocks. the

In addition, in the direct mode, by converting the second motion vector used when encoding the block at the same position in the second reference frame, it is not necessary to send motion vector information, and even if the block at the same position in the second reference frame only has Also in the case of the second motion vector, prediction efficiency can be improved. the

In addition, by forcibly setting the motion vector in the direct mode to 0, when the direct mode is selected, it is not necessary to transmit the motion vector information, and the conversion process of the motion vector is unnecessary, thereby reducing the amount of processing. the

Also, in the direct mode, by converting the motion vector of the succeeding P frame, when the second reference frame is a B frame, it is not necessary to store the motion vector of the B frame. In addition, it is not necessary to transmit information of motion vectors, and prediction efficiency can be improved. the

In addition, in the direct mode, if the second reference frame has the first motion vector, then the first motion vector is converted, and if the second reference frame does not have the first motion vector but only has the second motion vector, then the Since the second motion vector is scaled, it is not necessary to add motion vector information to the code string, and prediction efficiency can be improved. the

In addition, according to the moving picture decoding method of the present invention, decoding processing can be correctly performed when decoding a code sequence that is used when performing inter-frame predictive encoding processing using bidirectional prediction by using frames located nearby in the order of display time. The first reference frame and the second reference frame are coded and generated. the

Industrial availability

As described above, the moving picture encoding method and moving picture decoding method of the present invention are used, for example, in mobile phones, DVD devices, personal computers, etc., to encode image data corresponding to each frame constituting a moving picture to generate a code sequence, and simultaneously decode and generate The method used for the code column.

Claims (4)

1. An image decoding method, which decodes an encoded image, is characterized in that, comprising: In the decoding step, when decoding the block to be decoded (a), the motion vectors (E1, E2) of the block to be decoded (a) are determined based on the motion vector (C1) of the block (b) at the same position, the same position Block (b) is a block in the decoded frame (B7), and is a block at the same position as the decoding target block, And using the motion vector (E1, E2) of the decoding target block and the reference frame corresponding to the motion vector (E1, E2) of the decoding target block, the decoding target block (a) is subjected to motion compensation in direct mode and decode, In the decoding step, when decoding the co-located block (b) using one motion vector (C1) and one backward reference frame corresponding to the one motion vector (C1), The one motion vector (C1) used when decoding the co-location block (b) is converted using the difference of information indicating the display order of frames, thereby generating a motion vector for the decoding target block (a). Two motion vectors (E1, E2) obtained by performing motion compensation on the decoding object block in direct mode and decoding, Using the two generated motion vectors (E1, E2) and the two reference frames corresponding to the two generated motion vectors, the decoding target block (a) is decoded by performing motion compensation in direct mode. 2. The image decoding method according to claim 1, characterized in that: In the two reference frames respectively corresponding to the two motion vectors of the decoding target block, The first reference frame is a frame including the block at the same position, The second reference frame is a subsequent reference frame used when decoding the block at the same position, and is a reference frame corresponding to a motion vector to be converted when generating two motion vectors of the block to be decoded. 3. The image decoding method according to claim 2, characterized in that: The information representing the display order of the frames is: The first information indicates the display order of frames including the decoding target block; The second information indicates the display order of the second reference frame of the decoding target block; and The third information indicates the frame display sequence, and the frame is a frame including the block at the same position and is the first reference frame of the block to be decoded; The information difference is a difference between the first information and the second information, a difference between the first information and the third information, and a difference between the second information and the third information. 4. An image decoding device, which decodes an encoded image, is characterized in that it comprises: The decoding unit, when decoding the decoding object block, determines the motion vector of the decoding object block according to the motion vector of the same position block, the same position block is a block in the decoded frame, and is related to the decoding block at the same location as the object block, and using the motion vector of the decoding target block and the reference frame corresponding to the motion vector of the decoding target block, performing motion compensation and decoding on the decoding target block in direct mode, The decoding unit, when the co-location block is decoded using one motion vector and one backward reference frame corresponding to the one motion vector, The one motion vector used when decoding the block at the same position is converted by using the difference of the information indicating the display order of the frames, thereby generating the motion vector for directly converting the block to be decoded with respect to the block to be decoded. mode to perform motion compensation and decode the two motion vectors, And using the two generated motion vectors and the two reference frames respectively corresponding to the two generated motion vectors, the block to be decoded is subjected to motion compensation and decoded in direct mode.

Images