JP2007228560A - Moving picture coding method and moving picture coding apparatus - Google Patents

Abstract

The present invention provides a moving picture coding apparatus capable of preventing a deterioration in image quality due to a reduction in accuracy of motion vector prediction in temporal direct mode coding and efficiently compressing a moving picture.
A moving picture coding apparatus predicts and generates a motion vector of a target block by referring to a motion vector of a coded picture that is temporally nearby as direct mode processing of a B picture. Direct mode processing unit, a spatial direct mode processing unit that predicts and generates a motion vector of a target block with reference to a motion vector of an encoded block located in the spatial periphery of the target block, and encoding A temporal direct mode prohibition determining unit that determines prohibition of use of the temporal direct mode depending on the target conditions, and when the temporal direct mode is prohibited by the determination unit, the spatial direct mode for the encoding target Direct mode coding is performed using only the processing unit.
[Selection] Figure 1

Description

  The present invention relates to a moving picture coding method and a moving picture coding apparatus, and more particularly to a technique for preventing image quality deterioration due to a reduction in accuracy of motion vector prediction in the direct mode.

  In recent years, with the era of multimedia that handles voice and images in an integrated manner, traditional information media, that is, means for transmitting information such as newspapers, magazines, televisions, radios, and telephones to people, are taken up as multimedia targets. It has become like this. In general, multimedia refers to not only characters but also figures, sounds, especially images, etc. that are associated with each other at the same time. It is an essential condition.

  However, when the information amount of each information medium is estimated as a digital information amount, the amount of information per character is 1 to 2 bytes in the case of characters, whereas 64 Kbits (phone quality) per second in the case of speech. In addition, for a moving image, an information amount of 100 Mbits (current television reception quality) or more per second is required, and it is not realistic to handle the enormous amount of information in the digital format as it is with the information medium. For example, a video phone has already been put into practical use by an integrated services digital network (ISDN) having a transmission rate of 64 Kbit / s to 1.5 Mbits / s. It is impossible to send.

  Therefore, what is required is information compression technology. For example, in the case of a videophone, H.264 recommended by ITU-T (International Telecommunication Union Telecommunication Standardization Sector). 261 and H.264. H.263 standard video compression technology is used. In addition, according to the information compression technology of the MPEG-1 standard, it is possible to put image information together with audio information on a normal music CD (compact disc).

  Here, MPEG (Moving Picture Experts Group) is an international standard for video signal compression standardized by ISO / IEC (International Electrotechnical Commission). This is a standard for compressing information of TV signals up to 5 Mbps, that is, about 1/100. Further, since the MPEG-1 standard has set the target quality to a medium quality that can be realized mainly at a transmission rate of about 1.5 Mbps, the MPEG-2 standardized to meet the demand for higher image quality. Then, the TV broadcast quality is realized at 2 to 15 Mbps for the moving image signal. Furthermore, at present, the working group (ISO / IEC JTC1 / SC29 / WG11) that has been standardizing MPEG-1 and MPEG-2 achieves a compression ratio that exceeds MPEG-1 and MPEG-2, and further, in units of objects. MPEG-4 has been standardized that enables encoding, decoding, and operation and realizing new functions necessary in the multimedia era. In MPEG-4, it was originally aimed at standardizing a low bit rate encoding method, but now it has been extended to a more general encoding including a high bit rate including interlaced images.

  Further, in 2003, MPEG-4 AVC and ITU H.264 were jointly developed by ISO / IEC and ITU-T as next-generation image coding systems with higher compression rates. H.264 is standardized (for example, see Non-Patent Document 1). H. In the H.264 standard, a High Profile compatible standard suitable for HD (High Definition) images and the like has also been formulated, and it has been decided to adopt it as a compression standard for next-generation media such as BD-ROM (Blu-ray Disk ROM). .

  In general, in encoding of moving images, the amount of information is compressed by reducing redundancy in the time direction and the spatial direction. Therefore, in inter-picture predictive coding for the purpose of reducing temporal redundancy, motion is detected and a predicted image is created in block units with reference to the front or rear picture, and the obtained predicted image and coding are encoded. Encoding is performed on the difference value from the target picture. Here, a picture is a term representing a single screen, which means a frame in a progressive image and a frame or field in an interlaced image. Here, an interlaced image is an image in which one frame is composed of two fields having different times. In encoding and decoding of interlaced images, one frame may be processed as a frame, processed as two fields, or processed as a frame structure or field structure for each block in the frame. it can.

  A picture that does not have a reference picture and performs intra prediction coding is called an I picture. A picture that performs inter-frame predictive coding with reference to only one picture is called a P picture. A picture that can be subjected to inter-picture prediction coding with reference to two pictures at the same time is called a B picture. The B picture can refer to two pictures as an arbitrary combination of display times from the front or the rear. A reference picture (reference picture) can be specified for each block which is a basic unit of encoding and decoding, but the reference picture described earlier in the encoded bitstream is designated as the first reference picture. The one described later is distinguished as the second reference picture. However, as a condition for encoding and decoding these pictures, the picture to be referenced needs to be already encoded and decoded.

  Motion compensation inter-picture prediction coding is used for coding a P picture or a B picture. The motion compensation inter-picture prediction encoding is an encoding method in which motion compensation is applied to inter-picture prediction encoding. Motion compensation is not simply predicting from the pixel value of the reference frame, but detecting the amount of motion of each part in the picture (hereinafter referred to as a motion vector) and performing prediction in consideration of the amount of motion. This improves the prediction accuracy and reduces the amount of data. For example, the amount of data is reduced by detecting the motion vector of the encoding target picture and encoding the prediction residual between the prediction value shifted by the motion vector and the encoding target picture. In the case of this method, since motion vector information is required at the time of decoding, the motion vector is also encoded and recorded or transmitted.

  The motion vector is detected in units of macroblocks. Specifically, the macroblock on the encoding target picture side is fixed, the macroblock on the reference picture side is moved within the search range, and is most similar to the reference block. The motion vector is detected by finding the position of the reference block.

  H. In the H.264 system, an encoding mode called a direct mode can be selected for encoding a B picture. There are two types of direct mode: a temporal method (temporal direct mode) and a spatial method (spatial direct mode). In the direct mode, only one of a temporal method and a spatial method can be used for a slice (block group) to be encoded.

  In temporal direct mode, the current block itself does not have a motion vector. By using the motion vector of another encoded picture as a reference motion vector, scaling is performed based on the display temporal positional relationship between pictures. The motion vector used in the encoding target block is predicted and generated.

  FIG. 11 is a schematic diagram illustrating a motion vector prediction generation method in the temporal direct mode. In the figure, P indicates a P picture, B indicates a B picture, and the numbers attached to the picture types indicate the display order of the pictures. Each picture P1, B2, B3, P4 has display order information T1, T2, T3, T4, respectively. Here, a case where the block BL0 of the picture B3 shown in FIG. 11 is encoded in the temporal direct mode will be described.

  The motion vector MV1 of the block BL1 located in the vicinity of the display time of the picture B3 and included in the already encoded picture P4 and located at the same position as the block BL0 is used. This motion vector MV1 is a motion vector used when the block BL1 is encoded, and refers to the picture P1. In this case, the motion vector used when coding the block BL0 is the motion vector MV_F for the picture P1 and the motion vector MV_B for the picture P4. At this time, assuming that the magnitude of the motion vector MV1 is MV, the magnitude of the motion vector MV_F is MVf, and the magnitude of the motion vector MV_B is MVb, MVf and MVb are obtained by Expression (1) and Expression (2), respectively.

MVf = (T3-T1) / (T4-T1) × MV (1)
MVb = (T3-T4) / (T4-T1) × MV (2)

  Using the motion vector MV_F and the motion vector MV_B obtained by performing the scaling process from the motion vector MV1 in this way, motion compensation of the block BL0 is performed from the picture P1 and the picture P4 which are reference pictures.

  Note that when the block BL1 referred to for scaling processing is a block on which intra prediction encoding has been performed and does not have a motion vector, motion compensation is performed assuming that the magnitudes of the motion vectors MV_F and MV_B are both “0”. I do.

  In the spatial direct mode, similar to the temporal direct mode, the encoding target block itself does not have a motion vector, but refers to the motion vector of an encoded block located spatially around the encoding target block, Encode using it.

  FIG. 12 is a schematic diagram illustrating a motion vector prediction generation method in the spatial direct mode. In the figure, P indicates a P picture, B indicates a B picture, and the numbers attached to the picture types indicate the display order of the pictures. Here, a case will be described in which block BL0 of picture B3 shown in FIG. 12 is encoded in the spatial direct mode.

  Among the motion vectors MVA1, MVB1, and MVC1 of the encoded block including the three pixels A, B, and C around the block BL0 to be encoded, the motion vector is closest to the encoding target picture in display time. A motion vector referring to an already encoded picture is determined as a motion vector candidate of the encoding target block. When there are three determined motion vectors, the median value is selected as the motion vector of the block to be encoded. If there are two, the average value of them is obtained and used as the motion vector of the encoding target block. When there is only one, the motion vector is set as the motion vector of the encoding target block.

In the example shown in FIG. 12, the motion vectors MVA1 and MVC1 are obtained with reference to the picture P2, and the motion vector MVB1 is obtained with reference to the picture P1. Therefore, the average value of the motion vectors MVA1 and MVC1 that refer to the picture P2, which is the already encoded picture closest to the encoding target picture in terms of display time, is obtained, and the first motion vector of the encoding target block is obtained. MV_F which is The same applies to obtaining MV_B, which is the second motion vector.
ISO / IEC 14496-10, International Standard: "Information technology-Coding of audio-visual objects-Part 10: Advanced video coding" (2003-12-01).

  By the way, most film media such as movies are composed of 24 frames per second (24 fps). When these 24 fps materials are broadcast on television, since the television (NTSC) is 29.97 fps, it is necessary to convert the picture display time interval from 24 fps to 29.97 fps. This conversion is called telecine conversion (2-3 conversion).

FIG. 13 is a diagram illustrating a conversion method of telecine conversion.
In the telecine conversion, as shown in FIG. 13, the first frame of 24 fps is converted into 2 fields, the next frame is converted into 3 fields, and subsequently converted into 2 fields and 3 fields, and the converted fields are converted every 2 fields. By making one frame, conversion from 24 fps to 30 fps is performed.

  Here, in the case of telecine conversion, since 30 fps deviates from 29.97 fps of NTSC, a process of dropping frames every certain time and adjusting the timing is performed.

  When the above conversion is performed, images of the same field are displayed at different display times, such as field 1-0 of 30P frame 1 and field 1-0 of 30P frame 2 in FIG. This means that the display time interval of the image subjected to telecine conversion and the recording time interval of the image to be displayed do not completely match.

  When a moving image subjected to such telecine conversion is encoded using the temporal direct encoding mode, a problem occurs in scaling processing of motion vector prediction.

  That is, in the temporal direct mode encoding, as described above, the motion vector is predicted using the equations (1) and (2). This means that the calculation is based on the display order information of the moving image to be encoded, but not based on the recording time interval of the image to be displayed.

  Therefore, when the display order information and the recording time interval do not match, the scaling process in the motion vector prediction in the temporal direct mode does not make sense. An example of this is shown in FIG.

  FIG. 14 is a diagram showing temporal direct mode encoding in an image subjected to telecine conversion. Here, I means I picture, P means P picture, and B means B picture, and the numbers indicate the display order. In addition, which field in FIG. 13 corresponds to the parenthesis.

  In this case, in the temporal direct mode encoding of B4, the motion vector MV_P6 of the block BL3 at the same position as the block BL2 of the field P6, which is an already encoded picture located near the display time of the field B4, and the field A motion vector is predicted by the following expression using display order information Tb6, Tp2, Ti0 of the field I0 referred to by B4, fields P6, BL3.

MVf_b6 = (Tb6-Ti0) / (Tp2-Ti0) × MV_P2 (3)
MVb_b6 = (Tb6-Tp2) / (Tp2-Ti0) × MV_P2 (4)

  When the display order information interval of the field is Ta, the above formulas (3) and (4) are expressed by the following formulas (5) and (6).

MVf_b6 = (4 × Ta / 6 × Ta) × MV_P2 = (2/3) × MV_P2
... (5)
MVb_b6 = (− 2 × Ta / 6 × Ta) × MV_P2 = − (1/3) × MV_P2
(6)

  Since the results of MVf_b6 and MVb_b6 described above use display order information that deviates from the time when the picture was actually recorded, erroneous scaling is performed. For example, as shown in FIG. 15, when the objects in the picture are moving at regular intervals in the original material, the recording time intervals are constant in the field I0, the field B4, and the field P6. If the moving distance of the object in the field P6 is L, the moving distance in the field B4 is L / 2. Here, assuming that the above scaling process is performed with reference to the recording time, the intervals between the times Ti0, Tb6, and Tp2 are constant, so that the predicted motion vectors are expressed by the following equations (7) and (8). .

MVf_b6 = (1/2) × MV_P2 (7)
MVb_b6 = − (1/2) × MV_P2 (8)

  This is consistent with the scaling of the movement distance in the example of FIG. In general, in a moving image, a background or an object in a picture is moving at a constant speed in many cases. Therefore, if a motion vector is predicted based on a recording time, it can be predicted with high accuracy.

  However, if motion vector prediction in the temporal direct mode is performed on a moving image that has been subjected to telecine conversion as described above, the prediction is performed using display order information that deviates from the recording time interval. The prediction accuracy will deteriorate. As a result, the image quality is deteriorated.

  As described above, when the temporal direct mode is used for a moving image that has been subjected to display time interval conversion that causes the recording time interval to deviate from the display order information, such as telecine conversion, there is a problem that image quality deteriorates. is there.

  In addition, when an already-encoded picture that is located near the display time used for motion vector reference is an I picture, temporal direct mode encoding is performed in the same manner as when the block BL1 is intra-coded. The That is, motion vectors are not predicted in the (spatial) direct mode for all the blocks of the encoding target picture, resulting in a problem that the prediction accuracy is deteriorated and the image quality is deteriorated.

  Accordingly, the present invention provides a moving picture coding method and a moving picture coding apparatus capable of preventing image quality deterioration due to a reduction in accuracy of motion vector prediction in temporal direct mode coding and efficiently compressing a moving picture. The purpose is to do.

  In order to solve the above-described problem, in the moving picture coding method according to the present invention, a moving picture including a B picture that is subjected to predictive coding by referring to a plurality of coded pictures that are temporally forward or backward is referred to. In the encoding method, as the direct mode processing of the B picture, the temporal direct mode in which the motion vector of the target block is predicted and generated by referring to the motion vector of the encoded picture that is temporally nearby Including a processing step and a temporal direct mode prohibition determining step for determining prohibition of use of the temporal direct mode according to a condition to be encoded, and when the temporal direct mode is prohibited in the determination step, The prediction encoding is performed on the encoding target using processing steps other than the temporal direct mode processing step. It is characterized in.

  Thus, when it is predicted that image quality degradation will occur when temporal direct mode encoding is used, motion vector prediction in temporal direct mode encoding is not performed by performing temporal direct mode encoding. It is possible to prevent image quality deterioration due to a decrease in accuracy, and as a result, to efficiently compress moving images.

  Further, in the moving picture coding method according to the present invention, the processing steps other than the temporal direct mode processing step include coding that is located in the spatial periphery of the target block as the direct mode processing of the B picture. A spatial direct mode processing step of predicting and generating a motion vector of the target block with reference to a motion vector of the block, and when the temporal direct mode is prohibited in the determination step, The predictive encoding may be performed on the encoding target using the spatial direct mode processing step.

  As a result, it is possible to prevent image quality deterioration due to a decrease in the accuracy of motion vector prediction in temporal direct mode encoding, and as a result, it is possible to efficiently compress moving images.

  Further, in the moving picture encoding method according to the present invention, when the temporal direct mode prohibition determining step determines that the time interval of pictures constituting the encoding target is not constant, the temporal direct mode It is characterized in that it is determined that the use of is prohibited.

  As a result, it is possible to prevent deterioration in image quality due to a decrease in accuracy of motion vector prediction in temporal direct mode encoding that occurs when the time interval at which a picture is recorded as an image is not constant.

  Also, in the moving picture encoding method according to the present invention, when it is determined in the temporal direct mode prohibition determining step that the encoding target is a picture display time interval converted, the time It is characterized in that it is determined that the use of the general direct mode is prohibited.

  Even in this case, the time interval between pictures constituting a moving image in which the display time interval of the picture has been converted may not be constant. In this case, the image quality due to a decrease in the accuracy of motion vector prediction in temporal direct mode encoding may occur. It becomes possible to prevent degradation of the.

  Further, in the moving picture coding method according to the present invention, in the temporal direct mode prohibition determining step, an encoded picture used for referring to a motion vector in the temporal direct mode is an I picture for performing intra prediction encoding. If it is determined that the temporal direct mode is not used, it may be determined that the temporal direct mode is prohibited.

  This also makes it possible to prevent deterioration in image quality due to a decrease in the accuracy of motion vector prediction in temporal direct mode encoding that occurs when referring to an I picture.

  In the moving image encoding method according to the present invention, if it is determined in the temporal direct mode prohibition determining step that the time interval between pictures constituting the encoding target is not constant, the encoding target is the picture. When it is determined that the display time interval has been converted, and when it is determined that the encoded picture used for referring to the motion vector in the temporal direct mode is an I picture for performing intra prediction encoding It is determined whether or not any one of at least two cases is satisfied, and when any one of the at least two cases is determined, it is determined that use of the temporal direct mode is prohibited. Good.

  As a result, it is possible to reliably prevent a reduction in the accuracy of motion vector prediction in temporal direct mode encoding, and to further prevent deterioration in image quality.

  Note that the present invention can be realized not only as such an image encoding method but also as an image encoding apparatus using the characteristic steps included in such an image encoding method as a means, The means included in the encoding apparatus can be realized as an integrated circuit, or can be realized as a program for causing a computer to execute these steps. Needless to say, such a program can be distributed via a recording medium such as a CD-ROM or a transmission medium such as the Internet.

  As is apparent from the above description, according to the moving picture coding method of the present invention, it is possible to prevent image quality deterioration due to a decrease in the accuracy of motion vector prediction in temporal direct mode coding and to efficiently compress a moving picture. It becomes possible.

  Therefore, according to the present invention, it is possible to deliver a high-quality moving image with a high compression rate, and the practical value of the present invention in the present day when the Internet has become widespread is extremely high.

  Embodiments of the present invention will be described below with reference to the drawings.

(Embodiment 1)
FIG. 1 is a block diagram showing a functional configuration of a moving picture coding apparatus 100a according to Embodiment 1 of the present invention.

  The moving image encoding device 100a is a device that compresses and encodes an image input from an AV device or the like, and as illustrated in FIG. 1, a prediction residual encoding unit 101, a code string generation unit 102, and a prediction Residual decoding unit 103, in-plane prediction unit 104, frame memory 105, motion detection unit 106, motion compensation unit 107, motion vector storage unit 108, temporal direct mode processing unit 109, spatial Direct mode processing unit 110, direct processing determination unit 111, subtraction unit 112, mode selection unit 113, addition unit 114, frame memory 115, temporal direct mode prohibition determination unit 116, mode selection unit 121, Is provided.

  The motion detection unit 106 uses the encoded reconstructed image data as a reference picture, and detects a motion vector indicating a position predicted to be optimal in the search area in the picture.

  The motion compensation unit 107 determines a coding mode in inter-frame coding of a block using the motion vector detected by the motion detection unit 106, and generates predicted image data based on the coding mode. This encoding mode indicates how the macroblock is encoded.

  The motion vector storage unit 108 stores the motion vector detected by the motion detection unit 106.

  The temporal direct mode prohibition determination unit 116 determines whether or not the temporal direct mode can be used from the moving image information to be encoded, that is, whether or not the temporal direct mode is prohibited, and the determination result is sent to the direct processing determination unit 111. Notice.

  Based on the notification from the temporal direct mode prohibition determining unit 116, the direct processing determination unit 111 uses the temporal direct mode as the direct encoding mode for the encoding target image, or processes other than the temporal direct mode. Is used to determine whether to perform predictive coding.

  The temporal direct mode processing unit 109, when the direct encoding mode is the temporal direct mode, has already been encoded that is located near the display time of the encoding target image stored in the motion vector storage unit 108. A motion vector is predicted by performing scaling processing with reference to the motion vector of the block located at the same position as the encoding target block of the image.

  When the direct coding mode is the spatial direct mode, the spatial direct mode processing unit 110 refers to the motion vector of the coded adjacent block of the coding target block stored in the motion vector storage unit 108. Predict motion vectors.

  Based on the determination result of the temporal direct mode processing unit 109, the mode selection unit 121 performs either motion vector prediction by the temporal direct mode processing unit 109 or motion vector prediction by the spatial direct mode processing unit 110. To the motion compensation unit 107.

  The in-plane prediction unit 104 generates predicted image data using adjacent pixels of the encoding target block as an encoding mode for in-plane prediction.

  The mode selection unit 113 selects a mode with good coding efficiency among the coding mode of inter-frame prediction determined by the motion compensation unit 107 and the coding mode of the in-plane prediction unit 104.

  The subtraction unit 112 calculates a difference between the image data read from the frame memory 115 and the prediction image data of the motion compensation unit 107 or the in-plane prediction unit 104, and generates prediction residual image data.

  The prediction residual encoding unit 101 performs encoding processing such as frequency conversion and quantization on the input prediction residual image data to generate encoded data.

  The code string generation unit 102 performs variable-length coding or the like on the input encoded data, and generates a code string by adding the encoding mode information and the like input from the mode selection unit 113.

  The prediction residual decoding unit 103 performs decoding processing such as inverse quantization and inverse frequency transform on the input encoded data, and generates encoded data.

  The addition unit 114 adds the decoded difference image data input from the prediction residual decoding unit 103 and the predicted image data of the mode selected by the mode selection unit 113 to generate reconstructed image data.

The frame memory 105 stores the reconstructed image data generated by the adding unit 114.
Next, the operation of the moving picture coding apparatus 100a configured as described above will be described.

  FIG. 2 is an explanatory diagram showing the order of pictures in the frame memory 115. In particular, FIG. 2 (a) shows an input order, and FIG. 2 (b) shows an rearranged order. . Here, the vertical line indicates a picture, the symbol shown at the lower right of each picture is that the first letter of the alphabet indicates the picture type (I, P, B), and the numbers after the second letter are pictures in order of display time. Numbers are shown. In addition, the P picture uses a neighboring I picture or P picture that is forward in display time order as a reference picture, and the B picture is a neighboring I picture, P picture, and referenceable B picture that are forward in display time order; One nearby I picture or P picture in the display time order is used as a reference picture.

  For example, as shown in FIG. 2A, the input image is input to the frame memory 115 in picture units in order of display time. When the picture type to be encoded is determined, the pictures input to the frame memory 115 are rearranged in the order of encoding as shown in FIG. 2B, for example. This rearrangement to the coding order is performed based on the reference relationship in the inter-frame predictive coding, and is rearranged so that the picture used as the reference picture is first coded.

  Each picture rearranged in the frame memory 115 is read in units of macroblocks divided into groups of horizontal 16 × vertical 16 pixels, for example. Also, motion compensation and motion detection are performed in units of blocks divided into groups of, for example, horizontal 16 × vertical 16 pixels, horizontal 8 × vertical 16 pixels, horizontal 16 × vertical 8 pixels, and horizontal 8 × vertical 8 pixels.

  The subsequent operation will be described in the case where the picture to be encoded is a B picture.

  For B pictures, inter-picture predictive coding using two-way reference is performed. For example, when the encoding process of the picture B11 is performed in the example shown in FIG. 2A, the reference pictures ahead in the display time order are the pictures P10, P7, P4, and the reference pictures behind the display time order are the pictures. P13. Here, consider a case where a B picture is not used as a reference picture when another picture is encoded.

  The macroblock of the picture B11 read from the frame memory 115 is input to the motion detection unit 106 and the subtraction unit 112. The motion detection unit 106 uses the reference picture stored in the frame memory 105 to detect a forward motion vector and a backward motion vector for each block in the macroblock. Here, the reconstructed image data of the pictures P10, P7, and P4 stored in the frame memory 105 is used as a forward reference picture, and the reconstructed image data of the picture P13 is used as a backward reference picture. The motion detection unit 106 outputs the detected motion vector to the motion compensation unit 107.

  The motion compensation unit 107 uses the motion vector detected by the motion detection unit 106 to determine a coding mode in the inter-screen prediction of the macroblock. Here, the inter-picture coding mode of B picture is, for example, inter-picture predictive coding using a forward motion vector, inter-picture predictive coding using a backward motion vector, inter-picture predictive coding using a bidirectional motion vector, It is assumed that the encoding method can be selected from the direct mode. In the direct mode, the direct processing determination unit 111 determines whether to use the temporal direct mode or the spatial direct mode in a specific unit. The specific unit described above may be any unit that is larger than a slice, such as a slice unit, a picture unit, a GOP unit, or a sequence unit.

  The mode selection unit 113 receives the coding mode of the inter prediction determined by the motion compensation unit 107 and the coding mode of the intra prediction determined by the intra prediction unit 104 as an input, and selects an encoding mode with the highest encoding efficiency. And the mode becomes the encoding mode of the macroblock.

  Next, the processing operation in temporal direct mode prohibition determination unit 116 will be described. The temporal direct mode prohibition determination operation can be performed by the method 1 described below.

(Method 1)
FIG. 3 is a flowchart showing an operation of temporal direct mode prohibition determination according to method 1.

  The temporal direct mode prohibition determination unit 116 performs determination based on moving image information to be encoded. First, the temporal direct mode prohibition determination unit 116 determines whether or not the time interval between pictures constituting a moving image to be encoded is constant (step S201). If not constant (NO in step S201), the temporal direct mode prohibition determination unit 116 prohibits the use of the temporal direct mode (step S202). Here, when the time interval is not constant, for example, a frame drop occurs. On the other hand, if it is constant (YES in step S201), the temporal direct mode prohibition determination unit 116 permits the use of the temporal direct mode (step S203). Then, the temporal direct mode prohibition determination unit 116 notifies the direct processing determination unit 111 of whether the temporal direct mode can be used.

  Note that the determination in step S201 may be performed using time information for each picture given to the encoding device, or using information on whether the time interval given to the encoding device is constant. Alternatively, the encoding apparatus may use the time interval information obtained by determining whether the encoding is actually performed for each input picture or whether the encoding is skipped. That is, it may be information from the outside, or information obtained by the internal time management unit 120.

  Note that the temporal direct mode prohibition determination processing in the present embodiment may be performed in units of pictures, in units of GOPs in which a plurality of pictures are grouped into one group, or in sequence units delimited by a specific picture. Or in units of streams that are the entire moving image sequence to be encoded. In other words, the range in which the temporal direct mode is prohibited may be prohibited only when the frame drop occurs between the reference picture and the picture to be encoded, and the time exceeding the minimum range may be prohibited. You may make it prohibit also about the unit of a wide range when the frame drop | omission has generate | occur | produced in the scale.

  With the method 1 described above, a motion vector with low prediction accuracy is not calculated by scaling processing in motion vector prediction of temporal direct mode encoding, and deterioration in image quality can be prevented.

(Embodiment 2)
Next, another embodiment of the present invention will be described.

  FIG. 4 is a block diagram showing a functional configuration of the moving picture coding apparatus 100b according to Embodiment 2 of the present invention. In the figure, a telecine conversion device 200 is also shown. Also, the same reference numerals are given to the parts of the moving picture encoding apparatus 100b corresponding to the configuration of the moving picture encoding apparatus 100a shown in FIG. 1, the description thereof will be omitted, and different parts will be described in detail.

  Here, in the moving image encoding apparatus 100b, the temporal direct mode prohibition determining unit 116 determines that the encoding target is the picture display time based on the encoding target moving image information received from the telecine conversion apparatus 200 or the like. The configuration is different from the configuration of the video encoding device 100a shown in FIG. 1 in that it is configured to determine whether or not the video has undergone interval conversion.

  Next, the operation of the moving picture coding apparatus 100b configured as described above will be described.

(Method 2)
FIG. 5 is a flowchart showing an operation of temporal direct mode prohibition determination according to method 2.

  The temporal direct mode prohibition determination unit 116 performs determination based on moving image information to be encoded. First, it is determined whether or not the encoding target is a moving image that has undergone conversion of a picture display time interval (step S301). In the case of a converted moving image (YES in step S301), the temporal direct mode prohibition determination unit 116 prohibits the use of the temporal direct mode (step S302). On the other hand, in the case of a moving image that has not been converted (NO in step S301), the temporal direct mode prohibition determination unit 116 permits the use of the temporal direct mode (step S303). Then, the direct processing determination unit 111 is notified of whether or not the temporal direct mode can be used.

  Although the conversion of the display time interval of the picture is used for the determination here, the conversion of the display time interval may be limited to the telecine conversion (2-3 conversion).

  Information indicating whether or not the display time interval conversion of the picture used for the determination in step S301 has been performed may be given from the telecine conversion device 200 outside the encoding device, and the moving image encoding device 100a. It may be determined whether or not the display time interval of the picture has been converted from the feature of the image.

  When telecine conversion (2-3 conversion) is performed, use of the temporal direct mode may be prohibited for the entire range subjected to normal telecine conversion. Depending on the positional relationship with the picture for which the vector is referenced, the relationship between the time intervals before and after the conversion of the display time interval may be the same. In other words, there may be no problem in scaling the time interval. If such a condition is detected and the above condition is met, a determination process may be made that permits the temporal direct mode to be used as before.

  With the method 2 described above, a motion vector with low prediction accuracy is not calculated by scaling processing in motion vector prediction of temporal direct mode encoding, and deterioration of image quality can be prevented.

(Embodiment 3)
Next, still another embodiment of the present invention will be described.

  FIG. 6 is a block diagram showing a functional configuration of the moving picture coding apparatus 100c according to Embodiment 3 of the present invention. 1 and FIG. 4, the same reference numerals are assigned to the parts of the moving picture coding apparatus 100c corresponding to the configurations of the moving picture coding apparatuses 100a and 100b shown in FIG. 1, the description thereof is omitted, and different parts are described in detail. To do.

  Here, in the moving picture coding apparatus 100c, for example, based on the type of the reference picture sent from the mode selection unit 113, the temporal direct mode prohibition determination unit 116 uses the motion vector in the temporal direct mode. This is different from the configurations of the moving image encoding apparatuses 100a and 100b shown in FIGS. 1 and 4 in that it is configured to determine whether or not the encoded picture is an I picture.

  Next, the operation of the moving picture coding apparatus 100c configured as described above will be described.

(Method 3)
FIG. 7 is a flowchart showing the operation of temporal direct mode prohibition determination according to method 3.

  The temporal direct mode prohibition determination unit 116 performs determination based on moving image information to be encoded. First, the temporal direct mode prohibition determining unit 116 determines whether or not an encoded picture used for referring to a motion vector in the temporal direct mode is an I picture (step S401). This determination is performed based on the type of reference picture sent from the mode selection unit 113, for example. In the case of an I picture (YES in step S401), the temporal direct mode prohibition determination unit 116 prohibits the use of the temporal direct mode (step S402).

  On the other hand, if it is not an I picture (NO in step S401), the temporal direct mode prohibition determination unit 116 permits the use of the temporal direct mode (step S403). Then, the temporal direct mode prohibition determination unit 116 notifies the direct processing determination unit 111 of whether the temporal direct mode can be used.

  With the method 3 described above, the motion vector with low prediction accuracy is not calculated by the scaling process in the motion vector prediction of temporal direct mode encoding, and deterioration in image quality can be prevented.

(Embodiment 4)
In the first to third embodiments, the methods 1 to 3 have been described as separate methods, but at least two of these methods may be combined. Below, the case where the method 2 and the method 3 are combined is demonstrated as the example.

  FIG. 8 is a block diagram of moving picture coding apparatus 100d according to Embodiment 4 of the present invention. In addition, the same number is attached | subjected to the part of the moving image encoder 100d corresponding to the structure of the moving image encoder 100a, 100b, 100c shown by FIG.1, FIG.4, FIG.6, and a different part is demonstrated in detail. .

  Here, the moving picture coding apparatus 100d is configured to perform the method 2 and the method 3 in combination, and the moving picture coding apparatus 100a shown in FIGS. 1, 4, and 6 is used. , 100b, and 100c.

Next, the operation of the moving picture coding apparatus 100d configured as described above will be described.
FIG. 9 is a flowchart when the method 2 and the method 3 are combined.

  First, the temporal direct mode prohibition determination unit 116 determines whether or not the encoding target is a moving image that has undergone conversion of a picture display time interval (step S501). In the case of a converted moving image (YES in step S501), the temporal direct mode prohibition determination unit 116 prohibits the use of the temporal direct mode (step S503). On the other hand, in the case of a moving image that has not been converted (NO in step S501), the temporal direct mode prohibition determining unit 116 determines whether or not the encoded picture used for motion vector reference in the temporal direct mode is an I picture. (Step S502). In the case of an I picture (YES in step S502), the temporal direct mode prohibition determination unit 116 prohibits the use of the temporal direct mode (step S503).

  On the other hand, if it is not an I picture (NO in step S502), the temporal direct mode prohibition determination unit 116 permits the use of the temporal direct mode (step S504).

  Then, the temporal direct mode prohibition determination unit 116 notifies the direct processing determination unit 111 of whether the temporal direct mode can be used.

  By combining method 1, method 2, and method 3, method 1 and method 2, and method 1 and method 3 with such a method, the prediction accuracy is low due to scaling processing in motion vector prediction of temporal direct mode encoding. The frequency of calculating the motion vector can be further reduced, and the deterioration of the image quality can be prevented. Here, for example, in FIG. 9, the processing is performed in the order of steps S501 and S502, but the processing may be performed in the reverse order, and the order of processing is not limited in other combinations.

(Embodiment 5)
Further, the program for realizing the moving picture encoding method shown in each of the above embodiments 1 to 4 is recorded on a recording medium such as a flexible disk, so that the processing shown in each of the above embodiments is performed. Can be easily implemented in an independent computer system.

  FIG. 10 is an explanatory diagram when the moving image encoding method of each of the above embodiments is implemented by a computer system using a program recorded on a recording medium such as a flexible disk.

  FIG. 10B shows an appearance, a cross-sectional structure, and a flexible disk as viewed from the front of the flexible disk, and FIG. 10A shows an example of a physical format of the flexible disk that is a recording medium body. The flexible disk FD is built in the case F, and on the surface of the disk, a plurality of tracks Tr are formed concentrically from the outer periphery toward the inner periphery, and each track is divided into 16 sectors Se in the angular direction. ing. Therefore, in the flexible disk storing the program, the program is recorded in an area allocated on the flexible disk FD.

  FIG. 10C shows a configuration for recording and reproducing the program on the flexible disk FD. When the program for realizing the moving image encoding method is recorded on the flexible disk FD, the program is written from the computer system Cs through the flexible disk drive. When the above-described moving picture coding method for realizing the moving picture coding method by a program in a flexible disk is constructed in a computer system, the program is read from the flexible disk by a flexible disk drive and transferred to the computer system.

  In the above description, a flexible disk is used as the recording medium, but the same can be done using an optical disk. Further, the recording medium is not limited to this, and any recording medium such as an IC card or a ROM cassette capable of recording a program can be similarly implemented.

  Note that the processing for realizing the moving picture encoding method shown in each of the above embodiments may be realized as an LSI which is an integrated circuit. These may be integrated into one chip so as to include a part or all of them.

  The name used here is LSI, but it may also be called IC, system LSI, super LSI, or ultra LSI depending on the degree of integration.

  Further, the method of circuit integration is not limited to LSI's, and implementation using dedicated circuitry or general purpose processors is also possible. An FPGA (Field Programmable Gate Array) that can be programmed after manufacturing the LSI, or a reconfigurable processor that can reconfigure the connection and setting of circuit cells inside the LSI may be used.

  Furthermore, if integrated circuit technology that replaces LSI or the like appears as a result of progress in semiconductor technology or other derived technology, it is naturally also possible to perform integration of functional blocks using that technology.

  In the above embodiment, the telecine conversion apparatus 200 is provided outside the moving picture encoding apparatus 100a. However, the telecine conversion apparatus 200 may be provided inside the moving picture encoding apparatus 100a. Good.

  The moving image encoding method according to the present invention is useful as a method for generating a code string by encoding each picture constituting a moving image using, for example, a DVD device, a mobile phone, a personal computer, or the like.

It is a block diagram which shows the structure of the moving image encoder which concerns on Embodiment 1 of this invention. FIG. 2A is a diagram showing the order of pictures in the picture memory, FIG. 2A is a diagram showing the order of input, and FIG. 2B is a diagram showing the rearranged order. It is a flowchart which shows the operation | movement which determines the propriety of use of the temporal direct mode by the method 1 in the temporal direct mode prohibition determination part. It is a block diagram which shows the structure of the moving image encoder which concerns on Embodiment 2 of this invention. It is a flowchart which shows the operation | movement which determines the propriety of use of the temporal direct mode by the method 2 in the temporal direct mode prohibition determination part. It is a block diagram which shows the structure of the moving image encoder which concerns on Embodiment 3 of this invention. It is a flowchart which shows the operation | movement which determines the propriety of use of the temporal direct mode by the method 3 in the temporal direct mode prohibition determination part. It is a block diagram which shows the structure of the moving image encoder which concerns on Embodiment 4 of this invention. It is a flowchart which shows the operation | movement which determines the availability of the temporal direct mode by the combination of the method 2 and the method 3 in the temporal direct mode prohibition determination part. It is explanatory drawing about the recording medium for storing the program for implement | achieving the moving image encoding method of Embodiment 1-4 by a computer system. It is a schematic diagram which shows the prediction production | generation method of the motion vector in temporal direct mode. It is a schematic diagram which shows the prediction production | generation method of the motion vector in spatial direct mode. It is a figure which shows the example of telecine conversion (2-3 conversion). It is a figure which shows the example of the motion vector prediction of the temporal direct mode in the moving image which carried out telecine conversion (2-3 conversion). It is a figure which shows the example of the motion vector prediction of the temporal direct mode in the moving image which carried out telecine conversion (2-3 conversion).

Explanation of symbols

100a, 100b, 100c, 100d Moving picture encoding apparatus 101 Prediction residual encoding unit 102 Code sequence generation unit 103 Prediction residual decoding unit 104 In-plane prediction unit 105, 115 Frame memory 106 Motion detection unit 107 Motion compensation unit 108 Motion vector storage unit 109 Temporal direct mode processing unit 110 Spatial direct mode processing unit 111 Direct processing determination unit 112 Subtraction unit 113 Mode selection unit 114 Addition unit 116 Temporal direct mode prohibition determination unit 120 Time management unit 121 Mode selection unit 200 Telecine converter

Claims (9)

A method for encoding a moving image including a B picture for performing predictive encoding with reference to a plurality of encoded pictures that are temporally forward or backward,
As the direct mode processing of the B picture, a temporal direct mode processing step of predicting and generating a motion vector of a target block with reference to motion vectors of encoded pictures that are temporally nearby;
A temporal direct mode prohibition determining step for determining prohibition of use of the temporal direct mode according to an encoding target condition,
When the temporal direct mode is prohibited in the determination step, the predictive encoding is performed using a processing step other than the temporal direct mode processing step on the encoding target. Method. In processing steps other than the temporal direct mode processing step, as a direct mode processing of the B picture, referring to a motion vector of an encoded block located in the spatial periphery of the target block, It includes a spatial direct mode processing step that predicts and generates motion vectors,
The moving image according to claim 1, wherein when the temporal direct mode is prohibited in the determining step, the predictive encoding is performed on the encoding target using the spatial direct mode processing step. Encoding method. The use of the temporal direct mode is determined to be prohibited when it is determined in the temporal direct mode prohibition determining step that a time interval between pictures constituting the encoding target is not constant. Item 8. A moving image encoding method according to Item 1. In the temporal direct mode prohibition determining step, when it is determined that the encoding target is a conversion of a picture display time interval, it is determined to prohibit the use of the temporal direct mode. The moving picture encoding method according to claim 1. In the temporal direct mode prohibition determining step, when it is determined that an encoded picture used for reference to a motion vector in the temporal direct mode is an I picture to be subjected to intra prediction encoding, the temporal direct mode The moving image encoding method according to claim 1, wherein use is determined to be prohibited. In the temporal direct mode prohibition determining step, when it is determined that the time interval of the picture constituting the encoding target is not constant, it is determined that the encoding target is a conversion of the display time interval of the picture. And whether or not the encoded picture used for referring to the motion vector in the temporal direct mode corresponds to at least one of the two when it is determined that the picture is an I picture for intra prediction encoding The moving picture encoding method according to claim 1, further comprising: determining that the use of the temporal direct mode is prohibited when any one of the at least two conditions is satisfied. An apparatus for encoding a moving image including a B picture that performs predictive encoding with reference to a plurality of encoded pictures that are temporally forward or backward,
As the direct mode of the B picture, temporal direct mode processing means for predicting and generating a motion vector of the target block with reference to motion vectors of encoded pictures that are temporally nearby,
A temporal direct mode prohibition judging means for judging prohibition of use of the temporal direct mode according to a condition to be encoded;
When the temporal direct mode is prohibited by the determining unit, the predictive encoding is performed on the encoding target using a unit other than the temporal direct mode processing unit. apparatus.   The program which makes a computer perform the step contained in the moving image encoding method of Claim 1.   The integrated circuit which integrated the means contained in the moving image encoder of Claim 7.

Images