JP4345024B2 - Image encoding apparatus, image encoding method, and recording medium recording image encoding program - Google Patents

Description

  The present invention relates to an image encoding device, an image encoding method, and a recording medium on which an image encoding program is recorded, and is particularly suitable when applied to a case where an image is compressed and transmitted to an image decoding device.

  The image encoding device side encodes and compresses and transmits the image data, and the communication system that decompresses and decodes the image data transmitted on the image decoding device side, and compresses and records the image data. In addition, in a compressed image recording / reproducing apparatus that decompresses and outputs compressed image data during reproduction, as a method for compressing image data, for example, a bidirectional predictive code adopted in the MPEG (Moving Picture Experts Group) standard is used. Is used. In this bidirectional predictive encoding method, encoding efficiency is improved by using bidirectional prediction. In the bi-directional predictive coding method, three types of coding, that is, intra-frame coding, inter-frame forward predictive coding, and bi-directional predictive coding, are performed. coded picture), P picture (predictive coded picture) and B picture (bidirectionally predictive coded picture).

  By the way, an image encoding apparatus that compresses and encodes image data using the bidirectional predictive encoding method adopted in the above-mentioned MPEG standard has the image decoding apparatus that expands the compression-encoded image data. In order to verify that the capacity of the input buffer is not broken, a decoding guarantee buffer, that is, a VBV (Video Buffering Verfier) buffer, in which the input buffer on the image decoding apparatus side is virtually considered from the image encoding apparatus side, is assumed. Therefore, the amount of data generated in the image coding apparatus is controlled so that the data occupancy of the VBV buffer does not overflow or underflow the allowable value (“Television Society Multimedia Selection MPEG” published by Ohm Corporation). (See page 115).

  That is, when encoded data is transmitted from an image encoding device at a fixed bit rate, when a bit stream of image data is input to the VBV buffer at a specified bit rate, as shown in FIG. Image data is generated from the VBV buffer in a state where the data occupation amount does not underflow and overflow.

  In the image decoding apparatus, the bit stream transmitted from the image encoding apparatus is decoded in units of image data for one frame through the VBV buffer. Thus, even if the bit rate of the encoded data transmitted from the image encoding apparatus is constant, pictures with different data amounts can be reliably decoded. For example, a frame encoded using image data in a frame (that is, an intra-coded I picture is a frame encoded using motion compensation from a preceding and succeeding frame (that is, a P picture or a B picture). Compared with a large amount of code per frame, each picture can be reliably decoded if it can be decoded without VBV buffer underflow or overflow.

In Figure 1, the slope of each line segment x shows the fixed bit rate of the data stream to be transmitted, each position x ₁ of the line segment x represents the data occupancy of the VBV buffer. The line segment y indicates that the data occupancy amount of the VBV buffer is abruptly reduced by reading out data for one frame from the VBV buffer as an I-picture, B-picture, and P-picture.

  However, as shown in FIG. 2, when a picture having data exceeding the predetermined data occupancy range is generated from the VBV buffer, an underflow occurs in the VBV buffer. Although the target code generation amount is set by rate control, the actual code generation amount is larger than the target code generation amount due to the complexity of the transmitted image and the intensity of motion. In this case, it means that the image cannot be decoded because the data for decoding the picture is insufficient (corresponding to the portion below “0” in the figure).

  The operation of the VBV buffer is described in Annex C of ISO (International Organization for Standardization) / IEC (International Electrotechnical Commission) 13818-2 (MPEG2 video part). Underflow and overflow occurred in the VBV buffer. The operation of the image decoding apparatus at that time is not described.

  To prevent underflow from occurring in the VBV buffer, a fixed threshold is set for the VBV buffer, and skipping is performed for pictures whose data occupancy in the VBV buffer is below this threshold (for images). There has been proposed a method for preventing an underflow from occurring in the VBV buffer by stopping the encoding process.

  However, in this method, there is a possibility that the skip process is similarly performed for any picture coding type picture. In this case, the I picture including the image data most important for image quality at the time of decoding has a larger amount of code generation (that is, the data amount) than the P picture and the B picture, and therefore can be skipped more than the P picture and the B picture. It is conceivable that the property becomes higher. When the I-picture is skipped in this way, the image quality of the P-picture and B-picture that use the skipped I-picture as a predicted image is affected, so that the overall picture quality may be deteriorated.

  The present invention has been made in view of the above points, and intends to propose an image encoding apparatus and an image encoding method capable of preventing the occurrence of underflow in a decoding guarantee buffer without degrading the image quality. To do.

  In order to solve such a problem, in the present invention, the encoding control means calculates a code generation amount of the encoded image, and based on the calculation result, the data occupancy of the encoded image in the buffer of the image decoding apparatus A first amount set corresponding to a first image to be intra-encoded and a second image to be forward-predictive encoded and / or a third image to be bi-directional predictively encoded, or Of the second and / or third thresholds, less than the first, second, or third threshold corresponding to the first, second, or third image that is currently encoded When a code generation amount occurs, the encoding means is controlled so as to stop the encoding process for the first, second, or third image that is currently encoded.

  When the encoding process for the first, second, or third image that is currently encoded is stopped, the buffer amount for the next image is secured in the decoding guarantee buffer.

  Furthermore, by setting the values of the first and second and / or third threshold values according to the importance of the image encoding type when decoding an image, it is important in the decoding guarantee buffer. A buffer amount of an image coding type image having a high degree is preferentially secured.

  Thus, it is possible to realize an image encoding device and an image encoding method capable of preventing the occurrence of underflow in the buffer of the image decoding device without degrading the image quality.

  Further, when the input image data is compression-encoded by an encoding method according to the image encoding type by the encoding means and encoding of one image is completed, the occupation amount calculating means Thus, the data occupancy of the virtual buffer corresponding to the input buffer in the image decoding apparatus is calculated. In addition, when the encoding difficulty level calculating unit calculates the encoding difficulty level indicating the encoding difficulty level of the image subsequent to the encoded image, and when encoding of one image is completed, Coding in which the data occupancy calculated by the occupancy calculating means is less than a predetermined data occupancy determination threshold by the image skip control means and is calculated by the encoding difficulty level calculating means. When the difficulty level meets a predetermined standard, an image to be encoded next is encoded as an image composed of skip macroblocks.

  Thus, it is possible to prevent the underflow of the virtual buffer corresponding to the input buffer in the image decoding apparatus while suppressing the influence on the image quality as much as possible.

(1) First Embodiment In FIG. 3, reference numeral 1 denotes an image coding apparatus as a whole, and an input image signal S1 input from an input terminal W1 is input to an image type designation circuit 2.

  The image type designation circuit 2 designates which image coding type of the I picture, P picture, or B picture each frame of the inputted image signal S1 is to be processed to the image coding rearrangement circuit 3. Send it out. For example, the image encoding device 1 performs encoding processing using 15 frames as a group of pictures (GOP) as a unit.

  The image coding order rearrangement circuit 3 rearranges the frames in the order of coding according to the designated image coding type, sends the frames to the scan converter 4 as an image signal S2, and predicts the image coding type data S3. The data is sent to the mode determination circuit 5, the motion vector detection circuit 6, and the encoding control circuit 7. The image encoding / reordering circuit 3 detects a motion vector of a frame that is currently encoded, and an image temporally past the current image (hereinafter referred to as a past reference image). In addition, an image signal S4 representing an image temporally in the future from the current image (hereinafter referred to as a future reference image) is sent to the motion vector detection circuit 6.

  The scan converter 4 converts an input frame into a block format signal, and divides the signal of one frame into N slices with 16 lines as one unit. Each slice is divided into M macroblocks composed of luminance signals corresponding to 16 × 16 pixels. An address (variable length code (increment value)) is added to each macro block, and this address represents the position of each macro block. In this case, the address of the first macroblock transmitted in the picture represents the number itself, and the following macroblock address is the absolute address of the macroblock and the absolute address of the macroblock transmitted immediately before. Represents the difference.

  The motion vector detection circuit 6 performs I-picture by intra-frame coding, P-picture by inter-frame forward prediction coding, or both of the image data of each frame according to image coding type data S3 synchronized with each frame of the image signal S2. It is processed as a B picture by bidirectional predictive coding. Thus, the image data of the frame processed as the I picture is stored in the past reference image memory (not shown) that stores the past reference image from the motion vector detection circuit 6, and the image data of the frame processed as the B picture is Image data of a frame that is stored in a current image memory (not shown) that stores a current image and processed as a P-picture is stored in a future reference image memory (not shown) that stores a future reference image.

  Here, at the next timing, when a frame to be processed as a B-picture or P-picture is input to the motion vector detection circuit 6, the image data of the first P-picture stored in the future reference picture memory so far is past. Stored in the reference image memory. The image data of the next B picture is stored in the current image memory, and the image data of the next P picture is stored in the future reference image memory. Thereafter, such an operation is sequentially repeated.

  The motion vector detection circuit 6 sends to the prediction mode determination circuit 5 motion vector data S5 consisting of a motion vector in forward prediction and a motion vector estimation residual at that time. In the case of the B-picture, the motion vector detection circuit 6 transmits motion vector data S5 including the motion vector in the backward prediction and the motion vector estimation residual at that time.

  The prediction mode determination circuit 5 determines which prediction mode is selected from the intra mode, the forward prediction mode, the backward prediction mode, or the bidirectional prediction mode.

  Here, the intra mode is a process of transmitting image data of a frame to be encoded as transmission data as it is, and the forward prediction mode is a process of transmitting a prediction residual with a past reference image and a forward motion vector. It is. The backward prediction mode is a process of transmitting a prediction residual with a future reference image and a backward motion vector, and the bidirectional prediction mode is an average value of two predicted images of the past reference image and the future reference image. This is a process of transmitting the prediction residual and two motion vectors in the forward and backward directions. In the case of the B picture, these four types of prediction modes are switched in units of macroblocks.

  Based on the image coding type data S3 sent from the image coding rearrangement circuit 3, the prediction mode determination circuit 5 selects an intra mode in the case of an I picture, and an intra mode or a forward direction in the case of a P picture. Select one of the prediction modes, and in the case of the B-picture, select one of the prediction modes from the intra mode, the forward prediction mode, the backward prediction mode, or the bidirectional prediction mode, and select the selected prediction mode. Prediction mode data S6 representing the mode is sent to the arithmetic circuit 8.

  The arithmetic circuit 8 performs intra, forward prediction, backward prediction or bidirectional prediction on the macroblock S7 read from the scan converter 4 based on the prediction mode data S6.

  When image data of a frame to be processed as an I-picture is input as the macroblock S7, the arithmetic circuit 8 intra-codes the image data and sends it to a DCT (Discrete Cosine Transform) conversion circuit 9. The DCT conversion circuit 9 converts the intra-coded image data into DCT conversion coefficients and sends them to the quantization circuit 10.

  The quantization circuit 10 quantizes each DCT transform coefficient at the quantization step designated by the encoding control circuit 7 and sends it to the variable length encoding circuit 11 and the inverse quantization circuit 12.

  The variable length coding circuit 11 converts the quantized image data, the prediction mode data S6 sent from the prediction mode determination circuit 5 and the motion vector data S5 sent from the motion vector detection circuit 6 into a variable length code such as a Huffman code. And is transmitted to the image decoding device 13 via a transmission path (not shown) and given to the encoding control circuit 7. The variable length coding circuit 11 counts the increment value of each macro block.

  The inverse quantization circuit 12 inversely quantizes the quantized image data according to the quantization step at the time of quantization, and sends it to the inverse DCT conversion circuit 14. The inverse DCT conversion circuit 14 performs an inverse DCT conversion process on the output from the inverse quantization circuit 12. An output from the inverse DCT conversion circuit 14 is stored in a past reference image storage unit (not shown) that stores a past reference image in the frame memory 16 via the calculator 15.

  Next, when image data of a frame to be processed as a P-picture is input from the scan converter 4 to the arithmetic circuit 8, and when the prediction mode data S6 sent from the prediction mode determination circuit 5 is in the intra mode, the image data of the frame is In the same manner as in the case of the I-picture described above, intra coding is performed, and the data is transmitted to the image decoding device 13 via the DCT conversion circuit 9, the quantization circuit 10, and the variable length coding circuit 11, and is also supplied to the coding control circuit 7. Further, the data is stored in a future reference image storage unit (not shown) that stores a future reference image in the frame memory 16 via the inverse quantization circuit 12, the inverse DCT conversion circuit 14, and the arithmetic unit 15.

  When the prediction mode data S6 represents the forward prediction mode, the image data (in this case, I-picture image data) stored in the past reference image storage unit of the frame memory 16 is read and the motion compensation circuit is read out. 17 is sent out.

  The motion compensation circuit 17 performs motion compensation on the image data in correspondence with the forward motion vector data S5 sent from the motion vector detection circuit 6.

  That is, in the forward prediction mode, the motion compensation circuit 17 forwards the read address of the past reference image storage unit of the frame memory 16 from the position corresponding to the position of the macroblock currently output by the motion vector detection circuit 6. The data is read out by being shifted by the amount corresponding to the motion vector data S5, and a prediction reference image is generated and sent to the arithmetic circuit 8 and the arithmetic unit 15.

  The arithmetic circuit 8 subtracts the image data of the predicted reference image corresponding to the macroblock sent from the motion compensation circuit 17 from the macroblock data S7 of the reference image sent from the scan converter 4 to obtain a prediction residual. And the difference data is sent to the DCT conversion circuit 9. The difference data is transmitted to the image decoding device 13 via the DCT conversion circuit 9, the quantization circuit 10, and the variable length coding circuit 11, and is given to the coding control circuit 7. The difference data is locally decoded by the inverse quantization circuit 12 and the inverse DCT conversion circuit 14 and sent to the computing unit 15.

  The computing unit 15 adds the image data of the predicted reference image sent from the motion compensation circuit 17 to the difference data sent from the inverse DCT conversion circuit 14. As a result, locally decoded P-picture image data is obtained. The image data of the P picture is stored in a future reference image storage unit that stores a future reference image in the frame memory 16.

  Next, when image data of a frame to be processed as a B picture is input from the scan converter 4 to the arithmetic circuit 8, and the prediction mode data S6 sent from the prediction mode determination circuit 5 is in the intra mode or the forward prediction mode, the frame The image data is processed in the same manner as in the case of the P-picture. On the other hand, when the prediction mode data S6 is set to the backward prediction mode, the image data (in this case, P-picture image data) stored in the future reference image storage unit of the frame memory 16 is read out. To the motion compensation circuit 17.

  The motion compensation circuit 17 performs motion compensation on this image data in accordance with the reverse motion vector data S5 sent from the motion vector detection circuit 6.

  That is, in the backward prediction mode, the motion compensation circuit 17 sets the read address of the future reference image storage unit of the frame memory 16 from the position corresponding to the position of the macroblock currently output by the motion vector detection circuit 6. The data is read by shifting by the amount corresponding to the data S5 to generate a predicted reference image, which is sent to the arithmetic circuit 8 and the arithmetic unit 15.

  The arithmetic circuit 8 subtracts the image data of the prediction reference image corresponding to the macro block transmitted from the motion compensation circuit 17 from the macro block data of the reference image transmitted from the scan converter 4 to obtain a difference as a prediction residual. Data is obtained and this difference data is sent to the DCT conversion circuit 9.

  At this time, the arithmetic unit 15 adds the image data of the predicted reference image sent from the motion compensation circuit 17 to the difference data sent from the inverse DCT conversion circuit 14, thereby locally decoding the image data of the B picture. Get.

  In the bidirectional prediction mode, image data stored in the past reference image storage unit of the frame memory 16 (in this case, I-picture image data) and image data stored in the future reference image storage unit (this (P-picture image data) is read out and sent to the motion compensation circuit 17.

  The motion compensation circuit 17 performs motion compensation on the image data corresponding to the motion vector data S5 composed of the forward motion vector and the backward motion vector sent from the motion vector detection circuit 6.

  That is, in the bidirectional prediction mode, the motion compensation circuit 17 sets the read addresses of the past reference image storage unit and the future reference image storage unit of the frame memory 16 to the position of the macro block currently output by the motion vector detection circuit 6. The data is read out from the corresponding position by an amount corresponding to the motion vector data S5 composed of the forward direction motion vector and the backward direction motion vector to generate a predicted reference image, and is sent to the arithmetic circuit 8 and the arithmetic unit 15.

  The arithmetic circuit 8 subtracts the average value of the image data of the predicted reference image corresponding to the macro block transmitted from the motion compensation circuit 17 from the macro block data of the reference image transmitted from the scan converter 4 to predict the residual. Is obtained, and this difference data is sent to the DCT conversion circuit 9.

  At this time, the arithmetic unit 15 adds the image data of the predicted reference image sent from the motion compensation circuit 17 to the difference data sent from the inverse DCT conversion circuit 14, thereby locally decoding the image data of the B picture. Get. Here, since the B picture is not used as a predicted image of another image, it is not stored in the frame memory 16.

  The image decoding device 13 temporarily stores a bit stream transmitted from the image encoding device 1 at a fixed bit rate in the decoder buffer 18 and then reads out image data for one frame from the decoder buffer 18 at a predetermined timing. The decoding unit 19 sequentially decodes the decoded frames in the order of the original images and displays them on a display, for example.

  Here, the decoder buffer 18 constitutes a decoding guarantee buffer (that is, a VBV buffer) virtually conceived from the image encoding device 1, and the image encoding device 1 includes a decoder 19 that receives an image from the decoder buffer 18. The timing to read data is understood.

  In this embodiment, the encoding control circuit 7 calculates the code generation amount of the variable length encoded image data sent from the variable length encoding circuit 11 in units of macro blocks and counts the increment value. The encoding control circuit 7 controls the code generation amount per frame by giving the quantization step control signal S8 representing the target encoding generation amount to the quantization circuit 10.

  Further, the encoding control circuit 7 has a third threshold value, a second threshold value, and a first threshold value as threshold values of the data occupation amount in the decoder buffer 18 functioning as the VBV buffer. These third, second, and first threshold values correspond to the B picture, the P picture, and the I picture, respectively. In addition, these threshold values are set such that the third threshold value> the second threshold value> the first threshold value so as to increase in order of increasing importance of the image coding type when decoding the image. Is set to Thus, the encoding control circuit 7 uses the VBV buffer formed by the decoder buffer 18 based on the code generation amount of the image data sent from the variable length encoding circuit 11 when the B picture is encoded. When image data having a code amount such that the data occupancy amount of the image data is less than the third threshold value is generated, a skip execution signal S9 is sent to the quantization circuit 10 and the B picture being encoded is transmitted. Subsequent image data is skipped in units of macroblocks, and as a result, the decline in data occupancy is suppressed.

  Here, skipping means stopping the encoding process that is currently being executed.

  When such processing is performed, the data occupation amount in the VBV buffer 18 formed by the data buffer 18 changes as shown in FIG. The bold line portion shown in FIG. 4 indicates that the image data has been skipped, and the same applies to FIGS.

  In this case, when the quantizing circuit 10 receives the skip signal S9, the quantizing circuit 10 skips the image data by turning on the skip switch provided therein. In this case, the encoding control circuit 7 sends address information S10 about the macro block to be skipped to the variable length encoding circuit 11. Code (bit) generation from the quantization circuit 10 is eliminated. When the image data is not skipped, the quantization circuit 10 quantizes the image data by turning off the skip switch.

  Here, the encoding control circuit 7 knows the timing at which the decoding unit 19 of the image decoding device 13 reads data from the decoder buffer 18, and the code generation amount of the image data sent from the variable length encoding circuit 11. Thus, the amount of data currently stored in the decoder buffer 18 can be known.

  When the image data is not skipped, the variable length encoding circuit 11 counts the increment value and transmits the image data together with the address information to the encoding control circuit 7 and the image decoding device 13. On the other hand, when the image data is skipped, the variable length encoding circuit 11 counts the increment value but does not output the address information about the skipped macroblock, and is arranged in the row and column directions. Address information of the first macroblock in the row direction of a plurality of macroblocks (hereinafter referred to as head address information) and address information of the last macroblock in the row direction (hereinafter referred to as final address information) Is variable-length encoded and transmitted to the encoding control circuit 7 and the image decoding device 13.

  Thus, the encoding control circuit 7 and the image decoding device 13 can know which image encoding type of image data is skipped. Here, the designation of the start address information and the final address information for the variable length coding circuit 11 is performed by the coding control circuit 7 sending a designation signal S10 to the variable length coding circuit 11.

  Also, the encoding control circuit 7 determines that the data occupancy amount in the decoder buffer 18 is the second based on the code generation amount of the image data sent from the variable length encoding circuit 11 when the P picture is encoded. When image data having a code generation amount that falls below the threshold value is generated, a skip execution signal S9 is sent to the quantization circuit 10, and the subsequent image data of the P-picture being encoded is macro-coded. Skip in blocks.

Here, when the P-picture is skipped, the encoding control circuit 7 uses the skipped P-picture as a predicted image even if the data occupancy in the decoder buffer 18 exceeds the corresponding threshold value. As a result, all the B and P pictures used for decoding are skipped. This is because the P picture P ₃ uses the previous P picture P ₂ as a predicted image, as shown in FIG. That is, since the P picture P ₂ is skipped from the middle of encoding, the image data becomes insufficient, and the image quality of the decoded P picture P ₃ that is decoded using the P picture P ₂ is deteriorated. Because it does. The same applies to B picture B ₅ and B picture B ₆ . Therefore, the image data is skipped until the next I-picture encoding.

In FIG. 5, the data amount is reduced by the thick line even though B picture B ₃ , B picture B ₄ , P picture P ₃ , B picture B ₅ and B picture B ₆ are skipped. This is because the above-described head address information and end address information are read out. The same applies to FIG.

  Furthermore, the encoding control circuit 7 determines the buffer occupancy of the data in the decoder buffer 18 based on the amount of generated code of the image data sent from the variable length encoding circuit 11 when the I picture is encoded. When image data having a code amount lower than the threshold value of 3 is generated, a skip execution signal S9 is sent to the quantization circuit 10, and the subsequent image data of the I-picture being encoded is macroblocked. Coding is performed as a macroblock containing only DC components in units. In this case, the encoding control circuit 7 controls the quantization circuit 10 so as to skip the B picture and the P picture until encoding of the next I picture.

This is because, as shown in FIG. 6, for example, P picture P ₂ uses the previous I picture I ₁ as a predicted image. That is, since the I-picture I ₁ is coded with only the DC component from the middle of the coding, the image data is insufficient, and the decoded P-picture decoded using this I-picture I ₁ is used. image quality of the P ₂ is because the deterioration. The same applies to the B picture B ₂ , the B picture B ₅ , the B picture B _5, and the B picture B ₆ that are used for decoding the I picture I ₁ and the P picture P ₂ . Therefore, the image data is skipped until the next I-picture encoding.

  Here, the thick line shown in FIGS. 4 to 6 is composed of a skipped macro block or an intra macro block having only a DC component.

  In addition, when the image data is skipped as described above, the encoding control circuit 7 uses the target code generation amount control signal S8 so as to allocate a larger amount of target code generation for the subsequent image data by the skipped amount. Thus, the quantization circuit 10 is controlled. In this case, the encoding control circuit 7 assigns a larger amount of target code generation in the order of the image encoding types having the highest importance when decoding an image, that is, I-picture and P-picture.

  Even when the picture is skipped by the image encoding apparatus 1, the image decoding apparatus 13 can determine which macro block is skipped based on the transmitted address information, and when it is skipped by this. The previous frame is displayed as it is.

  The operation of the encoding control circuit 7 will be described with reference to FIG.

  The encoding control circuit 7 starts the operation from step SP1, and in step SP2, determines which image encoding type is the picture to be encoded (hereinafter referred to as encoding target picture). After the determination based on the image encoding type data S3 sent from the circuit 3, the process proceeds to step SP3.

  In step SP3, the encoding control circuit 7 determines, based on the image encoding type of the picture in which the skip determination flag is set (hereinafter referred to as skip picture) and the image encoding type of the encoding target picture. It is determined whether to skip the encoding target picture. Here, the skip determination flag represents a flag set for the skip picture, as will be described later.

In step SP3, it is determined whether or not the encoding target picture is skipped in order to determine whether or not the skip picture is a predicted image of the encoding target picture. That is, as described above, when the prediction image of the encoding target picture is skipped, the picture quality of the picture decoded using this prediction image is affected, so the encoding target picture is encoded. This is because it is necessary to skip even when the data occupation amount in the decoder buffer 18 does not fall below the threshold value corresponding to the image encoding type of the encoding target picture. For example, when the P picture P ₃ (FIG. 5) is the encoding target picture, the P picture P _{2 that is} a predicted image of the P picture P ₂ is skipped.

  If the encoding control circuit 7 obtains a negative result in step SP3, the process proceeds to step SP4, and the skip determination flag is reset. The reason for resetting the skip determination flag is to prevent the image data of the picture that should not be skipped from being skipped if the step determination flag remains set.

  Subsequently, the encoding control circuit 7 calculates the target code generation amount in step SP5, and then proceeds to step SP6 to encode the encoding target picture in units of macroblocks. Next, in step SP7, the encoding control circuit 7 calculates the current data occupancy in the decoder buffer 18, and then proceeds to step SP8.

  In step SP8, the encoding control circuit 7 determines whether or not the data occupation amount in the decoder buffer 18 calculated in step SP7 is below the threshold value corresponding to the encoding target picture, and obtains a negative result. Then, the process proceeds to step SP9. In step SP9, the encoding control circuit 7 determines whether or not the encoding target picture has been encoded for one frame. If a negative result is obtained, the encoding control circuit 7 returns to step SP6 and executes the processing loop from step SP6 to step SP9. Execute.

  When it is determined in step SP3 that the encoding target picture is skipped, the encoding control circuit 7 proceeds to step SP10, skips the encoding target picture except for the head address information and the final address information, and performs the encoding target. If the picture is an I picture, it is encoded as a macroblock with only a DC component. In step SP8, when the data occupancy in the decoder buffer 18 falls below the threshold value corresponding to the encoding target picture, the encoding control circuit 7 converts the image data of the encoding target picture thereafter to macro. Skip in blocks.

  Subsequently, the encoding control circuit 7 proceeds to step SP11 and determines whether or not encoding for one frame has been completed for the encoding target picture.

  If the encoding control circuit 7 obtains a negative result in step SP11, it returns to step SP10, executes the processing loop of step SP10 and step SP11, and if it obtains a negative result, it proceeds to step SP12. In step SP12, the encoding control circuit 7 sets the corresponding skip determination flag for the skipped picture, and then returns to step SP2 to repeat the above operation until all the image signal S1 is processed.

  In the above configuration, in this image encoding device 1, when the B picture is encoded based on the code generation amount of the image data from the variable length encoding circuit 11, the data occupation amount in the decoder buffer 18 is the first. When image data having a code amount that falls below the threshold value of 3 is generated, the subsequent image data of the B picture being encoded is skipped in units of macroblocks, so that the decoder buffer 18 The VBV buffer data occupation amount for the next P-picture image data is secured.

  Further, in this image encoding device 1, when the P picture is encoded based on the code generation amount of the image data from the variable length encoding circuit 11, the data occupation amount in the decoder buffer 18 is the second threshold. When image data having a code amount lower than the value is generated, subsequent image data of the P-picture being encoded is skipped in units of macroblocks, so that the I-picture image is decoded in the decoder buffer 18. The data occupation amount of the VBV buffer for data is secured.

  Further, when the P-picture is skipped, the P-picture and the B-picture having the P-picture as a predicted image are all skipped, so that the image quality can be prevented from deteriorating.

  Further, in this image encoding device 1, when the I-picture is encoded based on the amount of generated image data from the variable length encoding circuit 11, the data occupancy in the decoder buffer 18 is the first threshold. When image data having a code amount lower than the value is generated, the subsequent image data of the I-picture being encoded is skipped in units of macroblocks, so that the decoder buffer 18 uses the next I-picture. The data occupation amount of the VBV buffer for the image data is secured.

  When the I-picture is skipped, all P-pictures and B-pictures are skipped until the next I-picture is encoded, so that it is possible to prevent image quality deterioration.

  Therefore, in this image encoding apparatus 1, since the image encoding type picture having low importance when decoding the image is preferentially skipped, the decoder buffer 18 uses the next most important image. It is possible to secure the data occupation amount of the VBV buffer for the encoding type picture.

  Further, in this image coding apparatus 1, since the image coding type picture having low importance when decoding the image is skipped preferentially, the image coding type having high importance when decoding the image. It is possible to allocate a larger amount of target code generation by the amount skipped with respect to the picture, and it is possible to prevent image quality deterioration. In this case, it is particularly effective when the pattern of the image is complicated.

  Further, the freeze state of the image on the image decoding device 13 side can be controlled on the encoding device 1 side.

  According to the above configuration, as the threshold value of the data occupation amount in the VBV buffer, the third threshold value, the second threshold value, and the first threshold value corresponding to the B picture, the P picture, and the I picture, respectively. The threshold value is set, and the third, second, and first threshold values are set so that the threshold value increases in the order of the third threshold value, the second threshold value, and the first threshold value. By setting the threshold value, a picture encoding type picture having a low importance when decoding an image is preferentially skipped.

  As a result, in the VBV buffer, it is possible to preferentially secure the data occupancy amount of the image coding type picture that is highly important when decoding the image. As a result, the VBV buffer does not deteriorate the image quality. It is possible to realize an image encoding device that can reliably prevent occurrence of underflow.

  In the first embodiment described above, the case where the Huffman code is used as the encoding method in the variable length encoding circuit 11 is described. However, the present invention is not limited to this, and the code in the variable length encoding circuit 11 is used. Various other encoding schemes can be applied as the conversion.

  In the first embodiment described above, for example, a case where 15 frames are processed as a group of pictures (GOP) and processed as one unit has been described. However, the present invention is not limited to this. The GOP may be composed of various other numbers as the number of frames constituting the GOP.

  Furthermore, in the first embodiment described above, the case where each image is encoded in units of frames has been described. However, the present invention is not limited to this, and encoding is performed in units of fields or a mixture of units of frames and fields. You may make it do. In other words, a non-interlaced scanning method may be supported.

  Furthermore, in the above-described first embodiment, the image encoding rearrangement circuit 3 and the encoding control circuit 7 are used as image encoding type determination means for determining the image encoding type of the image to be encoded. Although the case has been described, the present invention is not limited to this, and various other image coding type determination means can be applied as the image coding type determination means.

  Further, in the above-described first embodiment, the image coding type designation circuit 2 is used as coding means for coding the first, second, or third image based on the output of the image coding type determination means. , Image coding order rearrangement circuit 3, scan converter 4, prediction mode determination circuit 5, motion vector detection circuit 6, coding control circuit 7, arithmetic circuit 8, DCT conversion circuit 9, quantization circuit 10, variable length coding Although the case where the circuit 11, the inverse quantization circuit 12, the inverse DCT conversion circuit 14, the arithmetic unit 15, the frame memory 16 and the motion compensation circuit 17 are used has been described, the present invention is not limited to this, and other encoding means may be used. Various first encoding means may be applied.

  Furthermore, in the first embodiment described above, the code generation amount of the encoded image is calculated, and the data occupation amount of the encoded image in the buffer of the image decoding device is calculated based on the calculation result. Of the first, second and third thresholds of the data occupation amount set in correspondence with the second and third images, respectively, the first, second or third currently encoded When a code generation amount that falls below the first, second, or third threshold corresponding to the image is generated, the encoding process for the first, second, or third image that is currently encoded is stopped. As described above, the case where the encoding control circuit 7 is used as the encoding control means for controlling the encoding means has been described. However, the present invention is not limited to this, and various other encoding controls can be used as the encoding control means. Means can be applied.

  Furthermore, in the first embodiment described above, image data is encoded into images of three encoding types, that is, I-picture, P-picture, and B-picture. However, the present invention is not limited to this. Even if an image is encoded using two encoding types out of the two encoding types, i.e., I-picture and P-picture, or I-picture and B-picture, the same as in the first embodiment described above. An effect can be obtained.

  Incidentally, the first threshold value corresponding to the I picture is set as the threshold value of the data occupation amount in the VBV buffer, and the second or third threshold value corresponding to the P picture or B picture is set as the second threshold value. By setting a threshold value greater than the threshold value of 1, it is possible to preferentially skip P-pictures or B-pictures that are less important than I-pictures when decoding an image.

  As a result, in the VBV buffer, it is possible to preferentially secure the data occupancy amount of the image coding type picture that is highly important when decoding the image. As a result, the VBV buffer does not deteriorate the image quality. It is possible to realize an image encoding device that can reliably prevent occurrence of underflow.

(2) Second Embodiment In FIG. 8, the image encoding device 25 performs processing such as preprocessing for compression encoding on the input image signal S25 in the encoder control unit 26, and then a FIFO (first-in first-out) memory. In 27, the output data is delayed by a predetermined time. The output data of the FIFO memory 12 is compressed and encoded by an encoder 28 as an encoding means for each picture by an image encoding type, that is, an encoding method corresponding to the picture type, and output as compressed image data S26. Is done.

  The motion vector detected by the motion detection circuit 14 based on the output data of the encoder control unit 11 is given to the encoder 28, and thus the generated bit amount data S 27 output from the encoder 13 and the output from the encoder control unit 26 are output. Is controlled by the encoding control unit 31 based on the intra AC data S28 to be performed and the ME residual data S29 output from the motion detection circuit 30.

  The image encoding device 25 shown in FIG. 8 is configured to perform feedforward type rate control that controls the amount of generated code based on the characteristics of image data before compression encoding.

  Here, the ME residual is the sum of absolute values or square sums of motion prediction errors for the entire picture, and the ME residual data S29 is data for obtaining the ME residual.

  The encoding control unit 31 is configured by a computer having a CPU (central processing unit) 33, a ROM (read only memory) 34, and a RAM (random access memory) 35 connected to each other via a bus 32. Each function in the encoding control unit 31 is realized by executing a program stored in the ROM 34 using the RAM 35 as a working area. The ROM 34 stores the image encoding control program according to the present invention, and may be an IC (integrated circuit) or a storage device using a magnetic disk such as a hard disk or a floppy disk as a recording medium, A storage device using an optical disk such as a CD (Compact Disk) -ROM or a storage device using another type of recording medium may be used. In any case, the IC and various recording media correspond to the media on which the image encoding control program according to the present invention is recorded.

  As shown in FIG. 9, the encoder control unit 26 receives an input image signal S25, and an image rearrangement circuit 41 that rearranges the order of pictures (I picture, P picture, B picture) according to the order of encoding. Then, the output data of the image rearrangement circuit 41 is inputted, it is discriminated whether it is a frame structure or a field structure, a scan conversion / macroblocking circuit 42 for performing a scan conversion and a macroblocking of 16 × 16 pixels according to the discrimination result Then, the output data of the scan conversion / macroblocking circuit 42 is input, the intra AC in the I-picture is calculated, the intra AC data S28 is sent to the encoding control unit 31, and the scan conversion / macroblocking circuit 42 An intra AV arithmetic circuit 43 that sends output data to the FIFO memory 27 and the motion detection circuit 30;

  Here, the intra AC is the sum of the absolute values of the differences between the pixel value of each pixel in the DCT transform (discrete cosine transform) block of 8 × 8 pixels and the average value of the pixel values in the DCT transform block in the I picture. And represents the complexity of the pattern.

  The encoder 28 takes the difference between the output data of the FIFO memory 27 and the predicted pixel data in the subtraction circuit 44, and performs DCT conversion on the output data of the subtraction circuit 44 in units of DCT conversion blocks in the DCT conversion circuit 45. As a result, the DCT transform coefficient is output, and the output data of the DCT transform circuit 45 is quantized by the quantization circuit 46 and then variable length coded by the variable length coding circuit 47. The output data of the variable length encoding circuit 47 is temporarily held by the buffer memory 48, and is output as compressed image data S26 composed of a bit stream having a constant bit rate.

  The output data of the quantization circuit 46 is inversely quantized by the inverse quantization circuit 49, and the inverse DCT conversion circuit 50 performs inverse DCT conversion on the output data of the inverse quantization circuit 49. The output data of the inverse DCT conversion circuit 50 is added to the predicted image data by the adder circuit 51, the output data of the adder circuit 51 is held in the motion compensation circuit 52, and the motion vector sent from the motion detection circuit 14 is stored. Accordingly, motion compensation is performed and the predicted image data is output to the subtraction circuit 44 and the addition circuit 51.

  The buffer memory 48 is configured to send generated bit amount data S27 representing the bit amount generated by the variable length encoding circuit 47 to the encoding control unit 31.

  Based on the output data of the encoder control unit 26, the motion detection circuit 30 calculates the absolute value of the pixel value difference between the target macroblock of the picture to be compression-encoded and the target macroblock in the referenced picture. A macro block that minimizes the sum or the sum of squares is searched for, and a motion vector is detected and sent to the motion compensation circuit 52. Further, when obtaining the motion vector, the motion detection circuit 30 sends the absolute value sum or the square sum of the pixel value differences between the macroblocks that have been minimized to the encoding control unit 31 as the ME residual data S29.

  The encoding control unit 31 calculates, in the ME residual calculation unit 61, an ME residual that is a value obtained by adding the ME residual data S29 from the motion detection circuit 30 for the entire picture in the ME residual calculation unit 61. Therefore, based on the ME residual calculated in this way and the intra AC data S28 from the intra AC arithmetic circuit 43, the encoding difficulty level calculation unit 42 calculates the encoding difficulty level representing the difficulty level of the picture encoding. To do. The generated bit amount data S27 from the buffer memory 48 is a virtual data corresponding to the input buffer in the image decoding apparatus that decompresses the image data compressed and encoded by the image encoding apparatus 28 according to the second embodiment. The VBV buffer occupancy calculation unit 62 calculates the data occupancy of the VBV buffer that is a buffer. Note that the ME residual represents the speed of motion of the video and the complexity of the pattern.

  The encoding control unit 31 further sets the target based on the encoding difficulty calculated by the encoding difficulty calculating unit 63 and the VBV buffer occupancy calculated by the VBV buffer occupancy calculating unit 62. A code amount is determined, and a target code amount determination unit 64 that determines whether or not to skip the encoding process for an image in order to prevent underflow of the VBV buffer is included. The quantization index determining unit 65 determines a quantization index corresponding to the quantization characteristic value in the quantization circuit 46 so that the quantization amount becomes the target code amount determined by the target code amount determining unit 64. Send to circuit 46. Thus, when the target code amount determination unit 64 determines that skipping is to be performed, the skip control unit 66 controls the quantization circuit 46 and the variable length coding circuit 47 to perform skipping.

  Here, the encoding difficulty level will be described. The encoding difficulty level represents the difficulty level of picture encoding, which can be rephrased as a ratio of the amount of data necessary for maintaining the same image quality. There are various methods for quantifying the encoding difficulty. In the second embodiment, the I difficulty is obtained by using the intra AC for the I picture, and the ME residual is obtained for the P and B pictures. Is used to obtain the encoding difficulty level. As described above, the intra AC represents the complexity of the picture, and the ME residual represents the speed of motion of the video and the complexity of the picture, and these have a strong correlation with the difficulty of encoding. The encoding difficulty level can be calculated from the intra AC or the ME residual by a linear function or the like using the ME residual as a variable.

  Next, a method for determining the risk of underflow of the VBV buffer based on the encoding difficulty will be described. As described above, the underflow of the VBV buffer occurs when the generated code amount greatly exceeds the target code amount. Such a case occurs when the difficulty of encoding the input image is very large with respect to the bit rate. Therefore, as a parameter for determining the risk of underflow of the VBV buffer, it is appropriate to use the ratio of the encoding difficulty level to the bit rate. Further, in order to determine the risk of underflow of the VBV buffer, it is important to look at the ratio of the encoding difficulty level to the bit rate in the number of pictures to be encoded from now on. This is because the encoding difficulty varies depending on the picture type. Therefore, in the second embodiment, a parameter x defined by the following equation is used as a parameter for determining the risk of underflow of the VBV buffer.

x = ΣD _k / G (1)

Here, D _k represents the coding difficulty of the picture k, k represents the coding order of the picture, and the picture to be coded next is k = 1. Σ means the sum from k = 1 to N (N is the number of pictures for 1 GOP (group of pictures)). G is the following formula:

  G = ([bit rate] × N) / [picture rate] (2)

Defined by

In the equation (2), [bit rate] represents the data amount (bit amount) per second determined based on the transmission capacity of the communication line and the recording capacity of the recording medium, and [picture rate] represents the picture amount per second. (For example, 30 in the NTSC zone and 25 in the PAL zone). Accordingly, G represents the amount of data (bit amount) assigned to the time corresponding to N pictures, and x represents N with respect to the data amount (bit quantity) assigned to the time corresponding to N pictures. This represents the ratio of the sum of the picture difficulty levels _Dk for the number of pictures.

  Note that it is not always necessary to obtain x by using N pictures, and x may be obtained by using pictures larger or smaller than N pictures. However, it should be noted that if the number is much larger than N, the accuracy of determination of the risk of underflow decreases, and if the number is smaller than N, the variation of the parameter x increases.

  Next, the operation of the image encoding device 25 according to the second embodiment will be described. The input image signal S25 is input to the encoder control unit 26. In the encoder control unit 26, first, the image rearrangement circuit 41 rearranges the order of pictures (I-picture, P-picture, B-picture) according to the order of encoding, and then scan conversion / macroblocking. The circuit 42 determines whether the frame structure or the field structure, scan conversion and macroblocking are performed according to the determination result. Next, in the case of I-picture, the intra AC arithmetic circuit 43 performs the intra AC calculation circuit 43. And the intra AC data S28 is sent to the encoding control unit 31. The output data of the scan conversion / macroblocking circuit 42 is sent to the FIFO memory 27 and the motion detection circuit 30 via the intra AC arithmetic circuit 43.

  The FIFO memory 27 delays the input image data by a time required for the encoding difficulty level calculation unit 63 to calculate the encoding difficulty level of the N pictures following the encoded pixels. To the encoder 28. The motion detection circuit 30 detects the motion vector and sends it to the motion compensation circuit 52, and sends the ME residual data S29 to the ME residual calculation unit 61.

  In the case of I-picture, the encoder 28 inputs the output data of the FIFO memory 27 as it is to the DCT conversion circuit 45 and performs DCT conversion without taking the difference from the predicted image data in the subtraction circuit 44, and performs the quantization circuit. 46, the DCT transform coefficient is quantized, the variable length coding circuit 47 variable length codes the output data of the quantization circuit 46, and the buffer memory 48 temporarily holds the output data of the variable length coding circuit 47. Then, it is output as compressed image data S26 composed of a bit stream having a constant bit rate.

  Further, the output data of the quantization circuit 46 is inversely quantized by the inverse quantization circuit 49, the output data of the inverse quantization circuit 49 is inversely DCT transformed by the inverse DCT transformation circuit 50, and the inverse DCT transformation is performed. The output image data of the circuit 50 is input to the motion compensation circuit 52 via the adding circuit 51 and held.

  In the case of the P-picture, the encoder 28 predicts the predicted image based on the stored image data corresponding to the past I-picture or P-picture and the motion vector from the motion detection circuit 30 by the motion compensation circuit 52. Data is generated and the predicted image data is output to the subtraction circuit 44 and the addition circuit 51. Further, the difference between the output data of the FIFO memory 27 and the predicted image data from the motion compensation circuit 52 is obtained by the subtraction circuit 44, DCT conversion is performed by the DCT conversion circuit 45, and the quantization circuit 46 is used. The DCT transform coefficient is quantized, the output data of the quantization circuit 46 is variable-length encoded by the variable-length encoding circuit 47, and the output data of the variable-length encoding circuit 47 is temporarily held by the buffer memory 48, and the compressed image data Output as S26. Further, the output data of the quantization circuit 46 is inversely quantized by the inverse quantization circuit 49, the output data of the inverse quantization circuit 49 is inversely DCT transformed by the inverse DCT transformation circuit 50, and the addition circuit 51 Therefore, the output data of the inverse DCT conversion circuit 50 and the predicted image data are added and input to the motion compensation circuit 52 to be held.

  In the case of the B-picture, the encoder 28 causes the motion compensation circuit 52 to use the two image data corresponding to the past and future I-picture or P-picture and the two motion vectors from the motion detection circuit 30. Based on the above, predicted image data is generated, and the predicted image data is output to the subtraction circuit 44 and the addition circuit 51. Further, the difference between the output data of the FIFO memory 27 and the predicted image data from the motion compensation circuit 52 is obtained by the subtraction circuit 44, DCT conversion is performed by the DCT conversion circuit 45, and the quantization circuit 46 is used. The DCT transform coefficient is quantized, the output data of the quantization circuit 46 is variable-length encoded by the variable-length encoding circuit 47, the output data of the variable-length encoding circuit 47 is temporarily held by the buffer memory 48, and the compressed image Output as data S26. The B picture is not held in the motion compensation circuit 52.

  The buffer memory 48 sends the generated bit amount data S27 representing the bit amount generated by the variable length encoding circuit 47 to the encoding control unit 31.

  Next, the operation of the encoding control unit 31 will be described with reference to the flowchart of FIG. In this operation, first, the encoding difficulty level calculation unit 63 calculates the encoding difficulty level from the intra AC data S28 from the intra AC calculation circuit 43 and the ME residual value calculated by the ME residual value calculation unit 61. The parameter x is obtained (step SP21). The parameter x is calculated based on the encoding difficulty level of N pictures following the picture to be encoded from now.

  Next, it is determined whether or not the skip flag is “1” (step SP22). When the skip flag is “1”, it indicates that the picture scheduled to be encoded is skipped. When the skip flag is “0”, it indicates that the picture scheduled to be encoded is not skipped.

  When the skip flag is “0” (when a negative result is obtained in step SP22), the target code amount determination unit 64 calculates the parameter x and the VBV buffer occupancy calculation unit 62 and VBV Based on the buffer occupancy, the target code amount of the picture j is calculated (step SP23) and sent to the quantization index determining unit 65. Note that the picture j means a picture to be encoded from now.

  Next, the quantization index determination unit 65 corresponds to the quantization specific value in the quantization circuit 46 so that the generated code amount in the encoder 28 becomes the target code amount determined by the target code amount determination unit 64. The quantization index to be determined is determined and sent to the quantization circuit 46. In response to this, encoding of picture j is performed (step SP24).

  On the other hand, when the skip flag is “1” (when a positive result is obtained at step SP22), the target code amount determination unit 64 instructs the skip control unit 66 to skip the picture j, and accordingly The skip controller 66 performs processing so as to display the previous frame (step SP25). That is, the skip control unit 66 controls the quantization circuit 46 to perform the skip in units of macro blocks, and sends the address information of the macro block to be skipped to the variable length coding circuit 47. In response to this, the variable length encoding circuit 47 encodes and outputs information for skipping. In this way, picture j is encoded as a picture composed of skip macroblocks, which are skipped macroblocks.

When step SP24 or step SP25 is completed, Yotsute the VBV buffer occupancy calculator 62 calculates the fullness _{B j} of the VBV buffer from the generation bit amount by step SP24 or step SP25 (step SP26). The VBV buffer occupancy B _j is obtained by subtracting the bit amount generated by the picture j from the VBV buffer occupancy before encoding the picture j and adding the bit quantity accumulated in the VBV buffer during the time between the pictures. Can be obtained. Next, the picture type of the next picture is determined (step SP27). Picture type is the case of B-picture, fullness _{B j} of the VBV buffer is determined whether occupancy of determining threshold value Th (B) is smaller than for the B picture (step SP28). Picture type is the case of P-picture, fullness _{B j} of the VBV buffer to determine whether the occupancy of determining threshold value Th (P) is smaller than for the P picture (step SP29). Picture type is the case of I-picture, fullness _{B j} of the VBV buffer to determine whether the occupancy of determining threshold value Th (I) is less than for an I-picture (step SP30). Note that the occupancy determination threshold is set such that Th (B)> Th (P)> Th (I).

When B _j is not smaller than Th (B) (when a negative result is obtained at step SP28), when B _j is not smaller than Th (P) (when a negative result is obtained at step SP29), or when B _j is If it is not smaller than Th (I) (when a negative result is obtained in step SP30), it is not necessary to skip the next picture, so the skip flag is set to "0" (step SP31).

When B _j is smaller than Th (B) (when a positive result is obtained at step SP28), when B _j is smaller than Th (P) (when a positive result is obtained at step SP29) or when B _j is Th ( If it is smaller than I) (if a positive result is obtained in step SP30), it is determined whether or not the parameter x exceeds a predetermined coding difficulty level threshold Th (x) (step SP32). . Note that the threshold value Th (x) is a threshold value for determining whether or not the picture is so difficult that the generated code amount greatly exceeds the target code amount. If x does not exceed Th (x) (N), it means that the pattern is not so difficult in the future, and it is not necessary to skip the next picture, so the skip flag is set to “0” (step SP31). Conversely, when x exceeds Th (x) (when an affirmative result is obtained), it means that a difficult pattern will continue in the future. In this case, there is a high risk of underflow of VBV buffer. Judgment is made and the skip flag is set to "1" to skip the next picture (step SP33).

  When the skip flag setting (step SP31, step SP33) is completed, j + 1 is set as a new j (step SP34) and the process returns to the main routine (step SP35) for the next picture processing.

  As described above, according to the image encoding device 25 according to the second embodiment, the VBV buffer occupancy determination threshold value is set for each picture type, and priority is given in the order of B picture, P picture, and I picture. Since it is determined whether to skip or not based on the encoding difficulty level of several pictures following the encoded picture, the design of the input image becomes difficult. When the underflow of the VBV buffer is likely to occur, the B picture that has little influence on the image quality is skipped preferentially. When the underflow of the VBV buffer does not actually occur, the picture skip is not performed. VBV buffer underflow can be prevented while minimizing the effect on image quality. .

  The present invention is not limited to the second embodiment described above. For example, when determining whether or not to perform skipping based on the degree of difficulty of encoding, in addition to the parameter x used in the embodiment. In addition, it is possible to further improve the accuracy of predicting the difficulty of the pattern of the input image in the future by determining whether the coding difficulty level is increasing or decreasing. The determination of whether or not to perform can be performed more appropriately. That is, for example, if the encoding difficulty level is currently decreasing but the encoding difficulty level is decreasing, the underflow of the VBV buffer may not occur even if skipping is not performed. By avoiding skipping, it is possible to minimize skipping. Note that the information of the change tendency of the encoding difficulty level can be obtained, for example, by linearly approximating the encoding difficulty level obtained in time series by the least square method or the like and by using the slope. In this case, the information of the change tendency of the encoding difficulty level is obtained by, for example, the encoding difficulty level calculation unit 63 in FIG.

  Further, in the second embodiment, as shown in FIG. 8, the amount of generated code is controlled based on the characteristics of image data before compression encoding without providing another encoder in addition to the encoder 28. However, in the present invention, a first bus encoder different from the encoder 28 is provided to estimate the amount of data after compression encoding. The feed forward type rate control for controlling the generated code amount in the encoder 28 of the second bus based on the data amount estimated by the compression encoding by the second encoder is also used. Can be applied. In this case, it is possible to obtain the encoding difficulty level based on the generated code amount obtained by compression encoding with the encoder of the first bus.

  Furthermore, the present invention is not limited to a configuration that performs feedforward type rate control, and predicts the future difficulty of encoding a picture based on the difficulty of encoding a picture that has already been compression-encoded. Of the generated code amount based on the generated code amount obtained by compression encoding in the past, as represented by TM5 (test model 5: ISO / IEC JTC / SC29 (1993)), which is a well-known compression algorithm. The present invention can also be applied to a configuration that performs feedback type rate control for performing control. In this case, for example, the tendency of the global complexity of the pictures that have been compression-encoded in the past is analyzed, and the difficulty of encoding the future several pictures is predicted by linear approximation or the like, The value of the parameter x used in the embodiment is obtained based on the predicted encoding difficulty level, and it is sufficient to determine whether to perform skipping based on the value of the parameter x as in the embodiment. .

  Global complexity is a parameter that indicates the complexity of the screen and correlates with the difficulty of encoding. Specifically, the amount of code generated during compression encoding of pictures and the compression encoding of pictures. It is obtained as the product of the average quantization scale code (quantization characteristic value) at the time (for example, see page 111 of “The Television Society of Japan Multimedia Selection MPEG” published by Ohm).

  In addition, in a configuration that performs feedforward type rate control, a change tendency of the encoding difficulty level is obtained by linear approximation, etc., and in a configuration that performs feedback type rate control, the number of When predicting the coding difficulty of a picture for a number of sheets, the continuity of the coding difficulty is lost during scene change. Therefore, during scene change, the process for determining the changing tendency of the coding difficulty and the prediction of the encoding difficulty are predicted. If the processing to be completed is completed before the scene change and a new process is performed after the scene change, the accuracy is further improved.

  In addition, the encoding difficulty level is not limited to the one using the intra AC, the ME residual, or the like described in the embodiment, but may be another parameter as long as the encoding difficulty level is expressed.

  In the second embodiment, the VBV buffer occupancy determination threshold is changed for each picture type. However, in the present invention, the VBV buffer occupancy determination threshold is constant regardless of the picture type. Including some cases.

  The present invention relates to a communication system that compresses and transmits image data on the transmission side, and decompresses the compressed image data on the reception side, and compresses and records image data, and stores the compressed image data during reproduction. The present invention can be used for a compressed image recording / reproducing apparatus that outputs an expanded image.

It is a basic diagram with which it uses for description of transition of the data occupation amount in VBV buffer. It is a basic diagram with which it uses for description of the underflow in VBV buffer. It is a block diagram which shows schematic structure of the image coding apparatus and image decoding apparatus which are the 1st Embodiment of this invention. It is a basic diagram with which it uses for description of transition of the data occupation amount when the data occupation amount in VBV buffer is less than the 1st threshold value. It is a basic diagram with which it uses for description of transition of the data occupation amount when the data occupation amount in VBV buffer is less than a 2nd threshold value. It is a basic diagram with which it uses for description of transition of the data occupation amount when the data occupation amount in VBV buffer is less than the 3rd threshold value. 3 is a flowchart for explaining the operation of the encoding control circuit. It is a block diagram which shows the structure of the outline of the image coding apparatus which is the 2nd Embodiment of this invention. It is a block diagram which shows the detailed structure of the image coding apparatus of FIG. 10 is a flowchart for explaining the operation of the encoding control unit in FIG. 9.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Image coding apparatus, 2 ... Image coding type designation circuit, 3 ... Image coding order rearrangement circuit, 4 ... Scan converter, 5 ... Prediction mode determination circuit, 6 ... Motion vector detection circuit , 7... Encoding control circuit, 8... Arithmetic circuit, 9... DCT conversion circuit, 10... Quantization circuit, 11. Decoding device, 14 ... inverse DCT conversion circuit, 15 ... arithmetic unit, 16 ... frame memory, 17 ... motion compensation circuit, 18 ... decoder buffer, 19 ... decoding unit, 26 ... encoder control unit, 27... FIFO memory, 28... Encoder, 30... Motion detection circuit, 31... Encoding control unit, 33... CPU, 34. ... Subtraction circuit, 45 ... D T conversion circuit 46... Quantization circuit 47. Variable length coding circuit 48. Buffer memory 49. Inverse quantization circuit 50. Inverse DCT conversion circuit 52 52 Motion compensation circuit 61. ... ME residual calculation unit, 62... VBV buffer occupancy calculation unit, 63... Encoding difficulty level calculation unit, 64... Target code amount determination unit, 65. Department.

Claims (10)

In an image encoding device that generates encoded data by encoding image data,
Encoding means for encoding the image data;
It is currently executed when the ratio of the sum of the difficulty levels of encoding of a plurality of pictures following the picture for which the encoding process has been completed by the encoding means and the bit rate assigned to the plurality of pictures is greater than a threshold value. An image coding apparatus comprising: control means for controlling the coding means so as to stop the coding processing. The control means controls the encoding means so that when the value of the parameter x is larger than the threshold value, the picture to be encoded next is encoded as an image composed of skip macroblocks. The image encoding device according to claim 1.
x = ΣDk / G,
G = ([bit rate] × N) / [picture rate])
Σ Dk: Sum of encoding difficulty levels of multiple pictures N: Number of pictures to be encoded The image encoding device according to claim 1, further comprising an encoding difficulty level calculating unit that calculates the encoding difficulty level. An occupancy calculation means for calculating the data occupancy of the virtual buffer corresponding to the input buffer of the decoding means for decoding the encoded data;
When the data occupancy calculated by the occupancy calculation means is smaller than the threshold, the control means encodes the picture to be encoded next as an image composed of skip macroblocks. The image encoding apparatus according to claim 1, wherein the encoding unit is controlled. The image encoding device according to claim 3, wherein the encoding difficulty level calculation unit calculates the encoding difficulty level based on image data before being encoded by the encoding unit. 6. The encoding difficulty level calculating unit calculates the encoding difficulty level based on a flatness or an intra AC calculated from a picture of the image data encoded by the encoding unit as an I-picture. Encoding device. 6. The encoding difficulty level calculating unit calculates the encoding difficulty level based on an ME residual calculated from a picture of the image data encoded by the encoding unit as a P picture or a B picture. The encoding device described in 1. The image encoding apparatus according to claim 1, wherein when the change tendency of the encoding difficulty level is decreasing, the encoding means is controlled to perform encoding processing without performing skipping. In an image encoding method for generating encoded data by encoding image data,
An encoding step for encoding the image data;
If the ratio of the sum of the coding difficulty levels of a plurality of pictures following the picture for which the above coding process has been completed and the bit rate assigned to the plurality of pictures is greater than a threshold value, the picture to be coded next is scanned. An image encoding method comprising: a control step for controlling the encoding process so as to perform encoding processing as an image composed of chip macroblocks. An encoding step for encoding image data;
If the ratio of the sum of the coding difficulty levels of a plurality of pictures following the picture for which the above coding process has been completed and the bit rate assigned to the plurality of pictures is greater than a threshold value, the picture to be coded next is scanned. A recording medium having recorded thereon a program for causing a computer to execute a process including a control step for controlling the encoding process so as to perform an encoding process as an image composed of chip macroblocks.

Images