JPH10285600A - Image encoder, image encoding method and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To prevent the degradation of the image quality of decoding images by detecting the change of images between screens, abandoning a prediction parameter corresponding to the detected result and performing control so as to output a prescribed basic quantization step. SOLUTION: A quantization step controller 141 calculates the basic quantization step to be the base of a quantization step for quantizing the image of an encoding object by using the prediction parameter as information obtained from already encoded image data and performing learning. In the meantime, a prediction parameter selection circuit 140 controls the quantization step controller 141 so as to abandon the prediction parameter obtained by then and output the prescribed basic quantization step in the case that the images are suddenly changed. Thus, after the images are suddenly changed, without using the prediction parameter obtained from the images before the change, the basic quantization step is obtained.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image encoding device, an image encoding method, and a recording medium, and more particularly to an image encoding device and an image encoding method for encoding image data by at least quantizing the image data. And a recording medium.

[0002]

2. Description of the Related Art FIG. 5 shows an example of the configuration of a conventional image encoding apparatus.

The input terminal 101 has, for example, a luminance component Y of 352 pixels (horizontal) × 240 pixels (vertical) and chrominance components Cb and Cr of 174 as shown in FIG.
One frame of image data digitized into pixels × 120 pixels is supplied in units of 30 frames per second.

[0004] The image data supplied to the input terminal 101 is stored in a block divider 110 via a frame memory 110 capable of temporarily storing the image data and replacing a plurality of images in a predetermined order. 111 and the motion detector 120. The block divider 111
The frame of the image data supplied from the frame memory 110 is divided into, for example, 8 × 8 pixel blocks of luminance components and chroma components Cb and Cr as shown in FIG. Here, as shown in the figure, four luminance components Y0 to Y3
Block and one corresponding chroma component C
A macroblock (MB) is composed of a total of six blocks including the blocks b and Cr.

[0005] From the block divider 111, image data is supplied to the differentiator 112 in macroblock units. The differentiator 112 calculates a difference between the image data from the block divider 111 and the inter-frame prediction image data described later, and uses the difference value as the data of a frame on which inter-frame encoding described later is performed. To the switched terminal b. Image data output from the block divider 111 is supplied to a switched terminal a of the changeover switch 113 as data of a frame on which intra-frame encoding described later is performed.

The changeover switch 113 selects one of the switched terminals a and b, and the image data supplied to the terminal selected by the switch a or b is converted into D-blocks in units of blocks.
The signal is supplied to a CT (discrete cosine transform) circuit 14. DC
The T circuit 114 converts the input image data into a DCT
And quantizes the resulting DCT coefficients into a quantizer 115
Output to The quantizer 115 quantizes the DCT coefficient from the DCT circuit 114 at a predetermined quantization step, and outputs the resulting quantized coefficient to the zigzag scan circuit 11.
6 is output.

The zigzag scan circuit 116 performs a so-called zigzag scan of the quantized coefficients in block units, for example, as shown in FIG. 8, and outputs them to a VLC (variable length coding) circuit 17 in that order. The VLC circuit 117 performs VLC processing on the quantized coefficient from the zigzag scan circuit 116 and supplies the resulting variable-length encoded data to the output buffer 118. The output buffer 118 is a VLC
By temporarily storing the variable length coded data from the circuit 117, the output data amount is smoothed and output from the output terminal 102. The data output from the output terminal 102 is recorded on, for example, a recording medium (not shown).

[0008] The output buffer 118 outputs the data storage amount to the quantization step controller 119. The quantization step controller 119 sets a quantization step based on the data accumulation amount from the output buffer 118 so that the output buffer 118 does not overflow or underflow, and outputs the result to the quantizer 115. In the quantizer 115 described above, quantization is performed according to the quantization step supplied from the quantization step controller 119 in this way.

On the other hand, the quantized coefficients output from the quantizer 115 are supplied not only to the zigzag scan circuit 116 but also to an inverse quantizer 126. The inverse quantizer 126 inversely quantizes the quantization coefficient from the quantizer 115, thereby
The T coefficient is output to the inverse DCT circuit 125. Inverse DCT
The circuit 125 performs an inverse DCT process on the DCT coefficient, and supplies the resulting data to the adder 124. Further, the adder 124 is also supplied with inter-frame predicted image data output from the motion compensator 121 via a changeover switch 123 which is turned on when processing an inter-frame encoded frame. The adder 124 adds these data and supplies the data to the frame memory 122 for storage.

The motion compensator 121 motion-compensates the data stored in the frame memory 122 in accordance with the motion vector supplied from the motion detector 120, and outputs the resulting inter-frame predicted image data to a differentiator. 1
12 and the changeover switch 123.

Next, the encoding process in the image encoding device of FIG. 5 will be further described.

In the image coding apparatus shown in FIG.
For example, a so-called MPEG (Moving Picture Exper), which is an international standardization working group for color moving picture coding,
ts Group) in accordance with the MPEG1 standard.

Here, each frame is defined as follows.

First, when the frames are arranged in the display order, the respective frames are referred to as I0, B1, B2, P3,
B4, B5, P6, B7, B8, I9, B10, B1
1, B12,... The above I, P, B are
This indicates that the frame is an I picture, a P picture, and a B picture, and the numbers following I, P, and B indicate the display order.

In MPEG, first, an image I0 is encoded. Next, although the image P3 is encoded, the difference between the image P3 and I0 is encoded instead of encoding the image P3 itself. Further, next, the image B1 is encoded. However, the image B1 itself is not encoded, but the image B1 and the average value of one or both of the images I0 and P3 or both are obtained. The difference is encoded. In this case, of the average values of the images I0, P3, or both, the one that minimizes the so-called prediction residual (the one that minimizes the amount of data obtained by encoding)
Is selected, and the image B1 and the difference are encoded.

After the picture B1 is coded, the picture B2 is coded. However, the picture B2 itself is not coded, and the picture B2 and one of the pictures I0 or P3, or The difference between the two averages is encoded. Also in this case, the one that minimizes the prediction residual among the average values of the images I0 and P3 or both is selected, and the difference between the prediction residual and the image B2 is encoded.

Thereafter, the image P6 is encoded. However, the image P6 is not encoded, but the images P6 and P6 are encoded.
3 is coded. Hereinafter, encoding is performed in a similar procedure.

Here, the correspondence between the image to be encoded and the image to be taken as a difference at that time is expressed in the order of encoding as follows:
It is shown below. Encoding order Image to be encoded Image to be taken as the difference partner (1) I0-(2) P3 I0 or P3 (3) B1 I0 or P3 (4) B2 I0 or P3 (5) P6 P3 (6) B4 P3 or P6 (7) B5 P3 or P6 (8) P9 P6 (9) B7 P6 or P9 (10) B8 P6 or P9 (11) I9-(12) P12 I9 (13) B10 I9 or P12 (14) B11 I9 or P12

As described above, the encoding order is I0, P
3, B1, B2, P6, B4, B5, P9, B7, B
8, I9, P12, B10, B11,..., Which is different from the display order. The encoded data is output in such an order.

As described above, the difference between a P picture and a B picture is usually encoded as described above. However, it is better to encode the image itself.
If the data amount is smaller than encoding the difference, the image itself is encoded.

In the image coding apparatus shown in FIG. 5, the coding process is performed as described above.

Therefore, when encoding the first image I0, the image data is read from the frame memory 110 and supplied to the block divider 111 to be divided into blocks. By the block division by the block divider 111, the image data becomes Y0 to Y3, Cb, Cr described above.
And output in that order. At the time of encoding an I picture, the changeover switch 113 has selected the switched terminal a, and therefore, the image data output from the block divider 111 is transmitted through the changeover switch 113 through the changeover switch 113.
It is supplied to the DCT circuit 114. The DCT circuit 114 performs a two-dimensional vertical / horizontal DCT process on the image data in block units supplied thereto, whereby the image data on the time axis is converted into DC data as data on the frequency axis.
It is converted to a T coefficient.

The DCT coefficient is supplied to a quantizer 115, where the DCT coefficient is quantized according to a quantization step from a quantization step controller 119 to be a quantized coefficient. This quantized coefficient is supplied to a zigzag scan circuit 116.
Then, zigzag scanning is performed as shown in FIG. Here, when the quantized coefficient, that is, the quantized DCT coefficient is zigzag scanned and output,
The later the output, the higher the DCT coefficient of the frequency component. In general, the values of the higher order DCT coefficients are small, so if a block is quantized with the same quantization step,
Of the quantized values obtained thereby, those corresponding to higher-order DCT coefficients have a higher frequency of becoming 0. As a result, when the quantized DCT coefficients are output by zigzag scanning, the frequency of later output becomes 0 frequently, and as a result, high-frequency components are cut off.

The quantized coefficients output from the zigzag scan circuit 116 are supplied to a VLC circuit 117, where they are subjected to variable-length coding such as Huffman coding. The variable-length coded data obtained as a result is temporarily stored in the output buffer 118 and then output at a constant bit rate. Therefore, the output buffer 118
Plays a role as a memory for buffering so that irregularly generated data can be output at a constant bit rate.

As described above, the I picture (Intra Pictur)
The image I0 which is e) is encoded by itself, but such encoding is performed within a frame (Intra).
Called encoding. In the decoder, the I picture is decoded according to the reverse procedure.

Next, the encoding of the second image P3 will be described. Although the second and subsequent images can be encoded as I-pictures, the compression ratio is reduced. Therefore, the second and subsequent images are encoded as follows by utilizing the fact that continuous images have a correlation.

That is, the motion detector 120 calculates the first image I for each macroblock constituting the second image P3.
From 0, a part very similar to a macroblock is detected,
A vector representing a shift in the relative positional relationship between the portion and the corresponding macroblock is detected as a motion vector. Here, a method of detecting a motion vector is described in, for example, ISO / ISC 11172-2 annex.
D. 6.2, etc., and the description is omitted here.

The block of the second image P3 is not supplied to the DCT circuit 114 as it is, but is subjected to motion compensation in accordance with the motion vector of each block, so that the block from the first image I0 is obtained. The difference from the obtained block is calculated by the difference unit 112,
It is supplied to the T circuit 114.

Here, if the correlation between the block obtained by motion-compensating the first image I0 according to the motion vector and the block of the second image P3 is high, the difference between them becomes small. The amount of data obtained as a result of encoding is smaller when the difference is encoded than when the block of the second image P3 is intra-coded.

The technique for encoding the difference in this way is called inter-frame (Inter) encoding.

It should be noted that encoding the difference does not always reduce the amount of data. Depending on the complexity of the image to be encoded and the degree of correlation with the preceding and succeeding frames, an inter-code encoding the difference may be used. The compression rate may be higher when intra coding is performed than when coding is performed. In such a case, intra coding is performed. Whether to perform intra coding or inter coding can be set for each macroblock.

By the way, in order to perform inter coding, it is necessary to obtain a decoded image obtained by a decoder.

Therefore, the image encoding device is provided with a so-called local decoder. 5, the motion compensator 121, the frame memory 122, the changeover switch 123, the adder 124, the inverse DCT circuit 125, and the inverse quantizer 126 constitute a local decoder. The image data stored in the frame memory 122 is
Local Decoded Pictur
e) or Local Decoded data
Data). On the other hand, the image data before being encoded is an original picture (Original Pictur).
e) or Original Data.

At the time of encoding the first image I0,
The output of the quantizer 115 is the inverse quantizer 126 and the inverse D
The signal is locally decoded through the CT circuit 125 (in this case, the changeover switch 123 is turned off, and as a result, the adder 124 does not substantially perform the processing) and is stored in the frame memory 122.

The image stored in the frame memory 122 is not the original picture, but the same picture as the picture obtained by encoding and then locally decoding the picture and obtained on the decoder side. Therefore, the frame memory 1
The image No. 22 has a slightly lower image quality than the original picture due to the encoding and decoding processes.

The second image P3 is a block-by-block difference from the frame memory 110 via the block divider 111 in a state where a locally decoded version of the first image I0 is stored in the frame memory 122. Is supplied to the vessel 112. Note that, by this time, the motion detector 120 must have finished detecting the motion vector of the image P3.

On the other hand, the motion detector 120 supplies the motion vector detected for the second image P3 in units of macroblocks to the motion compensator 121. Motion compensator 12
1 is already locally decoded according to the motion vector from the motion detector 120 and is stored in the frame memory 122.
The motion compensation (MC (MotionCo
mpensation)), and the resulting motion compensation data (MC data) (one macroblock) is supplied to the differentiator 112 as inter-frame prediction image data.

In the differentiator 112, the block divider 111
The difference between the corresponding pixels between the original data of the image P3 supplied via the P and the inter-frame prediction image data supplied from the motion compensator 121 is calculated. Then, the difference value obtained as a result is supplied to the DCT circuit 114 via the changeover switch 113, and is thereafter encoded in the same manner as in the case of the I picture. Therefore, in this case, the changeover switch 113 selects the switched terminal b.

As described above, the P picture (Predicted Pi
For the image P3 that is the current image, the difference between the predicted image obtained by motion-compensating the reference image and the I or P picture coded immediately before the reference image is basically coded. You.

That is, for a P picture, for a macro block (inter macro block) in which the data amount is smaller when the inter coding is performed, the switch-over terminal b is selected by the changeover switch 113 and the inter coding is performed. For a macroblock (intra-macroblock) whose data amount is smaller by intra-coding, the switch-over terminal a is selected by the changeover switch 113 and intra-coding is performed.

The intra-coded P-picture macroblocks are locally decoded in the same manner as the I-pictures and stored in the frame memory 122. In addition, the inter-coded data is obtained through an inverse quantizer 126 and an inverse DCT circuit 125,
The adder 124 adds the inter-frame prediction image data supplied via the changeover switch 123 that has been turned on, and the result is locally decoded and stored in the frame memory 122.

Next, the encoding of the third image B1 will be described.

At the time of encoding the picture B1 which is a B picture, the motion detector 120 determines whether the I picture or P picture displayed immediately before the picture B1 and the I picture or P picture displayed immediately after the picture B1. Are detected. Therefore, here
A motion vector of the image B1 with respect to each of the images I0 and P3 is detected. Here, a motion vector for an image I0, which is an I picture displayed immediately before the image B1, is a forward vector (Forward Vector), and a motion vector for an image P3, which is a P picture displayed immediately after that, is a backward vector. Bector).

As for the image B1, (1) the difference from the inter-frame prediction image data obtained by motion-compensating the locally decoded image I0 according to the forward vector, and (2) the image P3 according to the backward vector. (3) difference from inter-frame predicted image data obtained by motion compensation of decoded data, (3) difference from average value of two inter-frame predicted image data obtained in (1) and (2), (4) ) Image B1 itself,
Of the four, the one with the smallest data amount is selected and encoded.

When any of the data (1) to (3) is encoded (when inter encoding is performed), the necessary motion vector is sent from the motion detector 120 to the motion compensator 121. The data supplied and obtained by performing motion compensation according to the motion vector is supplied to the differentiator 112. Then, in the differentiator 112, the original data of the image B1 and the motion compensator 121
, And the difference from the data is obtained and supplied to the DCT circuit 114 via the changeover switch 113. Therefore, in this case, the changeover switch 113 selects the switched terminal b. On the other hand, when the data of (4) is encoded (intra-encoding is performed), the data, that is, the original data of the image B1 is
3 is supplied to the DCT circuit 114. Therefore, in this case, the changeover switch 113 selects the switched terminal a.

As for the picture B1 which is a B picture, the pictures I0 and P3, which have already been coded and locally decoded at the time of coding, are stored in the frame memory, so that the above-described coding becomes possible. .

For the fourth image B2, the process of replacing B1 with B2 in the above description of encoding the image B1 is performed.

Regarding the fifth image P6, in the description of encoding the image P3, P3 is replaced by P6,
A process is performed in which 0 is replaced with P3.

The above-mentioned image is repeated for the sixth and subsequent images, and the description is omitted.

In MPEG, a group of several pictures is defined as a GOP (Group Of Picture).
Is composed of pictures in which the sequence of data after encoding is continuous.

Since the GOP is defined in consideration of random access, the encoded data located at the beginning of the GOP must be an I picture. Furthermore, the last one of the pictures constituting the GOP must be an I or P picture.

FIG. 9 shows a configuration example of a GOP. In the figure, the first GOP is composed of four pictures, and subsequent GOPs are composed of six pictures. Also, FIG. 1A shows pictures constituting a GOP in display order (display order), and FIG. 2B shows pictures in encoding order.

For example, in FIG. 9, the second GOP
(GOP2), B4 and B5 are P3 and I6
Is encoded as a reference image, if random access to the first encoded I6 is performed in GOP2, subsequent B4 and B5 will not be normally performed because P3 in GOP1 has not been decoded. Can't decode. As described above, a GOP having a picture that cannot be normally decoded only by the pictures included therein is called an open GOP. On the other hand, if B4 and B5 are coded using only I6 as the reference image, when random access to I6 is performed, subsequent B4 and B5 can be decoded normally. A GOP that can be normally decoded only by the pictures contained therein is called a closed GOP.

By the way, in the image coding apparatus of FIG. 5, the picture of each screen is converted to any picture type (Picture Picture) of I picture, P picture or B picture.
Type) and what macro block type (Macro Block Type) to encode the macroblock of each picture is basically selected based on the amount of data generated as a result of the encoding. , The amount of data depends on the image to be encoded,
The exact value is not known.

However, in general, the bit rate of the bit stream as a coding result output from the image coding apparatus needs to be constant. As a method for this, for example, a quantization step (quantization Scale). That is, if the quantization step is increased, coarse quantization is performed, and the data amount (the generated code amount) can be reduced. Further, if the quantization step is reduced, fine quantization is performed, and the amount of generated codes can be increased.

The control of the quantization step is specifically performed, for example, as follows.

That is, in the image coding apparatus, an output buffer 118 is provided at the output stage, and temporarily stores the coded data to absorb a certain amount of change in the generated code amount. , The bit rate of the output bit stream can be made constant.

However, if encoded data (variable-length encoded data) continues to be generated at a rate exceeding a predetermined bit rate, the amount of data stored in the output buffer 118 increases and overflows. Conversely, if the generation of encoded data continues at a rate below the predetermined bit rate, the amount of data stored in the output buffer 118 will decrease, resulting in underflow.

Therefore, as described above, the output buffer 1
18 is fed back to the quantization step controller 119, and the quantization step controller 1
At 19, the quantization step is controlled based on the data storage amount so that neither overflow nor underflow occurs in the output buffer 118.

That is, the quantization step controller 119 increases the quantization step when the amount of data stored in the output buffer 118 approaches the capacity and is likely to overflow, thereby reducing the amount of generated code. Also,
When the amount of data stored in the output buffer 118 is close to 0 and is likely to underflow, the quantization step controller 119 decreases the quantization step, thereby increasing the amount of generated code.

Incidentally, the amount of generated codes also changes depending on whether the image is coded in a frame or between frames.

In general, when performing intra-frame encoding, a large amount of generated code is generated.
When the data storage amount of No. 8 is large, it is necessary to set a considerably large quantization step. However, in this case, even when the maximum quantization step is set, the output buffer 118 may overflow. In addition, when quantization is performed in a large quantization step, the quality of a decoded image basically deteriorates, so that the quality of an image encoded / decoded using the decoded image as a reference image also deteriorates. become. Therefore, when performing intra-frame encoding, it is necessary to secure a sufficient free space in the output buffer 118 in order to prevent the overflow of the output buffer 118 and prevent the deterioration of the image quality of the decoded image. There is.

Therefore, the quantization step controller 119
Based on the signal from the compression method selection circuit 132, the order in which intra-frame encoding and inter-frame encoding are performed is recognized in advance, and when intra-frame encoding is performed,
The quantization step is also controlled so that a sufficient free area is secured in the output buffer 118.

From the viewpoint of preventing the image quality of the decoded image from deteriorating, it is necessary to quantize a complicated image in a small quantization step and to quantize a flat image in a large quantization step. However, the quantization step set based only on the buffer feedback does not consider such a situation. If the quantization step is not an appropriate value from the viewpoint of the complexity of the image, an unreasonably large bit amount or a small bit amount is allocated to the image to be encoded. Can be If such an improper bit allocation is performed for a certain image, it also affects the bit allocation amount for another image, which is not preferable.

Therefore, in the quantization step controller 119, the quantization step is set in accordance with not only the feedback of the data accumulation amount from the buffer 118 (buffer feedback) but also the complexity of the image to be encoded. It has been made to be.

That is, in the image coding apparatus shown in FIG. 5, the picture evaluation circuit 130 reads out the picture to be coded from the frame memory 110 and calculates an evaluation value indicating the complexity of the picture. The change detection circuit 131 and the quantization step controller 119 are supplied.

A quantization step controller 119 includes a quantization step actually used for encoding an image, a data amount (generated code amount) obtained by performing quantization in the quantization step, and an image evaluation circuit. Supplied from 130,
The relationship between the evaluation values corresponding to the complexity of the image is learned, and based on the learning result, a basic quantization step that is the basis for setting the next quantization step is obtained.

That is, it corresponds to the quantization step actually used for encoding an image, the data amount (generated code amount) obtained by performing quantization in the quantization step, and the complexity of the image. Learning is performed by performing a regression analysis using the evaluation value and graphing the result of the regression analysis. Then, from the graph, an optimal basic quantization step to be used for encoding the image is predicted using the evaluation value regarding the complexity of the image to be encoded next as an argument.

Then, the quantization step controller 119
This basic quantization step is changed according to the buffer feedback, and the value is set as the quantization step.

The basic quantization step can be accurately predicted by learning, and its value takes into account the complexity of the image. By obtaining from the quantization step, it is possible to improve the image quality of the decoded image as compared with the case where the quantization step is controlled based only on the buffer feedback.

In FIG. 5, the scene change detection circuit 131 detects whether a scene change has occurred based on the evaluation value from the image evaluation circuit 130, and supplies the detection result to the compression method selection circuit 132. The compression method selection circuit 132 selects an image compression method using the evaluation value from the image evaluation circuit 130 and, if necessary, the output of the scene change detection circuit 131. In other words, the compression method selection circuit 132 determines, for example, which picture type of an I picture, a P picture, or a B picture is to be encoded,
A compression method is selected for the number of pictures constituting P, a macroblock type regarding whether a macroblock is to be intra-frame encoded or inter-frame encoded, and the like.

Upon selecting the compression method, the compression method selection circuit 132 controls the changeover switches 113 and 123 based on whether the macroblock is to be intra-frame encoded or inter-frame encoded. That is, as described above, when performing intra-frame encoding, the changeover switch 113 is switched to the switched terminal a and the changeover switch 123 is turned off. When performing inter-frame encoding, the changeover switch 113 is switched to the switched terminal b, and the changeover switch 123 is turned on.

Further, the compression method selection circuit 132 notifies the quantization step controller 119 whether to perform intra-frame coding or inter-frame coding. Based on this notification, the quantization step controller 119 recognizes the order in which the intra-frame coding and the inter-frame coding are performed, as described above.

Here, when the compression method selection circuit 132 selects encoding a picture as a P picture or a B picture continuously for a long time, the P picture and the B picture are basically Since the image is encoded between frames, when an image having low correlation between frames occurs due to a scene change or the like, the generated code amount increases,
The image quality of the decoded image deteriorates.

Therefore, as described above, a scene change detection result is supplied from the scene change detection circuit 131 to the compression method selection circuit 132, and the compression method selection circuit 132 detects the scene change. Upon receipt of the notification, the picture after the scene change is forcibly selected as an I picture.

As described above, a method of obtaining a basic quantization step by learning and setting a quantization step from the basic quantization step is described in, for example, Japanese Patent Application No. 8-108 filed by the present applicant. No. 102951,
The details are disclosed.

[0077]

As described above, the basic quantization step is, as described above, a quantization step actually used for coding an image, and data obtained by performing quantization in the quantization step. By learning the relationship between the amount (generated code amount) and the evaluation value corresponding to the complexity of the image, that is, by performing learning using information obtained from an already encoded image as a learning sample Desired.

Therefore, since learning is performed using information obtained from an image in a certain period in the past as a learning sample, if the period is short, the number of learning samples is reduced.
The prediction accuracy of the basic quantization step decreases. On the other hand, if the period is set too long, information obtained from an image that is considerably retroactive in time will be included in the learning sample, and the prediction accuracy of the basic quantization step will also deteriorate. Therefore, it is necessary to set an appropriate period for learning.

However, when an image having a low correlation with a past frame (an image that has changed sharply from the previous frame) occurs due to a scene change or the like, even if an appropriate period is set for learning, Since learning is performed using information obtained from an image having a low correlation with the image as a learning sample, the prediction accuracy of the basic quantization step deteriorates.

Further, after that, for a while, learning is performed using information obtained from an image of a past frame having low correlation, and during that time, encoding using the basic quantization step with low prediction accuracy is performed. Will be able to continue.

If the prediction accuracy of the basic quantization step is degraded, improper bit allocation is performed, and as a result, the image quality of the decoded image is degraded.

The present invention has been made in view of such circumstances, and prevents (reduces) a decrease in the prediction accuracy of the basic quantization step, thereby preventing (reducing) the deterioration of the image quality of a decoded image. )

[0083]

According to a first aspect of the present invention, there is provided an image coding apparatus comprising: a quantizing means for quantizing image data in a predetermined quantization step; and a basic for setting the predetermined quantization step. A basic quantization step that is obtained by performing learning using a prediction parameter as information obtained from image data that has already been encoded, and a basic quantization step. , A quantization step setting unit, a change detection unit that detects a change in an image between screens, and a predetermined basic quantization step is output while discarding a prediction parameter according to a detection result of the change detection unit. Control means for controlling the basic quantization step calculation means.

According to a second aspect of the present invention, there is provided an image encoding method comprising: a quantizing means for quantizing image data in a predetermined quantization step; and a basic quantization step serving as a basis for setting the predetermined quantization step. A basic quantization step calculating means obtained by performing learning by using a prediction parameter as information obtained from already encoded image data,
An image encoding method for an image encoding device, comprising: a basic quantization step; and a quantization step setting means for setting a predetermined quantization step, wherein the method detects an image change between screens and corresponds to the detection result. Then, the prediction parameter is discarded, and the basic quantization step calculating means is controlled so as to output a predetermined basic quantization step.

A recording medium according to a third aspect is characterized by recording image data encoded by the image encoding method according to the second aspect.

In the image encoding apparatus according to the first aspect, the quantization means quantizes the image data in a predetermined quantization step, and the basic quantization step calculation means sets the predetermined quantization step. Is obtained by learning using a prediction parameter as information obtained from already encoded image data. The quantization step setting means sets a predetermined quantization step from the basic quantization step, and the change detection means detects a change in an image between screens. The control means, in response to the detection result of the change detection means, to discard the prediction parameter, and to output a predetermined basic quantization step,
The basic quantization step calculating means is controlled.

According to the image coding method of the present invention, a change in an image between screens is detected, and a prediction parameter is discarded and a predetermined basic quantization step is output in accordance with the detection result. Thus, the basic quantization step calculation means is controlled.

The recording medium according to the third aspect includes the second aspect.
The image data encoded by the image encoding method described in (1) is recorded.

[0089]

FIG. 1 shows a configuration example of an embodiment of a personal computer to which the present invention is applied.

The personal computer (hereinafter, appropriately referred to as a personal computer) can directly receive images taken by the video camera 14, images received by the modem 6 or the tuner 5, images reproduced by the VTR 16, or the like. , And can be compressed and encoded by the compression unit 13 and recorded on the CD-R 11 or the hard disk 12.

That is, the microprocessor 1 executes an application program also recorded on the hard disk 12 under the control of the operating system recorded on the hard disk 12, for example, to perform image editing processing, encoding processing and other processing. Perform predetermined processing. The main memory 2 stores programs executed by the microprocessor 1 and data necessary for the operation of the microprocessor 1. The frame buffer 3 is, for example, a DRAM
(Dynamic Random Access Memory) and stores an image generated by the microprocessor 1 and the like. The bus bridge 4 includes an internal bus and, for example, a PCI (Peripher
al Component Interconnect) Controls the exchange of data with an expansion bus such as a local bus.

The above-described microprocessor 1, main memory 2, frame buffer 3, and bus bridge 4
The blocks are connected to each other via an internal bus, and the remaining blocks are connected to each other via an expansion bus. The bus bridge 4 is connected to both the internal bus and the expansion bus.

The tuner 5 receives, for example, a television broadcast signal broadcast using a terrestrial wave, a satellite line, or a CATV network. Here, images and sounds received by the tuner 5 can also be edited and encoded. The modem 6 controls communication via a telephone line. Here, in the modem 6, for example, images and sounds received from the Internet or the like can be edited or encoded, and the edited or encoded images or sounds can be transmitted to the outside. Can also.

The I / O (Input / Output) interface 7 outputs an operation signal corresponding to the operation of the keyboard 8 and the mouse 9. The keyboard 8 is used to input predetermined data or commands, and the mouse 9 is used to move a cursor displayed on a display (computer display) 17 or to indicate a position.
Each is operated.

The auxiliary storage interface 10 is a CD-ROM.
It controls reading / writing of data from / to an R (Compact Disc Recordable) 11 and a hard disk (HD (Hard Disk)) 12. For example, the compression unit 1
3 is recorded. The hard disk 12 stores an operating system, an application program for causing the microprocessor 1 to execute image encoding processing and other processing, and the like. Further, the hard disk 12 also records data compressed by the compression unit 13 and the like.

Under the control of the microprocessor 1, the compression section 13 converts the image or sound input thereto into, for example, an MP
Compression-encodes according to the EG standard. The compression unit 1
In 3, data supplied via an expansion bus, data supplied via an expansion unit 15, and data supplied from an external device such as a video camera 14 can be compressed. It has been made possible.

The video camera 14 captures an image, for example, and supplies it to the compression unit 13. Note that the compression unit 13 has an interface with the video camera 14, whereby images and sounds recorded by the video camera 14 can be input to the compression unit 13.

The expansion unit 15 decodes (expands) the data encoded (compressed) by the compression unit 13 and outputs the data. In addition,
The decompression unit 15 overlays the decoded image on the image stored in the frame buffer 3 and outputs the decoded image as necessary. Here, the supply of the image data from the frame buffer 3 to the decompression unit 15 is performed directly between them in the embodiment of FIG. 1, but the internal bus, the bus bridge 4 and the It is also possible to do via an expansion bus. However, the extension unit 1
In the case where the supply of image data to 5 is performed via the internal bus, the bus bridge 4 and the expansion bus, if the capacity of the internal bus or the expansion bus is low, data may be congested.

The VTR (video tape recorder) 16
Images and sounds output by the decompression unit 15 are recorded as necessary. The display 17 displays the image output by the extension unit 15 as necessary. The display of the image output by the expansion unit 15 can be performed not only by the display 17 which is a display for a computer but also by a TV (TeleVision) monitor or the like.

Next, FIG. 2 shows a configuration example of an embodiment of the compression section 13 of FIG. In the figure, portions corresponding to those in FIG. 5 are denoted by the same reference numerals, and a description thereof will be omitted as appropriate below. That is, the compression unit 13 is provided with a new prediction parameter selection circuit 140 (change detecting means) (control means), and further includes the quantization step controller 1 instead of the quantization step controller 119.
41 (basic quantization step calculation means) (quantization step setting means), except that it is newly provided.

Frame memory 110, block divider 1
11, differentiator 112, changeover switch 113, DCT circuit 114, quantizer 115 (quantizing means), zigzag scan circuit 116, VLC circuit 117, output buffer 11
8, motion detector 120, motion compensator 121, frame memory 122, changeover switch 123, adder 124, inverse D
In the CT circuit 125 and the inverse quantizer 126,
The image is MPEG-coded as in the case of FIG. That is, in the compression unit 13, the quantizer 1
Since the setting method of the quantization step used in 15 is different from that in FIG. 5, the following description focuses on this quantization step setting method.

In the image evaluation circuit 130, the following two parameters representing the complexity of the image are used as the evaluation values for evaluating the image to be encoded in the frame memory 110.
Is calculated by referring to.

That is, as the first parameter, it is possible to predict (estimate) the generated code amount when the image is coded in the frame (the generated code amount when the image is coded as an I picture). An evaluation value representing the information amount of the image itself is calculated. Specifically, as the first parameter, for example, an image is subjected to DCT processing for each block, and a sum of DCT coefficients and other statistics can be used. For example, for each block, the sum of the absolute values of the values obtained by subtracting the average value of the pixel values from the pixel values (hereinafter, appropriately referred to as the sum of the average absolute values) is obtained, and the sum of the sum of the average absolute values of the respective blocks Can be used as the first parameter. It should be noted that obtaining the sum of absolute values makes the circuit scale of the image evaluation circuit 130 relatively smaller than obtaining the sum of DCT coefficients, and
The processing load can be reduced.

Here, in the image evaluation circuit 130,
For example, the sum of the average absolute value sums as the first parameter is obtained as follows.

That is, for example, for a certain block S constituting an image to be encoded, the pixel value of the pixel located at the i-th position to the right and the j-th position to the right from the upper left of the block Let S _{i, j} be the mean absolute value sum MAD (Mean Absolute Diff
erence) is calculated according to the following equation (here,
For example, it is obtained for all of the luminance block and the chrominance block. However, for example, it is also possible to obtain only the luminance block.)

[0106]

(Equation 1) (1) In the expression (1), S _AVE represents an average value of the pixel values of the block S.

Then, the sum SMAD of the average absolute value sum is obtained as the first parameter according to the following equation.

SMAD = ΣMAD (2) where, in equation (2), Σ represents summation for all the blocks constituting the image.

In the image evaluation circuit 130, the expression (1)
The total sum of the average absolute value sums MAD expressed by the following formulas is also obtained for each macroblock. For example, whether each macro block is subjected to intra-frame coding or inter-frame coding (forward prediction coding, backward prediction coding, or bidirectional prediction coding) performed in the compression method selection circuit 132 It is used to determine

The second parameter is information on the difference between the image and a reference image used for inter-frame encoding, which can predict the amount of generated code when the image is inter-coded. An evaluation value representing the quantity is calculated. Specifically, as the second parameter, for example,
The sum of absolute values of the difference between the image and its predicted image (obtained by motion-compensating the reference image) (hereinafter, appropriately referred to as the sum of absolute differences) is obtained in block units, and the sum of absolute differences of each block is obtained. Can be used.

Here, the sum of absolute differences is calculated by the motion detector 12
It is obtained when a motion vector is detected at 0.
Thus, the image evaluation circuit 130 obtains, for example, the sum of the sum of absolute differences as a second parameter using the result of motion detection by the motion detector 120.

That is, for example, for the reference image,
Consider a block composed of 8 × 8 pixels in the horizontal and vertical directions, and the pixel value of the pixel at the i-th position to the right and the j-th position to the right from the upper left of the block is denoted by R _{i, j} . Further, regarding the image to be encoded, the x-axis or the y-axis is considered from the upper left to the right or downward, respectively, and the point (x,
From the upper left corner of a block where y) is the upper left pixel,
The pixel value of the pixel located at the i-th position in the right direction and the j-th position in the downward direction is represented as S _{x + i, y + j} .

In this case, the motion detector 120 obtains d (x, y) represented by the following equation by changing x and y by one.

[0114]

(Equation 2) ... (3)

Then, in the motion detector 120, the equation (3)
(X, y) that minimizes d (x, y) is detected as a motion vector, and the minimum d (x, y) is calculated as the sum of absolute differences AD.

The image evaluation circuit 130 uses the block-by-block difference absolute value sum AD obtained by the motion detector 120 as described above, and calculates the total sum SAD of the difference absolute value sums according to the following equation. Is required.

SAD = ΣAD (4) However, also in equation (4), Σ represents summation for all the blocks constituting the image.

In the image evaluation circuit 130, the expression (3)
The sum of the difference absolute value sum AD expressed by the following expression is also obtained in units of macroblocks. For example, whether each macro block is subjected to intra-frame coding or inter-frame coding (forward prediction coding, backward prediction coding, or bidirectional prediction coding) performed in the compression method selection circuit 132 It is used to determine

First parameter SMAD and second parameter SAD obtained by image evaluation circuit 130
Are supplied to a scene change detection circuit 131, a compression method selection circuit 132, a prediction parameter selection circuit 140, and a quantization step controller 141.

As described above, the scene change detection circuit 131 detects whether a scene change has occurred based on the output of the image evaluation circuit 130, and the compression method selection circuit 132 has the evaluation value from the image evaluation circuit 130 And, if necessary, a scene change detection circuit 1
Using the output of 31, an image compression method is selected.

Note that the scene change detection circuit 131 obtains, for example, the ratio between the second parameters SAD for continuous images.
It is detected whether a scene change has occurred. The compression method selection circuit 132 compares, for example, the sum of the average absolute value sum MAD and the difference absolute value sum AD supplied from the image evaluation circuit 130 in macroblock units for the P picture and the B picture. Then, it is determined whether the macroblock is to be intra-frame coded or inter-frame coded based on their magnitude relation.
That is, for a macroblock, if the sum of the average absolute value sums MAD is smaller than the sum of the differential absolute value sums AD, and therefore, it is expected that the amount of generated code will be smaller when intra-frame encoding is performed. It is selected to perform inner coding. Also, the sum of the average absolute value sums MAD is
If the sum of the absolute difference sums AD is larger than the sum of the absolute difference sums AD, and accordingly, it is expected that the generated code amount will be smaller when the interframe coding is performed, the interframe coding is selected.

On the other hand, in the prediction parameter selection circuit 140 and the quantization step controller 141, for example, the processing according to the flowchart of FIG.
The quantization step used in the quantization in the quantizer 115 is determined.

That is, when the first macro block in each frame is quantized, the quantization step controller 14
1 calculates a basic quantization step in step S1. Then, the process proceeds to step S2, where the quantization step controller 141 adjusts the basic quantization step obtained in step S1 based on the buffer feedback from the output buffer 118, and uses the adjustment result for actual quantization. It is set (determined) as a quantization step. This quantization step is supplied to the quantizer 115 and used for quantization of the macroblock.

After setting the macroblock quantization step in step S2, the process proceeds to step S3.
In the quantization step controller 141, it is determined whether or not quantization of all macroblocks constituting one frame has been completed.

Here, the compression method selection circuit 132 determines the picture type of the image to be encoded,
It also manages which macroblock in the picture is being processed, and so on.
At 41, for example, by making an inquiry to the compression method selection circuit 132, the determination processing of step S3 is performed.

If it is determined in step S3 that the quantization of all the macroblocks constituting one frame has not been completed, the process returns to step S2, and a quantization step for quantizing the next macroblock is set. You.
If it is determined in step S3 that the quantization of all the macroblocks constituting one frame has been completed, the process returns to step S1, and the basic quantization step for the next frame is calculated.

Next, referring to the flowchart of FIG.
The details of the calculation process of the basic quantization step in step S1 of FIG. 3 will be described.

In this case, first, in step S11, the prediction parameter selection circuit 140 determines whether an image is to be transmitted between frames based on the evaluation values (the first parameter SMAD and the second parameter SAD) from the image evaluation circuit 130. It is determined (detected) whether or not abruptly changed.

Here, when the image to be coded suddenly changes from the image of the previous screen (frame), the SMA
Either D or SAD, or both, change rapidly.

That is, for example, when an image displaying a state in which an object is moving in a certain direction changes to an image completely different from the image due to a scene change or the like, the SAD, which is a substantially constant value, greatly changes. In addition, for example, when an image in which a certain still image is displayed changes to an image in which a completely different still image is displayed, again,
The SMAD, which was almost constant, changes greatly.

As described above, the change of the image is determined by SMAD or S
Since this appears as a change in AD, if the amount of change exceeds a predetermined threshold, it is considered that a large image change has occurred. Thus, in step S11, it is determined whether or not the amount of change in SMAD or SAD has exceeded a predetermined threshold, thereby determining whether or not a large image change such as a scene change has occurred.

Note that even if the image does not change so much,
Since MAD and SAD change, it is desirable to set the threshold value in consideration of this.

In step S11, when it is determined that the image has not changed abruptly, that is, even if the image to be encoded is, for example, almost the same as the previous frame or not substantially the same. If the change is not so large, the prediction parameter selection circuit 140
As in the conventional case, the quantization step controller 141 performs learning using information (prediction parameters described later) obtained from an already-encoded image to calculate a basic quantization step. Control.

That is, in this case, steps S11 to S1
Proceeding to 2, the quantization step controller 141 calculates, for example, prediction parameters x and y represented by the following equations.

X = log (average_Q) y = log (generated_bit / di
ff1) (5) In equation (5), average_Q means an average value of quantization steps used for quantization of macroblocks constituting a previously encoded frame. Also, generate
The d_bit indicates the actual amount of generated code for the previously encoded frame, and is recognized based on the buffer feedback from the output buffer 118. And diff
1 represents the complexity (difficulty) of the previously encoded frame, and is obtained based on the SMAD or SAD from the image evaluation circuit 130.

After calculating the prediction parameters x and y in step S12, the process proceeds to step S13, in which the quantization step controller 141 discards the oldest prediction parameters x and y, and obtains them by the current processing in step S12. The latest predicted parameters x and y are newly stored as learning samples. That is, the quantization step controller 119 is used to store the prediction parameters x and y obtained from the previously coded images for the number n of frames appropriate for predicting the basic quantization step. When a new prediction parameter x, y is obtained, the oldest prediction parameter x, y is deleted from the memory (however, the deletion is performed only when n prediction parameters are stored in the memory). Only when x and y are stored), new prediction parameters x and y are stored.

Then, the quantization step controller 141
In step S14, a basic quantization step is calculated (predicted) by performing learning by regression analysis using the prediction parameters x and y as n learning samples stored in the memory.

That is, the quantization step controller 141 sets n
Using the prediction parameters x and y, A and B represented by the following equations are calculated.

A = (sum (xy) − (sum (x) × sum (y)) / n) / (sum (x ² ) − (sum (x)) ² / n) B = (sum (y) / nA) × sum (x) / n (6) In equation (6), sum () means the summation of the values in parentheses for n prediction parameters.

Then, the quantization step controller 141
Using A and B in equation (6), the basic quantization step Q_scal
e is calculated, for example, according to the following equation.

Q_scale = exp ((log (allocated_bit / diff2) −B) / A) (7) In Equation (7), “allocated_bit” means a bit allocation amount for the current encoding target frame. I do.
Diff2 represents the complexity of the current frame to be encoded, and is obtained based on the SMAD or SAD from the image evaluation circuit 130.

As described above, after the basic quantization step Q_scale is calculated in step S14, the process returns.

On the other hand, if it is determined in step S11 that the image has changed abruptly, the prediction parameter selection circuit 140 discards all the stored prediction parameters and outputs a predetermined basic quantization step. Next, the quantization step controller 141 is controlled.

That is, in this case, from step S11 to S1
Proceeding to 5, the prediction parameter selection circuit 140 obtains the complexity diff2 of the current frame to be coded, described in equation (7), based on the SMAD or SAD from the image evaluation circuit 130, and calculates the complexity diff2 The quantization step controller 141 is controlled so as to select the basic quantization step corresponding to.

Here, the bit allocation amount for one frame
It can be assumed that there is a relationship between the allocated_bit, the image complexity diff for that frame, and the basic quantization step Q_scale that is appropriate for quantizing the frame.

Log (allocated_bit / diff) = A × log (Q_scale) + B (8) In Expression (8), A and B are predetermined constants. A modified version of this equation (8) is the above-mentioned equation (7).

For the constants A and B in equation (8),
By performing learning using images of various complexity, an appropriate value can be obtained for each complexity, and the constants A and B obtained in advance in this way are transmitted to the quantization step controller 141. Is stored.

When the prediction parameter selection circuit 140 issues an instruction to select the basic quantization step corresponding to the complexity diff2 of the current frame to be encoded, the quantization step controller 141 Sa
Using the constants A and B corresponding to diff2, equation (7) is calculated, whereby the basic quantization step is obtained.

Therefore, when the image to be encoded changes rapidly due to a scene change or the like,
The basic quantization step corresponding to the complexity of the image to be coded is obtained instead of using the prediction parameters x and y obtained from the past coding results. It is possible to prevent the accuracy from being greatly reduced. As a result, deterioration of the image quality of the decoded image can be reduced.

After the processing in step S15, step S1
Proceeding to 6, the prediction parameter selection circuit 140 controls the quantization step controller 141 so as to discard (delete from the memory) all of the stored n prediction parameters x and y, and returns.

Thus, the prediction parameter selection circuit 14
At 0, all n prediction parameters x, y are deleted from memory. Therefore, after the image to be coded suddenly changes due to a scene change or the like,
In calculating the basic quantization step, the prediction parameters x and y obtained from the coding result before the change are not used, so that it is possible to prevent the prediction accuracy of the basic quantization step from being greatly reduced. as a result,
Deterioration of the image quality of the decoded image can also be reduced.

In the present embodiment, the image is
The encoding is performed according to the PEG standard. However, the encoding method of the image is not limited to the MPEG encoding as long as the image data is at least quantized.

Further, in the present embodiment, the prediction parameter selection circuit 140 determines whether or not a sharp change has occurred in the image based on SMAD and SAD. In addition, for example, in FIG. As shown in the figure, the scene change detection circuit 131 supplies a scene change detection result to the prediction parameter selection circuit 140. In consideration of the scene change detection result, it is determined whether a sharp change has occurred in the image. Can be determined.

Further, although not particularly mentioned in the present embodiment, the quantization step controller 141 is adapted to obtain a basic quantization step for each picture type.

[0155]

According to the image encoding apparatus of the first aspect and the image encoding method of the second aspect, a change in an image between screens is detected, and in accordance with the detection result,
Control is performed so that the prediction parameters are discarded and a predetermined basic quantization step is output. Therefore, it is possible to prevent (reduce) deterioration of the image quality of the decoded image.

The recording medium according to the third aspect includes the second aspect.
The image data encoded by the image encoding method described in (1) is recorded. Therefore, it is possible to obtain a decoded image of high quality.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration example of a personal computer according to an embodiment of the present invention.

FIG. 2 is a block diagram illustrating a configuration example of a compression unit 13 in FIG.

FIG. 3 is a flowchart for explaining processing of a prediction parameter selection circuit 140 and a quantization step controller 141 of FIG. 2;

FIG. 4 is a flowchart for explaining the process in step S1 of FIG. 3 in more detail;

FIG. 5 is a block diagram illustrating a configuration of an example of a conventional image encoding device.

FIG. 6 is a diagram showing image data input to an input terminal 101.

FIG. 7 is a diagram showing a macroblock and blocks constituting the macroblock.

FIG. 8 is a diagram showing a zigzag scan.

FIG. 9 is a diagram showing a GOP.

[Explanation of symbols]

1 microprocessor, 2 main memory, 3
Frame buffer, 4 bus bridge, 5 tuner, 6 modem, 7 I / O interface,
Reference Signs List 8 keyboard, 9 mouse, 10 auxiliary storage interface, 11 CD-R, 12 hard disk, 13 compression unit, 14 video camera, 15
Extension unit, 16 VTR, 17 computer display, 101 input terminal, 102 output terminal,
110 frame memory, 111 block divider,
112 Difference device, 113 Changeover switch, 114
DCT circuit, 115 Quantizer (quantization means), 1
16 zigzag scan circuit, 117 VLC circuit,
118 output buffer, 120 motion detector, 1
21 motion compensator, 122 frame memory, 12
3 changeover switch, 124 adder, 125 reverse D
CT circuit, 126 inverse quantizer, 130 image evaluation circuit, 131 scene change detection circuit, 132
Compression method selection circuit, 140 prediction parameter selection circuit (change detection means) (control means), 141 quantization step controller (basic quantization step calculation means) (quantization step setting means)

Claims (3)

[Claims] A quantizing means for quantizing the image data in a predetermined quantization step; and a basic quantization step which is a basis for setting the predetermined quantization step, comprising the steps of: A basic quantization step calculating means for obtaining by performing learning using a prediction parameter as information obtained from; a quantization step setting means for setting the predetermined quantization step from the basic quantization step; A change detecting means for detecting a change in an image in the calculation of the basic quantization step so as to discard the prediction parameter and output a predetermined basic quantization step in accordance with a detection result of the change detection means. Control means for controlling the means. 2. A quantization means for quantizing image data in a predetermined quantization step, and a basic quantization step which is a basis for setting the predetermined quantization step, comprising: An image comprising: a basic quantization step calculating unit that is obtained by performing learning using a prediction parameter as information obtained from the information processing unit; and a quantization step setting unit that sets the predetermined quantization step from the basic quantization step. An image encoding method for an encoding device, comprising: detecting a change in an image between screens, discarding the prediction parameter, and outputting a predetermined basic quantization step in accordance with the detection result. An image encoding method comprising controlling the basic quantization step calculating means. 3. A recording medium on which image data encoded by the image encoding method according to claim 2 is recorded.