JP2009033652A - Moving image encoding method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To perform moving image encoding with low power consumption by appropriately selecting an encoding mode. <P>SOLUTION: The present invention relates to a moving image encoding method for encoding a moving image by selecting one encoding mode from among a plurality of encoding modes, wherein the encoding mode to be selected is determined from a bit rate of an encoded motion picture and a target bit rate, and power consumption information corresponding to the respective encoding modes. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a moving image encoding method. It is suitable for power saving of a moving picture coding system that realizes high compression efficiency.

  For effective use of a digital image, it is necessary to efficiently store and transmit the digital image, and compression encoding of image data is essential. As a method for compressing image data, the following method is adopted in JPEG (Joint Photographic Experts Group), which is a standard coding method for image coding.

  First, image data to be compressed is divided into blocks, and converted into orthogonal transform coefficients composed of a DC component and an AC component using orthogonal transform for each block. A method of performing quantization to reduce the code amount for the transformed orthogonal transform coefficient and entropy-coding the quantized orthogonal transform coefficient is well known. At this time, low-frequency components that are sensitive to human visual characteristics are quantized using a small quantization value, and insensitive high-frequency components are quantized using a large quantization value so that there is little visual degradation and efficiency. In general, a method of performing quantization is generally used.

In addition, as used in MPEG-2 (Motion Picture Experts Group Phase 2), there is also an inter-frame predictive video encoding method that effectively reduces the amount of code using the correlation between video frames. Commonly used. FIG. 4 shows a conceptual diagram of inter-frame prediction. The current frame is divided into, for example, an area of 16 pixels × 16 lines, and a motion vector is searched for each area using the area as a processing target area. The reconstructed moving image data obtained by decoding the encoded moving image data is used as a reference frame, and the motion vector is based on the region at the position corresponding to the processing target region of the current frame in the reference frame. Then, the relative position of the region estimated to have the smallest code amount due to the prediction error of the orthogonal transform coefficient is obtained using the surrounding pixel group including the region as the motion search range. The code amount due to the prediction error of the orthogonal transform coefficient is estimated using block matching, and the relative position is an area of the same size as the current processing target area from the processing target area of the current frame and the motion search range (hereinafter referred to as reference). It is obtained by comparing with shifting the area.
In addition, for the block matching calculation, the sum of absolute differences (SAD) of pixels at positions corresponding to each other in the region is often used, and it is estimated that the smaller the SAD value, the higher the correlation and the smaller the code amount. .

  Also, in MPEG-2, MPEG-4, and H.264, the reference area in one reference frame in the forward direction (forward reference area) or the reference area in one reference frame in the backward direction (backward) Reference area) or the two reference areas at the same time. Then, bi-directional interframe predictive coding is defined in which difference data between a target picture and a predicted picture is encoded using an average value of two reference areas as a prediction area. In H.264, flexible prediction such as using two reference frames in the same direction is possible.

FIG. 5 shows a conceptual diagram of the bidirectional interframe predictive coding method.
In this bi-directional inter-frame prediction, motion information is generated from motion information of an already-encoded block together with a method of encoding motion vector information generally used (if a reference frame is determined, a prediction error is uniquely determined). Direct mode is defined.

FIG. 6 shows a conceptual diagram of the direct mode. In FIG. 6, MV _base indicates a motion vector of a region located at the same position as the processing target region when referring to Frame 0 that is the encoded forward frame from Frame 1 that is the encoded backward frame. . Using this MV _base , the forward motion vector MV _F and the backward motion vector MV _B of the processing target area in the direct mode are given as follows.
_{MV F = MV base × T F} / T D
MV _B = MV _F -MV _base
In the above equation, T _F and T _D represent the time interval between Frame 0 and the frame to be processed, and the time interval between Frame 0 and Frame 1, respectively.

In this way, since the direct mode can uniquely determine a motion vector from already encoded motion vectors, there is no need to perform a motion vector search. Since motion vector search is not performed, the code amount due to the orthogonal transform coefficient prediction error will generally increase and the compression efficiency will deteriorate. Can be improved.
Also, the advantage of this direct mode is the simplicity of processing. While general motion vector search performs dozens of block matching operations and reading reference areas from memory, direct mode evaluation requires only one block matching operation and reading of reference areas from memory. That's it.

In such bi-directional inter-frame prediction, whether or not to use the direct mode and the optimal motion vector are generally determined using the following cost function.
for (i = 0; i <search_number; i ++) {
COST [i] = SAD [i] + MV [i];}
In the above equation, search_number indicates the number of times that the direct mode evaluation and motion vector search are performed (i = 0 indicates the direct mode evaluation).

The search_number becomes larger as the motion vector search range is expanded in order to improve the coding efficiency and the number of motion vector searches (the number of times of memory reading and block matching) is increased.
SAD [i] is a block matching result for the direct mode and each motion vector search (code amount estimation information). As SAD [i] becomes smaller, the degree of correlation increases and the code amount due to the prediction error of the orthogonal transform coefficient decreases. It is estimated that.
MV [i] indicates the code amount information of the motion vector, and is 0 in the direct mode.
COST [i] is a cost function determined from the result of block matching and motion vector information. The motion vector having the smallest COST [i] is set as the optimal motion vector having the highest coding efficiency. Used for actual encoding. When COST [i] for the direct mode is the smallest, encoding is performed in the direct mode, and motion vector information is not encoded.

Thus, in the conventional example, for each divided region in the frame, many motion vector searches and direct mode evaluation are always performed, and the motion vector or direct mode that achieves the best coding efficiency is selected. It was.
In general, in video coding, a target bit rate to be controlled at a constant level is set. Here, there is a general known technique, for example, TM2 of MPEG-2 (ISO / IEC-JTC1 / SC29 / WG11: “Test Model 5 (Draft)”, MPEG93 / N0400, 1993). In this known technique, when the bit rate has a margin, the bit rate generated by decreasing the quantization scale value is increased.

  The present invention can be applied to mobile devices such as mobile phones, digital cameras, and digital video cameras. Many of such portable devices are battery-powered, and in order to use them for a long time, it is indispensable to save power in a moving image coding system that consumes a lot of power in the devices. However, when direct mode evaluation and a large number of motion vector searches are performed for each region in the frame as in the above-described known technique, the number of times of reading from the storage device and block matching increases. In general, the power consumed in the storage device is proportional to the number of times of reading and writing, and the power consumed in the block matching calculation is almost proportional to the number of times of performing block matching. Therefore, as a result, the moving picture coding method using the known technique has a problem that the power consumption increases because the motion vector search is performed many times without considering the power consumption.

  To deal with this problem, Patent Document 1 sets subsamples in the correlation degree calculation means in accordance with image feature information such as frequency components of the encoding target block and usage state information such as the remaining battery level, and the correlation degree. An invention is disclosed in which all or a part of operation blocks are selected and operated.

  Patent Document 2 discloses an invention that reduces power consumption by narrowing the motion vector search range when the shooting angle of view is wide or when the shooting distance is long.

Japanese Patent Laid-Open No. 11-136682 JP-A-11-205656

  However, the invention disclosed in Patent Document 1 does not take into account the bit rate and target bit rate of an already encoded moving image. For this reason, when the bit rate of the already encoded moving image is high, if the encoding efficiency is further deteriorated by performing sub-sampling, there is a problem that the image quality is deteriorated. In addition, there is a problem that the power consumption for reading from the memory does not change only by changing the setting of the subsample, and the effect of reducing the power consumption is small because the subsampling circuit must be operated.

  Similarly, in the invention disclosed in Patent Document 2, when the motion search range is narrowed, there is a problem that the encoding efficiency is deteriorated and as a result, the image quality is deteriorated. Further, when the shooting angle of view is narrow or when the shooting distance is short, there is a problem that the motion vector search is always performed in a wide range and the power consumption is not reduced.

  The present invention has been made in view of the above-described problems, and an object of the present invention is to provide a low-power-consumption video coding method by appropriately selecting a coding mode. Another object of the present invention is to set a cost function that does not cause extreme image quality degradation. Another object of the present invention is to select an encoding mode that further increases the power consumption reduction effect. Furthermore, an object of the present invention is to more appropriately select a low power consumption encoding mode.

  The present invention is a moving picture coding method for coding a moving picture by selecting one coding mode from a plurality of coding modes, and the coding mode includes a bit rate and a target of a moving picture that has already been coded. There is provided a moving picture coding method or the like characterized by being determined from a bit rate and power consumption information corresponding to each coding mode.

  According to the present invention, it is possible to reduce power consumption by appropriately selecting an encoding mode. More specifically, a description will be given with reference to FIG. FIG. 2 is a diagram showing the transition of the generated bit rate (code amount) in time series. In FIG. 2, the horizontal axis indicates time, the vertical axis indicates the accumulated code amount, the dotted line in the figure indicates the target bit rate, and the solid line indicates the bit rate of the already encoded video (the slopes of the dotted line and the solid line indicate the bit rate). ing). As shown in FIG. 2, the bit rate is allowed to increase only when there is a margin in the bit rate, and the low power consumption encoding mode such as the direct mode is selected to realize the low power consumption.

  Further, since the quantization scale value is not increased as in the conventional example, there is no phenomenon that the image quality is temporarily improved only when there is a margin in the bit rate. However, even if the image quality is temporarily improved, it does not have a great influence on the superiority or inferiority of the subjective image quality. Therefore, it can be said that reducing the power consumption has a greater merit for the user.

  In addition, according to the present invention, when the coding efficiency of the low power consumption coding mode is extremely low, the low power consumption coding mode is not applied, and image quality deterioration can be prevented. In addition, according to the present invention, a coding amount generated using image feature information is accurately predicted, and a coding mode with high coding efficiency (coding mode with high power consumption) or a coding mode with low power consumption ( It is possible to more appropriately select a mode with low coding efficiency.

(First embodiment)
In the moving picture coding system of the present invention, whether or not to use the direct mode and the optimum motion vector are determined using the following cost function.
for (i = 0; i <search_number; i ++) {
if (i = 0) {
COST [i] = SAD [i] + MV [i] −OFFSET;
if (COST [i] <Thr) break;
}
else {
COST [i] = SAD [i] + MV [i];}}
In the above equation, i = 0 indicates the evaluation for the direct mode. OFFSET is an offset amount defined as follows.

  In the above equation, the encoding difficulty level is image feature information indicating the ease of generation of the code amount, and generally an image activity is used. The activity of this image will be described with reference to FIG. FIG. 7 is a graph showing the relationship between the quantization scale value (Q scale) and the generated code amount, where the horizontal axis indicates the quantization scale value and the vertical axis indicates the generated code amount. As shown in FIG. 7, it can be seen that even when the same quantization scale value is used, the amount of code generated by the activity of the image increases. Therefore, the higher the activity (the higher the spatial frequency of the image, the higher the encoding difficulty), the more likely the code amount is generated, and the lower the activity (the lower the spatial frequency of the image, the lower the encoding difficulty). The amount of code is less likely to occur.

  By extracting the activity of the image that can predict the amount of code generated in this way and reflecting it in the determination of the encoding mode, the encoding mode can be determined more appropriately. If this activity is not used, there are disadvantages such as selecting a coding mode with high power consumption and high compression efficiency even though the image does not generate much code.

Also, alpha _POWER is the power consumption information determined and the power consumption in a normal motion vector search, the ratio of power consumption in the direct mode is a low power consumption. This power consumption information is a parameter specified in advance by the user in accordance with the configuration and mounting form of each encoding mode (α _POWER always takes a positive value). Α _POWER increases as the difference between the normal encoding mode and the low power consumption encoding mode increases, that is, as the power consumption reduction effect increases. As will be described later, this parameter can also be set according to the operation mode designated by the user. Since α _POWER is always a positive value, OFFSET increases as the bit rate has a margin and the encoding difficulty level decreases.

  When OFFSET increases, the cost function corresponding to the direct mode becomes lower than that in the normal motion vector search mode. Therefore, if the value of the cost function corresponding to the direct mode is equal to or less than the threshold value Thr, the encoding mode is determined as the direct mode, and the motion vector search is not performed. As a result, very little power is required for encoding the block. As described above, the direct mode can be easily selected according to the bit rate, the image feature information, and the power consumption information, thereby realizing low power consumption.

Next, FIG. 1 shows the configuration of the moving picture coding apparatus according to the present invention.
FIG. 1 is a block diagram of a moving picture coding apparatus according to the present invention. As shown in FIG. 1, the moving picture encoding apparatus includes a frame memory 101, a subtractor 102, an orthogonal transformer 103, a quantizer 104, a scan processor 105, an entropy encoder 106, and an inverse quantizer 107. , An inverse orthogonal transformer 108 and an adder 109 are included.
The moving picture coding apparatus includes a frame memory 110, a motion vector generator 111, a block matching calculator 112, an offset amount generator 113, an activity extractor 114, a cost function calculator 115, and a reference region generator 116.

In order to perform bidirectional inter-frame prediction, the frame memory 101 stores a plurality of frames of the input moving image and performs an operation of changing the processing order of the frames.
The subtracter 102 subtracts the reference region data output from the reference region generator 116 from the input region to be processed (motion prediction) in inter-frame prediction encoding for performing prediction from temporally different frames, The prediction error of the subtraction result is output to the orthogonal transformer 103.
The orthogonal transformer 103 performs orthogonal transformation on the input data from the subtractor 102 in units of blocks (the area where motion prediction is performed does not necessarily match this block), and outputs the orthogonal transformation coefficients to the quantizer 104. To do.
The quantizer 104 quantizes the orthogonal transform using the quantization table value corresponding to the position in the region and the quantization scale value of the region, and the scan processor 105 and inverse quantize all quantized orthogonal transform coefficients. Output to the device 107.
The scan processor 105 performs a scan process such as a zigzag scan in accordance with the encoding mode.
The entropy encoder 106 performs entropy encoding on the output of the scan processor 105 and outputs it as a code.

Here, in the moving picture encoding apparatus shown in FIG. 1, local decoding processing is performed using an inverse quantizer 107 and an inverse orthogonal transformer 108 in order to perform motion vector search and motion prediction. The inverse quantizer 107 performs inverse quantization of the quantized orthogonal transform coefficient of the region using the quantization scale value of the region, and outputs the inverse quantized coefficient to the inverse orthogonal transformer 108.
The inverse orthogonal transformer 108 performs inverse orthogonal transformation on the inversely quantized orthogonal transformation coefficient in units of the blocks, and outputs the decoded prediction error to the adder 109.
The adder 109 adds the prediction value output from the reference region generator 116 and the decoded prediction error from the inverse orthogonal transformer 108, and stores them in the frame memory 110 as reconstructed image data decoded.
The motion vector generator 111 calculates a motion vector corresponding to the direct mode and the motion vector search and outputs the motion vector to the cost function calculator 115 in order to perform processing of the next processing target region. At the same time, an address for reading a reference area corresponding to the direct mode or the motion vector from the frame memory 110 is output. The reference area read from the frame memory 110 is output to the block matching calculator 112.

The block matching calculator 112 performs a block matching calculation between the given reference area and the processing target area, and outputs the resulting SAD value to the cost function calculator 115.
The activity extractor 114 calculates the activity in the processing target area and outputs it to the offset amount calculator 113.
The offset amount calculator 113 calculates the offset amount using the target bit rate, the generated bit rate, α _POWER that is power consumption information, and the activity of the processing target area input from the activity extractor, and calculates the cost function. Output to the calculator 115.
The cost function calculator 115 determines an optimal motion vector or direct mode from the input SAD, motion vector information and offset amount, and outputs the determined motion vector or direct mode information to the reference region generator 116.
The reference region generator 116 outputs a corresponding reference region to the subtracter 102 for motion prediction according to the determined motion vector or direct mode information.

(Second Embodiment)
FIG. 3 shows a block diagram of a digital camera to which the moving picture coding system according to the present invention is applied.
FIG. 3 is a diagram showing an internal configuration of the digital camera. The digital camera includes a lens 301, an image sensor 302, a signal processing circuit 303, a storage device 304, a moving image encoding device 305 to which the present invention is applied, a recording medium 306, a battery 307, and a user interface 308.

An image input from the object to be imaged is input to the image sensor 302 through the lens 301 and converted into an image signal. The converted video signal is subjected to color space conversion, noise removal, and the like in the signal processing circuit 303.
The video signal output from the signal processing circuit 303 is stored in the temporary storage device 304 in order to adjust the processing speed. The video signal read from the storage device 304 is input to the moving image encoding device 305 and output as encoded data. The output encoded data is recorded on a recording medium 306 such as a flash memory.
The battery 307 supplies power to the entire apparatus including the moving image encoding device 305.

Here, the user designates the operation mode of the moving picture coding apparatus 305 through the user interface 308 along with processing such as shooting and reproduction. As this operation mode, the user can select either the high compression efficiency mode or the low power consumption mode. When the high compression mode is selected, the power consumption amount information α _POWER (see the first embodiment) in the moving picture coding apparatus 305 is small in order to achieve as high a compression efficiency as possible even if the power consumption increases. Become.
On the other hand, in the low power consumption mode, encoding is performed with low power consumption even when the compression efficiency is low, so that the power consumption amount information α _POWER in the moving picture coding apparatus 305 becomes large. In the low power consumption mode, the power consumed by the moving picture coding apparatus 305 is reduced, so that the battery 307 can be used for a longer time.

When the user has a small amount of media, the user can select the high compression mode because the shooting time is limited by the media capacity. If the capacity of the possessed media is sufficiently large, the low power consumption mode may be selected because the shooting time is limited by the battery capacity and power consumption mounted on the digital camera.
As described above, according to the present invention, the user can enjoy the advantage that a longer shooting time can be secured by appropriately selecting the operation mode according to the media or the like possessed.

  Each means constituting the moving picture coding apparatus and each step of the moving picture coding method in the embodiment of the present invention described above can be realized by operating a program stored in a RAM or ROM of a computer. This program and a computer-readable recording medium recording the program are included in the present invention.

  Further, the present invention can be implemented as, for example, a system, apparatus, method, program, or recording medium, and may be applied to an apparatus composed of a single device.

  The present invention supplies a software program for realizing the functions of the above-described embodiments directly or remotely to a system or apparatus. In addition, this includes a case where the system or the computer of the apparatus is also achieved by reading and executing the supplied program code.

  Accordingly, since the functions of the present invention are implemented by computer, the program code installed in the computer also implements the present invention. In other words, the present invention includes a computer program itself for realizing the functional processing of the present invention. In that case, as long as it has the function of a program, it may be in the form of object code, a program executed by an interpreter, script data supplied to the OS, and the like.

  Further, the functions of the above-described embodiments are realized by the computer executing the read program. Furthermore, based on the instructions of the program, an OS or the like running on the computer performs part or all of the actual processing, and the functions of the above-described embodiments can be realized by the processing.

  As another method, a program read from a recording medium is first written in a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer. Then, based on the instructions of the program, the CPU or the like provided in the function expansion board or function expansion unit performs part or all of the actual processing, and the functions of the above-described embodiments are also realized by the processing.

It is a figure which shows the structure of the moving image encoder which concerns on 1st Embodiment. It is a figure which shows the transition in the time series of the generated code amount which concerns on 1st Embodiment. It is a figure which shows the structure of the imaging device which concerns on 2nd Embodiment. It is a figure which shows the concept of inter-frame prediction. It is a figure which shows the concept of bi-directional inter-frame prediction. It is a figure which shows the direct mode in bidirectional | two-way inter-frame prediction. It is a figure which shows the relationship between a quantization scale value and the amount of generated codes.

Explanation of symbols

101 frame memory
102 Subtractor
103 Orthogonal converter
104 Quantizer
105 Scan converter
106 Entropy encoder
107 Inverse quantizer
108 Inverse orthogonal transformer
109 adder
110 frame memory
111 Motion vector generator
112 Block matching calculator
113 Offset generator
114 activity extractor
115 Cost function calculator
116 Reference region generator
301 lenses
302 Image sensor
303 Signal processing circuit
304 storage
305 Video encoding device
306 Recording medium
307 battery
308 User interface

Claims (4)

A moving picture coding method for coding a moving picture by selecting one coding mode from a plurality of coding modes,
The moving picture code characterized in that the selected coding mode is determined from a bit rate of a moving picture that has already been coded, a target bit rate, and power consumption information corresponding to each coding mode. Method.   The coding mode to be selected is determined using a cost function, and the cost function includes code amount estimation information calculated by block matching, motion vector code amount information, and power consumption corresponding to each coding mode. The moving picture encoding method according to claim 1, wherein the moving picture encoding method is determined based on an offset amount determined using the amount information.   The selected encoding mode includes a bidirectional inter-frame prediction encoding scheme, and the bidirectional inter-frame prediction encoding scheme includes a mode for encoding motion vector information and a mode for not encoding motion vector information. The moving picture encoding method according to claim 1, further comprising:   4. The moving image encoding method according to claim 1, wherein the selected encoding mode is determined using image feature information of an input moving image.

Images