JP4747109B2 - Calculation amount adjustment device - Google Patents

Description

  The present invention relates to operations used for encoding processing and decoding processing in a system that performs encoding, decoding, or both of moving images, such as a video conference system, a surveillance camera, a video recording / playback device, and a video creation device. The present invention relates to a calculation amount adjustment apparatus and method for adjusting coding information of an image so that the amount becomes a target calculation amount.

  A digitized moving image has an enormous amount of data as it is, and its handling is difficult. However, by using moving image encoding / decoding, which is a technology for reducing this data amount, It has become possible to record / play / edit moving images with devices such as video devices, personal computers (hereinafter referred to as PCs), and mobile phones. As a standard for the encoding / decoding method of the moving image, a standard standardized by MPEG (Motion Picture Image Coding Experts Group) such as MPEG-1, MPEG-2, MPEG-4, etc. 261, H.H. 262, H.C. 263, H.M. H.264, etc. recommended by ITU-T (International Telecommunication Union Telecommunication Standardization Sector). The 260 series standards are widely used. First, a video encoding / decoding system compliant with these standards will be described.

  FIG. 17 shows MPEG and ITU-T H.264. It is a block diagram which shows the functional structure of the moving image processing system based on the 260th moving image encoding / decoding system specification. These standards combine motion-compensated prediction that reduces temporally redundant information, spatial frequency conversion that reduces spatially redundant information, and entropy coding that reduces the amount of information by assigning a code amount according to the appearance frequency. A hybrid encoding scheme is defined.

  In an encoding unit 13010 that is an encoding device, first, an ME (Motion Estimation) 13011 and an MC (Motion Compensation, motion compensation) 13012 are converted into an original image 13001 with respect to an original image 13001 that is an encoding target image. On the other hand, temporally redundant information is reduced by performing motion compensation prediction with reference to the reference image A13002, which is a temporally preceding or following image.

  Next, DCT (Discrete Cosine Transform) 13013 converts the data with reduced temporal redundancy information into a spatial frequency, and Q (Quantize) 13014 quantizes it. Reduce spatially redundant information. Note that the amount of encoded information generated is adjusted by controlling the step width of the quantization performed by Q13014.

  Finally, VLC (Variable Length Coding) 13015 entropy-encodes the data with reduced spatial redundancy information, thereby reducing the amount of information and outputting encoded information.

  In a decoding unit 13020 which is a decoding device, VLD (Variable Length Decoding, variable length decoding) 13021, IQ (Inverse Quantize, inverse quantization) 13012, IDCT (Inverse Discrete Cosine Transform, Inverse Discrete Cosine Transform) 13023, (Inverse Motion Compensation, Inverse Motion Compensation) 13024 performs a reverse conversion to VLC13015, Q13014, DCT13013, and MC13012 to output a decoded image 13004 that is a reproduced image decoded from the encoded information.

  A reference image B13003 is an image that is referred to when the IMC 13024 performs inverse motion compensation, and a decoded image 13004 that is temporally forward or backward is used for this. The encoding unit 13010 also generates a reference image A13002, which is a reproduced image decoded from the encoded information, by performing IQ13016, IDCT13017, and IMC13018 on the data with reduced spatial redundancy information output by the Q13014. If the calculation errors of DCT and Q are ignored, the reference image A 13002 of the encoding unit 13010 and the reference image B 13003 of the decoding unit 13020 are in principle the same image.

  FIG. 18 shows MPEG and ITU-T H.264. The figure explaining the process data unit in the moving image encoding / decoding system specification of the 260 series is shown.

  Picture 14001 is a data unit representing the entire image at a certain time t, and has a size of (M × 16 pixels, N × 16 lines) (M and N are natural numbers). A Slice 14002 is a data unit representing an area obtained by dividing the Picture 14001 in a unit of 16 lines in the vertical direction, and has a size of (N × 16N pixels, 16 lines). MB (Macro Block) 14003 is a data unit representing an area obtained by dividing Slice 14002 in units of 16 pixels in the horizontal direction, and has a size of (16 pixels, 16 lines). Block 14004 is a data unit representing an area obtained by dividing MB 14003 into two parts in both the horizontal and vertical directions, and has a size of (8 pixels, 8 lines). Pixel 14005 is a data unit representing 16 × 16 pixels included in Block 14004, and the data after the spatial frequency conversion performed by DCT 13013 is also called a DC coefficient or an AC coefficient.

  Each of the encoding unit 13010 and the decoding unit 13020 sequentially encodes and decodes a plurality of pictures 14001 arranged in time order. At this time, each picture 14001 is processed in data units of MB14003 by ME13011, MC13012, IMC13018, and IMC13024, DCCT13013, IDCT13017, and IDCT13023 are processed in data units of Block14004. Process in units.

  Next, in a device having a limited amount of computing resources and communication resources, these MPEG and ITU-T H.264 standards are used. Execute one or more moving image encoding / decoding processes or simultaneously execute moving image encoding / decoding processes and other applications using a moving image encoding / decoding method according to the 260 series standards The technology to do is explained.

  Since the input / output timing of each picture 14001 of a moving image is predetermined (for example, each picture 14001 is about 1/30 second interval in the NTSC system), the encoding / decoding process of each picture 14001 and the encoding The moving image encoding / decoding process is required to be real-time so that the decoding unit 13020 needs to input / receive encoded information output / transmitted by the encoding unit 13010 via a communication line.

  However, if the computing resource amount and the communication resource amount possessed by the device cannot satisfy the real-time property of the moving image encoding / decoding process, the calculation amount and the communication amount are reduced with respect to the moving image encoding / decoding process. Some measures must be taken to make adjustments.

  Such difficulties related to real-time characteristics are caused by, for example, the fact that an application other than the decoding unit 13020 operates simultaneously in a computing device in which the decoding unit 13020 operates, and the encoding unit 13010 depending on the image content of the original image 13001 to be encoded. Alternatively, the calculation amount of the decoding unit 13020 fluctuates (this will be described in detail in the problem to be solved by the invention), an amount of calculation resources larger than expected, and the encoding unit 13010 or the decoding unit. This is caused by, for example, an increase in the amount of computing resources required for the function expansion / bug fix of the unit 13020.

  Although it is possible to prepare a sufficient amount of computing resources and communication resources in advance to satisfy the real-time nature of video encoding / decoding processing, embedded devices are limited in cost and have limited computing resources. There is a strong demand to perform real-time video encoding / decoding processing in an amount, and operations that operate as the amount of arithmetic resources used by the encoding unit 13010 or the decoding unit 13020 increases. Redesigning equipment takes too much development time and cost.

  Patent Document 1 discloses a technique for solving such a problem. In the technique disclosed in Patent Document 1, the frame rate that is the number of pictures 14001 per second included in the encoded information is lowered, and the image size (or resolution) that is the size of pictures 14001 is reduced to 1/2 or 1 /. By reducing the amount of encoded information output / transmitted by the encoding unit 13010 by controlling the quantization step width of Q13014, the amount of computation resources used by the video encoding / decoding process and Reduce the amount of communication resources.

The correspondence relationship between the frame rate, the image size, the quantization step width, the calculation resource amount and the communication resource amount is obtained by approximating a plurality of representative values measured in advance by a spline function.
Japanese Patent No. 2968789

  However, even with the technique disclosed in Patent Document 1, two problems still remain. Each problem will be described below.

  The first problem is that in the technique of Patent Document 1, the image size that is the size of Picture 14001 is reduced, which lowers the frame rate that is the number of Pictures 14001 per second of a moving image to be encoded / decoded. The amount of calculation required for the encoding / decoding process is reduced by setting it to 1/2 or 1/4, but the adjustment unit is coarse.

  If the unit of adjustment is simply calculated, the adjustment of the frame rate is 1/30 unit of the total amount of encoding / decoding, and the adjustment of the image size is 1/2 or 1/4 of the total amount of encoding / decoding. This is a coarse unit. In the technique of Patent Document 1, since the calculation amount of the encoding / decoding process cannot be reduced in units smaller than this unit, the encoding information is adjusted so that the calculation amount is excessively reduced. There can be a situation where a surplus of computing resources is not used. In that case, a decoded image having an image quality that is inferior to the image quality of the decoded image that just uses up the computation resources is provided to the user.

  As described above, according to the technique of Patent Document 1, since the unit of the adjustment amount for reduction is coarse, the calculation resource is used up without being used up, and a decoded image with better image quality cannot be provided to the user. There is a problem that things happen. The finer the unit of the calculation adjustment amount, the better.

  The second problem is that Patent Document 1 does not consider the amount of data for which temporal redundancy information is reduced by motion compensation prediction.

  MPEG and ITU-T H.264 as described above. In the processing of the 260th video coding / decoding scheme standards, for data in which temporal redundancy information is reduced by motion compensation prediction, DCT 13013 of coding section 13010 and subsequent IQ 13014, VLC 13015, IQ13016, IDCT 13017, All processes in the IMC 13018 and the decoding unit 13020 are performed.

  For this reason, the amount of calculation changes depending on the amount of data in which temporal redundancy information is reduced by motion compensation prediction. The amount of data reduced by motion compensation prediction depends on the image content (picture) of the original image 13001 to be encoded, and also changes in time within one image sequence (hereinafter referred to as content dependency). ). For example, the amount of data reduced by motion compensation prediction increases or decreases depending on the number of background regions that do not change with time in the image. In Patent Document 1, this point is not taken into consideration.

  It is difficult to determine in advance the correspondence table of calculation amount and communication amount for adjustment parameters such as frame rate, image size, quantization step width, etc. from the viewpoint of the content dependency described above, and statistical data of many contents Even if it is approximated from the above, it is necessary to take statistical data from all the contents existing in the world, which is also difficult. As described above, because of the content dependency, it is difficult to accurately estimate the correspondence between the calculation amount and the communication amount with respect to the adjustment parameter in advance.

  A worst-case image with the largest amount of computation and communication may be assumed, and a correspondence relationship may be created based on the amount. However, in this case, when an image other than the assumed worst case image is encoded / decoded, a calculation resource is left over, and a situation in which a decoded image with better image quality cannot be provided to the user occurs. .

  In addition, if the image content changes drastically, such as scene changes, it may be possible to let the neural network or fuzzy classification system learn the correspondence between the calculation amount and communication amount for the adjustment parameter. Depends on the situation and does not correspond properly during learning. When the calculation amount is adjusted based on the learning result, a result such as surplus calculation resources or a shortage of calculation resources is caused.

In order to solve the above-described conventional technique, one aspect of the calculation amount adjustment apparatus according to the present invention is to convert encoded images into encoded information composed of a plurality of data units each individually processed smaller than one image. A calculation amount adjustment device used in a moving image processing system including an encoding unit for encoding and a decoding unit for decoding the encoded information into a moving image, and the decoding unit can be used A computing resource amount measuring means for measuring a remaining computing resource amount that is a computing resource amount, the measured remaining computing resource amount, and a computing resource amount used for processing one data unit in the decoding unit; From this, the upper limit number of data units that can be processed by the decoding unit is determined, and the image quality of the image represented by the encoding information is less than the determined upper limit number and approaches the target image quality specified in advance. Included in the encoded information Target data unit number determining means for determining a target data unit number, which is the target number of data units, and data unit number control means for limiting the number of data units included in the encoded information to the target data unit number or less. With.
Also, one aspect of the calculation amount adjustment apparatus of the present invention is an encoding unit that encodes a moving image into encoding information composed of a plurality of data units each processed smaller than one image, A calculation amount adjustment device used in a moving image processing system including a decoding unit that decodes the encoded information into a moving image, the remaining calculation resource being an amount of calculation resources usable by the encoding unit Processing in the decoding unit from the calculation resource amount measuring means for calculating the amount, the calculated remaining calculation resource amount, and the calculation resource amount used for processing one data unit in the decoding unit The upper limit number of data units that can be determined is determined, and is included in the encoding information so that the image quality of the image represented by the encoding information is less than the determined upper limit number and approaches the target image quality specified in advance. The target number of data units Comprising the target data unit number determining means for determining a target number of data units, the number of the data units included in the coding information, and a data unit number control means for limiting the following number of the target data unit.

  In this case, the encoding unit is video data that conforms to a standard standardized by Motion Picture Image Coding Experts Group, or a video that is recommended by an international telecommunications union telecommunications standard standard video. The data unit may be at least one of a pixel, a coefficient, a block, a macroblock, and a slice.

According to this configuration, the encoding information is encoded or decoded within a calculation resource amount that can be used by the arithmetic device in a finer unit than the conventional technology and without content dependency. Can be adjusted.

  In addition, the data unit number control means preferentially removes data units that are less noticeable in image quality deterioration of an image obtained by decoding when removed from the encoded information, thereby determining the number of data units. You may control below the number of data units.

  According to this configuration, as a result of adjusting the encoding information, it is possible to suppress an impression of image quality degradation that occurs in an image obtained by decoding.

  Further, the target data unit number determining means is determined when generating the encoding information representing the image when the image quality of the image obtained by the previous decoding is worse than a predetermined target image quality. A value larger than the target data unit number may be determined as the target data unit number.

  According to this configuration, as a result of determining the target data unit number so that the image quality of the image obtained by decoding the encoded information is close to the target image quality, the amount of computation resources is adjusted by adjusting the encoded information. The possibility that the target image quality can be maintained is increased.

  In addition, the target data unit number determining means is configured to notify a user when a value to be determined to set the image quality of an image obtained by decoding the encoded information to the predetermined target image quality exceeds the upper limit number. Control for reducing the amount of computing resources used by an application program that performs a process other than the generation process or the decoding process in the computing device may be performed to notify the target image quality to be corrected downward.

  According to this configuration, when it is impossible to achieve both the limitation of the computational resource amount and the maintenance of the target image quality, a new trade-off between the computational resource amount and the target image quality can be achieved through user intervention or control over other application programs. Can be found.

  Whether the target data unit number determining means notifies the user of upward correction of the target image quality when variations in the image quality of the plurality of images obtained by the previous decoding exceed a certain value. Alternatively, control may be performed to reduce the amount of computing resources used by an application program that performs processing other than the generation processing or the decoding processing in the arithmetic device.

  According to this configuration, it is possible to reduce deterioration in image quality impression due to temporal variations in image quality.

  In addition, the target data unit number determination means notifies the user of a downward correction of the target image quality when there is not enough calculation resource amount to start a new generation process or a new decoding process in the arithmetic device. Alternatively, control may be performed to reduce the amount of computing resources used by the generation process or the decoding process that is operating earlier.

  According to this configuration, the possibility that the new encoding process or the new decoding process can be started is increased by user intervention or the control over the generation process or decoding process that is operating earlier.

  The present invention can also be realized as a moving image processing system including a calculation amount adjustment device. Further, the calculation amount adjustment method and a program for causing a computer to execute the method can also be realized. Further, it can be realized as a system in which a plurality of calculation amount adjusting devices operate. Needless to say, the program according to the present invention can be distributed via a recording medium such as a CD-ROM or a transmission medium such as the Internet.

  According to the calculation amount adjustment apparatus of the present invention, the encoding information generated by the encoding process in the data unit of Pixel, coefficient, Block, MB, and Slice, which is a data unit smaller than one image constituting one image. Since the number of data units included in the data unit is limited, it is possible to limit the amount of calculation resources necessary for processing the encoded information in a finer unit than the conventional technique.

  As a result, the computation resources are not excessively suppressed, so that the computation resources can be used up to the limit, and a decoded image with better image quality can be provided to the user. In addition, since the amount of computation can be adjusted to a certain value or less without depending on the content, the computation resources are excessively suppressed, insufficient computation resources are insufficient, or the image quality changes rapidly. There will be no degradation of the image quality impression.

  Furthermore, since the amount of computing resources used in the encoding or decoding process is adjusted based on the amount of computing resources of the computing device that performs the encoding process or the decoding process or the amount of computing resources remaining in the computing device, There is no need to estimate the exact amount of calculation. Also, in situations where other applications are operating at the same time, or in situations where the amount of computational resources required from the beginning increases due to functional enhancements or bug fixes of the encoding process or decoding process, suitable adjustments are made following the situation. Can be done.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(Embodiment 1)
In the present embodiment, a moving image processing system having a function of adjusting the calculation amount of the decoding unit and a calculation amount adjusting device included in the moving image processing system will be described with reference to FIGS.

First, a functional configuration of a moving image processing system including a calculation amount adjusting device will be described.
FIG. 1 is a block diagram showing a functional configuration of an image processing system according to Embodiment 1 of the present invention. In FIG. 1, ME 111, MC 112, DCT 113, Q 114, VLC 115, IQ 116, IDCT 117, IMC 118 in encoding unit 110, and VLD 121, IQ 122, IDCT 123, IMC 124 in decoding unit 120 are MPEG and MPEG described in the background section. ITU-TH. This is a functional block conforming to the 260th video coding / decoding scheme standards (see FIG. 17).

  The encoding unit 110 generates encoding information by performing encoding / decoding processing on the original image 101 to be encoded with reference to the reference image A102 and the reference image B103, and the decoding unit 120 A decoded image 104 that is a decoded image is obtained from the encoded information.

  The computing resource amount measuring unit 131 measures the computing resource amount and the remaining computing resource amount used by the decoding unit 120 of the computing device in which the decoding unit 120 is operating, for example, according to the following concept.

  As described in the background art, MPEG and ITU-T H.264. For each functional block of the 260th video coding / decoding scheme standard, for example, in the case of IDCT123, the data unit for processing is determined such as in Block 14004 unit. Also, when each functional block is implemented in software or hardware logic, the amount of computing resources (for example, the time for which the computing device operates) that each functional block uses to process one data unit is determined. Yes.

  Therefore, the amount of computing resources used by the decoding unit 120 is calculated by multiplying the number of data units to be processed by each functional block by the amount of computing resources used for processing of one data unit to obtain a sum. . Furthermore, the encoding unit 120 is operated by measuring all the calculation resource amounts that can be used by the calculation device in which the decoding unit 120 is operating, and subtracting the calculated calculation resource amount from the measurement result. In this case, the remaining computing resource amount that can be used by the computing device is calculated.

  This remaining computing resource amount is calculated as a negative amount when the computing resource amount used by the decoding unit 120 is larger than all the computing resource amounts of the computing device. The negative residual computing resource amount means that the encoding unit 120 cannot process all the data units to be processed even if all the computing resources that can be used by the computing device are used.

  The image quality evaluation unit 132 evaluates the image quality of the reference image A102. As described in the background art, the reference image A102 is a reproduction image decoded from the encoding information generated by the encoding unit 110, and the reference image B103 of the decoding unit 120 is the same image as the reference image A102. . The reference image B103 is an image temporally preceding or following the decoded image 104 reproduced by the decoding unit 120. Evaluation of the image quality of the reference image A102 is reproduced by the decoding unit 120. This is almost the same as evaluating the image quality of the decoded image 104.

  PSNR (Peak Signal to Noise Ratio), which is generally well-known as an image quality evaluation method, is used. However, any other method can be used as long as it can derive an index for determining the quality of an image. There may be. Alternatively, the decoded image 104 may be directly viewed by a human and the index may be indicated by subjective evaluation.

  The target image quality 133 is information representing the image quality target of the decoded image of the decoding unit 120 by the above-described index. This is prior to or during the processing of the decoding unit 120, such as a user, another application, etc. Set or changed by

  The target block number determination unit 134 outputs the calculation resource amount and the remaining calculation resource amount used by the decoding unit 120 in the calculation device in which the decoding unit 120 operates, the image quality evaluation unit, which is output from the calculation resource amount measurement unit 131. Based on the image quality of the reference image A102 output by 132 and the target image quality 133, the target number of blocks 14004 per picture 14001 included in the encoded information output by the encoding unit 110 is determined.

  The block number control unit 135 limits the number of blocks 14004 per picture 14001 included in the information output by the Q 114 according to the target number determined by the target block number determination unit 134, and outputs the limited information to the VLC 115.

  As described in the background art, in encoded information, temporally redundant information is reduced by DCT 113 and Q114 after temporally redundant information is reduced by ME 111 and MC 112.

  FIG. 2 is a diagram schematically illustrating reduction of temporally and spatially redundant information in encoded information. In FIG. 2, each square in the picture 14003 indicates a block 14004, and a block of four blocks 14004 indicates an MB 14003. A coefficient 14005 exists at a portion indicated by each black square (coefficient is not 0), and a coefficient 14005 does not exist (coefficient is 0) at a portion not painted black.

  As shown in FIG. 2, the information in which the temporally and spatially redundant information is reduced includes a block 14004 that does not have the coefficient 14005, and these blocks 14004 are not sent to the decoding unit 120 as encoded information. . Only the Block 14004 having the coefficient 14005 is sent to the decoding unit 120 as the encoded information, and is decoded.

  The IDCT 117 of the encoding unit 110 and the IDCT 123 of the decoding unit 120 process the encoded information in units of Block 14004. Therefore, by limiting the number of Block 14004 per Picture 14001 included in the encoded information, the IDCT 117 and the IDCT 123 It is possible to limit the amount of computing resources used when processing the encoded information.

  Next, the mounting and operating environment of the moving image processing system in the embodiment of the present invention will be described.

  As shown in FIG. 1, the moving image processing system according to the present embodiment includes an encoding unit 110, a decoding unit 120, an operation resource amount measurement unit 131, an image quality evaluation unit 132, a target block number determination unit 134, and the number of blocks. The control unit 135 is configured. All or a part of these units may be implemented as software, or may be implemented as hardware logic.

  Among these units, the calculation resource amount measurement unit 131, the image quality evaluation unit 132, the target block number determination unit 134, and the block number control unit 135 cooperate to limit the number of blocks included in the encoded information. , IDCT 117 and IDCT 123 reduce the amount of computing resources used when processing the encoded information. That is, the calculation resource amount measurement unit 131, the image quality evaluation unit 132, the target block number determination unit 134, and the block number control unit 135 constitute the calculation amount adjustment device in the embodiment of the present invention.

FIG. 3 is a diagram illustrating an example of an operating environment of the moving image processing system.
The PC 310 is a general personal computer, an arithmetic unit 301 realized by a CPU, DSP, hardware logic, etc., a storage unit 302 realized by a memory or a hard disk, an image input / output unit 303 such as a camera or a display, The communication unit 304 is a wired LAN adapter or a wireless LAN adapter.

  The communication unit 304 of each PC 310 is connected via a LAN 311 such as a wired LAN or a wireless LAN.

  Each unit of the moving image processing system including the calculation amount adjustment apparatus of the present invention is executed by the calculation unit 301 in each PC 310, and the reference image A 102 and the reference image B 103, intermediate information being executed, and encoded information are stored in the storage unit 302. May be temporarily stored.

  When each unit of the moving image processing system is implemented as software, the software program is stored in the storage unit 302. The original image 101 that is the encoding target image and the decoded image 104 that is the decoded image are acquired by the image input / output unit 303 and displayed. When the encoding unit and the decoding unit are mounted on different PCs 310, the encoded information is transmitted between the PCs 310 via the communication unit 304 and the LAN 311.

  In addition, it is not particularly specified in which PC 310 each part of the moving image processing system is mounted and operating. Since each PC 310 can communicate with the communication unit 304 and the LAN 311, data transfer between the functional blocks may be performed via the communication unit 304 and the LAN 311, and any part of the moving image processing system is mounted on any PC 310. It is possible to operate. All may be implemented / executed on one PC 310, and in an extreme example, all may be implemented / executed on different PCs 310.

  Furthermore, the encoding unit 110 and the decoding unit 120 may be executed at the same time or may be executed at different times. As an example to be executed at the same time, there is an example in which encoded information is decoded in real time and a decoded image is viewed like a TV broadcast or a TV conference. As described above, there is an example in which the encoded information is recorded in the storage unit 302 and is decoded later to see the decoded image.

  When the encoding unit 110 and the decoding unit 120 are executed at different times, the calculation resource amount measurement unit 131 performs the remaining calculation of the calculation device on which the decoding unit 120 is scheduled to operate during the execution of the encoding unit 110. Since the resource amount cannot be measured, by setting the calculation amount of the decoding unit 120 to 0, all the calculation resource amounts usable in the calculation device are measured as the remaining calculation resource amount.

  In addition, there is one set of the encoding unit 110, the decoding unit 120, the calculation resource amount measurement unit 131, the image quality evaluation unit 132, the target block number determination unit 134, and the block number control unit 135 that constitute the moving image processing system. Alternatively, a plurality of PCs 310 may be mounted and executed. When a plurality of sets of moving image processing systems are executed on the same PC 310, the computing resources of a common unit may be used by time-sharing. In that case, the calculation amount measured by the calculation resource amount measurement unit 131 should be measured in consideration of the time ratio of the calculation resources being time-sheared.

  Next, the basic concept regarding the adjustment of the calculation amount in the first embodiment will be described below.

  In this description, it is assumed that the calculation amounts of the VLD 121, IQ 122, and IMC 124 in the decoding unit 120 do not vary or are fixed. As described in the background art, the VLD 121 and IQ 122 process encoded information in units of coefficients 14005, and the IMC 124 processes encoded information in units of MB 14003. If the encoding unit 110 performs encoding at CBR (Constant Bit Rate), the number of coefficients 14005 per Picture 14001 included in the encoded information is substantially constant.

  Further, in order to configure the decoded image 104, the IMC 124 needs to refer to the MB 14003 that is not included in the encoded information from the reference image B103. If the calculation resource amount for this reference is equal to the calculation resource amount for referring to the MB14003 included in the encoded information, the IMC 124 has a constant calculation amount regardless of the encoded information. For this reason, the above assumption is correct in such a case.

  If the calculation amounts of the VLD 121, IQ 122, and IMC 124 in the decoding unit 120 do not vary or are fixed, the variation factor of the calculation resource amount used by the decoding unit 120 is only a change in the calculation amount in the IDCT 123. . Since IDCT 123 processes encoded information in units of Block 14004, if the number of Blocks 14004 per Picture 14001 included in the encoded information is limited, the amount of computation resources used by the decoding unit 120 may be limited to a certain value or less. It becomes possible.

Next, the operation of the moving image processing system will be described below with reference to FIG.
The encoding unit 110 sequentially encodes a series of original images 101 that are input from the image input / output unit 302 in terms of time, using ME111, MC112, DCT113, Q114, VLC115, IQ116, IDCT117, and IMC118, which are constituent elements thereof. , Output encoded information. The encoded information is input to the decoding unit 120, and the encoded information is decoded by the components VLD 121, IQ 122, IDCT 123, and IMC 124, and the decoded image 104 that is a decoded image is output from the image input / output unit 302. To do.

  In parallel with this movement, for each picture 14001, the computation resource amount measurement unit 131 sequentially measures the computation amount of the decoding unit 120 and the remaining computation resource amount of the computing device in which the decoding unit 120 is operating (step 15001). ), The image quality evaluation unit 132 sequentially evaluates the image quality of the reference image A102 (step 15002).

  Then, the measurement result of the calculation resource amount measurement unit 131 and the image quality evaluation result of the image quality evaluation unit 132 are sequentially input to the target block number determination unit 134. The target block number determination unit 134 determines the target block number for each picture 14001 as follows from the measurement result of the computing resource amount measurement unit 131, the image quality evaluation result of the image quality evaluation unit 132, and the target image quality 133 (step 15003). ).

If the decoding unit 120 is to be operated using all the computation amounts available on the PC 310 in which the decoding unit 120 is operating, the target number of blocks is determined according to (Equation 1).

In (Expression 1), T _{block — a} represents the target number of blocks (represents the upper limit of the number of blocks that can be processed by the IDCT 123), T ′ _block represents the previous number of target blocks (before 1 picture) (0 if there is no before 1 picture), and R _cal Is the amount of remaining computing resources of the PC 310 on which the decoding unit 120 is operating, measured by the computing resource amount measuring unit 131, and C _IDCT is the amount of computing resources used by the _IDCT 123 for processing of 1Block 14004. In (Formula 1), image quality is not taken into consideration.

  Here, (Equation 1) indicates that if the amount of remaining computing resources of the PC 310 (the amount of computing resources that can be used in the PC 310) is a positive value, that is, a residual, the target block number is increased by the number of blocks that can be processed by the residual, and a negative value is obtained. In other words, if it is insufficient, it should be noted that the target number of blocks is relatively determined by reducing the number of target blocks by the number of blocks that cannot be processed due to the shortage.

On the other hand, if the decoding unit 120 is operated by the PC 310 so that the image quality of the decoded image obtained by the decoding unit 120 approaches the target image quality 133 in consideration of the image quality, the number of blocks included in the encoded information Since the image quality improves as the value increases, the target number of blocks is determined according to (Equation 2).

In (Expression 2), T block — _b is the target number of blocks, Q _now is the current image quality evaluated by the image quality evaluation unit 132, Q _target is the target image quality 133, and α is a coefficient. However, since not exceed the amount of calculation can be used in _PC310, T block_b is limited to below T _{Block_a} of (Equation 1).

When T _{block_b in} (Expression 2) exceeds T _{block_a} , it means that the amount of remaining computing resources of the PC 310 in which the decoding unit 120 is operating is insufficient to obtain the target image quality 133. In such a case, although not shown in FIG. 1, the target block number determination unit 134 notifies the user who uses the decoding unit 120 to that effect, and prompts the user to reset the target image quality 133. May be.

  Further, in order to increase the amount of remaining computing resources in the PC 310 on which the decoding unit 120 is mounted / executed, other applications that are simultaneously operating on the PC 130 are stopped or the amount of computation is decreased. You may control. For example, when a moving image processing system including another set of calculation amount adjusting devices is operating on the PC 130, the moving image processing system is used by requesting the calculation amount adjusting device to lower the target image quality. It is possible to reduce the amount of computing resources to be performed and, as a result, increase the amount of remaining computing resources.

The coefficient α in (Expression 2) may be a fixed value or may be determined by the expression shown in (Expression 3).

In (Expression 3), β is a coefficient. According to (Equation 3), as the difference between the target image quality Q _target and the current image quality Q _now is larger, the coefficient α becomes a larger value and converges to the target number of blocks T block — _b that _{achieves the} target image quality earlier.

In (Equation 3), although the absolute value of the difference between the target quality Q _target and current quality Q _now, the square value of the difference between the target quality Q _target and current quality Q _now, even fourth power value Good. The larger the difference between the target image quality Q _target and the current image quality Q _now , the larger the coefficient α, and the smaller the difference between the target image quality Q _target and the current image quality Q _now , the coefficient α becomes 0. Any formula may be used.

  Such determination of the target number of blocks is performed by the target number-of-blocks determination unit 134 for each picture 14001. The target block number determination unit 134 makes this determination using the remaining calculation resource amount measured by the calculation resource amount measurement unit 131 and the image quality evaluated by the image quality evaluation unit 132.

  The target block number determined by the target block number determination unit 134 is input to the block number control unit 135. As described below, the block number control unit 135 removes the block 14004 exceeding the target block number from the information output from the Q 114, thereby limiting the number of blocks 14004 per picture 14001 included in the information (steps 15003 to 15007). ).

  First, the selection criteria for the removal target block to be removed by the block number control unit 135 will be described. The block to be removed is a block in which the decoded image 104, which is a decoded image, does not deteriorate as much as possible when it is removed.

  Since the information output by Q114 is information in which redundant information is reduced in time, the amount of information (the number of coefficients and the size of the coefficients) is the time of the original image 101 and the reference image A. The size of the difference. For this reason, if a block with a larger amount of information is removed, the image quality is greatly affected. Therefore, a block with a smaller amount of information is preferentially removed as a target block.

  The information output by Q114 is information converted into a spatial frequency, but the human eye is more sensitive to image quality degradation due to lack of low frequency components than image quality degradation due to lack of high frequency components. For this reason, a block with a small amount of low-frequency components (number of coefficients and amount of coefficients) is preferentially removed and set as a target block.

  As described above, a block having a smaller amount of information in the information of one block data unit and a block having a smaller amount of low-frequency components are preferentially removed as a target block.

  Note that the selection criterion described above is only an example, and the block to be removed is a criterion for selecting a block in which the decoded image 104, which is a decoded image, does not deteriorate as much as possible when it is removed. Other selection criteria may be used. For example, the degree of image quality degradation when a candidate block is removed may be evaluated by PSNR and selected based on the index.

  Next, an operation in which the block number control unit 135 removes the target block to be removed from the information output by the Q 114 based on the selection criterion will be described.

  The block number control unit 135 temporarily stores one or more blocks included in a predetermined accumulation unit. Here, the predetermined accumulation unit is, for example, a predetermined plurality of Blocks 14004, one or more predetermined number of MB 14003, one or more predetermined number of Slice 14002, or one Picture 14001.

  From the stored information, the information on the target block to be removed is removed based on the selection criteria until the number of blocks is equal to or less than the target number of blocks, and the remaining block information is output to the VLC 115 and IQ 116 (step 15005). ).

  Since the target block number determination unit 134 outputs the target block number per picture 14001, when the block is temporarily stored per picture 14001, the target block number is applied as it is.

  Further, when storing a predetermined number of Blocks 14004, a predetermined number of MB14003, or a predetermined number of Slice14002 as one storage unit and storing Blocks included in the storage unit, the target Block number output from the target Block number determination unit 134 is: Conversion is made to the target number of blocks corresponding to each storage unit (step 15004). Step 15004 shows an example in which one MB 14003 is used as a storage unit.

(Expression 4) shows an expression used for the conversion.

In (Expression 4), T _{block_X} is the target number of blocks per storage unit, T _{block_picture} is the target number of blocks per _picture 14001, and N _{X_MAX} is the number of storage units included in one _picture 14001.

In addition, during the processing of one picture 14001, when sequentially storing information output from Q114, removing blocks, and outputting information to VLC 115 and IQ 116 for a plurality of storage units, the expression shown in (Expression 5) is used. A suitable target block number may be determined according to the progress of the process.

In (Expression 5), N _X is the total number of storage units that have accumulated information so far (reset to 0 at the beginning of the picture), and O _block is the total number of blocks output from the block number control unit 135 so far (picture). Reset to 0 at the beginning of

  FIG. 5 shows an example of variation in the amount of information stored in the block number control unit 135 when information is accumulated, blocks are removed, and information is output according to the target number of blocks determined by (Equation 5). It is a graph represented by the number of blocks.

FIG. 5 shows an example in which N _{X_MAX} is 4, that is, one picture includes four storage units (for example, four slices may be included). The target number of blocks in the processing of the N _X (N _X = 1 to 4) -th accumulation unit is (N _X × T _{block_picture} / 4) −O _{block according} to (Equation 5).

  Information is sequentially stored for the four storage units, the blocks are removed according to the target number of blocks, and information on the remaining blocks is output. As a result, the amount of information stored in the block number control unit 135 changes as shown in FIG. When the output number of blocks falls below the target number of blocks (second storage unit in FIG. 5), the difference δ between the target number of blocks and the output number of blocks is added to the target number of blocks of the subsequent storage unit.

  The operation of limiting the number of data performed by the block number control unit 135 has been described above. In this operation, the number-of-blocks control unit 135 removes data in block 14004 units from the information output by the Q 114 so that the number of blocks is equal to or less than the target number of blocks, thereby the number of blocks 14004 per picture 14001 included in the information. Limit.

  In the foregoing, the calculation amount adjustment device according to the first embodiment and the moving image processing system including the calculation amount adjustment device have been described. As described above, the encoding unit 110 sequentially encodes a series of temporally different original images 101 input from the image input / output unit 302, outputs encoding information, and the decoding unit 120 outputs the encoded information. And the decoded image 104, which is a decoded image, is output / displayed from the image input / output unit 302.

  In parallel with this movement, the calculation amount of the decoding unit 120 measured by the calculation resource amount measurement unit 131, the remaining calculation resource amount of the calculation device in which the decoding unit 120 is operating, and the image quality evaluation unit 132 Based on the evaluated image quality of the reference image A102 and the target image quality 133, the target block number determination unit 134 determines the target number of blocks per one picture 14001 data unit.

  The number-of-blocks control unit 135 limits the number of blocks per picture 14001 included in the encoded information so as to be equal to or less than the determined target number of blocks.

  As a result, the amount of operation resources used by the IDCT 123 to process the encoded information is limited to a certain amount corresponding to the target number of blocks, and as a result, the operation amount of the decoding unit 120 is limited to a certain amount or less. The

  Here, the fixed amount is the amount of computing resources required by the decoding unit 120 measured by the computing resource amount measuring unit 131, the amount of remaining computing resources of the computing device in which the decoding unit 120 is operating, and the image quality evaluation unit 132. It is determined based on the evaluated image quality and the target image quality 133 and does not exceed the amount of remaining computing resources of the computing device in which the decoding unit 120 is operating, and the image quality of the decoded image is brought close to the target image quality 133. It is such a calculation amount.

  In this way, according to the moving image processing system including the present calculation amount adjusting apparatus, the image quality of the image to be decoded can be reduced without exceeding the remaining calculation resource amount of the calculation device in which the decoding unit 120 is operating. The encoding unit 110 generates suitable encoding information that can be decoded by the decoding unit 120 with an amount of computation that approaches the target image quality 133.

  In addition, according to the calculation amount adjustment device, the calculation amount is adjusted by a much smaller block 14004 unit than the picture 14001 which is a data unit in which the number of data is reduced in order to adjust the calculation amount by the conventional technique. Yes. For example, the size of the picture 14001 data unit is about 720 × 480 pixels in the case of a general NTSC TV image, and the size of the block 14004 data unit is MPEG and ITU-T H.264. According to the 260th video coding / decoding standard, the size is about 8 × 8 pixels.

  For this reason, the adjustment in units of Block 14004 can be adjusted in an extremely fine unit of 1/5400 units of the total encoding / decoding calculation amount. This is a few steps finer than the prior art, and as explained in the problem to be solved by the invention, the smaller the unit, the less the computing resources are left and the better the picture quality decoding is. An image can be provided to the user.

  For this reason, according to this calculation amount adjustment device, by adjusting the calculation amount in finer units as compared with the conventional technology, a decoded image with better image quality can be provided to the user without leaving a large amount of calculation resources. can do.

  Further, according to the calculation amount adjusting apparatus, the target number of blocks based on the calculation amount of the decoding unit 120 measured by the calculation resource amount measuring unit 131 and the remaining calculation resource amount of the calculation device in which the decoding unit 120 is operating. The block number control unit 135 limits the number of blocks per picture 14001 included in the encoded information to be equal to or less than the target number of blocks determined by the determination unit 134, thereby depending on the image content of the original image 101 that is the encoding target image. In other words, the amount of calculation is adjusted to the amount of calculation that falls within the amount of calculation resources of the calculation device in which the decryption unit 120 operates without content dependency.

  For this reason, this calculation amount adjustment device does not cause the problem due to the content dependency described in the section of the problem to be solved by the invention.

  In the background art and the first embodiment, for simplicity of explanation, the data is implicitly luminance data, and color data is not particularly mentioned.

  When color data is included in the original image 101 that is the encoding target data, the block 1414 for the processing data unit shown in FIG. 18 includes a block related to color data in addition to the luminance data. In this case, since the number of blocks included in the MB14003 is simply increased, the calculation amount adjustment device described in the first embodiment can be similarly applied. It goes without saying that the same effect can be obtained even when color data is included in the original image 101 which is the encoding target data.

  In the first embodiment, the number of blocks 14004 per picture 14001 is limited. However, the same effect can be obtained by limiting the number of blocks 14004 per one or more predetermined number of slices 14002 per predetermined number of pictures 14001. Needless to say.

  In the background art and the first embodiment, MPEG and ITU-T H.264 are used. The encoding / decoding schemes in conformity with the 260 series moving image encoding / decoding scheme standards have been taken up. However, encoding / decoding schemes other than these standardized standards, or encoding / decoding schemes to be developed in the future will be described. Even in the case of decoding schemes, motion compensation prediction that reduces temporally redundant information, spatial frequency conversion that reduces spatially redundant information, and frequency of appearance, similar to these video encoding / decoding schemes If this is a hybrid encoding / decoding scheme combined with entropy encoding that reduces the amount of information by assigning the amount of code, the calculation amount adjustment device can be applied, and the same effect can be obtained by applying the hybrid encoding / decoding method. Needless to say.

(Embodiment 2)
In the present embodiment, a modification example of the calculation amount adjustment device described in the first embodiment and a moving image processing system including the calculation amount adjustment device will be described with reference to FIGS.

  In the first embodiment, it is assumed that the calculation amount of the VLD 121, IQ 122, and IMC 124 in the decoding unit 120 does not vary or is fixed, but in this case, the calculation amount of the IMC 124 varies. Embodiments will be described below.

  First, functional blocks constituting the moving image processing system including the calculation amount adjusting device will be described below. FIG. 6 is a functional block diagram of a calculation amount adjustment device according to Embodiment 2 of the present invention.

  In FIG. 6, ME 111, MC 112, DCT 113, Q 114, VLC 115, IQ 116, IDCT 117, IMC 118 in encoding unit 110, and VLD 121, IQ 122, IDCT 123, and IMC 124 in decoding unit 120 are the units described in the first embodiment. Is the same. Further, the calculation resource amount measurement unit 131, the image quality evaluation unit 132, the target image quality 133, the target block number determination unit 134, and the block number control unit 135 are the same as those described in the first embodiment. The difference from the first embodiment is that a target MB number determination unit 136 and an MB number control unit 137 are added.

  Next, only the differences from the first embodiment will be described with reference to FIG. 7 for the configuration and operation of the calculation amount adjustment apparatus according to the second embodiment. The target MB number determination unit 136 operates in the same manner as the target block number determination unit 134 to determine the number of target MB 14003s, not the number of target Blocks 14004.

The target MB number determination unit 136 determines the target number of the number of MB 14003 per Picture 14001 according to (Expression 6) or (Expression 7) (Step 16003).

In (Expression 6) and (Expression 7), T _{MB — a} and T _{MB — b} are the target block numbers, T ′ _MB is the previous target MB number (before 1 picture) (0 if there is no previous one picture), and R _cal is the computational resource amount. The amount of remaining computing resources of the PC 310 on which the decoding unit 120 operating measured by the measuring unit 131, C _IMC is the amount of computation per MB 14003 of the _IMC 124, Q _now is the current image quality evaluated by the image quality evaluating unit 132, and Q _target is The target image quality 133 and α are coefficients.

However, T _{MB_b} is limited to T _{MB_a} or less because the amount of computation that can be used in the PC 310 must not be exceeded.

In addition, since the calculation amount that can be used in the PC 310 must not be exceeded, R _{cal of} (Expression 1) which is a determination formula of the target block number of the target block number determination unit 134 and the target MB number of the target MB number determination unit 136 The sum of R _{cal in} equation (6), which is the determinant of the above, must be limited so as not to exceed the amount of remaining computing resources of the PC 310.

For this reason, it is assumed that R _cal for each of Block and MB is obtained by (Equation 8).

In (Equation _8), R cal_block is operating _{R cal,} R cal_MB is R _cal, R _cal decoding unit 120 calculating the resource amount measurement unit 131 to measure the (Equation 6) (Equation 1) PC 310 C _IDCT is a computing resource amount used by the IDCT 125 to process one Block 14004, and C _IMC is a computing resource amount used by the _IMC 124 to process one MB 14003.

The assignment method shown in (Equation 8) is an example, and another assignment method may be used as long as the sum of R _{cal_block} and R _{cal_MB} does not exceed R _cal .

  Further, the number of Blocks 14004 constituting the MB14003 is determined to be a fixed number K (for example, K = 4 in FIG. 18). For this reason, even if the target block number determined by the target block number determination unit 134 is K times or more the target MB number determined by the target MB number determination unit 136, the MB number control unit 137 limits the MB number. Therefore, the block number control unit 135 is meaningless because information including the number of blocks equal to or larger than the target MB number is not input.

Therefore, the target MB number determination unit 136 and the target block number determination unit 134 set the target numbers so that the target block number does not exceed K times the target MB number and does not exceed the amount of computation usable by the PC 310. adjust. For example, as shown in the process flowchart of FIG. 8, the target MB number determination unit 136 notifies the target Block number determination unit 134 of the target MB number determined (Step 17001), and the target Block number determination unit 134 has the target Block number. limit below K times the number of target MB (step 17002～ step 17003), the result if, as long as the remaining operational resources of PC310 remain more C _IMC (step 17004), the target number of MB determination unit 136 The target number of MBs is incremented by 1 (step 17005), and if the result is that the remaining computational resource amount of the PC 310 remains equal to or greater than C _IDCT (step 17006), the target number of blocks is recalculated (step 17007).

The above is repeated until the amount of remaining computing resources of the PC 310 does not remain greater than C _IMC or C _IDCT .

  In the process shown in FIG. 8, the target block number determination unit 134 determines the target block number (step 15003 in FIG. 4), and the target MB number determination unit 136 determines the target MB number (step in FIG. 7). 16003) is performed immediately after the processing shown in FIG. 8 is completed, and subsequent processing (step 15004 in FIG. 4 and step 16004 in FIG. 7) is performed.

  The MB number control unit 137 operates in the same manner as the Block number control unit 135, thereby limiting the number of MB 14003 instead of the number of Block 14004.

The MB number control unit 137 converts the target number of MB14003 per picture to the target number of MB14003 per predetermined data unit (for example, slice) by (Equation 9) or (Equation 10) (step 16004). The number of MBs is limited to be equal to or less than the target number (step 16005), and as a result, the number of blocks 14004 per picture 14001 included in the information output by the MC 112 is limited.

In (Expression 9) and (Expression 10), T _{MB_X} is the target number of MBs per predetermined data unit (for example, Slice), T _{MB_picture} is the target number of MBs per _Picture 14001, and N _{X_MAX} is the number of data units included in 1 _Picture 14001. , N _X is the total number of the data units temporarily stored in the MB number control unit 137 so far (reset to 0 at the beginning of the picture), and O _MB is the MB output from the MB number control unit 137 so far (Reset to 0 at the beginning of the picture).

  However, as a result of the block number control unit 135 limiting the number of blocks, all blocks included in a certain MB may be lost, and as a result, the number of MB14003 per picture 14001 may decrease.

Therefore, when the number of MB 14003 decreases, the block number control unit 135 notifies the MB number control unit 137 of the decrease number, and the MB number control unit 137 takes into account the decrease number (formula 11) or (formula 12), it is desirable to determine the target number of data number of MB14003 data unit per predetermined data unit.

In (Expression 11) and (Expression 12), D _MB is the number of MBs decreased in the block number control unit 135.

  As described above, the calculation amount adjustment device according to the second embodiment and the moving image processing system including the calculation amount adjustment device have been described only in the points different from the first embodiment. From the above, it is apparent that the same effect as in the first embodiment can be obtained even when the calculation amount of the IMC 124 varies.

  Further, in Embodiment 1, it is assumed that the calculation amounts of VLD 121, IQ 122, and IMC 124 in decoding unit 120 do not vary or are fixed, but in this, the calculation amounts of VLD 121 and IQ 122 vary. Variations applicable to the case can also be considered.

  In that case, similarly to the first embodiment and the second embodiment, a target coefficient number determination unit and a coefficient number control unit are newly provided (the coefficient number control unit is newly provided after the block number control unit 135) to determine the target coefficient number. The unit determines the target coefficient number based on the outputs of the calculation resource amount measurement unit 131 and the image quality evaluation unit 132 and the target image quality 133, and the coefficient number control unit sets the coefficient 14005 per picture 14001 included in the encoded information. Limit the number. Thus, it goes without saying that the same effect as in the first embodiment can be obtained.

  The target coefficient number determination unit and the coefficient number control unit operate in the same manner as the target block number determination unit 134, the target MB number determination unit 136, the block number control unit 135, and the MB number control unit 137, respectively. Determine the target number and limit the number.

(Embodiment 3)
In the present embodiment, a moving image processing system having a function of adjusting the calculation amount of the encoding unit and a calculation amount adjusting device included in the moving image processing system will be described with reference to FIG.

  First, a functional configuration of a moving image processing system including a calculation amount adjustment device will be described below.

  FIG. 9 is a block diagram showing a functional configuration of the image processing system according to the third embodiment of the present invention. 9, ME 111, MC 112, DCT 113, Q 114, VLC 115, IQ 116, IDCT 117, IMC 118 in encoding section 110, and VLD 121, IQ 122, IDCT 123, IMC 124 in decoding section 120 are the first and second embodiments. It is the same as the part explained in.

  In addition, the calculation resource amount measurement unit 131, the image quality evaluation unit 132, the target image quality 133, the target block number determination unit 134, and the block number control unit 135 are also the same as those described in the first and second embodiments. However, the computing resource amount measuring unit 131 measures the computing resource amount and the remaining computing resource amount used by the encoding unit 110 of the computing device in which the encoding unit 110 is operating, not the decoding unit 120. .

  The difference between the first embodiment and the second embodiment is that the target MB number determination unit 136 is changed to the target MB number and target block number determination unit 138, and the MB number control unit 137 is the MB number and block. The point is that the number control unit 139 is changed.

  Next, the basic concept regarding the adjustment of the calculation amount in the third embodiment will be described below. In the first embodiment and the second embodiment, the number of blocks 14004 per picture 14001 is limited, and if necessary, the number of MB 14003 per picture 14001 and the number of coefficients 14005 per picture 14001 are limited, thereby decoding. The calculation amount of the unit 120 was controlled.

  In the encoding unit 110, the calculation amounts of the ME 111 and the MC 112 are fixed because the number of MB 14003 constituting the Picture 14001 is always processed. Further, since IMC 118, IDCT 117, IQ 116, and Q 114, VLC 115 are the same as or inversely converted to IMC 124, IDCT 123, IQ 112, and VLD 121 in decoding section 120, in the same manner as in the first and second embodiments, The amount of calculation can be controlled.

  Therefore, if the amount of calculation used by the DCT 113 remaining in the encoding unit 110 is controlled, the overall amount of calculation of the encoding unit 110 can be controlled.

  Next, the operation of the calculation amount adjustment apparatus according to the third embodiment will be described below only in matters different from those described in the first and second embodiments.

  The calculation resource amount measurement unit 131, the image quality evaluation unit 132, the target block number determination unit 134, and the block number control unit 135 perform the same operations as those described in the first and second embodiments, respectively.

  The target MB number and target block number determination unit 138, the MB number and block number control unit 139 perform the same operations as the target MB number determination unit 136 and the MB number control unit 137, respectively, and the target block number determination unit, respectively. The same operation as that of the block number control unit 135 is also performed.

  As described in the background art, since the DCT 113 process in the encoding unit 110 is performed in block 14004 units, the number of blocks 14001 per picture 14001 included in the information input to the DCT 113 is determined by the MB number and block number control unit 139. By limiting, the calculation amount of the DCT 113 is controlled.

  The target MB number and target block number determining unit 138 determines the target block number, and the MB number and block number control unit 139 limits the block number. The procedure for determining the number of blocks is the same as the procedure for limiting the number of blocks.

  As a result of limiting the coefficients by Q114, all the coefficients included in a certain block may disappear. Therefore, the target MB number and the target block number determined by the target block number determination unit 138 are the targets determined by the target block number determination unit 134. It is better to make it larger than the number of blocks. This is the same relationship as that in the second embodiment, when the target number of blocks is K times the target number of MBs or more, it becomes meaningless.

  Heretofore, only the matters different from the first embodiment and the second embodiment have been described regarding the calculation amount adjustment device in the third embodiment. As described above, the calculation amount adjustment apparatus of the present invention is necessary only for the encoding unit 110 to set the calculation amount to the remaining calculation resource amount of the calculation device in which the encoding unit 110 is operating or the target image quality 133. Obviously, it can be limited to less than the amount of computation.

(Embodiment 4)
In the present embodiment, the amount of encoded information and the quality of the decoded image in the calculation amount adjusting apparatus described in the first to third embodiments and the moving image processing system including the calculation amount adjusting apparatus are described. The relationship will be described with reference to FIGS. A method for alleviating the degradation of image quality recognized by the user will also be described.

  First, the result of simulating encoding / decoding by a moving image processing system including the calculation amount adjustment apparatus of the present invention is shown below. In this encoding / decoding simulation, a series of 4: 2: 0, 640 × 480 pixels, 600 images is used as an encoding target image.

  FIG. 10 shows a part of the encoding target image. This series of images is taken from the ceiling, and a person moves from right to left (Picture 70 to Picture 130) and from the front to the back and from the back to the front (Picture 310 to Picture 500).

  The moving image processing system uses the MPEG-2 system as an encoding system and encodes this series of images with a CBR having a motion detection range of ± 31.5, M = 1, N = 150, and a generated code amount of 4 Mbit / sec. To do.

  FIG. 11 shows decoded pictures of the Picture 70 for the four cases where the number of blocks is not adjusted or is adjusted at different degrees. Also, for each of the four cases, FIG. 12 shows the image quality (PSNR) of the luminance (Y) of the decoded image, and FIG. 13 shows the average value of the Q step width used in Q114 in one Picture 14001 period. .

  Here, “no block number adjustment” is a result of encoding / decoding with normal MPEG-2 without using the calculation amount adjusting apparatus of the present invention.

  Further, “adjusting the number of blocks to ½ or less” is a result of adjusting the number of blocks 14004 per picture 14001 included in the encoded information to ½ or less using the calculation amount adjustment apparatus of the present invention. Specifically, the number of blocks is adjusted to ½ or less, that is, 2400 or less, with respect to the maximum number of blocks 14004 of 8 × 8 pixels size included in one picture 14001 having a size of 640 × 480 pixels. ing.

  “Adjusting the number of blocks to 1/3 or less” and “adjusting the number of blocks to 1/6 or less” are the same as “adjusting the number of blocks to 1/2 or less”, and the number of blocks per picture respectively. Is adjusted to 1/3 or less (1600 or less) and 1/6 or less (800 or less). It should be noted that the average number of data per block 14004 data unit included in the encoded information at the time of “no block number adjustment” was about 4380.

  As shown in the decoded image of FIG. 11, as the number of blocks is reduced to 1/2, 1/3, and 1/6, the decoded image has a block noise (a phenomenon in which image quality deterioration is observed in an 8 × 8 size block shape). ) Appears, but the image quality is maintained at such a level that it is sufficiently practical for monitoring applications, for example.

  This result also appears in the image quality (PSNR) of luminance (Y) shown in FIG. The number of blocks when “adjusting the number of blocks to 1/6 or less” is 800 or less, and the number of blocks is 4 dB even though the number of blocks is greatly reduced from about 4380 when “no adjustment of block number” is set. The image quality has deteriorated only to the extent. This is because as the number of Blocks is reduced, the amount of other encoded information of the Block that has not been reduced increases by the amount of the encoded information of the Block that has been reduced. For this reason, the amount of encoded information increases for other blocks that have not been reduced, and as a result, the image quality is improved.

  This also appears in the average Q step width shown in FIG. As the number of blocks is reduced to 1/2 or less, 1/3 or less, or 1/6 or less, the average value of the Q step width used when the encoded information is quantized in Q114 is decreased (the step width is reduced). . That is, it can be seen that the amount of encoded information for the blocks that have not been reduced has increased.

  As described above, according to the encoding / decoding simulation result, even if the number of blocks is greatly reduced by the calculation amount adjustment apparatus of the present invention, the accompanying image quality deterioration is not so much as to impair the practicality. In addition, by performing the items described below, it is possible to further alleviate the impression of image quality degradation experienced by the user.

  First, as shown in the decoded image in FIG. 11, image quality degradation due to the calculation amount adjustment apparatus of the present invention limiting the data unit included in the encoded information appears as Block noise. When this block noise appears continuously at a specific position, or when it appears at a position that is noticed by the user, such as a moving object such as a person, the image quality impression received by the user is greatly reduced. In addition, this block noise appears at the reduced block.

  Therefore, the following items are added to the selection criteria for the block to be removed by the block number control unit 135 described in the first embodiment. First, a block in which a block removed in one or a plurality of previous pictures in time is present is excluded from the removal target block.

  This processing can be easily realized if the block number control unit 135 records the positions of the blocks removed in the previous one or a plurality of pictures in a memory or the like, and makes a determination based on this recording. Block noise can be prevented from continuously appearing at a specific position.

  Next, the block with large movement is removed and excluded from the target block. A user often pays attention to a part with a lot of movement such as a moving person or an object, and by removing a block with a large movement and removing it from the target block, it is possible to prevent a block noise from appearing at a position where the user pays attention. .

  This process can be easily realized by the block number control unit 135 determining the degree of movement using information on movement between images. Information detected by the ME 111 can be used as information on movement between images.

  It should be noted that not only the location where the movement is large, but also a position that is considered to be noticed by the user may be specified, and the Block existing at the specified position may be removed and excluded from the target Block. Such a position is, for example, a position of a person / object extracted by image recognition, and is a position detected using a technique for detecting a user's gaze position.

  Next, as shown by the luminance (Y) image quality (PSNR) in FIG. 12, the degree of variation in which the image quality changes with time as the number of blocks is reduced to 1/2 or less, 1/3 or less, or 1/6 or less. Is getting bigger. As this variation is larger, the difference in image quality before and after time can be seen by the user, and the impression of image quality may worsen.

  Therefore, the image quality evaluation unit 132 also measures the temporal variation degree of the image quality together with the image quality measurement of each picture, and when the variation degree exceeds a certain value, notifies the user to that effect. The user is prompted to reset the target image quality 133. Alternatively, the amount of computing resources used by other moving image processing systems and applications can be reduced by notifying the calculation amount adjusting device in other moving image processing systems operating in parallel or other applications, etc. The amount of calculation resources that can be used in the calculation amount adjustment device of the invention is increased. As a result, it is possible to prevent a deterioration in image quality impression due to image quality variations.

  Although the control of the number of blocks has been described above, the same applies to the control of the number of MBs, and the selection criterion of the removal target MB of the MB number control unit 137 described in the second embodiment is added to the items described above. It is clear that the same effect can be obtained.

(Embodiment 5)
In the present embodiment, some supplementary matters for use of the calculation amount adjusting apparatus described in the first to fourth embodiments will be described with reference to FIGS.

  First, in the calculation amount adjustment apparatus of the present invention, a part of coding information that has a function of eliminating coding errors that accumulate in time is excluded from a target to be removed. This will be described below.

  MPEG and ITU-TH In the standard of the moving picture coding / decoding scheme of the 260 series, the original picture 101 to be sequentially coded is I-Picture (Intra Picture, intra-frame coded picture), P-Picture (Predictive Picture, one-way motion predictive coding. Image) or B-Picture (Bi-directional Predictive Picture, bi-directional motion prediction encoded image).

  Of these pictures, only I-pictures are images that are encoded without using motion-compensated prediction that reduces temporally redundant information. The I-Picture is required to eliminate coding errors that accumulate over time, and the original image 101 that is sequentially encoded is encoded as this I-Picture at a constant period.

  Similarly to I-Picture, in order to eliminate coding errors accumulated in the time direction, all data of one or a plurality of Slice 14002 data units included in P-Picture are stored in Intra MB (Intra Macro Block, Intraframe). (Encoded macroblock) may be encoded (hereinafter referred to as Intra Slice). Intra MB is an MB that is encoded without using motion-compensated prediction that reduces temporally redundant information.

  The calculation amount adjustment apparatus of the present invention operates only by removing only slices other than Intra Slice in P-Picture and data units (MB, Block, Pixel, coefficient) included in B-Picture, and on the other hand, I-Picture. Are excluded from the objects of removal, that is, they do not operate on them.

  This is because the operation amount adjustment apparatus of the present invention inhibits the function of eliminating the accumulated coding error by discarding MB 14003, Block 14004, Pixel, or coefficient 14005 in I-Picture and Intra Slice. Is to prevent. However, this is not the case when it is desired to limit the calculation amount of the encoding unit 110 and the decoding unit 120 even if the function is intentionally weakened.

  Next, a description will be given of the difference in calculation amount and its setting when encoding with I-Picture, P-Picture, and B-Picture.

  As described above, in the intra slice included in the I-Picture and the P-Picture, the ME 111 and the MC 112 do not operate because they are encoded without using motion compensation prediction that reduces temporally redundant information. In addition, the calculation amount adjustment device of the present invention may not operate for these.

  Furthermore, in P-Picture and B-Picture, the calculation amounts of ME 111 and MC 112 may differ between P-Picture and B-Picture due to differences in the search range and prediction method of motion vectors used in motion compensation prediction. As a result, the calculation amount of the encoding process in I-Picture, P-Picture, and B-Picture may be different.

  However, since the intra slice portion included in I-Picture and P-Picture does not use motion compensation prediction that reduces temporally redundant information, the amount adjustment apparatus and method of the present invention operate on these. If not, the number of MB 14003 and the number of Block 14004 per Picture 14001 included in the encoded information are fixed. For this reason, the amount of calculation of the Intra Slice part included in I-Picture and P-Picture is a certain fixed value.

  Further, in the portion other than Intra Slice included in P-Picture and B-Picture, there is a difference in the amount of calculation of ME 111 and MC 112, but these amounts of calculation are predetermined constant values, respectively. The calculation amount is limited to a certain value or less by the calculation amount adjustment device. For this reason, the part other than Intra Slice included in the P-Picture and the calculation amount of the B-Picture are also constant values.

  In addition, although the calculation amounts in the encoding processing in I-Picture, P-Picture, and B-Picture are different, each calculation amount is a certain fixed value.

  These constant values can be easily set in advance by using the search range of the motion vector of the ME 111, the setting value of the prediction method of the MC 112, and the setting value (setting of the remaining calculation amount, etc.) of the calculation amount adjusting device of the present invention. Can do.

Therefore, in the decoding unit 120, respectively fixed amount of the above-described amount of calculation C _{p_D} when calculating the amount of C _{I_D,} the P-Picture is decoded at the time when the I-Picture is decoded, B-Picture There each computation amount C _{B_D} when being decoded, is set to a value which satisfies the relationship shown in equation (13).

MPEG and ITU-TH In the moving picture coding / decoding scheme standards of the 260 series, as shown in FIG. 14, the original picture 101 is coded with I-Picture, P-Picture, and B-Picture at a predetermined fixed period. This one cycle is a period from one I-Picture to the next I-Picture.

In (Equation 13), this one period, N _I is the number of images to be encoded / decoded number of images to be encoded / decoded in I-Picture, with N _{P P-Picture,} N _B is B -The number of pictures to be encoded / decoded with Picture, C _{picture_D} is the amount of computation of the decoding unit 120 per one Picture 14001 data unit.

Also, in the encoding unit 110, respectively fixed amount of the above-described amount of calculation C _{P_E} when calculating the amount of C _{I_E,} the P-Picture is coded when the I-Picture is coded, B-Picture There each computation amount C _{B_E} when it is encoded, is set to a value which satisfies the relationship shown in (equation 14).

  FIG. 15 shows a temporal flow of encoding / decoding processing for I-Picture, P-Picture, and B-picture of the encoding unit 110 and the decoding unit 120. As shown in FIG. 15, as long as the values satisfy the relations of (Equation 13) and (Equation 14), the image input, the encoding process of the encoding unit 110, the decoding process of the decoding unit 120, the image Output / display can be performed.

(Embodiment 6)
Finally, in the present embodiment, a more specific usage example of the calculation amount adjustment apparatus described in the first to fifth embodiments will be described with some usage scenes.

  First, instead of playing a program recorded on a video recording / playback device at home, such as the night before, by using the video recording / playback device, the encoded information of the program is transferred / stored on a mobile terminal or the like, and commuting There are many usage scenes such as playing on the go.

  A portable terminal needs to be reduced in size and power consumption in order to improve portability, and its calculation capability and recording capacity are less than that of a stationary device such as a video recording / reproducing apparatus. For this reason, when the encoded information of the program encoded so as to conform to the computing capability of the video recording / playback apparatus is played back on the mobile terminal, a part of the picture 14001 is not played back (so-called frame dropping). Will occur.

  Therefore, the encoded information of the program recorded by the video recording / reproducing apparatus is once decoded, encoded again based on the computing capability of the mobile terminal, and then transferred to the mobile terminal.

  Then, at the time of encoding, the calculation resource amount of the mobile terminal in which the decoding unit 120 operates is measured by the calculation resource amount measurement unit 131 using the calculation amount adjustment device of the present invention, and based on this, the target block number determination unit 134, the block number control unit 135, the target MB number determination unit 136, the MB number control unit 137, and the like limit the number of MB 14003 data units, Block 14004 data units, and the like per 1 Picture 14001 data unit included in the encoded information.

  Thereby, the encoded information transferred to the portable terminal is completely reproduced without dropping frames on the portable terminal. The fact that frames are not dropped means that the encoded information does not include information that cannot be reproduced by the mobile terminal, and the mobile terminal that records the encoded information has a small recording capacity. It is possible to prevent being pressed by information that cannot be reproduced.

  Next, there are many use scenes where a plurality of users exchange each other's images in real time, such as a TV conference.

  In this usage scene, for example, as shown in FIG. 16, while the user 12001A uses the PC 310A and the user 12001B is in the video conference using the PC 310B, the user 12001C newly participates in the video conference using the PC 310C. Think about the case.

  When the user 12001C newly participates in the TV start, the PC 310A newly activates the decoding unit 120CA, the PC 310B newly activates the decoding unit 120CB, and the PC 310C newly decrypts the decoding unit 120AC, the decoding unit 120BC, It is necessary to activate the encoding unit 110C.

  If the encoding unit and decoding unit that are operating earlier do not have a function for adjusting the amount of computing resources, the amount of computing resources used by the encoding unit and decoding unit that are newly activated in these PCs is secured. If not, the user 12001C cannot participate in the TV conference.

  Therefore, by adjusting the calculation amount of the encoding unit and the decoding unit using the calculation amount adjustment apparatus of the present invention, the user 12001C can participate in the TV conference.

  Before the user 12001C participates in the TV conference, as shown by solid lines in FIG. 16, two sets of moving image processing systems each composed of one encoding unit and one decoding unit operate in the PC 310A and the PC 310B. ing. One calculation amount adjusting apparatus of the present invention is included in each of these moving image processing systems.

  Further, when the user 12001C participates in the video conference, as shown by a solid line and a dotted line in FIG. The image processing system operates. One calculation amount adjusting apparatus of the present invention is included in each of these moving image processing systems.

  In FIG. 16, a calculation resource amount measurement unit 131, an image quality evaluation unit 132, a target block number determination unit 133, a block number limit unit 135, a target MB number determination unit 136, an MB number control unit 137, etc. that realize this calculation amount adjustment device. Is omitted.

  The description will be continued focusing on the PC 310A. When a user 12001C participates in a TV conference, the moving image processing system including the encoding unit 110C, the decoding unit 120CA, and the decoding unit 120CB has a calculation resource amount used by the decoding unit 120CA by the calculation amount adjustment device. It is activated and operates so as not to exceed the amount of computing resources available on the PC 310A.

  If there is no computational resource amount available in the PC 310A or the decoding unit 120CA cannot be activated with that amount, the encoding unit 110A reduces the computational resource amount used by the encoding unit 110A. Codes generated by the encoding unit 110B operating on the PC 310B in order to reduce the target MB number and the target block number included in the generated encoding information and to reduce the amount of computational resources used by the decoding unit 120BA The calculation amount adjustment device in the moving image processing system operating earlier works so as to lower the target MB number and the target Block number included in the conversion information.

  The encoding unit 110A and the encoding unit 110B reduce the number of MBs and the number of blocks included in the generated encoded information by lowering the target MB number and the target block number. As a result, the encoding unit 110A and the decoding unit 110B The amount of computing resources used by the conversion unit 120BA is reduced.

  Specifically, the target MB number determination unit 136 and the target block number determination unit 134 in the calculation amount adjustment apparatus subtract only the calculation resource amount necessary for starting the decoding unit 120CA from the calculation resource amount currently available in the PC 310A. The above control can be performed by determining the target number of MBs and the number of target blocks using the value as the remaining computing resource amount of the PC 310A.

  Note that the target MB number determination unit 136 and the target block number determination unit 134 prompt the user to reset the target image quality 133 by notifying the user, thereby resulting in the number of MBs and blocks included in the encoded information. The number may be reduced.

  As a result of the above control, the remaining computation resource amount of the PC 310A becomes the computation resource amount necessary for the operation of the decoding unit 120AC, so that the encoding unit 110A, the decoding unit 120BA, and the decoding unit 120CA all operate in the PC 310A. It becomes possible. The same applies to PC 310B and PC 310C.

  Further, after the start of the encoding unit / decoding unit, if necessary, the amount of calculation of the encoding unit / decoding unit is further increased depending on the image quality of the encoded / decoded image of each encoding unit / decoding unit. You may adjust.

  For example, if the image quality of the encoded / decoded image of each encoding unit / decoding unit is substantially equal, if the target image quality 133 in each PC is set to an equal value, the target image quality 133 and the image quality Based on the image quality of the encoded / decoded image of each encoding unit / decoding unit evaluated by the evaluation unit 132, the calculation amount adjusting apparatus of the present invention can encode / decode each encoding unit / decoding unit. Each encoding unit / decoding unit is operated with a calculation amount such that the image quality of the images is substantially equal.

  In addition, if the image quality of the encoded / decoded image of the speaker of the video conference is desired to be higher than the image quality of the other encoded / decoded images, similarly, an encoding unit for encoding the image of the speaker is provided. The target image quality 133 of the moving image processing system to be included may be set higher than the target image quality of other moving image processing systems. As a result, the calculation amount adjusting apparatus of the present invention enables each code to have a calculation amount that makes the image quality of the encoded / decoded image of the speaker of the TV conference higher than the image quality of the other encoded / decoded images. Activating the decoder / decoder.

  As described above, several usage scenes have been shown for more specific usage examples of the calculation amount adjustment apparatus of the present invention. However, the present invention is not limited to this, and video of a video phone, a video server, a VOD (Video on Demand), and a surveillance system. The same can be applied to a moving image processing system that performs one or a plurality of encoding or decoding processes such as distribution.

  The calculation amount adjustment apparatus according to the present invention is a moving image processing system that generates encoded information that is finely adapted to the calculation resource amount that can be used for encoding or decoding, such as a TV phone, a video server, a VOD, and a monitoring system. Widely available.

1 is a block diagram illustrating a functional configuration of a moving image processing system according to Embodiment 1. FIG. The figure explaining the reduction | restoration of the time and space redundant information in coding information The figure which shows an example of the operating environment of the moving image processing system in Embodiment 1-3 Flowchart showing an example of the operation of the calculation amount adjustment device in the first embodiment The graph showing an example of the fluctuation | variation of the information amount stored in the Block number control part in Embodiment 1. A block diagram showing a functional configuration of a moving image processing system according to Embodiment 2 The flowchart which shows an example of operation | movement of the calculation amount adjustment apparatus in Embodiment 2. The flowchart which shows an example of operation | movement of the target MB number determination part in Embodiment 2, and a target Block number determination part FIG. 7 is a block diagram illustrating a functional configuration of a moving image processing system in a third embodiment. The figure which shows a part of encoding object image in Embodiment 4. The figure which shows the decoded image at the time of adjusting the number of Blocks in the different degree in Embodiment 4. The graph which shows the image quality (PSNR) of the brightness | luminance (Y) at the time of adjusting the number of Blocks in the different degree in Embodiment 4. The graph which shows the Q step width | variety average value at the time of adjusting the number of Blocks in the different degree in Embodiment 4 MPEG and ITU-TH The figure explaining an example of the period of I-Picture, P-Picture, and B-Picture by the video encoding / decoding system standards of the 260 series The figure explaining the time flow of the encoding / decoding process with respect to I-Picture, P-Picture, and B-Picture in Embodiment 5. FIG. 10 is a diagram illustrating a usage example in a TV conference system in Embodiment 6. MPEG and ITU-TH Block diagram showing a functional configuration of a moving image processing system compliant with the 260th moving image encoding / decoding method standard MPEG and ITU-TH The figure explaining the process data unit in the moving image encoding / decoding system specification of the 260 series

Explanation of symbols

101 Original image 102 Reference image A
103 Reference image B
104 Decoded Image 110 Encoding Unit 111 Motion Detection Unit 112 Motion Compensation Unit 113 Discrete Cosine Transformer 114 Quantization Unit 115 Variable Length Coding Unit 116 Inverse Quantization Unit 117 Inverse Discrete Cosine Transformer 118 Inverse Motion Compensation Unit 120 Decoding Unit 121 Variable length decoding unit 122 Inverse quantization unit 123 Inverse discrete cosine transform unit 124 Inverse motion compensation unit 131 Computational resource amount measurement unit 132 Image quality evaluation unit 133 Target image quality 134 Target block number determination unit 135 Block number control unit 136 Target MB Number determining unit 137 MB number controlling unit 138 Target MB number and target Block number determining unit 139 MB number and Block number controlling unit

Claims (3)

  An encoding unit that encodes a moving image into encoded information composed of a plurality of data units that are individually processed smaller than one image, and a decoding unit that decodes the encoded information into a moving image; A calculation amount adjusting device used in a moving image processing system comprising:
  A computing resource amount measuring means for measuring a remaining computing resource amount that is a computing resource amount usable by the decoding unit;
  Determining the upper limit number of data units that can be processed by the decoding unit from the measured amount of remaining computing resources and the calculation resource amount used to process one data unit by the decoding unit; The target data unit number which is the target number of data units included in the encoded information so that the image quality of the image represented by the encoded information is less than the determined upper limit number and approaches the target image quality specified in advance. A target data unit number determining means for determining
  Data unit number control means for limiting the number of data units included in the encoded information to the target data unit number or less;
  A calculation amount adjustment device comprising:   An encoding unit that encodes a moving image into encoded information composed of a plurality of data units that are individually processed smaller than one image, and a decoding unit that decodes the encoded information into a moving image; A calculation amount adjusting device used in a moving image processing system comprising:
  A calculation resource amount measuring means for calculating a remaining calculation resource amount that is a calculation resource amount usable by the encoding unit;
  The upper limit number of data units that can be processed by the decoding unit is determined from the calculated remaining calculation resource amount and the calculation resource amount used to process one data unit by the decoding unit, The target data unit number which is the target number of data units included in the encoded information so that the image quality of the image represented by the encoded information is less than the determined upper limit number and approaches the target image quality specified in advance. A target data unit number determining means for determining
  Data unit number control means for limiting the number of data units included in the encoded information to the target data unit number or less;
  A calculation amount adjustment device comprising:   The data unit is at least one of a pixel, a coefficient, a block, a macroblock, and a slice.
  The calculation amount adjustment apparatus according to claim 1 or 2.

Images