JPH11239352A - Image processing method, image processing apparatus, and data storage medium - Google Patents

Abstract

(57) [Summary] An operation load caused by motion compensation of a decoded image signal is avoided by preventing the operation load amount from exceeding the operation processing capability of the system due to the motion compensation processing of the decoded image signal. Is prevented from having a large adverse effect on the image quality of the reproduced image. SOLUTION: A prediction signal generation unit 210 for predicting an image decoding signal of a target block to be decoded by a predetermined method to generate a prediction signal is provided in a decoding process of an input coded image signal. It has a load determiner 213 that detects the magnitude of the computation load, and switches the prediction method in the prediction signal generation processing according to the detected magnitude of the computation load.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing method, an image processing apparatus, and a data storage medium, and more particularly, to a method of processing a video signal even if the processing capability of the system is low. Method and apparatus capable of encoding or decoding an image signal without causing
Also, the present invention relates to a data storage medium storing a program for realizing encoding and decoding of an image signal by software using such a method and apparatus.

[0002]

2. Description of the Related Art In a conventional digital moving picture coding method, a moving picture has a strong temporal correlation, that is, a strong correlation of image information between screens. In order to reduce the temporal redundancy, the image signal of the target screen to be encoded is obtained from the image signal of the previous screen which has already been subjected to the encoding process before the target screen. And a process of encoding a difference value from the predicted signal to be obtained. Such an encoding process is called a differential encoding process, and is performed for each unit area that divides one screen. Here, the one screen is an MPEG (Moving P
icture Experts Group) 1 and 2 frames, MPE
G4 corresponds to an object area which is an image display space corresponding to each object image.

In the above-described differential encoding process, specifically, an error from an image signal (image input signal) of a target unit area to be subjected to the encoding process is minimized by motion estimation based on image information of the previous screen. A prediction area having a small image signal in the previous screen is detected, and the image signal of the prediction area is converted to a prediction signal (prediction image input signal) corresponding to the target unit area.
Then, a difference signal which is a difference value between the image signal of the target unit area and the prediction signal is encoded together with the prediction related information related to the motion prediction. Here, the prediction-related information is, for example, information indicating a method of motion prediction or a position of a prediction area, that is, a motion vector.

In a differential decoding process for reproducing an image signal from a differentially coded signal generated by the differential encoding process, a target to be decoded by motion compensation based on the prediction related information. A prediction signal (predicted image decoding signal) is generated by predicting an image reproduction signal (image decoding signal) of the unit area, and a difference coded signal corresponding to the target unit area is decoded to generate a difference decoding signal. Then, the prediction signal is added to the decoded difference signal to generate an image reproduction signal corresponding to the target unit area.

As described above, in the coding or decoding of an image signal, techniques for compressing image information such as motion prediction and motion compensation have been conventionally used. Practical examples of these techniques are as follows. , Specifically, M
Information compression processing based on international standards such as PEG1 and MPEG2 can be mentioned.

[0006] MPE which is currently being standardized is
In the technology based on G4, studies are being made on introducing a new motion prediction and motion compensation method as well as a motion prediction and motion compensation method similar to MPEG1 and MPEG2. Here, the motion prediction is a process of detecting information indicating the position of a prediction area, that is, a motion vector, in an encoding process, and the motion compensation is a process of encoding or decoding based on the motion vector in a coding process or a decoding process. This is a process for generating a prediction signal.

As an example of a method which is being considered for introduction to such an image processing technology based on MPEG4, overlap motion compensation is mentioned.
The normal motion compensation and the overlap motion compensation in the 2-compliant image processing technology will be described. Here,
For simplicity of description, only a luminance signal is shown as a signal to be subjected to motion compensation. In addition, the overlap motion compensation is generally performed only during the decoding process (including the local decoding process at the time of encoding), and thus the following description will be made regarding the decoding process.

First, in the decoding processing corresponding to the conventional MPEG1 and MPEG2 standards, motion compensation is performed for 16 pixels in the vertical direction.
This is performed using a two-dimensional image space (macroblock) consisting of 256 pixels in which 16 pixels are arranged in the horizontal direction as a unit area.

FIGS. 7A to 7D are diagrams for explaining the motion compensation processing in the decoding processing. As shown in FIG. 7 (a), on the decoding side, a motion vector is assigned to the target macroblock MB (n) in the frame F (n) being decoded at the current time t (n). MV
Using (n), the prediction region PR in the previous frame F (n-1) that has been decoded at the past time point t (n-1)
(N) is required. This predicted area PR (n) is located on the previous frame F (n-1) at the same position as the target macroblock MB (n) in the frame F (n) to be processed, and is located at the same position as the macroblock MB (n-1). Is located at a position apart from by the displacement amount indicated by the motion vector MV (n).

[0010] An image signal of the prediction area PR (n) is a prediction signal (predicted image decoding signal) corresponding to the image decoding signal of the target macroblock MB (n).
Is required. 7 (b), (c), and (d) show the co-located macroblock MB (n-1), the prediction area PR (n), and the target macroblock MB (n), respectively.
(a) is extracted and shown.

On the other hand, in the overlap motion compensation processing, eight pixels are arranged in the vertical direction and eight pixels are arranged in the horizontal direction.
This is performed using a two-dimensional image space (block) composed of pixels as a unit area. There are four such blocks, which correspond to one macroblock. In other words, one macro block is configured by arranging two blocks in the vertical direction and two blocks in the horizontal direction as shown in FIG.

In the overlap motion compensation processing,
If there is a motion vector corresponding to each block or to each macroblock, and this motion vector corresponds to a macroblock, it corresponds to four blocks in the macroblock. The motion vectors are all assumed to be the same motion vector.

Next, specific overlap motion compensation will be described with reference to FIG. Here, a case will be described in which a predicted signal (predicted image decoded signal) for the target block BC (n) of the frame to be processed F (n) is generated. First, the motion vector MVC of the target block BC (n) and its surrounding blocks BU (n), B
Motion vector MV of D (n), BR (n), BL (n)
U, MVD, MVR, MVL,
On (n-1), five prediction regions for the target block BC (n) are obtained based on the same-position block BC (n-1) at the same position as the target block BC (n). Here, the motion vectors MVU, MVD, MV
R and MVL are above the frame to be processed F (n),
Lower, right, left peripheral blocks BU (n), BD (n), BR
(N), regions PU (n), PD (n), PR in the previous frame F (n-1) as prediction regions for BL (n)
(N) and PL (n), the target block BC (n) calculated using these motion vectors
The four prediction regions are the regions PU shown in FIG.
(N) ', PD (n)', PR (n) ', PL (n)'
Becomes Further, a prediction area PC (n) for the target block BC (n) is obtained based on the motion vector MVC of the target block C (n) itself.

Next, five prediction areas PC (n), PU (n) ', PD for the target block BC (n)
From the pixels in (n) ', PR (n)', PL (n) ', the predicted images PGC, PGU, PG shown in FIG.
D, PGR, and PGL are obtained. However, here, unnecessary pixels in each prediction region are not used, and the prediction region PU
For (n) ', the prediction region PD
For (n) ', the prediction region PR
For (n) ', the prediction area PL
For (n) ', a predicted image is formed from the left half pixel.

Then, each predicted image PGC, PGU, PG
D, PGR, and PGL are represented by pixel values of respective pixels, and weighting matrices WVC, WVU, WVD, WVR, W
VL (see FIG. 10 (c)) is weighted with the value shown in FIG. Here, the weighting value indicated by each weighting matrix becomes 8 when all of the weighting values for one pixel are added. Therefore, each of the predicted images PGC, PGU, PGD, PGR, and PGL is calculated.
Are Pgc, Pgu, Pgd, Pgr, and Pgl, and the weighting values indicated by each weighting matrix are Wvc, Wvu, Wvd,
Assuming Wvr and Wvl, the pixel value Pg of the composite image PG is obtained by the following equation. Pg = (Pgc · Wvc + Pgu · Wvu + Pgd · Wvd + Pgr ·
Wvr + Pgl · Wvl) / 8 The composite image PG obtained by this synthesis processing is a prediction image obtained by overlap motion compensation, and the corresponding image signal is a prediction signal (predicted image decoded signal).
Becomes

In the above-described overlap motion compensation, when the target block to be subjected to motion compensation is located at the lower side of the macro block, the macro block including the peripheral block below the target block is Since the decoding process has not been performed yet, the motion vector of the target block is used as the motion vector of the lower peripheral block. Also, it goes without saying that the overlap motion compensation in the local decoding process at the time of encoding is performed in the same manner as the overlap motion compensation in the decoding process.

As described above, in the ordinary motion compensation, a predicted image (prediction signal) is generated based only on a motion vector of an area (macroblock) serving as a unit of motion compensation. In motion compensation, a prediction image (prediction signal) is generated based on not only a motion vector of a processing unit (target block) to be subjected to overlap motion compensation but also a motion vector of a peripheral block adjacent to the target block. I have. For this reason, in the overlap motion compensation processing, an image signal corresponding to the target block (strictly, an image value corresponding to each pixel) and a prediction signal thereof (strictly, corresponding to each pixel) are different from the normal motion compensation processing. (A predicted value to be obtained) is smoothed in one block, that is, a large local coding distortion in a macroblock can be dispersed and small, thereby improving visual image quality. On the other hand, more arithmetic processing is required as compared with normal motion compensation processing.

Next, a description will be given of the motion compensation on a per-object basis in the decoding processing corresponding to the conventional MPEG4 standard. In image display conforming to MPEG4, the position of the object area in the frame is relatively different between the frame to be processed and the previous frame. Briefly described with reference to FIG. 12, on the decoding side, as shown in FIG.
A predetermined object area OR in the frame to be processed F (n) that is being decoded at the current time t (n)
For the target macroblock MB (n) in (n), using the motion vector MV (n), a past time point t (n−
The prediction region PR (n) in the object region OR (n-1) corresponding to the object region OR (n) in the previous frame F (n-1) on which the decoding process is performed in 1).
Is required.

This prediction area PR (n)
On (n-1), the target macroblock MB in the object area OR (n) in the frame to be processed F (n)
Co-located macroblock MB (n) at the same position as (n)
(Indicated by a dotted line), it is located at a position separated by a displacement amount indicated by the motion vector MV (n).

Here, in the previous frame F (n-1), the position P of the object area OR (n) at the current time t (n)
(N) is the object area OR (n−n) of the past time point t (n−1) corresponding to this object area OR (n).
Although the position is different from the position P (n-1) of 1), the position Pmb (n) of the macroblock MB (n) in the object area OR (n) in the frame F (n) to be processed and the previous frame F Object area OR (n in (n-1))
-1), the position Pmb (n) of the macroblock MB (n-1)
-1), and the position Ppr (n) of the prediction region PR (n).
Can also be determined as an absolute position based on the left corner position P0 of each frame. Then, the prediction region PR
The image signal of (n) corresponds to the frame to be processed F
It is obtained as a prediction signal (predicted image decoded signal) corresponding to the image decoded signal of the target macroblock MB (n) in (n).

It should be noted that the overlap motion compensation processing corresponding to MPEG4 is performed in the same manner as in MPEG1 and MPEG2 by arranging 8 pixels in the vertical direction and 8 pixels in the horizontal direction.
Using a two-dimensional image space (block) composed of pixels as a unit area, an object area OR as shown in FIG.
The processing is performed between (n) and OR (n-1) as described with reference to FIG. As described above, in order to improve the coding efficiency of an image signal, various prediction signal generation methods have been proposed and incorporated into the coding method standards. Is considered to be devised.

[0022]

However, as described in the above description of the overlapped motion compensation, in many cases, new motion prediction processing and motion compensation processing contribute to improvement of coding efficiency, while The arithmetic processing becomes complicated as compared with the conventional one, and in order to perform real-time processing, a high-speed arithmetic performance is required for the image processing system.

In a conventional system for performing image encoding and image decoding processing, when decoding, motion compensation is performed by an arithmetic process based on a prediction signal generation method determined at the time of encoding. If the calculation processing capacity becomes insufficient in the conversion processing, the frames reproduced by the real-time processing may drop frames. In other words, in the conventional image processing system, if the operation processing capacity is insufficient due to the introduction of a new motion prediction process or a motion compensation process, the image quality of a reproduced image is greatly adversely affected.

SUMMARY OF THE INVENTION The present invention has been made to solve the above-described problems, and a target image signal to be subjected to image processing according to the state of a calculation load at the time of encoding or decoding an image signal. The signal generation process that generates the prediction signal can be switched between the one with a heavy computational load and the one with a light computational load, thereby minimizing the adverse effect on the reproduced video and operating with high coding efficiency. It is an object of the present invention to provide an image processing method and an image processing device capable of performing a prediction process or a motion compensation process, and a data storage medium storing a program capable of realizing image processing by the method and the device by a computer. I do.

[0025]

An image processing method according to the present invention (claim 1) encodes an image input signal for each unit area dividing a screen, outputs an image encoded signal, and outputs the image encoded signal. Based on an image decoded signal obtained by decoding the encoded signal, a signal generation process of generating a predicted image input signal corresponding to a target unit area to be subjected to the encoding process by an operation based on a predetermined prediction method, An image encoding method performed on a required unit area, comprising detecting a magnitude of an arithmetic load in the encoding processing of the image input signal, and performing the signal generation processing in accordance with the detected arithmetic load. The prediction method is switched.

According to a second aspect of the present invention, in the image processing method according to the first aspect, a threshold value of a calculation load based on a calculation processing capability necessary for the signal generation processing in real time; Based on the result of comparison with the size of the signal, the prediction method in the signal generation processing is performed between the first prediction method that requires a large amount of calculation processing and the second prediction method that requires a small amount of calculation processing. Is to switch.

According to a third aspect of the present invention, in the image processing method according to the first aspect, a threshold value of a calculation load based on a calculation processing capability required for the signal generation processing in real time, and the detected calculation load is calculated. And selecting a prediction method suitable for the detected calculation load from among a plurality of prediction methods with different amounts of calculation processing as the prediction method in the signal generation processing based on the comparison result with the size of the calculation load. .

According to a fourth aspect of the present invention, in the image processing method according to the first aspect, the signal generation processing includes a motion vector indicating a position of a prediction area corresponding to the target unit area with a predetermined pixel precision. The motion prediction process to be sought and the motion compensation process to obtain an image signal corresponding to a prediction region based on the motion vector as the predicted image input signal are included, and an arithmetic processing capability necessary for the signal generation process in real time is included. Based on a comparison result between the threshold value of the calculation load based on the calculation load and the magnitude of the detected calculation load, the motion prediction method in the signal generation process is required to be a motion prediction method that requires a large amount of calculation processing. It is necessary to switch between a motion prediction method having a small amount of calculation processing and a motion compensation method in the signal generation process based on the comparison result. A motion compensation method with a large amount of arithmetic processing,
This is to switch between a motion compensation prediction method requiring a small amount of calculation processing.

According to a fifth aspect of the present invention, in the image processing method according to the first aspect, the signal generation processing includes a motion vector indicating a position of a prediction area corresponding to the target unit area with a predetermined pixel precision. The motion prediction process to be sought and the motion compensation process to obtain an image signal corresponding to a prediction region based on the motion vector as the predicted image input signal are included, and an arithmetic processing capability necessary for the signal generation process in real time is included. A motion prediction method for obtaining a motion vector with high pixel accuracy, based on a comparison result between a threshold value of the calculation load based on the threshold value and the magnitude of the detected calculation load. And a motion prediction method for obtaining a motion vector having a low motion vector.

According to a sixth aspect of the present invention, in the image processing method according to the first aspect, the signal generation processing is performed by using a motion vector indicating a position of a prediction area corresponding to the target unit area with a predetermined pixel precision. The motion prediction process to be sought and the motion compensation process to obtain an image signal corresponding to a prediction region based on the motion vector as the predicted image input signal are included, and an arithmetic processing capability necessary for the signal generation process in real time is included. Based on the comparison result between the threshold value of the calculation load based on the calculation load and the magnitude of the detected calculation load, the method of the motion compensation process in the signal generation process is performed based on the motion vector with high pixel accuracy. Switching between a motion compensation method to be obtained and a motion compensation method to obtain the predicted image input signal based on a motion vector having low pixel accuracy.

According to a seventh aspect of the present invention, in the image processing method according to the sixth aspect, based on the comparison result between the threshold value of the operation load and the magnitude of the operation load, the image processing method is based on the motion vector with high pixel accuracy. The type of the motion compensation method to be performed is switched between a first motion compensation method having a large computation load and a second motion compensation method having a small computation load. On the basis of the comparison result, the type of the motion compensation method performed based on the motion vector having the low pixel precision is determined by comparing the type of the first motion compensation method with a large computation load with the second motion compensation method with a small computation load. Is to switch.

According to the present invention (claim 8), in the image processing method according to claim 1, the signal generation processing is performed by using a motion vector indicating a position of a prediction area corresponding to the target unit area with a predetermined pixel precision. The motion prediction process to be sought and the motion compensation process to obtain an image signal corresponding to a prediction region based on the motion vector as the predicted image input signal are included, and an arithmetic processing capability necessary for the signal generation process in real time is included. Based on a comparison result between the threshold value of the calculation load based on the calculation load and the magnitude of the detected calculation load, as a method of the motion compensation process in the signal generation process, a plurality of motion compensation methods each using a motion vector having a different pixel precision are used. One is to select one.

[0033] In the image processing method according to the present invention (claim 9), the input image coded signal is decoded for each unit area dividing the screen to output an image decoded signal, and the image decoding signal is output. Image decoding for performing a signal generation process for generating a predicted image decoded signal corresponding to a target unit region to be decoded by a calculation based on a predetermined motion compensation method on a required unit region based on the signal And detecting a magnitude of an operation load in the decoding process of the image-encoded signal, and switching a motion compensation method in the signal generation process according to the detected operation load. .

According to a tenth aspect of the present invention, in the image processing method according to the ninth aspect, a threshold value of an arithmetic load based on an arithmetic processing capability required for the signal generation processing in real time, and the detected arithmetic load is calculated. Based on the result of comparison with the size of
The motion compensation method in the signal generation process is switched between a first motion compensation method requiring a large amount of calculation processing and a second motion compensation method requiring a small amount of calculation processing.

According to a twelfth aspect of the present invention, in the image processing method according to the ninth aspect, a threshold value of a calculation load based on a calculation processing capability required for the signal generation processing in real time, and the detected calculation load is calculated. Based on the result of comparison with the size of
As the motion compensation method in the signal generation process, a motion compensation method suitable for the detected calculation load is selected from among a plurality of motion compensation methods having different calculation processing amounts.

According to a twelfth aspect of the present invention, in the image processing method according to the ninth aspect, a threshold value of a calculation load based on a calculation processing capability required for the signal generation processing in real time, and the detected calculation load is calculated. Based on the result of comparison with the size of
The motion compensation method in the signal generation processing includes a motion compensation method for obtaining the predicted image decoded signal based on a motion vector with high pixel accuracy, and a motion for obtaining the decoded image decoded signal based on a motion vector with low pixel accuracy. It switches between the compensation methods.

According to a thirteenth aspect of the present invention, in the image processing method according to the twelfth aspect, based on the comparison result between the threshold value of the calculation load and the magnitude of the calculation load, the image processing method is based on the motion vector with high pixel accuracy. The type of the motion compensation method to be performed is switched between a first motion compensation method having a large computation load and a second motion compensation method having a small computation load. On the basis of the comparison result, the type of the motion compensation method performed based on the motion vector having the low pixel precision is determined by comparing the type of the first motion compensation method with a large computation load with the second motion compensation method with a small computation load. Is to switch.

An image processing device according to the present invention (claim 14) comprises: an encoding processing unit that encodes an image input signal for each unit area for dividing a screen and outputs an image encoded signal; A signal generation process of generating a predicted image input signal corresponding to a target unit area to be subjected to an encoding process by an operation based on a predetermined prediction method based on an image decoded signal obtained by decoding a signal, An image coding apparatus having a prediction processing unit for a unit area, wherein the prediction processing unit includes a calculation load detection unit that detects a magnitude of a calculation load in a coding process of the image input signal. The prediction method in the signal generation process is switched in accordance with the magnitude of the detected calculation load.

According to a fifteenth aspect of the present invention, in the image processing apparatus according to the fourteenth aspect, the prediction processing unit comprises: an image input signal corresponding to a target unit area on a target screen to be coded; A motion detector that outputs a motion vector indicating a position of a prediction area in a previous screen that gives a predicted image input signal to the target unit area based on an image decoded signal corresponding to the previous screen that has been subjected to the encoding process; A motion compensator that calculates the predicted image input signal based on the motion vector of the target unit area output from the motion detector, and the motion detector detects the magnitude of the detected calculation load. And outputting either one of the first motion vector with low pixel accuracy and the second motion vector with high pixel accuracy to the motion compensator as the motion vector. It is intended.

An image processing apparatus according to the present invention (claim 16) decodes an input image coded signal for each unit area dividing a screen, and outputs an image decoded signal; A signal generation process of generating a predicted image decoded signal corresponding to a target unit region to be decoded by an operation based on a predetermined motion compensation method based on the image decoded signal, for a required unit region An image decoding apparatus having a prediction processing unit that performs the prediction processing unit.
It is provided with an operation load detecting means for detecting the amount of operation load in the decoding process of the image coded signal, and is configured to switch the motion compensation method in the signal generation process according to the detected operation load. Things.

According to a seventeenth aspect of the present invention, in the image processing apparatus according to the sixteenth aspect, the prediction processing unit is configured to perform the decoding process on the previous screen which has been subjected to the decoding process prior to the target screen to be decoded. Wherein, based on the motion vector information indicating the position of the prediction region that gives the predicted image decoded signal corresponding to the target unit region, the predicted image decoded signal is generated, and the motion vector of the target unit region is generated. Calculating a predicted image decoded signal corresponding to the target unit area based on the information, a first motion compensation process with a small amount of processing, and motion vector information of the target unit area and unit areas surrounding the target unit area; One of the second motion compensation processes that requires a large amount of calculation to calculate a predicted image decoded signal corresponding to the target unit area according to the detected calculation load. It is obtained by a structure having a motion compensator for performing selected Te.

A data storage medium according to the present invention (claim 18) is a data storage medium storing a program for causing a computer to perform an image encoding process, wherein an image input signal is converted into a unit area for dividing a screen. A predicted image input signal corresponding to a target unit area to be subjected to an encoding process, based on an image decoded signal obtained by decoding the image encoded signal while encoding the encoded image signal every time. Is performed on a required unit area by performing a calculation based on a predetermined prediction method. At this time, the magnitude of the calculation load in the coding process of the image input signal is detected, and the detected calculation is performed. The above-described program is configured so that a computer performs an image encoding process for switching a prediction method in the signal generation process according to a magnitude of a load.

A data storage medium according to the present invention (claim 19) is a data storage medium storing a program for causing a computer to perform an image decoding process. Decoding is performed for each unit area to be divided, and an image decoded signal is output. Based on the image decoded signal, a predicted image decoded signal corresponding to a target unit area to be decoded is subjected to predetermined motion compensation. A signal generation process generated by a calculation based on the method is performed on a required unit area. At this time, the magnitude of the computation load in the decoding process of the image coded signal is detected, and the magnitude of the detected computation load is detected. The program is configured so that a computer performs an image decoding process for switching a motion compensation method in the signal generation process in accordance with the above.

[0044]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiments of the present invention will be described below with reference to FIGS. Embodiment 1 FIG. FIG. 1 is a block diagram for explaining an image coding apparatus which is an image processing apparatus according to Embodiment 1 of the present invention. Image coding apparatus 100 according to the first embodiment
Is an MPEG1 or MPEG1 compliant image input signal.
Although the image signal corresponding to one object area compliant with MPEG4 as well as the image signal corresponding to the frame is configured to be processed, the basic processing is the same for any of the image signals. Typical explanation is 1
Only the processing of the frame image signal is performed. Note that M
In the case of PEG4, similarly to the encoding process of the image signal corresponding to one frame, the encoding process of the image signal corresponding to each object area is performed, and these are combined to form one image.
Output as two bit streams.

More specifically, the image encoding apparatus 100 receives an image input signal Sg, divides the image input signal Sg into a unit corresponding to a block serving as an encoding unit, and divides the image input signal Sg into an image signal (hereinafter, referred to as a unit). Based on the block generator 101 that outputs Bg and the output block Bg of the block generator 101, what mode of the encoding process is performed for the target block to be subjected to the encoding process. A mode determiner 105 that determines whether or not to apply the control signal and outputs control signals Cs1 and Cs2 according to the determination result;
A prediction signal generation unit (prediction process) that predicts a blocked image signal Bg corresponding to the target block and generates a prediction signal (predicted image input signal) Pg1 corresponding to the target block by an arithmetic process based on a predetermined prediction method. ) 110.

The image coding apparatus 100 further comprises a subtractor 106 for outputting a difference between the image signal Bg of the target block and its prediction signal Pg1 as a difference signal Bgd,
Based on the control signal Cs1 from the mode decision unit 105, the output Bg of the blocking unit 101 and the subtractor 106
Switch 102 for selecting one of the outputs Bgd
An information compressor 103 for compressing the output of the switch 102 to output an image compression signal Cg;
A variable length coder (VLC) 1 that performs a variable length coding process on the output Cg of No. 03 and outputs an image coded signal Eg
04.

Here, the information compressor 103 performs a DCT (discrete cosine transform) process, which is a kind of frequency transform process, on the image signal Bg or the difference signal Bgd of the target block, And a quantizer that quantizes frequency components (DCT coefficients) obtained by the conversion and outputs the quantized coefficients as the image compression signal Cg. The switch 102 sets the output Bg of the block generator 101 when the intra-frame encoding process using the intra-screen correlation is performed on the image signal of the target block as a result of the determination by the mode determiner 105.
Is selected, and as a result of the above-described determination, when the inter-frame encoding process using the inter-screen correlation is performed on the image signal of the target block, the output Bgd of the subtractor 106 is controlled to be selected.

The prediction signal generator 110 expands the information Cg of the information compressor 103 and outputs an image signal Bg or a difference signal Bgd as an image expansion signal LD. If the image expansion signal LD is the information-expanded image signal Bg, it is output as it is, and if the image expansion signal LD is the information-expanded difference signal Bgd, it is added to the prediction signal Pg1. And an addition processing section 111 for outputting.

Here, the information decompressor 112 is a dequantizer for dequantizing the output Cg of the information compressor 103 and a frequency domain data for the dequantized output Cg of the information compressor. Inverse DCT for transforming に into spatial domain data
IDCT that performs processing and outputs the image expansion signal LD
Container. The addition processing unit 111
Is composed of a second switch 111a that is switched and controlled by a control signal Cs2 from the mode determination unit 105, and an adder 111b that adds the output of the switch 111a and the output LD of the information decompression unit 112. I have. The switch 111a is connected to the mode determiner 105
As a result of the determination in, when the image signal of the target block is to be intra-frame encoded, the open input terminal T1 is selected. As a result of the above determination, when the image signal of the target block is inter-frame encoded, Is controlled to select the input terminal T2 to which the prediction signal is input.

The prediction signal generator 110 includes a frame memory 113 for storing the output LR of the adder 111b as an image reproduction signal for a frame preceding the frame to be processed (previous frame), and an output of the adder 111b. The load determination unit 114 receives the LR and determines the magnitude of the computational load in the encoding process for one frame, and performs the computation based on the load signal Cp indicating the magnitude of the computational load output from the load determination unit 114. And an arithmetic processing unit 120 for obtaining a prediction signal for the image signal Bg of the block.

Here, the load determiner 114 determines, for example, a load index R indicating the magnitude of the operation load by a unit time (1
The amount of calculation in the encoding process per frame) is described, and the threshold value TH of the computation load corresponding to the computation processing capability of the image encoding device 100 is used for the overlap motion compensation process using a 0.5-pixel precision motion vector. The amount of calculation in the required encoding processing per unit time is used, and the load index R is compared with the threshold value TH to determine whether or not the processing capacity is sufficient each time the motion compensation processing is performed. ing. The configuration of the load determiner 114 is not limited to the above. For example, the load determiner 114 receives only the timing at which the local decoding process for the image signal of one frame is completed from the frame memory 113 or the like. This timing interval is converted into a load index R which is a calculation amount,
A configuration may be adopted in which the processing index is determined by comparing the load index R with the threshold value TH of the calculation load. In this case, it is not necessary to send the locally decoded image signal, which is the output LR of the adder 111b, to the load determiner 114, and the amount of data sent to the load determiner 114 can be greatly reduced.

The arithmetic processing unit 120 determines, based on the output Rg1 of the frame memory 113 and the image signal Bg of the target block, an image signal having the smallest error between the image signal Bg of the target block and the previous frame. And a motion detector (ME) 121 that obtains and outputs motion displacement information (motion vector) MV indicating an area (prediction area) of the same size as the target block described above, and a motion vector MV from the motion detector 121. The above frame memory 1
And a motion compensator (MC) 122 that generates thirteen addresses Add1 and performs required arithmetic processing on the image signal in the memory area corresponding to the address to calculate the prediction signal Pg1.

In the first embodiment, the motion detector 121 receives the output Bg of the blocking unit 101 and the output Rg1 of the frame memory 113, and performs an arithmetic operation in accordance with the load signal Cp to obtain the accuracy. A low first motion vector (a motion vector with one pixel accuracy) or a high accuracy second motion vector (a motion vector with 0.5 pixel accuracy) is obtained, and either motion vector is output to the motion compensator 122. It has a configuration. That is, in the motion detector 121, when the calculation load is large, the first calculation process for generating a motion vector with one-pixel accuracy is performed. On the other hand, when the calculation load is small, the first calculation process is performed. In addition, from the 1-pixel accuracy motion vector, a second 0.5-pixel accuracy motion vector having a higher accuracy is calculated.
Is performed.

In the first embodiment, the motion compensator 122 outputs a prediction signal based on the motion vector output from the motion detector 121 based on the load determination output Cp from the load determiner 114. Is switched between normal motion compensation and overlap motion compensation. That is, in the motion compensator 122, when the calculation load is large, normal motion compensation processing with a small calculation processing amount is performed. On the other hand, when the calculation load is small, the image processing amount is large but the image quality is improved. The overlap motion compensation processing is performed.

Next, the operation will be described. FIG. 2 (a) is a flowchart for explaining an image signal encoding process performed by the image processing apparatus according to the first embodiment of the present invention.
When the image input signal Sg is input to the image encoding device 100, preprocessing is performed in step S0. That is,
The image input signal Sg input to the image encoding device 100
Are divided into blocks by a blocking unit 101 so as to correspond to a block area of a predetermined size, which divides one screen (frame) on which an image is displayed, and is output. here,
The image input signal Sg is (8 × 8) or (16 × 8)
16) The image input signal Sg is divided so as to correspond to a rectangular block area composed of samples.
It may be divided so as to correspond to a block region having an arbitrary shape.

The mode determination unit 105 determines an encoding processing mode based on the image signal (blocked image signal) Bg of each block, for example, an intra-screen encoding process using intra-screen correlation, A determination is made as to which encoding process, such as an inter-frame encoding process using correlation, is to be performed, and signals corresponding to the determination result are converted to control signals Cs1, Cs1.
Output as s2.

Next, in step S1, a process of generating a prediction signal is performed. That is, the information decompressor 112 of the prediction signal generator 110 performs the decompression process on the compressed image signal Cg and outputs the decompressed signal LD. And
When performing an inter-frame encoding process, the image expansion signal L
D is added to the prediction signal Pg1 by the adder 111b, and the image reproduction signal LR is output from the adder 111b to the frame memory 113 and the load determiner 114. When the intra-frame encoding process is performed, the prediction process is not performed.

At the same time as the image signal Bg of the target block is input to the arithmetic processing unit 120 of the prediction signal generation unit 110, the decompressed image signal LD from the frame memory 113 is already encoded. Is supplied as the image signal of the previous frame. In the arithmetic processing unit 120, based on the determination output Cp of the load determiner 114,
An arithmetic process for calculating a prediction signal is performed. Specifically, the image signal Bg of the target block is
Simultaneously with the input to the frame memory 21, the image reproduction signal Rg1 is read out from the frame memory 113 to the motion detector 121 as the image signal of the previous frame.

In the motion detector 121, the image signal Bg of the target block is obtained by a method such as block matching.
The region in the previous frame having the image signal with the smallest error is detected as the prediction region, and the position of the prediction region is determined by using the same position block in the previous frame at the same position as the target block in the processed frame (current frame). The motion displacement information (motion vector) MV shown as a reference is output. This motion vector MV is calculated by the motion compensator 122.
The motion compensator 122 generates a prediction signal from an image signal corresponding to the prediction region of the previous frame. At this time, the motion vector M for the target block
V is supplied to the variable-length encoder 104 and is converted into a corresponding variable-length code. Here, the detection of the motion vector by the motion detector 121 and the calculation of the prediction signal by the motion compensator 122 are performed based on the load determination output Cp from the load determiner 104.

More specifically, FIG. 2B is a flowchart showing the flow of the process of generating the prediction signal.
As shown in FIG. 2B, in the load determination step S11,
The load index R that indicates the magnitude of the computation load at that time is determined by the load determiner 114.

As a result of this determination, if the load index R is smaller than the threshold value TH determined by the arithmetic processing capability of the present image encoding apparatus, at step S12, a prediction signal which has a large arithmetic processing amount but can improve the encoding efficiency. Is generated. That is, the motion detector 121 first obtains a motion vector with 1-pixel accuracy, and further obtains a motion vector with 0.5-pixel accuracy based on this motion vector.
The 0.5-pixel-accuracy motion vector is the motion compensator 1
22. In addition, the motion compensator 122 combines the image signals corresponding to the plurality of prediction regions with a predetermined weighting ratio set for each pixel based on the 0.5 pixel motion vector, and generates a prediction signal. The generated overlap motion compensation is performed.

On the other hand, as a result of the load determination, when the load index R is larger than the threshold value TH, in step S13, a process of generating a prediction signal with a reduced calculation processing amount and a reduced calculation load is performed. That is, in the motion detector 121, the process of obtaining a 0.5-pixel accuracy motion vector is omitted, and the 1-pixel accuracy motion vector is calculated by the motion compensator 12.
2, the motion compensator 122 performs a normal motion compensation process of outputting an image signal corresponding to one prediction region as a prediction signal based on the one-pixel precision motion vector. Thereafter, in step S2, a compression encoding process is performed on the image signal Bg or the difference signal Bgd of the target block.

That is, the first switch 102 is controlled to be switched based on the output Cs1 of the mode determining unit 105. When the block image signal Bg is encoded in the intra-screen encoding mode, the image signal Bg is output to the information compressor 103 via the switch 102. The compression processing in the information compressor 103 is performed by frequency conversion by a DCT unit and quantization by a quantizer. For the compression processing, a method such as subband conversion or vector quantization may be used. Here, the quantized image signal Cg is supplied to the variable-length encoder 104 and subjected to variable-length encoding.

On the other hand, when the block image signal Bg is encoded in the inter-picture encoding mode, the output (difference signal) Bgd of the subtractor 106 for subtracting the prediction signal Pg1 from the image signal Bg is supplied to the switch 102. Is output to the information compressor 103 via The compression processing of the difference signal Bgd in the information compressor 103 is performed in the same manner as the image signal Bg.

As described above, in the first embodiment, the load determination unit 114 determines the magnitude (index) R of the computational load in the encoding process, and the index R of the computational load is used as the index of the computational load. If the threshold value TH is smaller than the threshold value TH corresponding to the capacity, the processing of generating a prediction signal having a large amount of processing but high encoding efficiency is performed. If it is larger than the above, the processing of generating a normal prediction signal having a smaller amount of calculation processing than the processing of generating the prediction signal is performed.
It is possible to perform a prediction signal generation process with good encoding efficiency while suppressing the adverse effect due to lack of arithmetic processing capability on a video reproduced by real-time processing on the decoding side. It is possible to realize an encoding process that is good and that reduces degradation in the image quality of a reproduced image.

For example, when ordinary motion compensation is performed instead of overlap motion compensation, the amount of data as pixel values input / output to / from the central processing unit of the image encoding device is reduced to 1/3. In addition, arithmetic processing such as multiplication processing in weighting processing in overlap motion compensation and processing for generating addresses based on motion vectors of peripheral blocks is reduced.

In the first embodiment, the load index R is a calculation amount in the encoding process. However, the load index R may be a time required for processing a predetermined calculation. .

In the first embodiment, a case has been described in which the load is determined each time a motion prediction process (detection of a motion vector) corresponding to each block is performed.
The determination of the load does not necessarily have to be performed every time the motion vector is detected.For example, the determination of the load may be performed once at the start of the encoding process, and thereafter, the same determination result may be used. Further, the processing may be performed periodically for each frame.

Further, in the first embodiment, the motion detector 121 and the motion compensator 1
In both cases, the arithmetic processing is switched in accordance with the arithmetic load.
21 or the motion compensator 122. For example, switching of pixel accuracy in the motion detector 121, that is, switching between processing for generating both a motion vector with 1-pixel accuracy and a motion vector with 0.5-pixel accuracy and processing for generating only a motion vector with 1-pixel accuracy. Or the motion compensator 12
Only the switching between the normal motion compensation processing and the overlap motion compensation in step 2 may be performed. Furthermore, as the normal motion compensation processing in the motion compensator 122, one of a plurality of motion compensation processes based on motion vectors having different pixel precisions may be selected and performed.

In the first embodiment, the motion vector generation process is realized by a hierarchical operation process.
That is, the process of generating a motion vector with 0.5 pixel accuracy is 1
This is realized by performing an additional second operation process in addition to the basic first operation process for generating a pixel-accurate motion vector. Switching of pixel accuracy is performed by the above-described additional operation process. However, the processing for generating the motion vector is not limited to the processing realized by the above-described hierarchical calculation processing. For example, as a module for generating a motion vector, a first module for performing an arithmetic process for generating a motion vector with 1-pixel accuracy;
A second module for performing an arithmetic process for generating a motion vector with 0.5 pixel accuracy may be provided, and the pixel accuracy may be switched by switching these modules.

Further, the switching of pixel precision is not limited to switching between 0.5 pixel precision and 1 pixel precision, but switching between 0.5 pixel precision and 2 pixel precision, or switching between 1 pixel precision and 2 pixel precision. Alternatively, one of 0.5 pixel accuracy, 1 pixel accuracy, and 2 pixel accuracy may be selected according to the calculation load. Further, the pixel precision is not limited to the above, and a very high precision such as 0.25 pixel precision or 0.125 pixel precision can be used.

Further, in the first embodiment, when the load index is larger than the threshold value, the processing of generating a 0.5-pixel-accurate motion vector is omitted, and the overlap motion compensation is changed to the normal motion compensation. As an example,
The method of changing the prediction method (processing for generating a prediction signal) to a prediction method with a small amount of computation processing may be to change bidirectional prediction or interlace prediction to forward prediction or frame prediction.

In addition, even if the overlapping motion compensation has the same process, if the mounting method is different,
Since the processing speeds are different, the overlap motion compensation processing may be switched between different mounting methods. The individual mounting method differs depending on what processing target (operations such as access to memory and addition / subtraction) by the CPU is optimized. For example, optimization is performed for access to memory. The mounting method is different from the mounting method optimized for the arithmetic processing.

More specifically, a first memory in which a plurality of data memories for storing image data are mounted in a memory space in software for performing overlap motion compensation.
Implementation method and the second with less data memory
It is possible to switch the processing between this and the implementation method. In this case, in the first mounting method, the memory speed used in the processing process is smaller than in the second mounting method, so that the processing speed is reduced, but the data access to the data memory is performed at high speed. If the first and second mounting methods are selectively used depending on the situation of the overlap motion compensation processing, the processing speed of the entire motion compensation processing is improved.

The calculation load in the first embodiment can be reduced by reducing the calculation accuracy in the encoding process to increase the speed of the calculation. In addition, the number of memories and access to the CPU can be reduced. The load can be reduced by increasing the calculation speed by implementing a high-speed calculation algorithm whose calculation speed depends on the amount of data, and the calculation load in the prediction signal generation processing is changed by switching the calculation accuracy and calculation algorithm. It is also possible.

Further, the switching of the processing by the motion detector or the motion compensator is not limited to the selection of one of the processing having a large amount of arithmetic processing and the processing having a small amount of arithmetic processing as in the first embodiment. One of three or more calculation processes having different calculation processing amounts may be selected. Hereinafter, such a configuration example will be described as a first modification of the first embodiment.

Modification 1 of Embodiment 1 In a first modification of the first embodiment, the arithmetic processing unit 120 is
1, one of the three or more arithmetic processings having different arithmetic processing amounts is selected according to the magnitude of the arithmetic load, and the motion compensator 122 selects the arithmetic processing regardless of the arithmetic load. The switching is not performed.

Briefly, in this modification, the load determiner 114 determines the calculation load based on the first threshold TH1 as an index of the calculation load and the second threshold TH2 which is larger than the first threshold TH1. It is configured to determine whether the value is the following value, a value between the two threshold values, or a value equal to or more than the second threshold value TH2. In addition, the motion detector 12
1 based on the determination result of the load determiner 114,
As motion displacement information indicating a prediction area for the target block, a low-precision two-pixel precision motion vector, a high-precision 0.5 pixel precision motion vector, and an intermediate precision 1
The configuration is such that any one of the pixel-accurate motion vectors is generated. Further, the motion compensator 12
2 is configured such that normal motion compensation processing using a motion vector with one-pixel accuracy is performed irrespective of the magnitude of the calculation load.

Next, the operation will be briefly described. FIG.
(a) shows a flow of a process in the arithmetic processing unit in the first modification of the first embodiment. In the first modification of the first embodiment, the detection of the motion vector by the motion detector 121 is performed by the load determination output Cp from the load determiner 104.
And the calculation of the prediction signal by the motion compensator 122 is performed by a predetermined method (motion compensation method) regardless of the load determination output Cp.

The details will be described below. As shown in FIG.
In the load determining step S11a, the load determiner 114 determines whether or not the load index R indicating the magnitude of the calculation load at that time is smaller than the first threshold TH1. As a result of this determination, when the load index R is smaller than the first threshold value TH1 determined based on the arithmetic processing capability of the present image encoding apparatus, the motion detector 121 generates a 0.5-pixel-accuracy motion vector. (Step S13a). On the other hand, as a result of the determination in the load determination step S11a, when the load index R is equal to or more than the first threshold value TH1, the load index R is determined based on the arithmetic processing capability of the present image encoding device in step S12a. A determination is made as to whether it is greater than the second threshold value TH2.

As a result of the determination in step S12a, if the load index R is not larger than the second threshold TH2,
The motion detector 121 generates a motion vector with one-pixel accuracy (step S14a). If the load index R is larger than the second threshold TH2, the motion detector 121
Then, a motion vector with two-pixel accuracy is generated (step S15a).

Then, in the motion compensator 122,.
A predetermined motion compensation process of outputting an image signal corresponding to one prediction region as a prediction signal is performed based on a motion vector of one of five-pixel accuracy, one-pixel accuracy, and two-pixel accuracy. In Modification 1 of Embodiment 1 having such a configuration,
Based on a comparison result between the first and second threshold values of the calculation load based on the calculation processing capacity necessary for the signal generation processing in real time and the magnitude of the detected calculation load, 0.5 pixel accuracy, 1 Since a motion vector with a predetermined pixel accuracy is generated from the pixel accuracy and the two-pixel accuracy, the amount of calculation processing on the encoding side can be finely switched.

Modification 2 of Embodiment 1 In a second modification of the first embodiment, the arithmetic processing unit 120 is
1, one of the two arithmetic processes having different arithmetic processing amounts is selected as in the first embodiment, and the motion compensator 122 is selected.
In this configuration, one of four arithmetic processes having different arithmetic load amounts is selected according to the magnitude of the arithmetic load.

Briefly, in the second modification, the load determiner 114 is provided with a first, second, and third thresholds THs, THm, THb (THs <TH
Based on m <THb), the calculation load is a value smaller than the first threshold, a value not less than the first threshold and less than the second threshold, or not less than the second threshold and less than the third threshold Or a value equal to or greater than a third threshold value.

Further, based on the result of determination by the load determiner 114, the motion detector
When the value is equal to or larger than the threshold value THb, a low-precision 1-pixel-accuracy motion vector is generated as motion displacement information indicating a prediction area for the target block, and the calculation load is reduced to the third threshold value TH.
When the value is smaller than the value of b, a high-precision 0.5-pixel-accuracy motion vector is generated as the motion displacement information indicating the prediction area for the target block.

When the operation load is smaller than the first threshold value THs, the motion compensator 122 performs overlap motion compensation based on a 0.5-pixel-precision motion vector, and the operation load is equal to or larger than the first threshold value THs. And the second threshold T
When it is less than Hm, overlap motion compensation is performed based on a motion vector with one pixel accuracy, and when the calculation load is equal to or more than the second threshold THm and less than the third threshold THb,
Normal motion compensation is performed based on a 0.5-pixel accuracy motion vector, and when the calculation load is equal to or greater than the third threshold value THb, normal motion compensation is performed based on a 1-pixel accuracy motion vector. I have.

Next, the operation will be briefly described. FIG.
(b) shows the flow of the processing in the arithmetic processing unit in the second modification of the first embodiment. In the second modification of the first embodiment, the motion estimation process (detection of a motion vector) in the motion detector 121 is performed based on the load determination output Cp from the load determiner 114, and includes a process with a large calculation load and a calculation load. Of the motion compensator 1
In the motion compensation process (calculation of a prediction signal) by 22, one of four motion compensation processes having different calculation loads is selected according to the load determination output Cp. More specifically, as shown in FIG. 3B, in the load determination step S11b,
The load determiner 114 determines whether or not the load index R indicating the magnitude of the calculation load at that time is smaller than the second threshold value THm.

If the result of this determination is that the load index R is smaller than the second threshold THm, then at step S12b, the load index R is set to the first threshold THs (THs <T
Hm) is determined. On the other hand, as a result of the determination in the load determination step S11b, the load index R
Is greater than or equal to the second threshold value THm, step S13b
, The load index R is equal to the third threshold value THb (THb>
THm) is determined.

If the result of the determination in the load determination step S12b is that the load index R is smaller than the first threshold value THs, the motion detector 121 uses the 0.5-pixel precision motion vector together with the 1-pixel precision motion vector. Is generated, and the motion compensator 122 performs an overlap motion compensation process based on the 0.5-pixel precision motion vector (step S14b).

If the result of the determination in the load determination step S12b is that the load index R is equal to or greater than the first threshold value THs, the motion detector 121 calculates a 0.5-pixel precision motion vector together with a 1-pixel precision motion vector. A vector is generated, and the motion compensator 122 performs overlap motion compensation processing based on the motion vector with one-pixel accuracy (step S15b).

If the load index R is smaller than the third threshold value THb as a result of the determination in the load determination step S13b, the motion detector 121 calculates the 0.5-pixel motion vector together with the 1-pixel precision motion vector. Is generated, and the motion compensator 122 performs a normal compensation process based on the 0.5-pixel-precision motion vector (step S16b).

If the result of the determination in the load determination step S13b is that the load index R is equal to or larger than the third threshold THb, the motion detector 121 generates only a motion vector with one-pixel accuracy, and At 122, a normal motion compensation process is performed based on a motion vector with one pixel accuracy (step S17b).

Modified Example 2 of Embodiment 1 Having Such Configuration
Then, based on a comparison result between the first to third threshold values of the calculation load based on the calculation processing capacity necessary for the signal generation processing in real time and the detected calculation load, the motion detector Of the pixel accuracy and the type of motion compensation in the motion compensator, so that the amount of calculation on the encoding side can be gradually changed, and the variable range of the amount of calculation can be changed. Can be wider.

Embodiment 2 FIG. 4 is a block diagram for explaining an image decoding apparatus according to Embodiment 2 of the present invention. The image decoding device 200 according to the second embodiment corresponds to the image coding device 100 according to the first embodiment, and the image decoding device 200 uses MPEG1, MPEG1,
2 can process not only an image coded signal corresponding to one frame compliant with MPEG-2, but also an image coded signal corresponding to one object area compliant with MPEG4. Since the basic decoding processing is the same, a specific description will be given only of the processing of an image coded signal corresponding to one frame. Note that M
In the case of PEG4, similarly to the decoding process of the image coded signal corresponding to one frame, the decoding process of the image coded signal corresponding to each object area is performed, and these are combined to form one screen. A corresponding image reproduction signal is obtained.

More specifically, the image decoding apparatus 200 receives a coded signal (image coded signal or differential coded signal) Eg corresponding to a target area (target block) to be decoded, A data analyzer 201 for analyzing the encoded signal Eg and performing variable length decoding, and an information expander (DEC) 2 for performing an expansion process on the output AEg of the data analyzer 201 and outputting an image expansion signal DEg.
02, an output DEg of the information decompressor 202, an adder 203 that adds the prediction signal Pg2 or the reference signal (0 level) corresponding to the target block, and a block corresponding to the output PEg of the adder 203. A deblocking unit 204 that outputs an image output signal REg having a scanning line structure by integrating image reproduction signals to be reproduced, and a prediction signal generation unit 210 that generates a prediction signal Pg2 corresponding to a predetermined block.

Here, the information decompressor 202 is an inverse quantizer for performing an inverse quantization process on the output AEg of the data analyzer 201, and a type of inverse frequency conversion process for the output of the inverse quantizer. And an IDCT converter that performs an IDCT (Inverse Discrete Cosine Transform) process and outputs an image expansion signal DEg.

The prediction signal generator 210 receives the frame memory 211 for storing the output PEg of the adder 203 as an image reproduction signal and the output PEg of the adder 203, and determines the magnitude of the calculation load in the decoding process. And a load determiner 213 for determination.

Here, the load determiner 213 has the same configuration as the load determiner 114 of the first embodiment. That is, the load determiner 213 expresses, for example, a load index R indicating the magnitude of the calculation load as a calculation amount in the decoding processing per unit time (one frame), and indicates the calculation processing capability of the image decoding apparatus 200. The corresponding calculation load threshold value TH is the calculation amount in the decoding process per unit time required for the overlapped motion compensation processing using the 0.5-pixel precision motion vector, and the load index R and the threshold value TH are set. Are compared each time the motion compensation processing is performed to determine whether the processing capacity is sufficient. In addition,
The configuration of the load determiner 213 is not limited to the configuration described above.
For example, the load determiner 213 includes the frame memory 21
Only the timing at which the decoding process for the image signal of one frame from 1 or the like is completed is received, this timing interval is converted into a load index R which is a calculation amount, and this load index R
May be compared with the threshold value TH of the calculation load to determine the processing capacity. In this case, the decoded image signal, which is the output PEg of the adder 203, is output to the load determiner 21
3, the amount of data to be sent to the load determiner 213 can be greatly reduced.

The prediction signal generator 210 generates an address Add2 of the frame memory 211 based on the motion vector MV corresponding to the target block decoded by the data analyzer 201, and generates the address Add2.
Based on d2, an image reproduction signal corresponding to an area (prediction area) in the previous frame having an image reproduction signal with the smallest error from the image reproduction signal of the target block is detected from the frame memory 211, and based on the image reproduction signal. A motion compensator 212 that calculates a prediction signal Pg2 from the data analyzer 201. The prediction signal Pg2 is output to the adder 203 via a switch 214 controlled by a control signal ACs from the data analyzer 201. It has become. The switch 214 selects one of the input terminal I1 to which the prediction signal Pg2 is input and the opened input terminal I2 according to the control signal ACs. Note that this switch 214
Selects the open input terminal I2, the switch 21
The reference signal is supplied to the adder 203 as the output of the reference numeral 4. Here, the reference signal is a 0-level signal.

In the second embodiment, the motion compensator 212 performs the following operations based on the load determination output Ch from the load determiner 213 and the motion vector MV output from the data analyzer 201. The motion compensation processing for obtaining a prediction signal for the image decoded signal of the target block is switched between normal motion compensation and overlap motion compensation. That is, when the calculation load is large, the motion compensator 212 uses the value of the motion vector of 1-pixel accuracy, and rounds the value of the motion vector of 0.5-pixel accuracy to obtain the 1-pixel accuracy. A normal motion compensation process with a small amount of processing is performed using a vector value corresponding to the motion vector. On the other hand, when the calculation load is small, a 1-pixel precision motion vector or a 0.5-pixel precision motion vector is calculated. Using the values as they are, an overlap motion compensation process that can improve the image quality although the amount of calculation processing is large is performed.

Next, the operation will be described. FIG. 5A is a flowchart for explaining the decoding processing of the image signal by the image processing apparatus according to the second embodiment of the present invention.
First, an image decoding process is performed in step S3. That is, the coded signal (image coded signal or differential coded signal) Eg input to the image processing apparatus 200 is subjected to data analysis by the data analyzer 201 and is subjected to variable-length decoding. Is output to the information decompressor 202. At this time, the data analyzer 201 outputs the motion vector MV of the target block to be subjected to the decoding process to the motion compensator 212 of the prediction signal generation unit 210, and also corresponds to the coding mode of the target block. The control signal ACs is output from the switch 214 of the prediction signal generation unit 210.
Supplied to

In the information decompressor 202, a decompression process is performed on the variable-length-decoded image coded signal, and the image signal or the difference signal is restored as a decompression signal DEg. Specifically, in the information decompressor 202, the variable-length-decoded image encoded signal is subjected to inverse quantization by an inverse quantizer, and further, by an inverse discrete cosine transformer,
Inverse frequency conversion processing for converting a signal in the frequency domain into a signal in the spatial domain is performed.

Next, a motion compensation process is performed in step S4. In the motion compensator 212, the motion vector M
An address Add2 for accessing the frame memory 211 is generated from V, and a prediction signal Pg2 corresponding to the target block is read from the image reproduction signal of the previous frame stored in the frame memory 211. The generation of the prediction signal in the motion compensator 212 is performed based on the control signal Ch corresponding to the magnitude of the calculation load.

When the mode signal ACs indicates the inter-picture coding mode, the switch 214 selects the input terminal I1 of the prediction signal, and the read prediction signal Pg2 and the output of the information decompressor 202 are output. DEg is input to the adder 203, and the adder 203 outputs a decoded image signal PEg as an added value of these signals. On the other hand, when the mode signal ACs indicates the intra-frame encoding mode, the switch 214 selects the open input terminal I2, and the output DE of the information decompressor 202 is output.
g is directly passed through the adder 203 to the deblocker 20
4 is output.

The output PEg of the adder 203 is supplied to the deblocking device 204 and integrated by the deblocking device 204 to output an image output signal REg having a scanning line structure.
Is converted to At this time, the image reproduction signal PEg output from the adder 203 is output to the frame memory 211 and the load determiner 213.
A calculation processing amount in the decoding process is determined, and a control signal Ch corresponding to the determination result is output to the motion compensator 212.

Hereinafter, the motion compensation processing in the motion compensator 212 will be described in detail. FIG. 5B is a flowchart showing the motion compensation processing. In the load determination step S41, the load index R is determined by the load determiner 213. As a result, if the calculation load index R is smaller than the threshold value TH corresponding to the processing capability of the image decoding apparatus 200, in step S42, the data analyzer 20
Based on the motion vector from 1 (0.5-pixel accuracy or 1-pixel accuracy motion vector), overlap motion compensation processing with a large amount of arithmetic processing but high coding efficiency is performed. On the other hand, when the index R of the calculation load is larger than the above TH, in step S43, a normal motion compensation process having a smaller calculation processing amount than the overlap motion compensation process is performed. At this time, if the motion vector from the data analyzer 201 is a 0.5-pixel precision motion vector, the value is rounded, and a vector value corresponding to a 1-pixel precision motion vector is used.

As described above, according to the second embodiment, when the amount of calculation load is larger than the load threshold corresponding to the calculation processing capability of the present image decoding device 200 during the decoding processing of the image coded signal, the reproduction is performed. Instead of the overlap motion compensation processing that is effective for improving the image quality of the image, the normal motion compensation processing with a smaller amount of calculation processing is performed, so that the decoding processing of the image coded signal in real time is performed. In addition, the motion compensation processing is switched depending on the state of the calculation load, and the adverse effect on the reproduced image due to the lack of the calculation processing ability can be avoided.

Specifically, in the conventional decoding method, a prediction signal is generated by a motion compensation process corresponding to the motion compensation process selected at the time of encoding, whereas in the second embodiment, If the arithmetic processing capability is insufficient during the decoding process, the motion compensation process corresponding to the motion compensation process selected at the time of encoding is not performed, and a motion compensation process that is different from the motion compensation process is performed. For example, when the overlap motion compensation process is performed at the time of encoding, a normal motion compensation process with a small calculation processing amount is performed at the time of decoding, instead of the overlap motion compensation process, according to the calculation load.

Therefore, according to the second embodiment, even if the image processing obtained by the real-time decoding processing is greatly degraded due to the shortage of the arithmetic processing capacity in the conventional image decoding processing, the processing capacity is not changed. Motion compensation can be performed, and adverse effects on a video due to real-time decoding processing due to lack of arithmetic processing capability can be suppressed.

In the second embodiment, an example in which the overlap motion compensation is changed to the normal motion compensation when the load index is larger than the threshold value has been described. This is similar to the first embodiment. The motion compensation processing corresponding to bidirectional prediction or interlace prediction may be changed to the motion compensation processing corresponding to forward prediction or frame prediction.

Further, the switching of the motion compensation processing by the motion compensator is performed by interlocking the switching of the pixel precision and the switching of the type of the motion compensation processing by the motion compensator as in the second embodiment. Not only the switching method but also various switching methods are conceivable, and a switching method different from that of the second embodiment will be described below.

Modification 1 of Embodiment 2 In the first modification of the second embodiment, the prediction signal generator 210 according to the second embodiment is different from the motion compensator 212 in that the data analyzer 2
01, whether the motion vector obtained is 0.5 pixel accuracy or 1 pixel accuracy, a motion vector of 1 pixel accuracy is used as the motion vector, and switching between overlapped motion compensation and normal motion compensation is performed. Is performed.

That is, in this modification, the motion compensator 212 receives the judgment result of the load judgment unit 213,
When the calculation load R is smaller than the threshold value TH determined based on the calculation processing capability of the present image decoding device, an overlap motion compensation process using a motion vector with one-pixel accuracy is performed.
When the calculation load R is larger than the load threshold TH, a normal motion compensation process using a motion vector with one-pixel accuracy is performed. Also in this modified example, when the motion vector obtained by the data analyzer 201 has 0.5-pixel accuracy, this value is rounded to generate a 1-pixel accuracy motion vector.

In the motion compensation process according to the first modification of the second embodiment, when a one-pixel precision motion vector is sent from the encoding side, the one-pixel precision motion vector is used as it is. When a 0.5-pixel precision motion vector is sent from the encoding side, the value of the 0.5-pixel precision motion vector is rounded to generate a 1-pixel precision motion vector.

Next, the operation will be briefly described. FIG.
(a) shows the flow of the processing in the arithmetic processing unit in the first modification of the second embodiment. In the first modification of the second embodiment, the motion compensation processing (calculation of the prediction signal) by the motion compensator 212 is performed based on the load determination output Ch from the load determiner 213 to perform the overlap motion compensation and the load Is switched to the normal motion compensation in which is reduced.

In brief, as shown in FIG. 6A, in the load determination step S41a, the load determiner 21
According to 3, it is determined whether or not the load index R indicating the magnitude of the calculation load at that time is smaller than a threshold value TH determined based on the calculation processing capability of the present image decoding apparatus.

As a result of this determination, when the load index R is smaller than the threshold value TH, the motion compensator 212
Overlap motion compensation is performed based on a one-pixel accuracy motion vector to generate a prediction signal (Step S).
42a). On the other hand, as a result of the determination in the load determination step S41a, when the load index R is equal to or larger than the threshold value TH, the normal motion compensation with reduced load (load reduction motion compensation) is performed based on the motion vector with one-pixel accuracy. ) Is performed to generate a prediction signal (step S43a).

As described above, in the first modification of the second embodiment, based on the load determination output Ch from the load determiner 213, the motion compensation in the motion compensator 212 is performed based on the motion vector of one pixel accuracy. Since the switching is performed between the wrap motion compensation and the load reduction motion compensation based on the one-pixel accuracy motion vector, the pixel accuracy is not switched, and only the type of the motion compensation processing is switched. With the circuit configuration, it is possible to suppress deterioration in the image quality of the reproduced image due to an increase in the operation load.

Modification 2 of Embodiment 2 In the second modification of the second embodiment, the prediction signal generator 210 according to the second embodiment is different from the motion compensator 212 in that the motion compensator 212 performs overlap motion compensation and normal motion compensation (load Switching of (reduced motion compensation) is not performed, and switching between pixel accuracy, that is, normal motion compensation based on a motion vector with one pixel accuracy and normal motion compensation based on a motion vector with 0.5 pixel accuracy is performed. Things.

That is, in this modification, the motion compensator 212 outputs the load determination output C from the load determiner 213.
h, when the calculation load is smaller than a threshold value TH determined based on the calculation processing capability of the image decoding apparatus, 0.
A normal motion compensation process is performed based on a 5-pixel precision motion vector, and when the calculation load is larger than the threshold value TH, a normal motion compensation process is performed based on a 1-pixel precision motion vector.

In the motion compensation processing according to the second modification of the second embodiment, the above-described switching of the pixel precision is performed only when a 0.5-pixel precision motion vector is transmitted from the encoding side. When a motion vector with one-pixel accuracy is sent from the encoding side, the above-described switching of pixel accuracy is not performed.

Next, the function and effect will be described. Fig. 6 (b)
Shows a flow of a process performed by the prediction signal generation unit in the second modification of the second embodiment. In the second modification of the second embodiment, the motion compensation processing (calculation of the prediction signal) by the motion compensator 212 is performed based on the load determination output Ch from the load determiner 213, and the motion of 0.5 pixel accuracy is obtained. Switching is performed between normal motion compensation based on a vector and normal motion compensation based on a motion vector with one-pixel accuracy.

Briefly, as shown in FIG. 6B, in the load determination step S41b, the load determination unit 213 is used.
Accordingly, it is determined whether or not the load index R indicating the magnitude of the computational load at that time is smaller than a threshold value TH determined based on the computational processing capability of the present image decoding device.

As a result of this judgment, when the load index R is smaller than the threshold value TH, the motion compensator 212
Normal motion compensation is performed based on a 0.5-pixel precision motion vector to generate a prediction signal (step S42).
b). On the other hand, as a result of the determination in the load determination step S41b, when the load index R is equal to or larger than the threshold TH,
Normal motion compensation is performed based on the motion vector with one-pixel accuracy, and a prediction signal is generated (step S43).
b).

As described above, in the second modification of the second embodiment, based on the load determination output Ch from the load determiner 213, the motion compensation by the motion compensator 212 is converted into a 0.5-pixel-precision motion vector. Since the switching between the normal motion compensation based on the motion vector based on the motion vector with one-pixel accuracy and the normal motion compensation based on the one-pixel accuracy is performed, the type of motion compensation is not switched, and only the switching on the pixel accuracy is performed. With a simple circuit configuration, it is possible to suppress deterioration in the image quality of the reproduced image due to an increase in the calculation load.

In the motion compensation processing according to the second modification of the second embodiment, the above-described switching of the pixel precision is performed only when a 0.5-pixel precision motion vector is transmitted from the encoding side. In the case where a one-pixel precision motion vector is sent from the encoding side, the above-described switching of the pixel precision is not performed. However, the switching of the pixel precision in the motion compensation processing is not limited to this. Absent.

For example, the above-described switching of the pixel accuracy is performed regardless of whether a 0.5-pixel precision motion vector or a 1-pixel precision motion vector is transmitted from the encoding side. It may be.

More specifically, when a 0.5-pixel precision motion vector is sent from the encoding side, in the 1-pixel precision motion compensation, the value of the 0.5-pixel precision motion vector is rounded. Using a 1-pixel precision motion vector,
In motion compensation with pixel accuracy, a motion vector with 0.5 pixel accuracy is used as it is. On the other hand, when a motion vector with one-pixel accuracy is sent from the encoding side, the motion vector with one-pixel accuracy uses the motion vector with one-pixel accuracy as it is, and A 0.5 pixel precision motion vector generated based on a 1 pixel precision motion vector is used.

Further, the motion compensation for each pixel accuracy may be realized by a hierarchical operation process or by an operation process by an operation module corresponding to each pixel accuracy. For example, a motion compensation process with 0.5 pixel accuracy is realized by performing an additional second calculation process in addition to a basic first calculation process for performing motion compensation with 1 pixel accuracy. The switching of the accuracy may be performed depending on whether or not to perform the additional arithmetic processing. Further, as a module for performing arithmetic processing for motion compensation, a first module for performing motion compensation with 0.5 pixel accuracy and a second module for performing motion compensation with 1 pixel accuracy are provided. By switching, the motion compensation of each pixel accuracy may be switched.

Further, the switching of pixel precision is not limited to switching between 0.5 pixel precision and 1 pixel precision, but switching between 0.5 pixel precision and 2 pixel precision, or switching between 1 pixel precision and 2 pixel precision. , And further, 0.5 pixel accuracy,
One of the one-pixel accuracy and the two-pixel accuracy may be selected according to the calculation load. The pixel accuracy is not limited to the above, but may be 0.25 pixel accuracy or 0.
It is also possible to use very high precision such as 125 pixel precision.

Modification 3 of Embodiment 2 In a third modification of the second embodiment, the prediction signal generation unit 210 according to the second embodiment is configured such that the motion compensator 212 switches the arithmetic processing in four stages according to the magnitude of the arithmetic load. It is. Briefly, in the third modification, the load determiner 213 according to the second embodiment is used.
With the first, second, and third threshold values TH, which are indexes of the operation load.
Based on s, THm, THb (THs <THm <THb), whether the calculation load is a value smaller than the first threshold, a value equal to or more than the first threshold and less than the second threshold, A value greater than or equal to two thresholds and less than a third threshold,
Is determined to be a value equal to or larger than the threshold value.

In the third modification, the motion compensator 212 according to the second embodiment has the operation load of the first
When the threshold value is smaller than the threshold value THs, overlap motion compensation is performed using a 0.5-pixel accuracy motion vector. When the calculation load is equal to or more than the first threshold value THs and less than the second threshold value THm, the overlap is performed using a one-pixel accuracy motion vector. When motion compensation is performed and the calculation load is equal to or more than the second threshold value THm and smaller than the third threshold value THb, normal motion compensation is performed using a 0.5-pixel accuracy motion vector. At one time, the configuration is such that normal motion compensation is performed using a motion vector with one-pixel accuracy.

Next, the operation will be briefly described. FIG.
(b) shows a flow of a process performed by the prediction signal generation unit according to the third modification of the second embodiment. Embodiment 2
In the third modification, the motion compensation processing (calculation of the prediction signal) by the motion compensator 212 is one of four motion compensation processings having different calculation loads in accordance with the load determination output Ch from the load determiner 213. One is selected.

In detail, as shown in FIG. 6C,
In the load determining step S41c, the load determiner 213 determines whether or not the load index R indicating the magnitude of the calculation load at that time is smaller than the second threshold THm.

If the result of this determination is that the load index R is smaller than the second threshold THm, then at step S42c, the load index R is set to the first threshold THs (THs <T
Hm) is determined. On the other hand, as a result of the determination in the load determination step S41c, the load index R
Is greater than or equal to the second threshold value THm, Step S43c
, The load index R is equal to the third threshold value THb (THb>
THm) is determined.

When the load index R is smaller than the first threshold THs as a result of the determination in the load determination step S42c, the overlap motion compensation processing is performed based on the 0.5-pixel precision motion vector (step S44c). ).
If the result of the determination in the load determination step S42c is that the load index R is equal to or greater than the first threshold value THs, an overlap motion compensation process is performed based on a one-pixel accurate motion vector (step S45c). If the result of the determination in the load determination step S43c is that the load index R is smaller than the third threshold value THb, normal motion compensation processing (load reduction motion compensation) is performed based on a 0.5-pixel precision motion vector. (Step S46c). As a result of the determination in the load determination step S43c, the load index R
If it is equal to or greater than the threshold value THb, normal motion compensation processing (load reduction motion compensation) is performed using a motion vector with one-pixel accuracy (step S47b).

Modification 3 of Embodiment 2 with Such a Configuration
Then, based on the comparison result between the first to third threshold values of the calculation load based on the calculation processing capacity necessary for the signal generation processing in real time and the magnitude of the detected calculation load, the pixel accuracy and the motion Since the type of compensation is switched and the magnitude of the computational load is switched in four steps, the computational processing amount on the decoding side can be gradually changed, and the variable range of the computational processing amount can be widened. Can be.

Further, an encoding or decoding program for realizing the image encoding process and the image decoding process described in each of the above embodiments and the modifications thereof is recorded on a data storage medium such as a floppy disk. By doing so, the processing described in each of the above-described embodiments and the modifications thereof can be easily performed by an independent computer system.

FIG. 11 shows an image encoding process by the image encoding device according to the first embodiment or its modification, or an image decoding process by the image decoding device according to the second embodiment or its modification. Using a floppy disk storing a program corresponding to the image processing of
FIG. 4 is a diagram for explaining a case where the present invention is implemented by a computer system.

FIG. 11B shows the external appearance, cross-sectional structure, and main body of a floppy disk as a recording medium when viewed from the front of the floppy disk FD. FIG. 11A shows an example of the physical format of the main body D of the floppy disk. Is shown.
The floppy disk main body D is housed in a case F. On the surface of the disk main body D, a plurality of tracks Tr are formed concentrically from the outer circumference toward the inner circumference, and each track has 16 sectors Se in the angular direction. Is divided into Therefore, in the floppy disk body D storing the program, data as the program is recorded in an area allocated on the floppy disk body D.

FIG. 11C shows a configuration for recording and reproducing the program on the floppy disk FD. When recording the program on the floppy disk FD, data as the program is written from the computer system Cs via the floppy disk drive FDD. In the case where the above image encoding process or image decoding process is constructed in the computer system Cs by a program in the floppy disk FD,
The program is read from the floppy disk FD by the floppy disk drive FDD and transferred to the computer system Cs.

In the above description, image processing by a computer system using a floppy disk as a data recording medium has been described. However, this image processing can be similarly performed using an optical disk. Further, the recording medium is not limited to this, and may be an IC card, a ROM cassette, or the like.
The present invention can be similarly implemented as long as the program can be recorded.

[0143]

As described above, according to the image processing method of the present invention (claim 1), the magnitude of the arithmetic load in the encoding process of the image input signal is detected, and the magnitude of the detected arithmetic load is detected. The prediction method in the process of generating the prediction signal of the image input signal is switched according to the above, so that the calculation load in the prediction signal generation process exceeds the calculation processing capability of the system that performs the encoding process of the image input signal. Thus, it is possible to prevent the decoding side from having a problem such as dropping of frames in the video reproduced by the real-time processing on the decoding side. As a result, it is possible to prevent the occurrence of the system caused by the prediction signal generation processing. It is possible to prevent an increase in the calculation load from having a large adverse effect on the image quality of the reproduced image.

According to the present invention (claim 2), claim 1
In the image processing method described above, an arithmetic processing is performed based on a comparison result between a threshold of an arithmetic load obtained based on an arithmetic processing amount required for the prediction signal generation processing in real time and the magnitude of the detected arithmetic load. Since the prediction method in the prediction signal generation processing is switched between the first prediction method having a large amount and the second prediction method having a small amount of operation processing, the coding efficiency is not deteriorated without deteriorating the image quality of the reproduced image. Can be improved.

According to the present invention (claim 3), claim 1
In the image processing method described above, based on a comparison result between a threshold value of a calculation load based on a calculation processing capability required for the signal generation process in real time and the magnitude of the detected calculation load, As the prediction method, a prediction method suitable for the magnitude of the detected computation load is selected from a plurality of prediction methods having different computation processing amounts, so that the computation processing amount can be finely switched. In addition, the motion prediction processing, in which the signal processing amount accounts for a large proportion of the entire coding processing signal processing amount, is switched, and the calculation load in the image transmission system can be greatly reduced.

According to the present invention (Claim 4), Claim 1
In the image processing method described above, based on a comparison result between a threshold value of a calculation load based on a calculation processing capability required for the signal generation process in real time and the magnitude of the detected calculation load, The method of the motion prediction process is switched between a motion prediction method that requires a large amount of calculation processing and a motion prediction method that requires a small amount of calculation processing, and based on the comparison result, Since the method of motion compensation processing is switched between a motion compensation method requiring a large amount of calculation processing and a motion compensation method requiring a small amount of calculation processing, the switching of the amount of calculation processing can be performed in a large range. Can be performed relatively finely.

According to the present invention (Claim 5), Claim 1
In the image processing method described above, based on a comparison result between a threshold value of a calculation load based on a calculation processing capability required for the signal generation process in real time and the magnitude of the detected calculation load, Since the method of the motion prediction process is switched between a motion prediction method for obtaining a motion vector with high pixel accuracy and a motion prediction method for obtaining a motion vector with low pixel accuracy, by a relatively simple process of switching pixel accuracy, Encoding efficiency can be improved without deteriorating the quality of a reproduced image.

According to the present invention (claim 6), claim 1
In the image processing method described above, based on a comparison result between a threshold value of a calculation load based on a calculation processing capability required for the signal generation process in real time and the magnitude of the detected calculation load, The motion compensation method is performed between a motion compensation method for obtaining the predicted image input signal based on a motion vector with high pixel accuracy and a motion compensation method for obtaining the predicted image input signal based on a motion vector with low pixel accuracy. Therefore, the calculation load in the image transmission system can be greatly reduced by the relatively simple process of switching the pixel accuracy.

According to the present invention (claim 7), claim 6
In the image processing method described above, the type of the motion compensation method that is performed based on the motion vector with high pixel accuracy is determined based on a comparison result between the threshold value of the computation load and the magnitude of the computation load. And the second motion compensation method with a small computational load, and based on the comparison result between the threshold value of the computational load and the magnitude of the computational load, based on the motion vector with a low pixel accuracy. Is switched between the first motion compensation method having a large computational load and the second motion compensation method having a small computational load. Can be performed relatively finely.

According to the present invention (claim 8), claim 1
In the image processing method described above, based on a comparison result between a threshold value of a calculation load based on a calculation processing capability required for the signal generation process in real time and the magnitude of the detected calculation load, Since one of a plurality of motion compensation methods using motion vectors having different pixel precisions is selected as a method of the motion compensation processing, the switching of the amount of arithmetic processing can be finely performed by a relatively simple process of switching the pixel precision. Can be.

According to the image processing method of the present invention (claim 9), the magnitude of the calculation load in the decoding process of the coded image signal is detected, and the image processing is performed in accordance with the detected calculation load. Since the motion compensation method in the process of generating the prediction signal for the decoded signal is switched, it is possible to prevent the calculation load in the prediction signal generation process from exceeding the calculation processing capability of the system for decoding the image coded signal. As a result, it is possible to prevent a problem such as dropping frames in a video reproduced by real-time processing. As a result, it is possible to prevent an increase in the computational load of the system caused by the prediction signal generation processing from having a large adverse effect on the image quality of the reproduced image.

According to the present invention (claim 10), in the image processing method according to claim 9, a threshold value of a calculation load obtained based on a calculation processing amount necessary for the prediction signal generation processing in real time; Based on the result of the comparison with the detected magnitude of the computation load, the first motion compensation method having a small computation processing amount and the second motion compensation method having a large computation processing amount are:
Since the motion compensation method in the prediction signal generation process is switched, it is possible to realize a decoding process corresponding to the encoding process capable of improving the encoding efficiency without deteriorating the image quality of the reproduced image.

According to the present invention (claim 11), in the image processing method according to claim 9, the threshold value of the calculation load based on the calculation processing capability necessary for the signal generation processing in real time and the detection of the detected load are performed. As a motion compensation method in the signal generation processing based on the comparison result with the magnitude of the calculation load,
Since a motion compensation method suitable for the magnitude of the detected computation load is selected from among a plurality of motion compensation methods having different computation processing amounts, switching of the computation processing amount can be performed finely.

According to the present invention (claim 12), in the image processing method according to claim 9, the threshold value of the calculation load based on the calculation processing capability required for the signal generation processing in real time and the detection of the detected load are performed. A motion compensation method in the signal generation processing based on a comparison result with the magnitude of the calculation load; a motion compensation method in which the predicted image decoded signal is calculated based on a motion vector with high pixel accuracy; and a motion vector with low pixel accuracy. Therefore, the amount of calculation can be switched by a relatively simple process of switching the pixel accuracy by switching between the motion compensation method for obtaining the predicted image decoded signal based on the above.

According to the present invention (claim 13), in the image processing method according to claim 12, the motion vector with high pixel accuracy is obtained based on a comparison result between the threshold value of the calculation load and the magnitude of the calculation load. The type of the motion compensation method performed based on the first motion compensation method with a large computational load,
A motion compensation method performed based on the motion vector with low pixel accuracy based on a comparison result between the threshold value of the computation load and the magnitude of the computation load, while switching between the second motion compensation method having a small computation load. Are classified into a first motion compensation method having a large calculation load and a second motion compensation method having a small calculation load.
, The amount of computation can be relatively finely switched over a large range.

According to the image processing apparatus of the present invention (claim 14), the prediction processing unit for predicting the image input signal of the target unit area to be encoded by a predetermined method to generate a prediction signal is provided. Has an operation load detecting means for detecting the amount of operation load in the encoding process of the image input signal,
Since the prediction method in the prediction signal generation process is switched according to the magnitude of the detected calculation load,
It is possible to prevent the calculation load in the above-described prediction signal generation processing from exceeding the calculation processing capacity of the system that performs the coding processing of the image input signal, whereby the video reproduced on the decoding side by the real-time processing can be prevented. Can prevent a problem such as dropping frames. As a result,
It is possible to prevent an increase in the computational load of the system caused by the generation processing of the prediction signal from having a large adverse effect on the image quality of the reproduced image.

According to the present invention (claim 15), in the image processing apparatus according to claim 14, the motion detector constituting the prediction processing unit includes a target unit area in a screen to be processed and a corresponding unit area. Either a first motion vector with a low pixel accuracy or a second motion vector with a high pixel accuracy as a motion vector indicating the positional relationship with the prediction area in the previous screen, depending on the magnitude of the detected calculation load. , The coding efficiency can be improved without deteriorating the image quality of the reproduced image.

According to the image processing apparatus of the present invention (claim 16), a prediction in which a decoded signal of a target unit area to be decoded is predicted by a predetermined motion compensation method to generate a predicted signal. The processing unit includes an operation load detection unit that detects the amount of operation load in the decoding process of the image encoded signal, and performs a motion compensation method in the prediction signal generation process according to the detected operation load. Since the configuration is switched, it is possible to prevent the amount of arithmetic processing in the prediction signal generation processing from exceeding the arithmetic processing capability of the system that performs the decoding processing of the coded image signal. It is possible to prevent such a problem that a dropped frame occurs in the image to be displayed. As a result, it is possible to prevent an increase in the computational load of the system caused by the prediction signal generation processing from having a large adverse effect on the image quality of the reproduced image.

According to the present invention (claim 17), in the image processing apparatus according to claim 16, the motion compensator constituting the prediction processing unit is controlled based on the motion vector information included in the coded image signal. The process of generating a prediction signal for a target unit area is performed by selecting between a coding efficiency high level and a processing level low according to the detected calculation load. Therefore, it is possible to realize a decoding process corresponding to the encoding process that can improve the encoding efficiency without deteriorating the image quality of the reproduced image.

According to the data storage medium of the present invention (claim 18), the program for performing the encoding process of the image signal includes a process of detecting a magnitude of a calculation load in the encoding process of the image input signal, And a program for causing a computer to perform a process of switching a prediction method in a process of generating a prediction signal of an image input signal in accordance with the magnitude of the detected computation load. In order to avoid exceeding the arithmetic processing capability of the system that performs the encoding process of the image input signal,
On the decoding side, it is possible to realize, by a computer, encoding processing capable of preventing a problem such as dropping of frames in a video reproduced by real-time processing.

The data storage medium according to the present invention (claim 19) detects a magnitude of an operation load in the decoding process of the image coded signal as a program for performing the decoding process of the image coded signal. A program for causing a computer to perform a process and a process of switching a motion compensation method in a process of generating a prediction signal of an image decoded signal in accordance with the magnitude of the detected calculation load is stored. It is possible to prevent the calculation load from exceeding the calculation processing capability of a system that performs decoding processing of an image coded signal, and to prevent a problem such as dropping of frames in a video reproduced by real-time processing. A possible decoding process can be realized by a computer.

[Brief description of the drawings]

FIG. 1 is a block diagram for describing an image encoding device according to a first embodiment of the present invention.

FIG. 2 is a diagram showing, by a flow chart, encoding processing (FIG. (A)) and prediction signal generation processing (FIG. (B)) by the image encoding apparatus according to the first embodiment.

FIG. 3 is a diagram showing, by a flowchart, generation processing (FIGS. (A) and (b)) of a prediction signal by an image encoding device according to modifications 1 and 2 of the first embodiment;

FIG. 4 is a block diagram for explaining an image decoding device according to a second embodiment of the present invention.

FIG. 5 is a diagram showing, by a flowchart, decoding processing (FIG. (A)) and motion compensation processing (FIG. (B)) by the image decoding apparatus according to the second embodiment.

FIG. 6 is a flowchart showing the motion compensation processing (FIGS. (A), (b), and (c)) by the image decoding apparatus according to Modifications 1, 2, and 3 of the second embodiment.

FIG. 7 is a diagram for explaining a motion compensation process in a conventional image decoding method corresponding to MPEG1 and MPEG2.

FIG. 8 shows a macro block composed of 16 × 16 pixels and 8 ×
FIG. 3 is a diagram illustrating a relationship with a block including eight pixels.

FIG. 9 is a diagram for explaining an overlap motion compensation method in a conventional image decoding method.

FIG. 10 is a diagram for explaining a process of generating a prediction signal in the overlap motion compensation method.

11 (a), 11 (b) and 11 (c) show data storage media storing a program for implementing a coding or decoding process by the image processing apparatus of each of the above-described embodiments by a computer system. It is a figure for explaining.

FIG. 12 is a diagram for explaining motion compensation processing in a conventional image decoding method compatible with MPEG4.

[Explanation of symbols]

 REFERENCE SIGNS LIST 100 Image encoding device 101 Blocker 102 First switch 103 Information compressor 104 Variable length encoder 105 Mode determiner 106 Subtractor 110, 210 Predicted signal generator 111 Adder 111a Second switch 111b, 203 Adders 112, 202 Information decompressors 113, 211 Frame memories 114, 213 Load determiners 120 Arithmetic processing units 121 Motion detectors 122, 212 Motion compensators 200 Image decoding devices 201 Data analyzers 204 Deblockers 214 Switches Cs Computer system D Floppy disk FD Floppy disk FDD Floppy disk drive

Claims (19)

[Claims] An image input signal is encoded for each unit area for dividing a screen, and an image encoded signal is output.
A signal generation process for generating a predicted image input signal corresponding to a target unit area to be subjected to an encoding process by an operation based on a predetermined prediction method based on an image decoded signal obtained by decoding the image encoded signal Is performed on a required unit area, wherein the magnitude of the computational load in the encoding process of the image input signal is detected, and the signal is supplied in accordance with the magnitude of the detected computational load. An image processing method characterized by switching a prediction method in a generation process. 2. The image processing method according to claim 1, wherein a threshold value of a calculation load based on a calculation processing capability required for the signal generation processing in real time and a magnitude of the detected calculation load are compared. Based on the above, the prediction method in the signal generation processing is switched between a first prediction method that requires a large amount of arithmetic processing and a second prediction method that requires a small amount of arithmetic processing. Image processing method. 3. The image processing method according to claim 1, wherein a comparison result between a threshold value of a calculation load based on a calculation processing capability required for the signal generation processing in real time and the magnitude of the detected calculation load is calculated. An image processing method according to claim 1, wherein a prediction method suitable for the detected calculation load is selected from among a plurality of prediction methods having different amounts of calculation processing as the prediction method in the signal generation processing. 4. The image processing method according to claim 1, wherein the signal generation processing includes a motion prediction processing for obtaining a motion vector indicating a position of a prediction area corresponding to the target unit area with a predetermined pixel precision. A motion compensation process for obtaining an image signal corresponding to the prediction area as the predicted image input signal based on the motion vector, and a calculation load based on a calculation processing capability necessary for the signal generation process in real time. On the basis of the comparison result between the threshold value and the detected magnitude of the computation load, the motion prediction method in the signal generation processing is performed by: Switching between a small motion prediction method and a motion compensation method in the signal generation process based on the result of the comparison is performed for a motion that requires a large amount of arithmetic processing. Compensation method and an image processing method characterized by switching between motion compensation process operation processing amount is small required. 5. The image processing method according to claim 1, wherein the signal generation processing includes a motion prediction processing for obtaining a motion vector indicating a position of the prediction area corresponding to the target unit area with a predetermined pixel precision. A motion compensation process for obtaining an image signal corresponding to the prediction area as the predicted image input signal based on the motion vector, and a calculation load based on a calculation processing capability necessary for the signal generation process in real time. Based on the comparison result of the threshold value and the magnitude of the detected computation load, the motion prediction method in the signal generation process is performed by using a motion prediction method that determines a motion vector with high pixel accuracy, and a motion vector with low pixel accuracy. An image processing method characterized by switching between a desired motion prediction method and a desired motion prediction method. 6. The image processing method according to claim 1, wherein the signal generation processing includes a motion prediction processing for obtaining a motion vector indicating a position of a prediction area corresponding to the target unit area with a predetermined pixel accuracy. A motion compensation process for obtaining an image signal corresponding to the prediction area as the predicted image input signal based on the motion vector, and a calculation load based on a calculation processing capability necessary for the signal generation process in real time. A threshold value and a motion compensation method for obtaining the predicted image input signal based on a motion vector having a high pixel accuracy, based on a comparison result between the detected computation load and a magnitude of the detected computation load. And a motion compensation method for obtaining the predicted image input signal based on a motion vector having low pixel accuracy. 7. The type of motion compensation method according to claim 6, wherein the motion compensation method is performed based on a motion vector with high pixel accuracy based on a comparison result between the threshold value of the computation load and the magnitude of the computation load. To the first calculation load
And the second motion compensation method with a small computational load, and based on the comparison result between the threshold value of the computational load and the magnitude of the computational load, based on the motion vector with a low pixel accuracy. The type of motion compensation method performed by
And a second motion compensation method having a small calculation load. 8. The image processing method according to claim 1, wherein the signal generation processing includes a motion prediction processing for obtaining a motion vector indicating a position of a prediction area corresponding to the target unit area with a predetermined pixel accuracy. A motion compensation process for obtaining an image signal corresponding to the prediction area as the predicted image input signal based on the motion vector, and a calculation load based on a calculation processing capability necessary for the signal generation process in real time. Based on the comparison result between the threshold value and the detected magnitude of the calculation load, one of a plurality of motion compensation methods using motion vectors having different pixel precisions is selected as a method of the motion compensation processing in the signal generation processing. An image processing method comprising: 9. An input image coded signal is decoded for each unit area dividing a screen to output an image decoded signal, and is subjected to a decoding process based on the image decoded signal. An image decoding method for performing a signal generation process of generating a predicted image decoded signal corresponding to a target unit area by an operation based on a predetermined motion compensation method for a required unit area, comprising: An image processing method comprising: detecting a magnitude of an operation load in a decoding process; and switching a motion compensation method in the signal generation process according to the detected operation load. 10. The image processing method according to claim 9, wherein a comparison result between a threshold value of a calculation load based on a calculation processing capability necessary for the signal generation processing in real time and the magnitude of the detected calculation load. Based on the above, switching the motion compensation method in the signal generation process between the first motion compensation method that requires a large amount of calculation processing and the second motion compensation method that requires a small amount of calculation processing Characteristic image processing method. 11. The image processing method according to claim 9, wherein a threshold value of a calculation load based on a calculation processing capability required for the signal generation processing in real time and a magnitude of the detected calculation load are compared. Image processing, wherein a motion compensation method suitable for the magnitude of the detected computation load is selected from among a plurality of motion compensation methods having different amounts of computation processing as the motion compensation method in the signal generation processing based on the image processing. Method. 12. The image processing method according to claim 9, wherein a threshold value of a calculation load based on a calculation processing capability required for the signal generation processing in real time and a magnitude of the detected calculation load are compared. A motion compensation method in the signal generation process, a motion compensation method for obtaining the predicted image decoded signal based on a motion vector with high pixel accuracy, and a motion compensation method for obtaining the predicted image decoded signal based on a motion vector with low pixel accuracy. An image processing method characterized by switching between a motion compensation method and a motion compensation method. 13. The type of motion compensation method according to claim 12, wherein the motion compensation method is performed based on the motion vector with high pixel accuracy based on a comparison result between the threshold value of the computation load and the magnitude of the computation load. To the first calculation load
And the second motion compensation method with a small computational load, and based on the comparison result between the threshold value of the computational load and the magnitude of the computational load, based on the motion vector with a low pixel accuracy. The type of motion compensation method performed by
And a second motion compensation method having a small calculation load. 14. An encoding processing unit which encodes an image input signal for each unit area for dividing a screen and outputs an image encoded signal, and an image decoding signal obtained by decoding the image encoded signal. A prediction processing unit that performs a signal generation process for generating a predicted image input signal corresponding to a target unit region to be encoded based on a predetermined prediction method for a required unit region based on the predetermined prediction method An image encoding device, comprising: a prediction processing unit that includes a computation load detection unit that detects a magnitude of a computation load in the encoding process of the image input signal, and according to the detected computation load, An image processing apparatus configured to switch a prediction method in the signal generation processing. 15. The image processing apparatus according to claim 14, wherein the prediction processing unit performs an encoding process on an image input signal corresponding to a target unit area on a processing target screen to be encoded. A motion detector that outputs a motion vector indicating a position of a prediction area in the previous screen that provides a predicted image input signal for the target unit area based on an image decoded signal corresponding to the previous screen; A motion compensator that calculates the predicted image input signal based on the motion vector of the target unit area to be calculated. The motion detector has a low pixel accuracy according to the detected calculation load. An image processing unit configured to output one of the first motion vector and the second motion vector with high pixel accuracy to the motion compensator as the motion vector. Apparatus. 16. A decoding processing unit that decodes an input image coded signal for each unit area dividing a screen and outputs an image decoded signal, and a decoding process based on the image decoded signal. Image decoding apparatus having a prediction processing unit that performs signal generation processing for generating a predicted image decoded signal corresponding to a target unit area to be processed by a calculation based on a predetermined motion compensation method on a required unit area The prediction processing unit further comprises: a calculation load detecting unit configured to detect a size of a calculation load in the decoding process of the image coded signal. The signal generation unit generates the signal according to the detected calculation load. An image processing apparatus having a configuration for switching a motion compensation method in processing. 17. The image processing apparatus according to claim 16, wherein the prediction processing unit is configured to perform the decoding on the target unit area in the previous screen on which the decoding process has been performed before the target screen to be decoded. Generating the predicted image decoded signal based on the motion vector information indicating the position of the prediction area for providing the corresponding predicted image decoded signal, wherein the target image is generated based on the motion vector information of the target unit area. Calculating a predicted image decoded signal corresponding to the unit area;
A first motion compensation process having a small arithmetic processing amount, and calculating a predicted image decoded signal corresponding to the target unit region based on motion vector information of the target unit region and unit regions around the target motion region. An image processing device comprising: a motion compensator that performs one of the large second motion compensation processes in accordance with the detected calculation load. 18. A data storage medium storing a program for causing a computer to perform an image encoding process, wherein the program encodes an image input signal for each unit area for dividing a screen. Calculating a predicted image input signal corresponding to a target unit area to be encoded based on a decoded image signal obtained by decoding the encoded image signal, based on a predetermined prediction method. Is performed on a required unit area. At this time, the magnitude of the computation load in the encoding process of the image input signal is detected, and according to the magnitude of the detected computation load, A data storage medium characterized in that a computer is configured to perform image encoding processing for switching a prediction method in the signal generation processing. 19. A data storage medium storing a program for causing a computer to perform an image decoding process, wherein the program decodes an input image coded signal for each unit area for dividing a screen. A decoded image signal corresponding to a target unit area to be decoded, based on the decoded image signal, and a prediction image decoded signal generated by an operation based on a predetermined motion compensation method. The generation process
The processing is performed for a required unit area. At this time, the magnitude of the operation load in the decoding processing of the image coded signal is detected, and the motion compensation in the signal generation processing is performed in accordance with the detected operation load. A data storage medium, wherein a computer is configured to perform an image decoding process for switching a method.