JPH02192378A - Inter-frame predictive coding system - Google Patents

Abstract

PURPOSE:To attain prediction adaptive to the change of a picture and to improve coding efficiency by predicting the signal of a non-independent frame based on a predictive signal corresponding to the signal and executing coding for the predicted error. CONSTITUTION:When the signal of a moving picture inputted from a picture signal input terminal 1 is the signal of the non-independent frame, changeover switches 37 and 38 are connected to a (b) side, and the signal is delayed by (N-1) frames in an (N-1) frame memory 31. Further, after the predicted signal is subtracted in a predicted signal subtracter 2, the signal is guided to an orthogonal transformer 3, and the coding is executed for the predicted error of the signal. The signal of the non-independent frame is delayed by the memory 31, and the picture signal of an independent frame used for the prediction is stored by frame memories 32 and 33 by 2 frames before prediction processing for the non-independent frame is executed. Thus, the predictive signal can be obtained from front and rear (new and old) frames. Consequently, the prediction adaptive to the change of the picture can be executed, and the coding efficiency can be improved.

Description

Detailed Description of the Invention (Industrial Field of Application) The present invention can be used to efficiently process moving image signals with a smaller amount of code in various equipment such as recording, transmission equipment, and other display devices that perform signal processing of digital signals. Among high-efficiency coding methods for encoding data, the present invention particularly relates to interframe predictive coding methods.

(Prior Art) Among high-efficiency encoding methods for encoding a moving image signal with a smaller amount of code, interframe predictive encoding is an encoding method that utilizes correlation between frames of an image signal.

This is because normal video images are quite similar between frames, so the signal of the frame to be encoded is predicted from the signal of the previous encoded frame, and the prediction error (residual error) is calculated.
It encodes only the

A typical conventional configuration of interframe predictive coding is shown in FIG. 4 for an encoder and in FIG. 5 for a decoder.

In FIG. 4, a moving image signal that is continuously input from an image signal input terminal 1 has a predicted signal (predicted value) subtracted by a predicted signal subtracter 2, and its prediction error (residual difference) is encoded. . Note that the method for forming the prediction signal will be described later.

Here, the prediction error (residual) may be quantized as is, but in order to obtain higher coding efficiency, it is generally orthogonally transformed by the orthogonal transformer 3 and then quantized by the quantizer 4. It has become. Since the distribution of the quantized signal concentrates around O (zero), it is converted into a variable length code such as a Huffman code by the variable length encoder 5, and outputted from the data output terminal 6 as variable length digital data. , recorded or transmitted.

On the other hand, on the decoder side, as shown in FIG. 23, the data is replaced with a representative value (representative value setting), and an orthogonal inverse transformer 24 performs inverse orthogonal transform processing.

Since the signal obtained in this way is a prediction error (residual error), the same signal as the prediction signal in the encoder is sent to the adder 25.
and output the reproduced image signal to output terminal 2 as a reproduced image signal.
It is output from 6.

Note that this predicted signal is obtained by delaying the reproduced image signal by one frame in the frame memory 27 and passing it through a spatial low-pass filter (LPF) 28 which is the same as the encoder.

On the other hand, the predicted signal at the encoder needs to be the same as the one at the decoder, and needs to be created from a quantized signal. Otherwise, in frame cyclic prediction processing as shown in FIG. 4, differences in prediction signals between the encoder and decoder will accumulate for each frame, resulting in a large error.

For this purpose, in the encoder shown in FIG. 4, the quantized signal is changed to a representative value by the inverse quantizer 7 (representative value setting), as in the decoder shown in FIG. 8 performs inverse transform processing of the orthogonal transform.

The signal obtained in this way is the decoded prediction error (
The predicted signal of one frame before is added to this by the adder 9 to obtain a decoded image signal. Furthermore, this signal is delayed by one frame by the frame memory 10, and is multiplied by a coefficient different depending on the spatial frequency using the spatial signal pl''11 to obtain a predicted signal.

The reason why the spatial L P F 11 is used here is to reduce the quantization error remaining in the prediction error (residual error), but in the encoding process, the quantization error is mostly in the high frequency range of the spatial frequency. On the other hand, inter-frame correlation also decreases in high frequencies due to noise, etc., so this is effective.

In addition, in cyclic interframe predictive coding as shown in FIG. 4, calculation errors accumulate due to spread of code errors occurring on the transmission path and mismatch between the transmitting side and the receiving side of the orthogonal inverse transformer 8. Interframe prediction is reset once in an appropriate interval (30 to 100 frames), and the prediction signal is fixed for that frame, which is essentially intraframe coding. This operation is performed by periodically switching the changeover switch 12. In this case, from the standpoint of error countermeasures, it is better to perform the reset in a short period, but since intra-frame encoding occurs at the time of reset, encoding efficiency decreases in image transmission with high interframe correlation, such as in video conferences. do.

(Problem to be solved by the invention) In such a cyclic interframe predictive coding method using the previous frame, when trying to decode a certain frame, since the data is accumulated from the past, all past Data is required. Therefore, there is no major inconvenience when transmitting images continuously, such as in a video conference, but in storage media such as information recording disks and information recording tapes, arbitrary locations within the media can be used for random access or search. It is necessary to reset interframe prediction in small units so that it can be decoded. In particular, when visual search is attempted, it is necessary to decode every few frames, which means that prediction is reset each time, which unilaterally causes a decrease in encoding efficiency.

On the other hand, in the case of reverse reproduction in which the data is reproduced in the reverse order in time compared to normal reproduction, decoding cannot be performed by conventional prediction using the previous frame because a prediction signal for decoding cannot be obtained.

In addition, prediction from the previous frame is prediction from one side of the time axis, which is not sufficient from the point of view of prediction efficiency.
Particularly when the image changes significantly, such as a scene change, appropriate prediction cannot be made.

Furthermore, since the predicted signal must be obtained by the same decoder as the decoding system, the encoder system also needs to perform decoding processing, which increases the size of the apparatus.

Another problem is that if there is a difference in the calculation precision of the decoding process, deviations occur in the predicted signals and accumulate.

Therefore, an object of the present invention is to provide an interframe predictive coding method that solves the problems of the conventional techniques described above.

(Means for Solving the Problems) In order to achieve the above object, the present invention sets independent frames at regular intervals (several frames) from among continuous frames of image signals that are continuously input. a first encoding means for encoding independent frames independently within the frame;
The predicted signal of the non-independent frame between the independent frames is
a prediction signal type adult stage that is formed based on the signals of the preceding and succeeding independent frames; and a second stage that predicts the signal of the non-independent frame based on the corresponding prediction signal and encodes the prediction error. The present invention provides an interframe predictive encoding method characterized by comprising: encoding means.

(Function) In the interframe predictive coding method with the above configuration,
Independent frames are set in advance at regular intervals (several frames) to be encoded independently within a frame without using interframe prediction from among consecutive frames of image signals that are continuously input, and frames in between are encoded independently within a frame without using interframe prediction. (old and new) are predicted and encoded using independently encoded independent frames.

This situation is shown in FIG. 6, where A is the conventional interframe prediction method and B is the interframe prediction method of the present invention. In the same figure, the rectangles are consecutive frames of a moving image signal that is input continuously, and the shaded frames are frames that are encoded independently within the frame.
(or at reset) is an independent frame, but in B, there are independent frames periodically. The arrows indicate the directional relationship of interframe prediction; in A, prediction is performed only from the previous frame as in each frame, but in B, prediction is performed from two independent frames before and after.

Furthermore, prediction is performed based only on independent frames, and predicted frames are not used for other predictions.

(Embodiment) An embodiment of the interframe predictive coding method according to the present invention will be described below with reference to the drawings.

1 and 2 show the structure of the encoder, and FIG. 3 shows the structure of the decoder. The basic configuration of this encoder and decoder is similar to the conventional example, and the same components in FIGS. 4 and 5 described above are given the same numbers.

1 and 2, it has (N-1) frame memories 31 [N is an integer of 2 or more] for encoding non-independent frames after the independent frames used for prediction have been encoded. .

In addition, in order to form a predicted signal (predicted value) based on the two frames before and after, two frame memories 32 and 33 and two coefficient multipliers (Xα, X( 1-α) ) 34.35 and their adder 36. [However, 0kuαku1] Furthermore, the changeover switch 37 is connected to the image signal input terminal 1 and (
N-1> between the frame memory 31, the changeover switch 38 between the predicted signal subtracter 2 and the orthogonal transformer 3, and the changeover switch 39 between the pattern generator 4 and the inverse quantizer 7. ,
Changeover switch 40 to two frame memories 32 and 33
provided between each.

Here, as will be explained in detail later, in Fig. 1, the same predicted signal is obtained on the decoder side as in the conventional case, but in Fig. 2, the predicted signal is obtained from the original image signal to be encoded. Therefore, the predicted signals on the encoder side and the decoder side are different. Although the configuration shown in FIG. 2 does not require decoding processing on the encoder side, it is possible because the system of the present invention is not a cyclic type of processing unlike the conventional example.

In FIG. 3, there is a predicted signal adder 41 that adds a predicted signal to the signal obtained from the orthogonal inverse transformer 24.

In addition, in order to form a predicted signal based on the two frames before and after, two frame memories 42 and 43 and two coefficient multipliers (xα,
x (1-1) 44.45 and their adder 4G.

Further, a changeover switch 47 is placed between the orthogonal inverse transformer 24 and the predicted signal adder 41, a changeover switch 48 is placed between the predicted signal adder 41 and the reproduced image signal output terminal 26, and a changeover switch 49 is placed between the two frames. They are provided between the memories 42 and 43, respectively.

In the configuration of the embodiment shown in FIG. 1, the moving image signal (continuous frames) input from the image signal input terminal 1 is
With the changeover switches 37 and 38, in the case of independently encoded frames, it is connected to the a side, and the (N-1) frame memory 3
1 and the predicted signal subtractor 2, and are guided to the orthogonal transformer 3.

Orthogonal transformer 3. Quantizer 4. The operation of the variable length encoder 5 is basically the same as in the conventional example.

On the other hand, since the remaining non-independent frames are inter-frame predicted, the predicted signal is subtracted, but in the method of the present invention, it is necessary to encode the independent frames first, so the remaining frames are delayed by that amount.

Here, if the number of independent frames is one frame in N frames [N is an integer of 2 or more], then the delay fil(N-
1) It will be for a frame. That is, for the remaining non-independent frames, selector switches 37 and 38 are connected to side b,
The signal is delayed by (N-1> frames) in the (N-1) frame memory 31, the predicted signal is subtracted in the predicted signal area n unit 2, and then guided to the orthogonal transformer 3, where the prediction error (residual ) is encoded.

Here, the changeover switches 37 and 38 are periodically connected to the a side for one frame every N frames, and otherwise connected to the b side. The following orthogonal transformer 3. Quantizer 4. The operation of the variable length encoder 5 is the same as for independent frames. The above operation is the same in the case of FIG. 2, which is another embodiment of the encoder.

In the case of the encoder having the configuration shown in FIG. 2, the predicted signal is obtained not from the encoded reproduced image signal but from the original image signal before encoding. The operation when forming a predicted signal is the same as in FIG. 1 except that the original image signal is input instead of the encoded reproduced image signal.

In the configuration of FIG. 2, the inverse quantizer 7 and the orthogonal inverse transformer 8 in FIG. 1 are not required. In this case, the predicted signal will be different between the transmitting side and the receiving side, but in the method of the present invention, the error is not accumulated for each frame.

Rather, since the quantization error no longer remains in the prediction error (residual), the need for the spatial LPF in the conventional example is eliminated, and prediction efficiency is improved.

Next, how to generate a prediction signal in the method of the present invention will be described. First, as in the conventional example, the same predicted signal is obtained on the encoder side and the decoder side, an example of which is shown in FIG. The difference here from the conventional example is that in the conventional example, all frames are used to obtain the predicted signal, whereas in the conventional example, all frames are used to obtain the predicted signal.
In the method of the present invention, a prediction signal is generated only from independently encoded frames (independent frames), so the changeover switch 39 is connected to the a side only for independent frames, and subsequent processing is performed.

The quantized signal is replaced by a representative value by an inverse quantizer 7 (representative value setting), as in the conventional example, and then an inverse orthogonal transform is performed by an orthogonal inverse transformer 8.

Since the signals obtained in this way are encoded independently, they are stored in the frame memory 32 in order to be used as they are to create a predicted signal without being added to the previous frame etc.
will be written to. At this time, the changeover switch 40 is connected to the a side, and the signal of the previous independent frame that had been held in the frame memory 32 is transferred to the frame memory 32.
can be replaced by Through this operation, reproduced frame signals used for prediction are prepared in the frame memories 32 and 33 at the same time as the independent frame encoding process.

This reproduced frame signal is held until the next independent frame signal is supplied, and is repeatedly output (N-1) times for prediction processing.

The predicted signal is obtained by multiplying these two reproduced frame signals by coefficients .alpha.

Here, the weighting coefficients are determined by the time relationship between the frame input to the prediction signal subtracter 2 to be encoded and the frame used for prediction.

The method considered to be the most general is a method using quadratic linear prediction, which is given by the following equation.

α=(m-ml) )/N where m is the frame number to be encoded (1°2.3
．． ), mp is the past independent frame number (0, N
, 2N, ·), m>mp t'', and N is an integer of 2 or more.

An example of a predicted signal (predicted value) created in this way is N=
Case 4 is shown in FIG. This gives greater weight to frames that are closer in time, and provides a more appropriate predicted value when the signal changes in a nearly linear manner from frame to frame.

In FIGS. 1 and 2 described above, the input image signal waits for independent frames every few frames by the switches 37 and 38, and then the inter-frame correlation of the encoded data is cut off. Therefore, visual search becomes possible by random access or by decoding only the data of independent frames.

On the other hand, the signal of the non-independent frame is delayed by (N-1) frame memory 31, and before the prediction process of the non-independent frame is performed, the image signal of the independent frame used for prediction is sent to the frame memory 32 and 33 for two frames. By storing the data, predicted signals can be obtained from the previous and subsequent (old and new) frames.

In addition, since the coefficient multiplication is performed by an appropriate coefficient depending on the time relationship between the frame predicted by the calculator 34 and 35 and the independent frame, it is possible to make predictions that are compatible with changes in the image, and the S/N of the prediction signal is also improved. , higher prediction efficiency is obtained.

Furthermore, since the encoded data obtained in this manner has a symmetrical structure on the time axis, reverse playback can also be easily realized.

Next, processing on the decoder side is performed in the configuration of the embodiment shown in FIG. 3. First, as in the conventional example, variable length digital data input from the data input terminal 21 is input to the variable length encoder 22. Reverse protonation amount 23. The orthogonal inverse transformer 24 converts the reconstructed image signal into independent frames and the pre-image signal into non-independent frames.
A IIl error (residual) signal is obtained. Since the independent frame signal is used for prediction, the changeover switch 47 is connected to the a side and the signal is written into the frame memory 42. At this time, the changeover switches 48 and 49 are also connected to the a side, and the signal of the previous independent frame is replaced in the frame memory 43, and simultaneously outputted from the reproduced image output terminal 26. In this way, reproduced frame signals used for prediction are prepared in the frame memories 42 and 43 at the same time as the independent frame decoding process.

On the other hand, when the reproduction prediction error (residual) signal is generated by a non-independent frame, the changeover switches 44 and 45 are connected to the b side, and the prediction signal subtractor 41 adds the same prediction signal (prediction value) as that on the encoder side. Then, the reproduced image is output from the reproduced image output terminal 26.

Further, the method of forming the predicted signal by multiplying the coefficient by 344.45 and adding g$346 is the same as that on the encoder side.

Note that the data transmitted from the encoder side is sent in advance of independent frames, so in order to compensate for this on the decoder side, the reproduced image signal of the independent frame is transmitted when the prediction process is completed. is output from the frame memory 42. That is, the frame memory 42 also serves as time correction.

(Effects of the Invention) As described above, in the method of the present invention, independent frames are encoded independently within a frame without using inter-frame prediction from among continuous frames of continuously inputted image signals. Since the frames in between are predicted and encoded using independent frames that are independently encoded before and after (old and new), there are independent frames at regular intervals (several frames). Since the frame correlation of the data is broken, random access can be made in storage media in units of data, and a decoded image can be obtained immediately. On the other hand, when attempting to perform a visual search, independent frames exist at regular intervals (several frames), so by decoding only the data of the independent frames, data is not wasted and a smooth search image is created. can get. Furthermore, since the encoding is basically symmetrical on the time axis, reverse reproduction is also possible by decoding in the reverse order. However, since the independent frame data is recorded before the other frames, correction is necessary during reverse playback, but since it is sufficient to time-correct the encoded and compressed data, small-scale data It's okay to delay.

On the other hand, since the images are encoded independently at regular intervals (several frames), encoding efficiency may drop for images with high interframe correlation; however, for images with such high interframe correlation, the basic It is easy to obtain good reproduced image quality, and this does not pose much of a problem. On the other hand, when the image moves, the prediction error (residual error) between frames increases, so the image quality tends to deteriorate, and it is desired to improve the prediction accuracy.

Furthermore, in the method of the present invention, since prediction is performed using the previous and previous (old and new) frames for frames between independent frames, it is possible to make predictions that are compatible with changes in the image, improving coding efficiency.

Furthermore, since the predicted signal is formed by adding a plurality of frames, the S/N of the predicted signal is improved and the prediction accuracy is improved.

Furthermore, even if there is an error in the prediction signal between the encoder and the decoder, it is not accumulated and the encoder uses the original image signal to perform prediction, rather than the encoded reproduced image from other signals. is also possible, in which case no signal processing is required in the encoder.

As described above, according to the method of the present invention, it becomes possible to perform random access, visual search, reverse playback, etc. in storage media, and it is also possible to encode images with scene changes and movement with high efficiency. .

[Brief explanation of the drawing]

1 and 2 are block diagrams showing the configuration of an encoder in an embodiment of the interframe predictive coding method according to the present invention, and FIG. 3 is a block diagram showing the configuration of an encoder in an embodiment of the interframe predictive coding method according to the present invention. Figure 4 is a block diagram showing the configuration of a decoder; Figure 4 is a block diagram showing the configuration of an encoder for interframe coding in a conventional example; Figure 5 is a block diagram showing the configuration of a decoder for interframe encoding in a conventional example. FIG. 6 is a diagram showing the interframe prediction method of the present invention and a conventional example, and FIG. 7 is a diagram showing an example of the interframe prediction value of the present invention. 1... Image signal input terminal, 2... Prediction signal subtractor,
3... Orthogonal transformer, 4... Quantizer, 5... Variable length encoder, 6... Data output terminal, 7.23...
Inverse quantizer, 8.24... orthogonal inverse transform, 21...
Data input terminal, 22...For variable length decoding, 26...
Playback image signal output terminal, 27, 32.33.42.43... frame memory,
28... Spatial LPF, 31... (N-1) frame memory, 34, 35.44.45... Coefficient multiplication calculator,
36.46... Adder, 37, 38.39.40.4
7.48.49... Changeover switch, 41... Prediction signal adder. Patent Applicant: Japan Victor Co., Ltd. Representative Kakiki
Kunio Diagram Diagram ()("s(>()("'*(1r)r) Diagram Revised=025 0.75 Diagram

Claims (1)

[Scope of Claims] A first encoding means that sets independent frames at regular intervals from among continuous frames of image signals that are continuously input, and encodes the independent frames independently within the frame; The predicted signal of the non-independent frame between the independent frames is
a prediction signal forming means for forming a prediction signal based on the signals of the preceding and following independent frames; and a second code for predicting the signal of the non-independent frame based on the corresponding prediction signal and encoding the prediction error. 1. An interframe predictive coding method comprising: