JPH0817492B2 - Variable rate video coding device - Google Patents

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a variable rate moving image coding apparatus used for a Delevi telephone, a video conference system or the like.

2. Description of the Related Art FIG. 4 shows the configuration of a conventional variable rate moving image coding apparatus. In FIG. 4, 40 is a moving image signal input in frame units, 40a is obtained by subjecting this moving image signal and a previous frame reproduction signal stored in a frame memory 48 described later to motion compensation and the like, A subtractor to which the predicted signal of the current frame, which is the next frame, is input,
The frame-by-frame moving image signal 40 is subtracted from the current frame prediction signal to output an N × N prediction error signal.

Reference numeral 41 denotes a discrete cosine transform circuit that performs a discrete cosine transform on the N × N prediction error signal input from the subtractor 40a and outputs a transform coefficient. Reference numeral 42 denotes a transform coefficient input from the discrete cosine transform circuit 41. Is a quantization circuit that outputs the N × N quantized representative value.

Further, 43 is a portion to which the quantized representative value from the quantization circuit 42 is input and which includes a low frequency component of the prediction error signal (hereinafter, referred to as MS
P) is separated and transmitted on the variable rate high priority channel 44, and the part containing the high frequency component of the prediction error signal (hereinafter referred to as LSP) is separated and transmitted on the variable rate low priority channel 45. It is a hierarchical circuit.

46 is a dequantization circuit for local decoding, which dequantizes the MSP input from the layering circuit 43 via the variable rate high priority channel 44, and 47 is input from the dequantization circuit 46. Is an inverse discrete cosine transform circuit for performing inverse discrete cosine transform on the inversely quantized transform coefficient, which is also for local decoding.

40b is the inverse discrete cosine transform output from the inverse discrete cosine transform circuit 47, and the prediction of the current frame obtained by subjecting the previous frame playback signal output from the frame memory 48 to motion compensation and the like. It is an adder that adds the signal and outputs it to the frame memory 48 as a current frame reproduction signal.

Next, the operation of the above conventional example will be described. In FIG. 4, motion compensation or the like is applied to the previous frame reproduction signal stored in the frame memory 48, whereby a prediction signal of the current frame is obtained. The prediction signal of the current frame and the moving image signal 40 in frame units are subtracted by a subtractor 40a to become a prediction error signal, and hereinafter, processing is performed for each block of size N × N.

This N × N prediction error signal is a discrete cosine transform circuit.
In 41, discrete cosine transform is performed to obtain transform coefficients, which are input to the quantization circuit 42.

The quantization circuit 42 quantizes the transform coefficient obtained by the discrete cosine transform, and the quantized representative value is input to the layering circuit 43. The layering circuit 43 separates the N × N quantized representative value into the MSP of the prediction error signal and the LSP of the prediction error signal, where MSP is the variable rate high priority channel 44 and LSP is the variable rate low priority channel. Transmit at 45.

The variable rate high-priority channel 44 and the variable rate low-priority channel 45 are both variable rate channels capable of supporting arbitrary transmission rates, and are the wideband ISDN (digital service integrated services) currently being studied as a next-generation communication network. Network).

In wideband ISDN, a plurality of variable rate channels with different priorities are prepared, and when congestion occurs in a communication network (hereinafter referred to as network), it is planned to perform control such that packets of low priority channels are discarded. Correspondingly, the variable rate high priority channel 44 has a high priority, that is, the packet is not discarded even when the network is congested, and the variable rate low priority channel 45 is the network congestion. At times, it is as a channel on which packets may be dropped.

Local decoding is performed using only MSP, and decoding on the receiving side is performed by adding MSP to the value of the previous frame and then adding the decoded signal of LSP.

Therefore, when the network is congested, the LSP is stochastically discarded, but the LSP itself has a small influence on the image quality and does not spread over time, and therefore deterioration of the image quality can be suppressed to a small extent.

The MSP is inversely quantized by an inverse quantization circuit 46, and the inversely quantized transform coefficient is an inverse discrete cosine transform circuit 47.
To the inverse discrete cosine transform.

This inverse discrete cosine transformed transform coefficient is added by the adder 40b.
Is added to the prediction signal of the current frame output from the frame memory 48, and the addition result is output to the frame memory 48.

DISCLOSURE OF THE INVENTION Problems to be Solved by the Invention However, in the above-mentioned conventional variable rate moving image coding device, the MSP and LSP code amount ratio, or the S / N ratio (signal to noise ratio) in a block is used as a threshold value, Since the LSP and the LSP are separated from each other, there is a problem that the MSP and the LSP are not separated from each other in consideration of the influence of the image quality deterioration when the LSP is discarded on the visual sense.

The present invention is to solve such a conventional problem, in consideration of the visual spatial frequency characteristics, the image quality deterioration amount that occurs when a packet in a certain area within a block is discarded due to network congestion. It is possible to set the image quality deterioration amount set for each area. Therefore, by using the intra-block error weighted by the visual spatial frequency characteristic, the inside of the block is divided into M areas and hierarchized, and the packetization buffer is used. It is an object of the present invention to provide a variable rate moving image coding apparatus that can be transmitted via variable rate channels having different priorities.

Means for Solving the Problems In order to achieve the above object, the present invention subtracts a current frame prediction signal obtained by multiplying a previous frame reproduction signal stored in a frame memory by a prediction coefficient and a moving image signal in frame units. A predictive error signal obtained from the result is subjected to a discrete cosine transform for each block of size N × N to output a transform coefficient, and the transform coefficient output from the discrete cosine transform circuit is quantized. A quantization circuit for outputting the quantized transform coefficient and the quantized representative value of the quantized transform coefficient; and the transform coefficient obtained by dequantizing the quantized representative value output from the quantization circuit. And an inverse discrete cosine transform circuit for inverse discrete cosine transform of the transform coefficient output from the inverse quantum circuit, and an inverse discrete cosine transform circuit. Force and the current frame prediction signal are added, and the adder for accumulating the addition result in the frame memory, and N × N number of N × N obtained by performing the discrete cosine transform for each block in the discrete cosine transform circuit. Based on the spatial frequency characteristic of the visual sense, the image quality deterioration amount that occurs when information of a predetermined area of the M areas is discarded when the conversion coefficient is divided into M areas and hierarchized A layering circuit that calculates and divides and hierarchizes the conversion coefficient so that the image quality deterioration amount is approximately equal to the allowable value of the image quality deterioration amount set in advance for each area, and M The data in each region of the transform coefficient, which is divided into a plurality of regions and hierarchized, is provided with a bucketing buffer that transmits the variable rate channels with different priorities.

Effect of the Invention With the above-described configuration, the image quality deterioration amount that occurs when a packet in a certain area in a block is discarded due to network congestion is set in advance for each area in consideration of visual spatial frequency characteristics. Therefore, by using the intra-block error weighted by the visual spatial frequency characteristic, the block is divided into M regions and hierarchized, and the variable rate channels with different priorities are transmitted by the packetization buffer. can do.

Embodiment An embodiment of the present invention will be described below with reference to the drawings.

FIG. 1 is a block diagram showing the configuration of a variable rate moving image coding apparatus according to an embodiment of the present invention. In FIG. 1, 10 is a moving image signal input in frame units, and 10a
Is a subtracter to which this moving image signal and the prediction signal of the current frame obtained by multiplying the reproduction signal of the previous frame stored in the frame memory 18 described later by the prediction coefficient a in the prediction coefficient multiplier 19 are input. Yes, these video signals in frame units 10
And the prediction signal of the current frame are subtracted to output an N × N prediction error signal.

Reference numeral 11 denotes a discrete cosine transform of the N × N prediction error signal input from the subtractor 10a to obtain an N × N transform coefficient D (u,
The discrete cosine transform circuit for outputting v), 12 quantizes the transform coefficient D (u, v) input from the discrete cosine transform circuit 11, and quantizes the transformed coefficient Dq (u, v) Quantized representative value Ix (u, v) of the quantized transform coefficient Dq (u, v)
This is a quantization circuit that outputs and.

13 is a transform coefficient D (u, from the discrete cosine transform circuit 11
v) and the quantized transform coefficient Dq from the quantization circuit 12
(U, v) and the quantized representative value Ix (u, v) are input to the hierarchical circuit.

In the layering circuit 13, the image quality deterioration caused when the information of a certain area Lm in the block of the N × N conversion coefficient D (u, v) is discarded is an allowable value TH in consideration of the visual spatial frequency characteristic.
Transform coefficients quantized to be approximately equal to _Lm
Using Dq (u, v) and the quantized representative value Ix (u, v), the above N
The conversion coefficient D (u, v) of × N is divided into M areas (L1 to LM) and hierarchized.

Reference numeral 14 is a packetizing buffer that packetizes the data layered by the layering circuit 13 for each layer in order to transmit the data using the variable rate channel 15 corresponding to each layer.

Reference numeral 15 is a variable rate channel for transmitting data packetized for each layer in the packetization buffer 14 and capable of supporting an arbitrary transmission rate, and has a plurality of channels having different priorities.

16 is the quantized representative value input from the quantization circuit 12
An inverse quantization circuit for local decoding, which inversely quantizes Ix (u, v) and outputs an inversely quantized transform coefficient, 17,
A circuit for local decoding, which is an inverse discrete cosine transform circuit that outputs an inverse discrete cosine transformed transform coefficient by performing an inverse discrete cosine transform on the inverse quantized transform coefficient input from the inverse quantization circuit 16. is there.

10b is the inverse discrete cosine transform circuit output transform coefficient of the inverse discrete cosine transform circuit 17, and adds the current frame prediction signal output from the prediction coefficient multiplier 19 as a current frame reproduction signal. It is an adder that outputs to the frame memory 18.

Next, the operation of the above embodiment will be described. In FIG. 1, the previous frame reproduction signal stored in the frame memory 18 is subjected to motion compensation and the like, and further, a prediction coefficient multiplier
The prediction coefficient a (0 <a ≦ 1) is multiplied by 19 to obtain the prediction signal of the current frame.

This current frame prediction signal and the frame-by-frame moving image signal 10 are subtracted by a subtractor 10a to become a prediction error signal,
Hereinafter, processing is performed for each block of size N × N.

This N × N prediction error signal is a discrete cosine transform circuit.
After the discrete cosine transform in 11, the transform coefficient D (u,
It is input to the quantization circuit 12 as v) and also to the layering circuit 13.

The quantization circuit 12 uses a transform coefficient D that has been discrete cosine transformed.
(U, v) is quantized, and the quantized transform coefficient Dq (u, v) and the quantized representative value Ix (u, v) thus obtained are input to the layering circuit 13. At the same time, the quantized representative value Ix (u, v) of the quantized transform coefficient Dq (u, v) is
It is also input to the inverse quantization circuit 16 for local decoding.

The transform coefficient obtained by the inverse quantization of the quantized representative value Ix (u, v) by the inverse quantization circuit 16 is inverse discrete cosine transformed by the inverse discrete cosine transform circuit 17, and then the prediction coefficient multiplier 19
The predicted signal of the current frame output from the adder 10b
In the frame memory 18 as the current frame playback signal.
Is accumulated in

On the other hand, the layering circuit 13 outputs the quantized transform coefficient Dq (u, v) and the quantized representative value Ix (u,
v) is used to output N of the output of the discrete cosine transform circuit 11.
The block of the conversion coefficient D (u, v) of × N is divided into M areas in block units and hierarchized.

The packetizing buffer 14 packetizes the data layered in each layer in order to transmit the layered data in the layering circuit 13 using a channel corresponding to each layer, and sequentially packetizes the packetized data into a variable rate channel. Output to 15.

Next, a method of dividing each block into M areas and hierarchizing them in the hierarchization circuit 13 will be described with reference to FIGS. 2 and 3.

FIG. 2 is an explanatory diagram showing a method of dividing an N × N transform coefficient D (u, v) into M regions, and FIG. 3 is a diagram showing a visual spatial frequency characteristic of an error in a discrete cosine transform region. FIG. 8 is an explanatory diagram showing a numerical example of obtaining a weighting coefficient for multiplying an error for a block size of 8 × 8 in order to evaluate in consideration of

The size of the block as the processing unit is N × N, and the transform coefficient output from the discrete cosine transform circuit 11 is D (u,
v), the transform coefficient after quantizing the transform coefficient D (u, v) by the quantizing circuit 12 is Dq (u, v), and the quantized representative value is Ix.
(U, v).

Further, in order to evaluate the error in the discrete cosine transform region in consideration of the visual spatial frequency characteristic, the weight for multiplying the error is W (u, v). However, u = 0, 1, 2, ..., N-1 v = 0, 1, 2, ..., N-1.

Further, for simplification, as shown in FIG. 2, a case where a block is divided into three areas and hierarchized will be described.

First, consider the case where no packet discard occurs. When the packet is not discarded, the error of the transform coefficient D (u, v) generated in the block is only the quantization error.

The error occurring in the block is determined by the visual spatial frequency characteristic W
If the mean squared error calculated by weighting with (u, v) is defined as WMSE and the mean squared error (WMSE) when packet discard does not occur is WMSE _min , WMSE _min is calculated by the following formula. It

In addition, the S / N ratio calculated using WMSE is defined as WSNR,
WSN is the signal-to-noise ratio (WSNR) when no packet drops occur
Assuming R _max , WSNR _max is calculated according to the following equation.

Next, consider a case where the packet P ₁ (α, β) in the area L3 shown in FIG. 2 is discarded.

When the packet P ₁ (α, β) in the area L3 is discarded, the error of the transform coefficient D (u, v) generated in the block is the error due to the discarded information and the error in the areas L1 and L2 that are not discarded. It is the sum of the quantization errors.

Therefore, if the WMSE in this case is WMSE _L31oS , WMSE
_L31oS is calculated according to the following formula.

Becomes However, the scanning of u and v in the calculation of Σ in the equation (3) is performed by zigzag scanning as shown in FIG. 2, for example.

here, WS when the packet P ₁ (α, β) in the region L3 is discarded
_Assuming that NR is WSNR _L31oS , from the above equations (2) and (4), Becomes

In the second term on the right side of the above equation (5), the packet in the region L ₃ is P ₁
(Α, β) represents the amount of image quality deterioration caused by discarding. When this is set to ΔD _L3 , γ = EXP10 (ΔD _L3 / 10) (6) However, EXP10 (X) = 10 ^X.

On the other hand, the above equation (3) can be rewritten as the following equation.

Furthermore, from equations (4), (6), and (7), Get.

Therefore, in order that the image quality deterioration amount when the packet P ₁ (α, β) of the region L3 is discarded becomes a certain allowable value TH _{L3 in} consideration of the preset visual spatial frequency characteristic, equation (8) is used. , ΔD _L3 = TH _L3 , and this is satisfied (α,
β) may be defined as a boundary point between the regions L2 and L3.

However, in general, in the expression (8), it is not possible to obtain (α, β) such that the left side and the right side completely match, so that the value on the left side and the value on the right side are the closest ( α,
Let β) be the boundary point.

Considering the division of the region L2 in the same manner, the boundary point between the regions L1 and L2 is a point (ζ, η) that satisfies the following equation, where P (ζ, η) is the packet of the region L2.

Regions L1, L2, and L3 divided in this way are divided into regions L1
Indicates a low frequency component of the prediction error signal, the region L3 includes a high frequency component, and the region L2 includes an intermediate frequency component between the regions L1 and L2. Therefore, the regions L1, L2, and L3 are transmitted in the order of high priority change rate channels.

Also, the allowable value TH _L2 considering the spatial frequency characteristics of vision,
When TH _L3 is changed, the amount of information generated in each area L1, L2, L3 changes.

For example, if the permissible value TH _L2 of the visual frequency characteristic is reduced, the area L2 determined by the equation (9) becomes smaller, so that the amount of information generated in the region L2 decreases and the amount of information generated in the region L1 decreases. To increase.

As a result, feedback control is performed from the packetization buffer 14 to the layering circuit 13, and the allowable values TH _L2 and TH _L3 of the visual spatial frequency characteristics are appropriately changed to control the amount of information generated in each variable rate channel. be able to.

Therefore, according to the above-mentioned embodiment, when the network is congested, the packets in the region L3 having a lower priority are discarded in order, but the image quality deterioration amount caused when the packets are discarded is considered in consideration of the preset visual spatial frequency characteristic. It is possible to set the allowable value TH _Lm .

Also, the allowable value TH _L2 considering the spatial frequency characteristics of vision,
By appropriately changing TH _L3 , the amount of information generated in each variable rate channel can be controlled.

Further, the packet data in a certain area is discarded, which causes a difference in image data between transmission and reception, but this influence decreases with time due to the prediction coefficient a.

In the above embodiment, the block is divided into three areas.
I explained the case of layering, but if you think in the same way,
It is easy to divide this into M areas and divide them into layers.

Further, in the above embodiment, u when calculating Σ of each equation,
Although the zigzag scanning as shown in FIG. 2 is used as the v scanning order, the present invention is not limited to this scanning order.

Further, in order to evaluate the error in the discrete cosine transform domain in consideration of the visual spatial frequency characteristic, FIG. 3 shows an example of the weighting coefficient by which the error is multiplied, in the case of the block size 8 × 8 in the present embodiment. Although such a numerical value is mentioned, the present invention is not established to this numerical value.

As described above, according to the present invention, the prediction obtained from the subtraction result of the current frame prediction signal obtained by multiplying the previous frame reproduction signal stored in the frame memory by the prediction coefficient and the moving image signal in frame units A discrete cosine transform circuit that outputs a transform coefficient by performing a discrete cosine transform for each block of size N × N, and the transform coefficient output from the discrete cosine transform circuit is quantized and quantized. And a quantization circuit that outputs the quantized representative value of the quantized transform coefficient, and an inverse quantum that inversely quantizes the quantized representative value output from the quantization circuit and outputs a transform coefficient. Circuit, an inverse discrete cosine transform circuit for performing an inverse discrete cosine transform of the transform coefficient output from the inverse quantizer circuit, an output of the inverse discrete cosine transform circuit and the current frame An adder for adding the expected signal and accumulating the addition result in the frame memory, and N × N transform coefficients obtained by performing the discrete cosine transform for each block in the discrete cosine transform circuit When dividing or hierarchically dividing into M areas, the image quality deterioration amount that occurs when information of a predetermined area of the M areas is discarded is calculated based on the visual spatial frequency characteristic. A hierarchical circuit that divides and hierarchizes the conversion coefficients so that the deterioration amount is approximately equal to the allowable value of the image quality deterioration amount set in advance for each area, and the hierarchy circuit divides the area into M areas. The bucketed buffer for transmitting the data in each region of the layered transform coefficients through the variable rate channels having different priorities is adopted.

Therefore, the image quality deterioration amount that occurs when a packet in a certain area in a block is discarded due to network congestion can be set to the image quality deterioration amount that is set in advance for each area in consideration of visual spatial frequency characteristics. Therefore, by using the intra-block error weighted by the visual spatial frequency characteristic, the inside of the block is divided into M areas and hierarchized, and the packetized buffer can transmit the variable rate channels having different priorities.

[Brief description of drawings]

FIG. 1 is a block diagram showing a schematic configuration of a variable rate moving picture coding apparatus according to an embodiment of the present invention, and FIG. 2 is a block diagram showing N × N transform coefficients in the hierarchical circuit of FIG. FIG. 3 is an explanatory diagram showing a method of dividing the region into individual regions, and FIG. 3 is for weighting the quantization error when calculating the mean square error in the discrete cosine transform region of the variable rate moving image coding apparatus of FIG. 4 is a schematic block diagram of a conventional variable rate moving image coding apparatus. 10 ... Moving image signal, 10b ... Adder, 11 ... Discrete cosine transform circuit, 12 ... Quantization circuit, 13 ... Hierarchical circuit, 14
...... Packetizing buffer, 15 …… Variable rate channel,
16 ... Inverse quantization circuit, 18 ... Frame memory, a ... Prediction coefficient, L1, L2, L3.

Continuation of the front page (51) Int.Cl. ⁶ Identification number Office reference number FI Technical indication location H04N 7/32 9466-5K H04L 11/20 102 E G06F 15/66 330 D (56) References -15791 (JP, A) JP 63-73789 (JP, A) ICC 88, Intl Symp on Communication (1988) P. 1257-1261 SPIE, Visual Communication and Image Processing 88 (1988) P.M. 991 −998

Claims (1)

[Claims] 1. A N × N prediction error signal obtained from a subtraction result of a current frame prediction signal obtained by multiplying a previous frame reproduction signal stored in a frame memory by a prediction coefficient and a moving image signal in frame units. A discrete cosine transform circuit that outputs a transform coefficient by performing a discrete cosine transform for each block of size, the above transform coefficient output from this discrete cosine transform circuit is quantized, and the quantized transform coefficient and this quantized transform coefficient are quantized. A quantization circuit that outputs the quantized representative value of the transformed coefficient, an inverse quantization circuit that inversely quantizes the quantized representative value output from the quantization circuit, and outputs the transformed coefficient, and an inverse quantization circuit The inverse discrete cosine transform circuit for performing the inverse discrete cosine transform of the transform coefficient output from the digitizing circuit, the output of the inverse discrete cosine transform circuit and the current frame prediction signal are added, and the addition is performed. The adder for accumulating the result in the frame memory and the N × N transform coefficients obtained by performing the discrete cosine transform for each block in the discrete cosine transform circuit are divided into arbitrary M regions and hierarchized. At this time, the image deterioration amount that occurs when information of a predetermined one of the M regions is discarded is calculated based on the visual spatial frequency characteristic, and this image deterioration amount is set in advance for each region. A layering circuit that divides and hierarchizes the conversion coefficient so that it is approximately equal to the allowable value of the image quality deterioration amount, and each region of the conversion coefficient that is divided into M areas by this layering circuit. Variable-rate video encoding device having a packetization buffer for transmitting the data of 1. by a variable-rate channel with different priority.