JPH0591499A - Difference encoder with movement compensation - Google Patents

Abstract

PURPOSE:To provide a movement compensation function, to reduce the scale of construction and to reduce cost. CONSTITUTION:Data for prescribed period of the preceding data written in a frame memory 23 are stored in a small-capacity memory 25, movement compensation calculation is performed from the current frame data and the data read out of the small-capacity memory 25 by a movement compensation circuit 26, and the difference between the data of the small-capacity memory 25 and the current frame data is calculated and encoded corresponding to the output of the movement compensation circuit 26.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a differential encoding apparatus having a motion compensation function, and is particularly suitable for application to, for example, so-called video conferences or video telephone systems.

[0002]

2. Description of the Related Art As a high-efficiency coding apparatus for compressing and coding and outputting an image signal, for example, a so-called video conference or a video telephone system, for example, H.264 in the so-called CCITT (International Telegraph and Telephone Consultative Committee) recommendation. There is a H.261 standard system.

This H.264 In the H.261 standard, the CIF format (intermediate signal format) is 360 pixels x 2
88 lines × 29.97 Hz, Y, C _B , C _R according to the non-interlaced method. In addition, the video source coding method is a hybrid coding in which the basic algorithm is motion compensation inter-frame prediction + orthogonal transform (DCT). Here, in the motion compensation, one vector is transmitted to a macro block of 16 × 16 pixels, and the search range is ± 1.
With 5 pixels x ± 15 lines, the color signal has no motion compensation, and the transform coding is DC with a block size of 8 x 8 (pixels).
T (discrete cosine transform), the applied quantization is selected from a maximum of 8 types of scans, the maximum number of quantizers is 32, and the in-loop filter is applied to the predictive code block (8 × 8) and 121 low pass types are applied. Is becoming In the video multiplex coding, the data structure has a four-layer structure of a frame, GOB, macroblock, and block. The macroblock type defines 10 types according to the inter / intra coding mode, motion compensation (MC) presence / absence, DCT coefficient presence / absence, quantizer change presence / absence, and filter presence / absence. The motion vector information is encoded by 32 code by encoding the difference vector and folding back the vector value. The coding of DCT coefficients is a two-dimensional coding for zero coefficient runs and non-zero coefficient values. Furthermore, the transmission format is
Bit rate is p × 64 kbit / sec, p = 1 to 3
It becomes 0. In buffering, the maximum number of bits in one encoded frame is 256 kbits in CIF, QCIF ((1
/ 4) CIF) results in 64 kbits.

In FIG. 4, the H. The structure of the encoding device in the H.261 standard is shown. The encoding apparatus of FIG. 4 uses an input video signal (input video signal) or a differential signal described below as a DC signal.
T (discrete cosine transform) is performed, the DCT-processed data is quantized, and then the quantized data is variable-length coded to realize compression coding of the input video signal. That is, the coding apparatus of FIG. 4 makes a determination of so-called inter-frame predictive coding (inter-frame predictive coding) / intra-frame predictive coding (intra-frame predictive coding), and according to this determination, the input video signal To DCT
(Intra coding mode) or DC
T (inter coding mode).

First, the case of the intra coding mode will be described. That is, in FIG. 4, the H.V. A video signal (digital video signal) of a block format of 261 standard is supplied.
This video signal is sent to a CPU (Central Processing Unit) 8 which will be described later.
A switched signal from 0 (switching signal according to the intra coding mode of the inter / intra coding mode) supplies the switched terminal a to the discrete cosine transform (DCT) circuit 83 through the selected selector 82. The DCT circuit 83 outputs a frequency component obtained by subjecting the input video signal to the discrete cosine transform process. This DCT
The output of circuit 83 is sent to quantizer 84.

The quantizer 84 has its quantization step controlled by the CPU 80, and quantizes the frequency component from the DCT circuit 83. The output of the quantizer 84 (quantization output index q of the transform coefficient) is the terminal 74.
Via a variable length coding (VLC) process such as Huffman coding which adaptively uses so-called run length coding.

The data variable-length coded by the variable-length coding circuit 91 is serially output to the communication line side via the output terminal 77.

In the inter coding mode, the output of the quantizer 84 is sent to the predictive coding processing unit with motion compensation. That is, this predictive coding processing section is composed of the respective components after the inverse quantizer 85, and the output of the quantizer 84 sent to the predictive coding processing section is first the inverse quantizer. The quantizer 85 performs a process (inverse quantization process) opposite to the quantization process of the quantizer 84, and thereafter, an inverse discrete cosine process that is a process opposite to the discrete cosine transform in the DCT circuit 83. It is stored in an image memory (frame memory) 88 having a variable delay function for motion compensation through an IDCT circuit 86 which performs conversion processing and further through an adder 87. Note that the processing up to the frame memory 88 is performed even in the intra coding mode.

The input video signal of the current frame is supplied to the input terminal 70 and sent to the subtractor 81. At this time, the output of the frame memory 88 (data of the previous frame with respect to the current frame) is sent to the subtractor 81 via the loop filter 89 for noise removal. Therefore, the output of the subtractor 81 becomes difference data between the current frame and the previous frame. The output of the subtracter 81 passes through the path of the DCT circuit 83, the quantizer 84, the dequantizer 85, and the IDCT circuit 86 via the selector 82 whose terminal b is selected according to the inter-encoding mode. It is then supplied to the adder 87.

At this time, in the adder 87, the data of the previous frame stored in the frame memory 88 is passed through the loop filter 89, and further the switching signal (inter-encoding) from the CPU (central processing unit) 80. Selector 9 whose selected terminal b is selected by a switching signal according to the mode)
It is supplied via 0. That is, the adder 87 works in accordance with the inter / intra coding mode, and the output of the adder 87 is the difference data between the previous frame and the current frame obtained from the subtractor 81 and the data of the previous frame. It becomes the added data (that is, the data of the current frame). The data of the current frame from the adder 87 is again stored in the frame memory 88.

As described above, whether to use the inter coding mode or the intra coding mode depends on the above C.
It is controlled by the PU 80. Specifically, the CPU 8
At 0, the energy of the difference data is compared with the energy of the input video data of the current frame. For example, when the energy of the input video data of the current frame becomes larger, the selector 82 performs the processing in the inter coding mode. , 9
0 is controlled, and in the opposite case, the selectors 82 and 90 are controlled so that the processing in the intra coding mode is performed.

Further, the quantizer 84 is provided with a buffer memory (not shown) which is normally connected to the output terminal 77 after the output terminal 77 in order to adjust the difference between the transmission rate inside the encoding device and the transmission rate of the communication line. The quantization step is controlled in order to prevent (omitted) overflow and the like. That is,
The CPU 80 is supplied with accumulated amount data indicating the accumulated amount of the buffer memory from the buffer memory, and the CPU 80 controls the quantizing step of the quantizer 84 based on the accumulated amount data. There is.

In FIG. 4, an inter / intra coding mode identification flag p is output from the terminal 71 to the subsequent configuration, and a transmission / non-transmission identification of whether or not to transmit a signal is performed from the terminal 72. The flag t is output and the terminal 73
Outputs the quantization characteristic designation data qz, which is the control information of the quantization step in the quantizer 84, from the terminal 75.
Outputs the motion vector data v from the frame memory 88 having a variable delay function for motion compensation, and the terminal 76 outputs data f indicating ON / OFF of the filter processing in the loop filter 89.

Further, each processing in the above-mentioned encoding device is performed by a block of 8 × 8 pixels or a macroblock consisting of four blocks of 8 × 8 pixels (that is, 16 blocks).
X16 pixel unit).

[0015]

By the way, the above H. Two
In the 61 standard, an encoding device (differential encoding device) is configured so as to support both the case where motion compensation is performed and the case where motion compensation is not performed.

Here, when performing the above motion compensation, One data block (eg, 1) to the frame memory used in the above-described encoding device to which the H.261 standard is applied.
The following four accesses are required for access in the processing of (macroblock). That is, writing of data of a new image frame (writing of data of the current frame), reading of data of a frame of an old image for motion compensation (reading of data of a previous frame), and difference calculation (difference calculation in DPCM) 4), that is, the reading of the data of the frame of the old image for (4) and the reading of the data for the display set as an option are required.

As described above, when the number of times of access to the frame memory increases, the normal frame memory cannot support the access capability. Therefore, in the conventional coding device for motion compensation, a frame memory having two or more planes (two) is usually provided as the frame memory, so that the frame memory per one (one plane) can be accessed. It is designed to reduce the burden of speed. That is, in the conventional encoding device, the load of access speed is reduced by writing the image data being decoded on one surface and reading the data from the other surface.

In the case of the conventional apparatus of FIG. 4 described above,
For example, the frame memory 88 needs a RAM of about 2 Mbits on one side, and this frame memory is used on two sides (4 Mbits in total) as described above, and one of the one side writes image data being decoded. And on the other side D
Data for CPM and motion compensation (MC) is read out.

However, as described above, the two-sided frame memory also requires high-speed access, and therefore requires an expensive frame memory. On the other hand, when an inexpensive frame memory is used, it is difficult to operate at a high speed, so a frame memory having a plurality of surfaces is required, which leads to an increase in circuit scale. Furthermore,
Considering to make the frame memory and the like into an LSI (Large Scale Integrated Circuit), the number of frame memories to be controlled increases, so that the number of pins also increases. When the number of pins increases like this, the LSI configuration ( It will adversely affect the design).

The encoder (differential encoder) without the motion compensation function does not need to access the frame memory four times as described above. unnecessary. However, this differential encoding apparatus is described in the above H.264. In order to support the H.261 standard, it is necessary to be able to support both the case where motion compensation is not performed and the case where motion compensation is performed. Therefore, normally, the above-described two-sided frame memory is provided. Therefore, the circuit scale is increasing.

Therefore, the present invention has been proposed in view of the above situation, and has a motion compensation function, a structure can be downsized, and the cost can be reduced. That is, it is an object of the present invention to provide a differential encoding device with motion compensation.

[0022]

A motion compensation differential encoding apparatus of the present invention has been proposed in order to achieve the above-mentioned object, and includes a frame memory for storing image frame data and the frame. A memory control unit for reading the data of the previous frame written in the memory and for sequentially writing the data of the current frame in the frame memory, and data for a predetermined period of the data of the previous frame read from the frame memory. It has an auxiliary memory for accumulating and arithmetic means for supplying data of the current frame and read data of the auxiliary memory to arithmetically operate data relating to the movement of the image in the predetermined period, and the output of the arithmetic means According to the above, the data in the auxiliary memory is read and the difference from the data in the current frame is calculated and encoded. It is obtained by way.

[0023]

According to the present invention, a part of the function of the frame memory is assigned to the auxiliary memory which is sufficiently smaller than the frame memory, so that a small frame memory can be used. At the same time, we are trying not to increase the size of the equipment.

[0024]

Embodiments of the present invention will be described below with reference to the drawings.

As shown in FIG. 1, the differential encoding apparatus with motion compensation according to the embodiment of the present invention includes a frame memory 23 for storing data of one frame of an image and data of a previous frame read from the frame memory 23. In a small capacity memory 25 as an auxiliary memory for accumulating data for a predetermined period of time (for example, data corresponding to one GOB in the above-mentioned bit multiplex encoding data structure of the H.261 standard) and the frame memory 23. The controller 27 as a memory control circuit for controlling the writing and reading of the small-capacity memory 25 while reading the written data of the previous frame and sequentially writing the data of the current frame in the frame memory 23, and the current frame And the read data of the small capacity memory 25 are supplied. And a motion compensation (MC) circuit 26 as a calculation means for calculating data related to the motion of the image in the predetermined period, and the data of the small capacity memory 25 according to the output of the motion compensation circuit 26. Is read out and the difference from the data of the current frame is calculated.

Here, the memory capacity of the small-capacity memory 25 depends on the complexity of the configuration of the controller 27, but is 256 k in the simplest configuration, for example.
The capacity is 2 bits. That is, the small-capacity memory 25 of the present embodiment is the same as the H.264 standard. It is used as a memory for buffering 1 GOB worth of data of the data structure of the H.261 standard video multiplex coding. The small-capacity memory 25 can be used as another memory by changing the clock or the like.

Further, in the present embodiment, as the frame memory 23, two 1 Mbit dual port RAMs are provided, and the clock transfer of data for 3 macroblocks can be performed in the processing time TB of 1 macroblock (clock CK is 14).
MHz to 15 MHz, 1 macroblock processing time TB
It operates at a timing of about 1200 clocks). The mapping of the frame memory 23 is such that one row address corresponds to one macroblock of data.

The configuration shown in FIG. 26
The present invention is applied to the high-efficiency coding apparatus as shown in FIG. 4 which compresses and outputs an image signal according to the H.1 standard, and FIG. 1 shows only the configuration of the main part of the high-efficiency coding apparatus. In addition, the same components as those in the configuration of FIG. 4 are designated by the same reference numerals, and detailed description thereof is omitted.

Further, FIG. 2 shows a block diagram of FIG.
3 is a timing chart of the configuration of FIG. 3 and the configuration of FIG. DEV ₁ of FIG. 2 shows the timing of the configuration of FIG. 1, and DEV ₃ of FIG. 2 shows the timing of the configuration of FIG.

In FIG. 1, the terminal 10 has the same structure as that shown in FIG.
The output of the inverse quantizer 85 having the above construction is supplied, and the output from the terminal 12 is sent to the subtractor 81 shown in FIG. The data from the terminal 10 (differential data from the inverse quantizer 85) is passed through the IDCT circuit 86, and further through the adder 87 that operates according to the inter / intra coding mode, It is supplied to the frame memory 23 of this embodiment. That is, in the configuration of FIG. 1 (timing chart of DEV ₁ in FIG. 2), I
Writing of data from the DCT circuit 86 to the frame memory 23 (writing W _IDCT from the IDCT circuit 86 shown in DEV ₁ of FIG. 2) is performed in the page mode, and the portion of the frame memory 23 which has not been rewritten is serial port. The difference calculation data, the motion compensation calculation data, and the display data set by the option are read out via. The data read from the frame memory 23 is transferred through the multiplexer 24 to the difference calculation data and the motion compensation calculation data to the small capacity memory 25, and the display data is supplied to the terminal 11. (The transfer _FL M to the small-capacity memory 25 and the display data read R _I shown in DEV ₁ in FIG. 2).

Here, as the small-capacity memory 25, as described above, there are two memories having a capacity of 256 kbits (see FIG. 2).
A) and (B) shown in DEV ₁ of
Data corresponding to the calculation is stored, and reading and writing of each surface is switched for each GOB. In the DEV ₁ of FIG. 2 of the present embodiment, one side A of the small capacity memory 25 accesses the motion compensation circuit 26, and the next GOB data is input to the other side B of the small capacity memory 25. Shows the case. In addition, since 1 GOB + 27 macroblocks of data are required for motion compensation calculation of 1 GOB data, the frame memory 2 of the present embodiment is used.
From 3 to the small capacity memory 25, data of 2 macroblocks is transferred during the processing time TB of 1 macroblock.

That is, in this embodiment, D in FIG.
As shown in EV ₁ , the writing of data from the IDCT circuit 86 in the frame memory 23 is performed in the page mode, but the writing of data in the frame memory 23 (writing W _IDCT ) is performed in the page mode. However, for example, in the case of 30 Hz CIF, about 1 macroblock processing time TB is required (one access in page mode is 200 ns on average). Also, the DEV of FIG.
As shown in transfer F _LM and the display data read R _I to ₁ of a small capacity memory 25, the difference as calculation data from the frame memory 23, the macro blocks that are not yet rewritten in the frame memory 23 Data is read (for example, n at time t-1 of DEV ₁ in FIG. 2).
The + mth macroblock B _{n + m} and the n + pth macroblock B _{n + p} ), and the display data from the frame memory 23 is a macroblock of a new plane (for example, n at time t of DEV ₁ in FIG. 2). The data is sequentially read from the _−kth macro block B _nk ) and transferred to the small capacity memory 25.

Further, in the small-capacity memory 25, the data of two macroblocks for the difference calculation and the motion compensation calculation, which are continuously supplied from the frame memory 23, are written on the surface B, for example (FIG. 2). Writing to the other surface B of the small capacity memory 25 shown in DEV ₁ of _No. W _FM ). Further, from one surface A of the small capacity memory 25, data corresponding to the two macroblocks for the motion compensation calculation previously written from the frame memory 23 (see FIG. 2).
(N + mth and n + pth macroblocks B _{n + m} and B _{n + p} ) at time t−2 of DEV ₁ of the DEV ₁ (motion compensation from one side A of the small capacity memory 25 shown in DEV ₁ of FIG. 2). Read operation data R _MC ). However, in the case of motion compensation calculation, since only Y (luminance) data is required, 1
The macroblock becomes 256 bytes.

The Y (luminance) data for motion compensation calculation read from the small capacity memory 25 is used as the motion compensation circuit 2.
Sent to 6. The result of the motion compensation calculation in the motion compensation circuit 26 is sent to the controller 27. The controller 27 performs address control according to the result of the motion compensation calculation in the motion compensation circuit 26 so as to read the data whose motion amount has been shifted from the small capacity memory 25.
The data for one macroblock of Y and C, which is read out from the small capacity memory 25 and whose amount of movement is shifted, is sent to the loop filter 89. This loop filter 8
The output data of 9 is supplied to the adder 87 via the selector 90.

Therefore, the IDC is added by the adder 87.
By adding the output of the T circuit 86 and the data that has been subjected to the motion amount shift, difference calculation data sent to the subtractor 81 of FIG. 4 is obtained. The small capacity memory 25
The difference calculation data (read R _DC of the difference calculation data from one surface A of the small-capacity memory 25 shown in DEV ₁ in FIG. 2) read from the terminal 12 is transferred to the subtracter 81 via the terminal 12. Sent.

FIG. 3 shows a block circuit diagram of the differential encoding apparatus with motion compensation according to the embodiment of the present invention, excluding the function of motion compensation.

In FIG. 3, the data subjected to the inverse discrete cosine transform in the IDCT circuit 86 is added by the adder 87 and the decoded data is written in the frame memory 23 according to the inter / intra coding mode (see FIG. 2). DEV
Write W _IDCT from IDCT circuit 86 shown in ₃ ). Further, as the difference calculation data, the data is read from a portion of the frame memory 23 that has not been rewritten (reading the difference calculation data R shown in DEV ₃ in FIG. 2).
_DC ) and sent to the loop filter 89. The data passed through the loop filter 89 is output from the terminal 12 and sent to the subtracter 81 and the like shown in FIG.

In the configuration of FIG. 3, the difference calculation data and the display data (not shown) are read out from the frame memory 23 (display data read R _I shown in DEV ₃ in FIG. 2). As described above, since one data can be read from the serial port in one clock, the time margin is large, and therefore the small capacity memory shown in FIG. 1 is not required.

As described above, according to the motion compensation differential encoding apparatus of this embodiment, as described with reference to FIG. 1, the frame memory 23 for storing the data for one frame of the image.
And a small-capacity memory 25 that stores data for a predetermined period of the data of the previous frame read from the frame memory 23.
And a controller 27 for controlling writing / reading of the frame memory 23 and writing / reading of the small capacity memory 25, and an image in the predetermined period from the data of the current frame and the read data of the small capacity memory 25. A motion compensating circuit 26 for calculating data relating to motion, and reading the data of the small capacity memory 25 according to the output of the motion compensating circuit 26 to calculate the difference from the data of the current frame. By doing so, the frame memory 23 smaller than the conventional frame memory and the small-capacity memory 25 sufficiently smaller than the frame memory 23 can have the function of replacing the conventional frame memory. It has become possible to downsize the device configuration. Since the frame memory 23 and the small capacity memory 25 have different functions,
When integrated into an LSI, it can be configured without concentrating pins in one LSI, and even if the number of pins increases, it will be 2
Fewer pins than using face frame memory.

Further, in the case where the motion compensation function is not added as shown in FIG. 3, only the frame memory is required and the small capacity memory is not required. Therefore, the size of the structure can be further reduced and the cost is low. The number of pins in the case of an LSI can be further reduced, and the load on the memory control means can be reduced.

Further, in the case of an LSI, since the frame memories are the same with and without the motion compensation function, they can be realized on the same LSI, and the size of the structure can be reduced.

In this embodiment described above, 30 Hz CI
Although the case where the F and the clock CK are 14 MHz to 15 MHz is taken as an example, in addition to this, since only Y is required for the motion compensation calculation as described above, if Y / C is separately used, 128 k bits × 2 + 64 k It is also possible to use a small capacity memory of 1 bit.

Further, the number of small-capacity memories can be reduced by increasing the number of clocks in one macroblock processing time TB such as setting 30 Hz CIF to 15 Hz CIF or less or raising the clock CK to be changed to QCIF.

Further, in consideration of making into an LSI, a memory control means (controller 2) of a small capacity memory.
By bringing 7) to the motion compensation circuit 26 side, it becomes possible to disperse the number of pins of the LSI (the number of pins for memory is generally large), and the configuration is changed from one with motion compensation to one without motion compensation. Can be realized by omitting the LSI of the motion compensation circuit and the small-capacity memory. Therefore, the structure of the frame memory 1 without motion compensation can be easily realized.

[0045]

As described above, in the motion-compensation differential encoding apparatus of the present invention, the frame memory for storing the image frame data and the reading of the previous frame data written in the frame memory are performed. Memory control means for sequentially writing frame data, an auxiliary memory for accumulating data of a previous frame of data for a predetermined period, and data of the current frame and read data of the auxiliary memory are related to the movement of an image. And a motion compensation function by calculating the difference between the data of the auxiliary memory and the data of the current frame in accordance with the output of the calculation unit. , The configuration can be downsized,
Further cost reduction is possible.

[Brief description of drawings]

FIG. 1 is a block circuit diagram showing a schematic configuration of a differential encoding device with motion compensation according to an embodiment.

FIG. 2 is a timing chart showing the operation of each part of the apparatus of this embodiment.

FIG. 3 is a block circuit diagram showing a schematic configuration of a motion compensation-less differential encoding apparatus according to the present embodiment.

FIG. It is a block circuit diagram which shows the structure of the high efficiency encoding device corresponding to the H.261 standard.

[Explanation of symbols]

 23 ... Frame memory 24. Multiplexer 25 .. Small capacity memory 26 .. Motion compensation circuit 27 .. Controller 86 .. ..IDCT circuit 87..adder 89..loop filter 90..selector

Claims (1)

[Claims] 1. A frame memory for storing image frame data, and memory control means for controlling the reading of the previous frame data written in the frame memory and the sequential writing of the current frame data in the frame memory. And an auxiliary memory for accumulating data for a predetermined period of the data of the previous frame read from the frame memory, and data of the current frame and read data of the auxiliary memory are supplied to move the image in the predetermined period. And an arithmetic means for arithmetically operating data related to the data, the data of the auxiliary memory is read according to the output of the arithmetic means, and the difference from the data of the current frame is arithmetically operated and encoded. Differential encoding apparatus with motion compensation.