CN101888554B - VLSI (Very Large Scale Integration) structure design method for parallel flowing motion compensating filter - Google Patents

Abstract

The invention discloses a VLSI structure design method of a parallel pipeline motion compensation filter. The method of parallel calculation of luma pixel and chroma pixel of the motion compensation module is adopted, and the internal pipeline processing method of adaptive block is adopted, which reduces the processing time of the whole decoding and improves the system throughput. The layered architecture design of the on-chip memory is adopted, and the access method is further optimized to reduce the access to the off-chip memory. Using the data multiplexing technology of reference pixels and the update sub-macroblock processing technology can significantly save the time for reading data and reduce the bandwidth of data reading, thereby improving the decoding efficiency of the decoder and improving the decoding performance.

Description

Design Method of Parallel Pipeline Motion Compensation Filter VLSI Structure

technical field

The invention belongs to the field of video decoding, in particular to a VLSI structure design method of a parallel pipeline motion compensation filter.

Background technique

The H.264 video codec standard is currently the mainstream codec standard in the world. As a new generation of video coding standard, H.264 video coding standard has basically the same coding framework as previous video coding standards, but the coding efficiency has been greatly improved. The price is that the calculation process is more complicated and the calculation amount is larger, which increases the difficulty of hardware implementation of the video decoder. Therefore, when designing a video decoding chip, efforts should be made to improve decoding efficiency and reduce area and power consumption. However, in the field of video decoding, the power consumption and on-chip area of the device are always subject to various constraints. It is necessary to implement more complex decoding algorithms to improve efficiency, but also to reduce area and power consumption, so there are still many challenges. Motion compensation is a technology that uses reference images to restore the current image, which is used to reconstruct temporal redundant information, and then restore the entire video sequence. During the entire decoding process of H.264, motion compensation takes a lot of time. The characteristic of this module is that data reading is time-consuming and the amount of calculation is large. H.264 decoding motion compensation operation consumes the most time. Especially for high-definition and ultra-high-definition video decoding applications, designing efficient and real-time motion compensation filters has become a research hotspot.

Contents of the invention

In view of the above-mentioned deficiencies in the prior art, the purpose of the present invention is to provide a VLSI structure design method of a parallel pipeline motion compensation filter. Using parallel processing and data multiplexing calculation methods to perform decoding motion compensation, reduce the processing time of the entire decoding, thereby improving the decoding efficiency of the decoder and improving the decoding performance.

In order to realize above-mentioned task, the technical solution that the present invention adopts is:

Read the decoded motion vector and block division type control flow information of a macroblock from the off-chip memory, and store it in the on-chip memory;

According to the control flow information that has been stored in the on-chip memory, read the reference pixel data of the corresponding sub-block that needs motion compensation from the off-chip memory, and store it in the reference pixel data memory layer of the layered on-chip memory for motion compensation and motion compensation It is divided into two modules: luminance component and chroma component. The chroma component includes U component and V component. The luminance component adopts six-tap filter operation, and the chroma component adopts bilinear interpolation filter operation. The components are processed in parallel pipeline. After the brightness component reference pixel data is read from the off-chip memory, it directly enters the reading stage of the U component reference pixel data of the chroma, and the operation of the brightness component and the U component reference pixel data of the chroma are read. Start at the same time. After the U component of the chroma is calculated, the reference pixel of the V component of the chroma is read and calculated. After the calculation of the two components of the chroma is completed, the reference pixel data of the luminance component is read again. Read, after the motion compensation of a macro block is completed, the control flow information of the next macro block is read from the off-chip memory again for the motion compensation of the macro block.

The motion vector describes the result of the motion estimation operation of the current block. The more severe the motion, the larger the value of the motion vector, and the block division type represents the length and width of the current block.

The layered on-chip memory is divided into three layers, including a reference pixel data memory layer, an intermediate result memory layer, and a data multiplexing memory layer. The fetched luminance and chrominance refer to pixel data. In the process of luminance calculation, intermediate results are often needed to be saved. These data need to be stored on the intermediate result memory layer as the input of the next filtering operation. The sub-macroblock luminance and After the calculation of the chrominance component is completed, read the reference pixel data of the luma component again. It is very time-consuming to read and update the luma reference pixel data of the sub-macroblock from the off-chip memory, and the reference pixels of the adjacent sub-macroblocks have a large For a large degree of overlap, the update order should be arranged reasonably when updating the reference frame data, and the reference pixel data of the previous sub-macroblock existing in the on-chip memory is multiplexed and stored in the data multiplexing memory layer of the layered on-chip memory , when the motion compensation calculation of the current sub-macroblock is completed and the motion compensation calculation of the next sub-macroblock starts, a judgment is made first, and if data multiplexing can be realized, the data in the data multiplexing memory is first written into the reference pixel data memory, The remaining reference pixels are then read from off-chip memory.

The invention realizes parallel decoding of chroma and brightness, improves the efficiency and speed of parallel decoding, reduces access to off-chip memory, saves the time used for reading data, and reduces data bandwidth, so as to realize real-time high-definition video decoding and adapt to high-definition video application development.

Description of drawings

FIG. 1 is a schematic diagram of the architecture of the motion compensation module.

Fig. 2 is a schematic diagram of motion compensation internal pipeline calculation processing.

FIG. 3 is a schematic diagram of a three-layer memory structure of a luminance component.

Fig. 4 is a schematic diagram of sequential data multiplexing in update processing.

Figure 4(a) shows the first 8x8 block reference pixels.

Figure 4(b) shows the second 8x8 block reference pixels.

Figure 4(c) shows the common reference pixels of the first 8x8 block and the second 8x8 block.

Figure 4(d) shows the second 8x8 block reference pixels.

Figure 4(e) shows the 4th 8x8 block reference pixels.

Figure 4(f) shows the common reference pixels of the 2nd 8x8 block and the 4th 8x8 block.

Detailed ways

Figure 1 shows the overall architecture design of the motion compensation module. As can be seen in the figure, this module can be roughly divided into three parts. The following are introduced separately.

The first part is the module input and output interface. Among them, the input interface is an off-chip memory, from which information such as reference frame pixel data, motion vector and block division type is read. The output interface is to write the reconstructed current frame data into the external memory, which contains the values of two components of brightness and chrominance.

The second part is the hierarchical on-chip memory, which is used to store the required data read from the external memory. In order to reduce the read bandwidth and improve the data multiplexing rate, at the same time, in order to realize the self-adaptive variable size block processing, this paper adopts a three-layer on-chip buffer structure, including the reference pixel data memory layer, the intermediate result memory layer and data multiplexing storage layer. The first layer of on-chip memory is used as a reference pixel data memory layer to store the read-in reference frame data, the second layer of on-chip memory is used as an intermediate result memory layer to store intermediate results of filter operations, and the third layer of on-chip memory is used as a data multiplexing memory layer , storing part of the reference pixels so as to be rewritten into the first layer of on-chip memory to participate in the next operation.

The third part is the interpolation filter bank, which is specifically divided into two sub-modules of luma operation and chroma operation. Among them, the interpolation operation of the luminance part is realized by a 6-tap filter, and a plurality of 6-tap filters constitute a filter bank, and a double filter bank is further composed of a horizontal filter bank and a vertical filter bank to realize parallel operation. This design structure can implement pipeline processing at the sub-macroblock level, reducing the interpolation calculation time of the entire macroblock. The interpolation operation of the chroma part is realized by a bilinear interpolator, and a design structure of a filter bank composed of multiple filters is also adopted. During the entire operation process, the brightness and chrominance are processed in parallel, thereby reducing the entire decoding time and improving the operation efficiency. In the process of processing, regardless of brightness or chrominance, the adaptive variable block size pipeline is used for processing.

FIG. 2 is a schematic diagram of motion compensation flow calculation processing.

Video processing is divided into three components of YUV. According to the regulations in H.264, the Y component is calculated using a six-tap filter, and the UV component is calculated using bilinear interpolation. It can be seen from the figure that while the Y component is operating, the U component has already started to read the reference data. This structure saves computation time. In the previous serial operation structure, the data of the chrominance component is read for operation after the operation of the luminance component is completed.

FIG. 3 is a schematic diagram of the three-layer memory structure of the brightness component.

As can be seen from the figure, the entire processing process includes three layers of memory. Each layer is responsible for its own different functions.

First, read the data required for decoding the current macroblock from the off-chip memory, and store it in the first layer of reference pixel data memory layer, which is divided into two parts: luminance storage slice and chrominance storage slice, specifically including reference frame pixel data, Block type division, sub-macroblock motion vectors, etc. Next, refer to the data in the data memory as input, and perform operations in the filter bank. In the calculation process, it is often encountered that intermediate results need to be saved, and these data need to be stored on the intermediate result storage layer as the input of the next filtering operation. After the operation is completed, the reconstructed pixels are written to the external memory as the final result. In order to achieve data multiplexing, a third layer of on-chip memory-data multiplexing memory layer is added to store part of the reference frame data. These data may be used for the operation of the next sub-macroblock. When the operation of the current sub-macroblock is completed and the operation of the next sub-macroblock starts, a judgment is made first. If data multiplexing can be realized, the data of the third-level memory is first written into the first-level memory, thereby reducing the number of external storage operations. The time to read the data. The coordination between different levels of memory optimizes the performance of the entire module.

FIG. 4 is a schematic diagram of sequential data multiplexing in update processing.

Reading and updating data for motion compensation is time consuming. However, after analysis, it is found that the reference pixels of these sub-macroblocks overlap to a large extent. Therefore, when updating the reference frame data, the update sequence should be arranged reasonably, and the existing data in the on-chip memory should be reused as much as possible, thereby reducing the time for data reading.

Fig. 4(a), Fig. 4(b) and Fig. 4(c) show the reference pixel blocks required for processing the first two 8x8 sub-macroblocks, and the shared reference data area between them. When performing motion compensation on the second sub-macroblock, it is not necessary to read all the reference frame data, but save the multiplexed part (5x13 pixel block) of the reference data of the first sub-macroblock, and only update the data of the 8x13 block on the right . In this way, the amount of update data is reduced from 13x13 bytes to 8x13 bytes, which is reduced by nearly 40%, and the time required to read data is significantly shortened. Therefore, when the first 8x8 sub-block is processed, the data of the 5x13 pixel block on the right side will be stored in the on-chip memory of the third layer, which can be accessed when processing the next sub-block to update the reference data.

In order to further improve the data multiplexing rate, the processing order of the sub-macroblocks can be changed, and after the second sub-macroblock is processed, the fourth sub-macroblock is processed first. FIG. 4(d), FIG. 4(e), and FIG. 4(f) show the reference pixel blocks required by these two sub-macroblocks, and the reference data area shared between them. Since the second sub-macroblock and the fourth sub-macroblock are spatially adjacent, the number of shared pixels is large, and the size is still a 5x13 block. In this way, after the fourth sub-macroblock is processed, the third sub-macroblock is processed. At this time, the reference pixel for multiplexing is the 5×13 pixel block on the left side of the fourth sub-macroblock. Due to the adjustment of the processing order of the sub-macroblocks, not only the reading order of the reference pixels needs to be changed, but also the corresponding current sub-block motion vector and the like need to be adjusted.

The invention provides a VLSI structural design method of a parallel pipeline motion compensation filter, which can realize the parallel pipeline calculation of brightness pixels and chrominance pixels of a motion compensation module, and save overall calculation time. In addition, a hierarchical memory architecture design is proposed, the access method is further optimized, and the data reuse rate on the chip is improved, thereby significantly saving the time for reading data and reducing the data bandwidth.

In order to realize above-mentioned task, the technical solution that the present invention adopts is:

1) Through "parallel technology", realize parallel motion compensation decoding of luma component and chrominance component, improve decoding speed and efficiency;

2) Through the "intrinsic pipeline technology", the system throughput is improved without increasing the data bandwidth.

3) Through the "layered on-chip storage technology", the access to off-chip storage is reduced, and the three-layer on-chip memory structure is used to multiplex data.

4) Through "data multiplexing technology", reduce the time for the module to read reference pixels.

5) Through the "updating and processing sub-macroblock sequence technology", the maximum multiplexing of reference pixel data is realized.

The "parallel technique" mentioned is the time-parallel processing of luma and chrominance motion compensation calculations. Since there is no data dependency between the processing of the luma component and the processing of the chrominance component, the luma and chrominance can be processed in parallel. Under the condition of not increasing the data reading bandwidth, the preparation and calculation phase of the chrominance component data can be directly entered after the preparation of the luminance component data is completed. The calculation time of the whole macroblock is further shortened by simultaneous operation of luma and chrominance.

The "intrinsic pipeline technology" refers to the pipeline technology that the system adopts the arithmetic processing of adaptive blocks, and the pipeline operation is performed on the read data of brightness and chrominance respectively, which improves the throughput of the system, reduces the operation and decoding time, and improves Reusability of many function blocks. The process of processing each sub-block is divided into two parts: data reading and data operation. The operation adopts parallel processing, while the reading adopts pipeline processing. Read the reference pixels for the luma and chroma components separately, and read the reference pixels for the chroma when the luma component data is ready and the motion compensation operation is started, so as to realize pipeline without increasing the data bandwidth. For the luminance component, when the current pixel block is greater than or equal to 8x8, it is processed according to the 8x8 size block, and the operation needs to be iterated 4 times; when it is smaller than 8x8, it is processed according to the 4x4 block, and the operation needs to be iterated 16 times. For the chrominance component, when the current pixel block size is above 4x4, it is processed as a 4x4 block, and iteratively completes 4 times; when the block size is less than 4x4, it is processed as a 2x2 block, and iteratively completes 16 times.

The "hierarchical on-chip storage technology" means that if the reference pixels of a macroblock are all stored in the off-chip memory, it will impose a serious burden on the bandwidth of the off-chip memory. And the continuous access to the off-chip memory will generate huge power consumption. The first layer of on-chip memory of "layered on-chip memory technology" solves the problem of continuous access to off-chip. In order to increase the throughput of the system, the intermediate results that often need to be saved during the calculation process are stored in the second layer of memory as the input of the next filtering operation. In order to achieve data multiplexing, a third layer of on-chip memory is added to store part of the reference frame data. The coordination between different levels of memory optimizes the performance of the entire module.

The "data multiplexing technology" means that the first step in the motion compensation process is to read a large amount of reference frame data for the subsequent filter bank to perform calculations. Before each motion compensation, the reference frame data needs to be written into the on-chip memory. After the operation of each sub-macroblock is completed, the on-slice reference frame data is updated for calculation of the next sub-macroblock. Reading and updating data is time consuming. However, after analysis, it is found that the reference pixels of these sub-macroblocks overlap to a large extent. Therefore, when updating the reference frame data, the update sequence should be arranged reasonably, and the existing data in the on-chip memory should be reused as much as possible, thereby reducing the time for data reading.

The "updating and processing sub-macroblock order technology" is to reasonably arrange the sub-macroblock processing order, and more fully multiplex the pixel data of the reference block. The previous sub-macroblock processing is "Z"-shaped scanning processing, the second sub-macroblock and the third sub-macroblock are not adjacent in space, and reference pixels cannot be reused. The present invention updates the processing sequence, and after the second sub-macroblock is processed, the fourth sub-macroblock is processed first. Since the second sub-macroblock and the fourth sub-macroblock are spatially adjacent, shared pixel values can be multiplexed. In this way, in the most ideal case, the value of the existing reference pixel can be multiplexed three times, which greatly shortens the time for reading the reference pixel.

Claims (2)

1. VLSI (Very Large Scale Integration) structure design method for parallel flowing motion compensating filter, is characterized in that, specifically comprises the following steps:

Read motion vector and the piece classified types of a macro block of having decoded out from chip external memory and control stream information, be stored on the layering on-chip memory;

according to these control stream informations that are stored in the layering on-chip memory, read the needed reference pixel data of corresponding sub-block motion compensation from chip external memory, and leave the reference pixel data storage layer of layering on-chip memory in, carry out motion compensation, motion compensation is divided into luminance component and the large module of chromatic component two, chromatic component comprises U component and V component, luminance component adopts six tap filtering computings, chromatic component adopts the bilinear interpolation filtering operation, whole processing procedure is that the parallel pipelining process processing is carried out in brightness and chromatic component, luminance component reference pixel data from chip external memory read complete after, directly enter the fetch phase of the U component reference pixel data of colourity, the U component reference pixel data of the computing of luminance component and colourity read in synchronization and begin to carry out, after the U components operation of colourity is complete, the V component reference pixel data of and then carrying out colourity read and computing, after the computing of two components completing colourity, again carry out reading of luminance component reference pixel data, a macro block motion compensation is complete, again read the control stream information of next macro block from chip external memory, be used for this macro block motion compensation,

described layering on-chip memory is divided into three layers, comprise reference pixel data storage layer, scratch-pad storage layer and data-reusing memory layer, reference pixel data storage layer is divided into brightness and colourity memory feature, deposit the brightness and the reference chroma pixel data that read from chip external memory, often encounter the situation that needs to preserve intermediate object program in the brightness calculating process, these intermediate object programs are deposited on the scratch-pad storage layer, as the input variable of filtering operation next time, sub-block brightness and chromatic component computing are complete, again carry out reading of luminance component reference pixel data, the brightness reference pixel data of sub-block from chip external memory read with upgrade very consuming time, and that the reference pixel of adjacent sub-block has is overlapping significantly, should arrange update sequence when upgrading the reference pixel data, the reference pixel data of an existing upper sub-block in multiplexing layering on-chip memory, be stored on the data-reusing memory layer of layering on-chip memory, complete in current sub-block motion compensation computing, when the computing of next son block motion compensation begins, first judge, if can realize data-reusing, first the data of data-reusing memory layer are write in reference pixel data storage layer, read all the other reference pixel data from chip external memory again.

2. VLSI (Very Large Scale Integration) structure design method for parallel flowing motion compensating filter according to claim 1, it is characterized in that, described motion vector has been described the result of current block estimation computing, and motion Shaoxing opera is strong, motion vector numerical value is larger, and the piece classified types represents the length of current block and wide.

Images