CN102790884B - A kind of searching method based on hierarchical motion estimation and realize system - Google Patents

Abstract

The present invention discloses a search method based on layered motion estimation and its implementation system. The method performs a downsampling of the current frame and the reference frame according to the ratio. When the downsampling layer performs motion estimation, the step size of the candidate motion vector is set as 2 integer pixel unit vectors after downsampling. The system comprises a luminance reference pixel cache module, a layered and multiplexed PE array module, and a PE array register data update operation control module. The scheme proposed by the present invention changes the 16x16 blocks of the original image layer into 8x8 blocks only by one 4:1 downsampling, and the feature of block matching is more obvious; meanwhile, the search step size of the downsampling layer is set to 2 to reduce the calculation amount, satisfying Real-time performance of video encoders.

Description

A Search Method Based on Hierarchical Motion Estimation and Its Implementation System

technical field

The invention belongs to the field of hardware implementation of multimedia video encoders, in particular, it is a search method based on layered motion estimation and its implementation system, which is suitable for FPGA and ASIC implementation.

Background technique

In video encoders, inter-frame predictive coding with motion compensation is the main technology of video compression, and the size of search window, search strategy, and matching criteria directly affect the performance of the encoder. Among them, the whole pixel motion estimation is one of the most complicated modules in the video encoder. In order to meet the real-time performance of high-definition video encoding, the encoding can be realized through hardware acceleration. The hardware implementation of the video encoder needs to be able to achieve a good balance between complexity and performance.

The search methods based on block matching mainly include full search, fast search, and hierarchical search. The full search is a point-by-point search in the search area. Through the criterion function such as the SAD (Sum of Absolute Difference) value, that is, the absolute error sum of the initial value and the predicted value, the minimum SAD value is selected by calculating each candidate motion vector. best match block. When the image resolution is high and the motion is relatively complex, it is necessary to search within a large search window, and the amount of calculation is quite large. In order to realize real-time calculation, parallel processing must be adopted. In order to reduce the number of searches, a variety of fast search methods have been proposed, such as two-dimensional logarithmic search method, three-step search method, conjugate direction search method, orthogonal search method, diamond search method and so on. The common feature of the fast search method is that the direction that makes the criterion function (SAD) tends to the minimum is regarded as the direction of the minimum distortion, and it is assumed that the criterion function is monotonically increasing when it deviates from the direction of the minimum distortion, that is, it is considered to be in the entire search area is a single-pole function of the motion vector with a unique minimum. The fast search method can reduce most of the calculations and significantly increase the search speed, but the rate-distortion performance will decrease to a certain extent, which is not conducive to hardware implementation, mainly because the data flow control of the fast search method is irregular and cannot effectively utilize the on-chip storage. data.

The hierarchical search method proposed for this purpose can achieve the accuracy and performance close to the full search while reducing the amount of computation, and obtain the real motion displacement vector. In the hierarchical search method, the low-resolution representation of the image sequence is obtained through low-pass filtering and sub-sampling of the original image, and then a full search is performed on the obtained low-resolution image. Due to the reduction of the resolution, the number of searches is doubled, and the computing resources required for a single SAD calculation are also doubled, so that an optimal motion vector can be obtained by the full search. The motion vector is used as the starting point of searching in the original image, and the fine search is started. In this step, the size of the search window can be relatively reduced, and the number of searches can be correspondingly reduced. Finally, the estimated value of the motion displacement vector can be obtained through fine search. The advantage of using hierarchical search is that the number of searches is between full search and fast search, and it can be regarded as a full search when searching at each layer, which has a relatively regular data flow and control flow, which is conducive to hardware implementation. Existing hardware based on layered motion estimation searches through two downsamplings, which improves the search speed, but the downsampling layer obtained after two consecutive 4:1 downsamplings changes from 16x16 blocks of the original image layer to 4x4 blocks , so it is easy to find the pseudo-best matching block because the features are not obvious enough when matching blocks, which affects the search performance.

Contents of the invention

No matter whether the existing hardware-based video encoder adopts the full search method or the layered search method, the hardware area is sacrificed to a certain extent in exchange for speed. If the search range is increased, the hardware consumption will be huge, which is not conducive to FPGA or ASIC circuits. The purpose of this invention is to propose a layered search method with nearly full search performance and a corresponding hardware implementation system, so that it has higher search speed and better encoding performance, and meets the real-time encoding requirements of high-definition encoders.

In order to achieve the above object, the present invention adopts the following technical solutions:

The search method based on layered motion estimation in the present invention is specifically as follows: the current frame and the reference frame are down-sampled according to the ratio, and when the down-sampling layer performs motion estimation, the step size of the candidate motion vector is set as the down-sampled 2 integer pixel unit vectors.

Further, the down-sampling obtains 4 channels of down-sampled images, and the multi-PE array performs 4 channels of parallel search in each channel of down-sampled images, while the other 3 channels of down-sampled images are searched in parallel.

Further, in the method, the on-chip luminance pixel cache adopts classified pixel storage to realize down-sampling search, that is, 4 types of pixel types are interleavedly stored according to down-sampling, and are divided into odd-even rows and odd-even macroblock columns for separate storage, a total of 16 types of pixels Types are interleaved.

Further, the PE array reads data from the luminance reference buffer and multiplexes, and reads 16-byte wide data of two pixel types at a time for data update of the 4-way PE array.

The idea of layered search in the present invention is: first, the current frame and the reference frame are down-sampled according to the ratio, the L1 layer is the image obtained by the 4:1 down-sampling of the L0 layer, and the L0 layer is the original image layer. The search is first performed on the L1 layer, and the candidate MV search step is 2. When calculating the SAD, a 4:1 downsampling precision is used, that is, 64 pixels participate in the calculation of the SAD value. The scheme proposed by the present invention changes the 16x16 blocks of the original image layer into 8x8 blocks only by one 4:1 downsampling, and the feature of block matching is more obvious; meanwhile, the search step size of the downsampling layer is set to 2 to reduce the calculation amount, satisfying Real-time performance of video encoders.

In the present invention, four channels of downsampled images are obtained after downsampling, and parallel searches of four regions are realized by adopting different types of pixel points for the downsampled images of four regions. At the same time, when searching for each area at the L1 layer, a 4-way parallel search is further used to improve the search speed. Using rate-distortion optimization to support variable-size block motion estimation, the block matching cost function takes the weighted sum of the SAD value of the current block and the reference block and the coding bit consumption of the motion vector. The block size of the L1 layer is 8x8, and the optimal motion vector can be obtained by finding the minimum matching cost function. When searching at the L0 layer, the optimal motion vector of the L1 layer and the motion vector obtained by the motion prediction module of the video encoder are used as the starting point of the motion search, and a full search is performed in a small search window to obtain a variable size The block matching cost function value of the block, and the matching costs of various modes will be used for sub-pixel motion estimation and prediction mode selection of the encoder.

A system for realizing a search method based on layered motion estimation according to the present invention includes:

Luminance reference pixel cache module: Zigzag scanning sequence encoding and on-chip data interleaved storage are adopted to obtain interleaved storage of 4 pixel types according to down-sampling, and at the same time, the odd and even rows and odd and even macro block columns are stored separately, and a total of 16 pixel types are interleaved storage;

Hierarchically multiplexed PE array module: 4 downsampled images are obtained by downsampling, and the multi-PE array performs 4 parallel searches in each downsampled image, while the other 3 downsampled images are searched in parallel; the PE array can be Each clock gets valid data from the brightness reference pixel buffer module and calculates the SAD value;

PE array register data update operation control module: used to control the PE array to read reference pixels from the luminance reference pixel buffer module to the register, and read 16-byte wide data of two pixel types at a time for data update of 4-way PE array, The data update operation of the current brightness pixel register of the PE array realizes fast update by shifting different current pixel registers.

The luma reference pixel cache module is the core of hardware implementation. The luma cache meets the needs of PE array to read data through pixel interleaved storage, making it suitable for hardware implementation. The luminance cache must be easy to read and update data during PE array calculations; at the same time, it is easy to store off-chip data in the storage format of the luminance cache. The present invention adopts Zigzag scanning sequence coding and a caching mode of on-chip data interleaved storage. When the video encoder reads the original video sequence from the off-chip memory DDRSDRAM, it does not use the traditional raster order to encode macroblocks, but uses Zigzag macroblock sequential encoding, which can reduce the external memory access bandwidth by about 2/3. By dividing the reference pixels into four types of pixels (square, triangle, circle, and pentagon), divide them into odd and even rows, and store them separately in odd and even macroblock columns. A total of 16 on-chip RAMs store brightness reference pixels, and the bit width of each RAM is 64 bits. Each address line of RAM corresponds to 8 pixel point values.

The hierarchically multiplexed PE array module is a calculation module for motion estimation. The present invention designs the corresponding array distribution according to the motion estimation search method, and makes full use of each PE unit. It is assumed here that one PE unit will only be used for the absolute difference calculation of two 8-bit pixel values. The present invention uses 2*1024 PE units in total, of which 1024 PE units are used for forward and backward prediction. Calculating a 16x16 matching block requires 256 PEs, and at the same time there are 4 channels of 16x16 block SAD values to be calculated in parallel, and 16 channels of 8x8 block SAD values to be calculated in parallel. The PE array reads reference pixels from the luminance reference pixel module to the register, and finally the PE array completes the calculation work. It is necessary to ensure that the PE array can obtain valid data from the luminance cache at each clock and perform SAD value calculations to ensure the pipeline. Clock cycle constraints.

In the hierarchically multiplexed PE array module, the PE array is distributed in three levels: PEA_32pels array unit, PEA_64pels array unit, and PEA_256pels array unit. Among them, the PEA_32pels unit is the most basic array unit, consisting of 32 PE units, used to calculate the SAD value of 4x8 blocks; the PEA_64pels unit includes two PEA_32pels units, composed of 64 PE units, used to calculate the SAD value of 8x8 blocks; The PEA_256pels unit includes a four-way PEA_64pels unit, which consists of 256 PE units and is used to calculate the SAD value of 16x16 blocks and their variable size blocks.

The hardware system of the present invention mainly solves the data organization format of on-chip luminance pixel cache, adopts classified pixel storage to realize down-sampling search; designs data flow and control flow with moderate complexity, which makes hardware implementation simple; without adding additional storage Under the premise of space, increase the PE array to achieve high-throughput computing.

The present invention has the following advantages due to the adoption of the above technical scheme:

1. The number of downsampling layers is small, and the search accuracy is high;

2. Set the search step size of the candidate motion vector of the downsampling layer to 2 to improve the search speed;

3. Make full use of the on-chip luminance buffer for all readable pixel data at the same time, and process in parallel with multiple PE arrays;

4. On-chip RAM and Register fully consider data multiplexing to reduce bandwidth requirements;

5. Reduce the data interaction between the same PE array and different RAMs, and reduce the complexity of hardware implementation.

Description of drawings

Fig. 1 is a schematic diagram of the overall hardware architecture of hierarchical search;

Fig. 2 is a schematic diagram of two-layer pixel relationship of hierarchical search;

Figure 3 is a schematic diagram of two-layer motion estimation;

Fig. 4 is a schematic diagram of Zigzag macroblock encoding sequence;

Fig. 5 is the storage structure of on-chip brightness cache;

Fig. 6 is the structure of PEA unit;

Figure 7 is the composition of the unit structure of PEA_64pels and PEA_256pels;

Fig. 8 is the storage mode of the current pixel register of the PE array;

FIG. 9 is a schematic diagram of searching for one of the downsampled images in the downsampling layer.

detailed description

The embodiments of the present invention are described in detail below. Based on the premise of the technical solution of the present invention, the present embodiment provides detailed implementation and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

This embodiment provides a search method based on layered motion estimation that is suitable for FPGA and ASIC implementation, specifically: the current frame and the reference frame are down-sampled according to the ratio, and the L1 layer is an image obtained by 4:1 down-sampling of the L0 layer. The L0 layer is the original image layer. In this example, the search range of the L0 layer is [-160160)x[-104104) as an example, and the search range of the L1 layer becomes [-8080)x[-5252). The search is first performed on the L1 layer. The candidate MV search step is 2. There are a total of 160x104/4=4160 candidate MVs in the full search [-8080)x[-5252) interval. When calculating SAD, 4:1 is used Downsampling accuracy, that is, 64 pixels are involved in the SAD value calculation. The schematic diagram of the pixel relationship between the two layers is shown in Figure 2.

In the above method of this embodiment, 4 channels of down-sampled images are obtained by down-sampling, and the multi-PE array performs 4-channel parallel search in each channel of down-sampled images, and simultaneously searches the other 3 channels of down-sampled images in parallel.

As shown in Figure 2, (b) is a four-way downsampling image (square, triangle, circle, pentagon) obtained after (a) horizontal and vertical 2:1 downsampling. Different types of pixels to realize the parallel search of 4 regions. At the same time, when searching for each area at the L1 layer, a 4-way parallel search is further used to improve the search speed. As shown in Figure 3, the motion estimation based on rate-distortion optimization that supports variable-size blocks is used, and the block matching cost function takes the weighted sum of the SAD values of the current block and the reference block and the coding bit consumption of the motion vector. The block size of the L1 layer is 8x8, and the optimal motion vector can be obtained by finding the minimum matching cost function. When searching at the L0 layer, the optimal motion vector of the L1 layer and the motion vector obtained by the motion prediction module of the video encoder are used as the starting point of the motion search, and a full search is performed in a small search window to obtain a variable size The block matching cost function value of the block, and the matching costs of various modes will be used for sub-pixel motion estimation and prediction mode selection of the encoder.

As shown in Figure 1, the hardware system used to implement the above search method is composed of a number of different functional modules, mainly including a brightness reference pixel buffer module, a hierarchically multiplexed PE array module, and a PE array register data update operation control module.

The brightness reference pixel on-chip cache is shown in Figure 5, a circle in the figure represents a reference pixel point, and the characters in it represent the pixel type. The reference image is divided into 16 types of reference pixels from 0 to f, among which the reference pixels from 0 to 7 are the reference pixels of the even macroblock column, and the reference pixels from 8 to f are the reference pixels of the odd macroblock column. Each type of reference pixel corresponds to a corresponding type RAM, luma reference pixel buffer consists of 16 RAMs with a bit width of 64 bits. This embodiment adopts Zigzag macroblock scanning sequence encoding as shown in FIG. 4 , and only reads the data of one macroblock column each time the data is updated, and only needs to update 8 even (odd) macroblock column RAMs at a time.

The PEA_32pels structure is shown in Figure 6. This unit is the most basic array unit and calculates the SAD value of a 4x8 block; simultaneously generates four SAD values of a 2x4 block (upper left, upper right, lower left, and lower right units of a 4x8 block), which are used for SAD value calculation for variable size blocks. The PEA_32pels unit is composed as follows: 5 8-byte wide reference pixel shift registers, 4 8-byte wide shift registers for pixels to be encoded, 32 PE basic units, and 5 accumulators. As shown in Figure 7, 2 PEA_32pels units form a PEA_64pels unit, and 4 PEA_64pels units form a PEA_256pels unit. The storage method of the current macroblock pixel in the register is shown in Figure 8. When block matching motion estimation is performed at the L1 layer or L0 layer, the current macroblock pixel register of the PEA_32pels unit always stores one of the current pixel types from 0 to 7 pixels.

The steps of motion estimation at the L1 layer and the L0 layer are described in detail below. The following description is only for 1024 PE basic units, involving the processing of one frame. For a P frame, another 1024 PE units process another reference frame in parallel; for a B frame, another 1024 PE basic units process a reference frame in the opposite direction in parallel.

The L1 layer performs parallel motion estimation on 4 channels of downsampled images obtained by 4:1 downsampling of the original image, in which each channel of downsampled images performs 4 channels of parallel motion estimation on 8x8 blocks. After downsampling, four types of pixels i, (i+4), (i+8), (i+12) (i=0, 1, 2, 3) each constitute a downsampled image. In this example, it is agreed to adopt a two-dimensional coordinate system, the origin is the upper-left pixel of the current macroblock to be encoded, the positive direction of the x-axis is horizontally to the right, and the positive direction of the y-axis is vertically downward. Assume the candidate motion vector (x, y), where x and y are both negative, the downsampled image composed of i=0 is used for motion estimation; when x is non-negative and y is negative, the downsampled image composed of i=1 Image motion estimation; when x is negative and y is non-negative, the down-sampled image composed of i=2 is used for motion estimation; when x and y are both non-negative, the down-sampled image composed of i=3 is used for motion estimation. In the following, only one of the downsampled images will be analyzed, and i=0 is enough. PEA1_64pels, PEA2_64pels, PEA3_64pels, and PEA4_64pels are used for 4-way parallel motion estimation. As shown in FIG. 9 , PEA1_64pels, PEA2_64pels, PEA3_64pels, and PEA4_64pels are horizontally parallel processed, and the SAD value calculation of candidate motion vectors is searched in a serpentine scanning manner.

At T=cycle1, PEA1_64pels calculates the SAD value of the candidate motion vector (-160, -80); PEA2_64pels moves 4 integer pixel unit vectors in the positive direction relative to the motion vector x-axis of PEA1_64pels, and calculates (-156, -80) SAD value; similarly, PEA3_64pels and PEA4_64pels respectively calculate the SAD values of (-152, -80), (-148, -80) candidate motion vectors. The four PEA_64pels units move the calculated candidate motion vectors by 4 integer pixel unit vectors in the positive direction of the y-axis in the next cycle. PEA1_64pels consists of two PEA units, PEA1_64pels_PEA0 and PEA1_64pels_PEA1; PEA1_64pels is calculated in the next cycle (-160, -76), and the required data only needs to update 8bytes of data compared with the reference pixel data for calculation (-160, -80). Yes, the reference pixel register of PEA is 5x8bytes, then four 8bytes registers are used to participate in the operation, and the fifth 8bytes register is used to read in the 8byte data that needs to be updated, ensuring that the PE array can immediately participate in the operation at T=cycle2 . No update is required for the current pixel register. Do the same for PEA2_64pels, PEA3_64pels, and PEA4_64pels.

At T=cycle2, PEA1_64pels calculates the candidate motion vector (-160, -76); PEA2_64pels calculates the candidate motion vector (-156, -76); PEA3_64pels calculates the candidate motion vector (-152, -76); PEA4_64pels calculates the candidate motion vector ( -148, -76). Update the 8bytes data of the reference pixel register, and the current pixel register does not need to be updated.

Operate as above, continue to scan in the positive direction of the y-axis, then at T=cycle19, PEA1_64pels, PEA2_64pels, PEA3_64pels, PEA4_64pels respectively calculate candidate motion vectors (-160, 4), (-156, -4), (-152, -4 ), (-148, -4) SAD values.

At T=cycle20, if the motion vectors (-160,0), (-156,0), (-152,0), and (-148,0) that continue to move in the positive direction of the y-axis belong to another downsampling image The range of candidate motion vectors. Here, it is necessary to move 16 integer pixel unit vectors in the positive direction of the x-axis. Data cannot be multiplexed. The PE array needs to be empty until the required data is read into the reference pixel register. It takes 4 cycles to update 4 8-byte wide data at the same time, minus the operation time of the fifth 8-byte wide register updated in the previous cycle, a total of 3 cycles is required. There is no need to update the current pixel register.

At T=cycle23, PEA1_64pels, PEA2_64pels, PEA3_64pels, PEA4_64pels respectively calculate the SAD of candidate motion vectors (-144, -4), (-140, -4), (-136, -4), (-132, -4) value. In the next cycle, the candidate motion vectors calculated by each PEA_64pels array move 4 integer pixel unit vectors in the negative direction of the y-axis relative to the motion vector at T=23cycle time, so the reference pixel data of 8 bytes is updated in this cycle. The current pixel register needs to be updated.

At T=cycle24, PEA1_64pels, PEA2_64pels, PEA3_64pels, PEA4_64pels respectively calculate the SAD of candidate motion vectors (-144, -8), (-140, -8), (-136, -8), (-132, -8) value. At the same time, update the 8bytes data of the reference pixel register without updating the current pixel register.

Do the same as above, continue to scan in the negative direction of the y-axis, then at T=cycle41, PEA1_64pels, PEA2_64pels, PEA3_64pels, PEA4_64pels respectively calculate candidate motion vectors (-144, -80), (-140, -80), (-136, - 80), (-132, -80) SAD values. In the next cycle, it is necessary to calculate the SAD value of the candidate motion vector moving 16 integer pixel unit vectors in the positive direction of the x-axis. The data cannot be effectively multiplexed, and the PE array needs to be empty until the required data is read into the reference pixel register. There is no need to update the current pixel register. Same as above, need to wait for 3 cycles.

At T=cycle45, PEA1_64pels, PEA2_64pels, PEA3_64pels, PEA4_64pels respectively calculate the SAD of candidate motion vectors (-128, -8), (-124, -8), (-120, -8), (-116, -8) value. Update 8byte data at the same time, no need to update the current pixel register.

Same operation as above, until T=cycle217, PEA1_64pels, PEA2_64pels, PEA3_64pels, PEA4_64pels respectively calculate the SAD of candidate motion vectors (-16, 4), (-12, 4), (-8, 4), (4, -4) value. So far, the block matching motion estimation of the down-sampled image ends.

Perform the same operation for the other 3 downsampled images. According to the selection of the smallest block matching cost, 16 optimal candidate motion vectors can be obtained. Comparing the block matching costs of these 16 optimal candidate motion vectors, if the restriction condition is met: the search area of the vector as the search starting point at the L0 layer and the search area with the predicted motion vector as the search starting point do not overlap each other, Select the motion vector with the smallest matching cost as the last candidate motion vector of L1 layer.

So far, the motion estimation of the L1 layer is completed, and the related description of the motion estimation of the L0 layer will be described below.

In the L0 layer, the optimal motion vector and the predicted motion vector obtained from the L1 layer search are used as the starting point of the search, which is set to (cenx, ceny), and the motion estimation is performed in the search window of [-88)x[-66) to find the whole Optimal motion vector for pixel motion estimation. The calculation of each candidate motion vector at the L0 layer requires 256 PE units, that is, a PEA_256pels unit, which is composed of 4 PEA_64pels units, and 4 PEA_256pels units in the front and back directions, which are recorded as PEA1_256pels, PEA2_256pels, PEA3_256pels, PEA4_256pels. The L0 layer motion estimation will search four ways in parallel, which is the same as the L1 layer serpentine scanning principle.

First determine that due to the need for data reading, you need to expand the search to: [-8-(cenxmod4),12+(cenxmod4)) or [-12-(cenxmod4),8+(cenxmod4)), there is no substantial difference between the two , you can choose one of the two according to the actual situation. To simplify the subsequent description, here it is assumed that cenx=16, ceny=16, and the extended search window is [-8,12).

At T=cycle1, PEA1_256pels, PEA2_256pels, PEA3_256pels, PEA4_256pels respectively calculate the SAD of candidate motion vectors (-8, -6), (-7, -6), (-6, -6), (-5, -6) value. Among them, PEA1_256pels is composed of four PEA_64pels structural units, which are recorded as aPEA1_64pels, aPEA2_64pels, aPEA3_64pels, aPEA4_64pels, and assume that the current macroblock pixel types stored by them are (0, 4), (1, 5), (2, 6), (3, 7). The pixel coordinates of the upper left corner of the reference macroblock corresponding to the candidate motion vector (-8, -6) are (8, 10), and the pixel type of this point is 3. aPEA1_64pels, aPEA2_64pels, aPEA3_64pels, APEA4_64pels respectively calculate the SAD value of the 8x8 downsampling block with (8, 10), (9, 10), (8, 11), (9, 11) as the upper left corner, and accumulate to get the L0 layer SAD for variable size blocks. At the same time, during this period, each PE array needs to update the pixels of the register. Assume that aPEA1_64pels is composed of aPEA1_32pels and aPEA2_32pels, respectively storing the pixel types of the current macroblock to be encoded as (0, 4), and the corresponding types of the reference pixel registers as (3 and b, 7 and f). These two PEA_32pels arrays update the coded pixel type of the current macroblock as (2, 6), and the corresponding type of the reference pixel as (3 and b, 7 and f), and need to update 8-byte wide data. The other 3 PEA_64pels arrays of aPEA1_256pels and the other 3 PEA_64pels arrays of PEA_256pels do the same.

At T=cycle2, PEA1_256pels, PEA2_256pels, PEA3_256pels, PEA4_256pels respectively calculate the SAD of candidate motion vectors (-8, -5), (-7, -5), (-6, -5), (-5, -5) value. At the same time, during this period, each PE array needs to update the pixel value in the register.

Operate as above, continue to scan in the positive direction of the y-axis, then at T=cycle12, PEA1_256pels, PEA2_256pels, PEA3_256pels, PEA4_256pels respectively calculate candidate motion vectors (-8, 5), (-7, 5), (-6, 5), (-5, 5) Position SAD value. Here, it is necessary to move 4 integer pixel unit vectors in the positive direction of the x-axis, the data cannot be effectively multiplexed, and the PE array needs to be empty until the required data is read into the reference pixel register. A total of 3 cycles are required.

At T=cycle16, PEA1_256pels, PEA2_256pels, PEA3_256pels, PEA4_256pels respectively calculate the SAD values of candidate motion vectors (-4, 5), (-3, 5), (-2, 5), (-1, 5). At the same time, within this period, each PE array needs to update the pixel value of the register. The next clock cycle needs to move an integer pixel unit vector to the negative direction of the y-axis for motion estimation.

At T=cycle17, PEA1_256pels, PEA2_256pels, PEA3_256pels, PEA4_256pels respectively calculate the SAD values of candidate motion vectors (-4, 4), (-3, 4), (-2, 4), (-1, 4). At the same time, within this period, each PE array needs to update the pixel value of the register. The next cycle needs to move an integer pixel unit vector to the negative direction of the y-axis for motion estimation.

Same operation as above, at T=72cycle time, PEA1_256pels, PEA2_256pels, PEA3_256pels, PEA4_256pels respectively calculate the SAD values of candidate motion vectors (8, 5), (9, 5), (10, 5), (11, 5).

So far, the local fine search motion estimation of the L0 layer based on the search starting point is over. According to the selection of the smallest block matching cost, 4 optimal candidate motion vectors can be obtained, and then the block matching costs of these 4 optimal candidate motion vectors are calculated. By comparison, an optimal motion vector corresponding to the search starting point can be obtained.

Similarly, by performing the same operation on another search starting point, an optimal motion vector corresponding to the search starting point can be obtained. Finally, the cost of the respective optimal motion vectors corresponding to the two search starting points is compared, and the optimal motion vector for the whole pixel motion estimation can be obtained. At this point, the whole pixel estimation ends.

Although the content of the present invention has been described in detail through the above preferred embodiments, it should be understood that the above description should not be considered as limiting the present invention. Various modifications and alterations to the present invention will become apparent to those skilled in the art upon reading the foregoing disclosure. Therefore, the protection scope of the present invention should be defined by the appended claims.

Claims (3)

1. A search method based on layered motion estimation, characterized in that: the current frame and the reference frame are down-sampled in proportion, and by a 4:1 down-sampling, the 16x16 blocks of the original image layer become 8x8 blocks, When the downsampling layer performs motion estimation, the step size of the candidate motion vector is set to be 2 integer pixel unit vectors after downsampling; The down-sampling obtains 4-way down-sampled images, and the multi-PE array searches in one-way down-sampled images, while the other 3-way down-sampled images are searched in parallel; In the method, the on-chip luminance pixel cache adopts classified pixel storage to realize down-sampling search, that is, 4 types of pixel types are interleavedly stored according to down-sampling, and are divided into odd-even rows and odd-even macroblock columns for separate storage, and a total of 16 pixel types are interleavedly stored ; The PE array reads data multiplexing from the brightness pixel cache, and reads two types of pixel types of 16-byte wide data for data update of the 4-way PE array; the data update of the current brightness pixel register of the PE array The operation implements fast updates by applying shifts to different current brightness pixel registers; The on-chip luminance pixel cache divides the reference image into 16 types of reference pixels from 0 to f, wherein the reference pixels of types 0 to 7 are reference pixels of even macroblock columns, and the reference pixels of type 8 to f are reference pixels of odd macroblock columns. Each type of reference pixel corresponds to a corresponding type of RAM. The brightness reference pixel cache is composed of 16 RAMs with a bit width of 64 bits. It adopts Zigzag macroblock scanning sequence encoding, and only reads the data of one column of macroblock columns each time the data is updated. It only needs to update 8 even or odd macroblock column RAMs at a time; The PE array is distributed in three levels: PEA_32pels array unit, PEA_64pels array unit, PEA_256pels array unit, wherein: PEA_32pels array unit is the most basic array unit, which is composed of 32 PE units and is used to calculate the SAD value of 4x8 blocks; PEA_64pels The array unit includes two PEA_32pels array units, consisting of 64 PE units, used to calculate the SAD value of 8x8 blocks; the PEA_256pels array unit includes four PEA_64pels array units, composed of 256 PE units, used to calculate 16x16 blocks and Change the SAD value of the large block; The PEA_32pels array unit calculates the SAD value of the 4x8 block; at the same time, the SAD value of four 2x4 blocks is generated, that is, the upper left, upper right, lower left, and lower right units of the 4x8 block, which are used for the SAD value calculation of the variable size block; the PEA_32pels array unit constitutes As follows: 5 8-byte wide reference pixel shift registers, 4 8-byte wide shift registers for pixels to be encoded, 32 PE basic units, 5 accumulators; block matching is performed at the L1 or L0 layer During motion estimation, the current macroblock pixel register of the PEA_32pels array unit always stores one of the current pixel types from 0 to 7. 2. The search method based on layered motion estimation according to claim 1, characterized in that: said method adopts motion estimation based on rate-distortion optimization to support variable size blocks, and the block matching cost function takes the ratio of the current block and the reference block Weighted sum of SAD value, coded bit consumption of motion vector. 3. A realization system based on the search method of layered motion estimation as claimed in claim 1, characterized in that comprising: Luminance reference pixel cache module: Zigzag scanning sequence encoding and on-chip data interleaved storage are adopted to obtain interleaved storage of 4 pixel types according to down-sampling, and at the same time, the odd and even rows and odd and even macro block columns are stored separately, and a total of 16 pixel types are interleaved storage; Hierarchically multiplexed PE array module: downsampling to obtain 4 channels of downsampled images, and multi-PE arrays perform 4 channels of parallel search in each channel of downsampled images; the PE array can be obtained from the brightness reference pixel buffer module at each clock Valid data and calculate SAD value; PE array register data update operation control module: used to control the PE array to read reference pixels from the luminance reference pixel buffer module to the register, and read 16-byte wide data of two pixel types at a time for data update of 4-way PE array, The data update operation of the current brightness pixel register of the PE array realizes fast updating by shifting different current brightness pixel registers; The layered multiplexing PE array module has three levels of PE array distribution: PEA_32pels array unit, PEA_64pels array unit, PEA_256pels array unit, wherein: PEA_32pels array unit is the most basic array unit, consisting of 32 PE units, It is used to calculate the SAD value of a 4x8 block; the PEA_64pels array unit includes two PEA_32pels array units, which are composed of 64 PE units, and is used to calculate the SAD value of an 8x8 block; the PEA_256pels array unit includes four PEA_64pels array units, which consist of 256 PE units Composition, used to calculate the SAD value of 16x16 blocks and their variable size blocks; The brightness reference pixel cache module divides the reference pixels into 4 types of pixels, divides the odd and even rows, and stores the odd and even macro block columns separately, and a total of 16 on-chip RAMs store the brightness reference pixels, and the bit width of each RAM is 64 bits. One address line corresponds to 8 pixel values.