CN100481963C - Visual difference calculating method for video image in multi-vision point system - Google Patents

Abstract

This invention relates to a parallax computing method for video images in a multi-viewing point system, which utilizes a block matching method to compute the complete parallax of the first video image in the target video images takes the parallax value of the related image block in the above one target video image as the current position to carry out limit search and directional search to the reference video image, if the correspondent image block is searched, then the parallax is got, if not, the search is continued to the end, the parallax is got by fetching the image block with the smallest mean pixel gray difference in the target video images as the corresponding block.

Description

Disparity Calculation Method of Video Image in Multi-viewpoint System

technical field

The invention designs a parallax calculation method of video images in a multi-viewpoint system, and belongs to the technical field of video image processing.

Background technique

The block matching method used in multi-view video systems is the most commonly used disparity calculation method between two video images. best match. The classic search technique is a complete search within the search window, and its basic algorithm is as follows (set the block size to be N times N):

like $f(T\′)=\min (\▿f\left(T\right)),$ Then the parallax d=T',

in

$\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; T</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; N</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; N</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}<span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; l</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; r</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; |</span>$

$\mathrm{<span\; class="notranslate">\; T</span>}<span\; class="notranslate">\; =</span>\sqrt{{\mathrm{<span\; class="notranslate">\; m</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; no</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}$

T is the spatial displacement of the image block; T' is the spatial displacement when the correlation between the two corresponding image blocks in the target image and the reference image reaches the maximum; m and n are the abscissa and ordinate of the spatial displacement T respectively ,

f _l , f _r are the pixel gray values of two corresponding image blocks in the target image and the reference image.

The traditional block-matching disparity estimation algorithm is widely used because of its simple algorithm and small amount of calculation. However, it is computationally expensive to perform a full search on the entire target image.

Contents of the invention

The purpose of the present invention is to propose a disparity calculation method for video images in a multi-viewpoint system. On the basis of traditional block matching, an adaptive algorithm based on stereo camera geometry and video characteristics is proposed, which improves the efficiency of disparity calculation.

The parallax calculation method of the video image in the multi-viewpoint system that the present invention proposes, comprises the following steps:

(1) Utilize the block matching method to calculate the complete disparity of the first frame video image in the target video image;

(2) The disparity value of the corresponding image block in the above frame target video image is the current position, and the reference video image is limitedly searched with LW as the search radius;

(3) if searching for the image block corresponding to the target video image in the reference video image, then proceed to step (8), if not searching for the corresponding image block in the target video image, then proceed to step (4) ;

(4) The disparity value of the corresponding image block in the above frame target video image is the current position, and the direction search is carried out to the left for the reference video image, and the search range is LW+1～LW+DW1;

(5) if searching for the image block corresponding to the target video image in the reference video image, then proceed to step (8), if not searching for the corresponding image block in the target video image, then proceed to step (6) ;

(6) The parallax value of the corresponding image block in the above frame target video image is the current position, and the direction search is carried out to the right for the reference video image, and the search range is LW+1～LW+DW2;

(7) If in the reference video image, search for the image block corresponding to the target video image, then proceed to step (8), if not find the corresponding image block in the target video image, then get the image block corresponding to the target video image The image block with the smallest average pixel gray level difference is used as the corresponding image block, and then step (8);

(8) Obtain the disparity between the image block in the target video image and the corresponding image block in the reference video image according to the position of the image block in the target video image and the position of the searched reference image block.

The disparity calculation method of video images in the multi-viewpoint system proposed by the present invention has the advantage of improving the calculation efficiency of the algorithm, and only needs less calculation than the traditional block matching algorithm to obtain the same accuracy of the disparity calculation results .

Detailed ways

The method for calculating the disparity of video images in the multi-viewpoint system proposed by the present invention first utilizes the block matching method to calculate the complete disparity of the first frame video image in the target video image; the disparity value of the corresponding image block in the above frame of target video image is At the current position, with LW as the search radius, a limited search is performed on the reference video image; if the image block corresponding to the target video image is found in the reference video image, according to the position of the image block in the target video image and the searched Refer to the position of the image block to obtain the disparity between the image block in the target video image and the corresponding image block in the reference video image. If the image block corresponding to the target video image cannot be searched, then The disparity value of the corresponding image block is the current position, search the direction of the reference video image to the left, and the search range is LW+1～LW+DW1; if the image corresponding to the target video image is found in the reference video image block, then according to the position of the image block in the target video image and the position of the searched reference image block, the disparity between the image block in the target video image and the corresponding image block in the reference video image is obtained. The corresponding image block in the image, then the disparity value of the corresponding image block in the target video image of the previous frame is the current position, and the direction search is performed on the reference video image to the right, and the search range is LW+1～LW+DW2; if Search for the image block corresponding to the target video image in the reference video image, then according to the position of the image block in the target video image and the position of the searched reference image block, obtain the image block in the target video image and the reference video image The disparity between the corresponding image blocks in the target video image, if the corresponding image block in the target video image cannot be searched, the image block with the smallest average pixel gray level difference in the target video image is taken as the corresponding image block, according to the target video image The position of the image block and the searched position of the reference image block are used to obtain the disparity between the image block in the target video image and the corresponding image block in the reference video image.

In a multi-viewpoint camera system, directionality and uniformity are two properties related to parallax. If these geometric characteristics can be fully utilized in parallax calculation, the algorithm of parallax matching can be simplified and fast matching can be achieved:

1. Direction:

For the same scene, the left image of its perspective projection moves to the opposite direction of the horizontal direction locally relative to the right image. That is to say, if the positive horizontal direction is to the right, the projected position of the same spatial point in the left image is to the left of the right image. This feature provides a basis for the directional search of this algorithm, for example, when searching for the matching blocks of the blocks in the left image in the right image, you can search to the right first.

2. Uniformity:

The disparity difference between adjacent image sub-blocks on the polarization line is actually very small. Therefore, when disparity matching is performed, the disparity vector of the adjacent block is used as a reference to search for the disparity vector of the current block in a small range. The property that can quickly and accurately search out the current disparity is called limited search.

The purpose of disparity calculation is to apply to video encoding and decoding, so it can combine the characteristics of video, that is, the disparity correlation between adjacent video frames is very high, and further improve the efficiency of block matching; in addition, because the algorithm is applied to video, it can Use the search results of the previous video frame to modify the parameter value of this frame, so that the parameter itself can automatically adapt to the characteristics of the video.

For the first frame of the video, the traditional search algorithm is used to obtain its complete disparity; starting from the second frame, each frame is calculated based on the disparity result of the previous frame.

The search radii LW, DW1 and DW2 in the method of the present invention must be constantly adjusted according to the search results of the target image, so as to improve the search efficiency. The adjustment methods of the three search radius parameters are exactly the same, and the following uses LW as an example to illustrate:

An adjustment parameter ALW is set for each frame image, then the LW=(ALW of the nth frame+the ALW of the n-1 frame+the ALW of the n-2 frame)/3 of the n+1th frame image, wherein the nth frame The ALW of the frame depends on whether the search result of this frame in the LW range satisfies the matching criterion. If the matching criterion is satisfied, the ALW of the nth frame is the absolute value of the difference between the searched disparity value and the previous frame disparity value; If the matching criterion is not met, the ALW of the nth frame is the ALW of the n-1th frame plus 9.

The above-mentioned matching criterion may adopt the mean absolute difference criterion (MAD) or other similarity criterion. Here, a dispersion criterion is defined, and its performance is better than that of the general MAD criterion. Let f _l (i, j), f _r (i, j) be the pixel grayscale of the reference video image and the target video image block, the size of the block is N times N, and under the condition of satisfying the stereo parallel camera system, define the discrete The degree function is:

When B(d) is the minimum, the two image blocks are matched, and d at this time is the disparity between the two image blocks.

$\mathrm{<span\; class="notranslate">\; B</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; d</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; N</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; N</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; l</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; r</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; d</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}{\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; N</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; N</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; l</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; r</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; d</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}$

Claims (1)

1, the parallax calculation method of video image in a kind of many view system is characterized in that this method may further comprise the steps:

(1) utilize BMA to calculate the complete parallax of first frame video image in the target video image;

(2) parallax value with the respective image piece in the previous frame target video image is a current location, with LW is search radius, reference video image is searched for, wherein the initial value of LW is made as the search box size in the BMA when first frame, be defaulted as 32 pixels, the LW=of n+1 two field picture (ALW of ALW+ n-2 frame of ALW+ n-1 frame of n frame)/3, whether the result after wherein the ALW of n frame searches in the LW scope according to this frame satisfies matching criterior is decided, if satisfy matching criterior, then the ALW of n frame is the parallax value that searches and the absolute value of previous frame parallax value difference; If do not satisfy matching criterior, then the ALW of n frame is that the ALW of n-1 frame adds 9;

(3) if in reference video image, search with target video image in corresponding image block, then carry out step (8), if the search less than with target video image in corresponding image block, then carry out step (4);

(4) parallax value with the respective image piece in the previous frame target video image is a current location, left reference video image is searched for, and search radius is LW+1～LW+DW1, and wherein the obtaining value method of DW1 is identical with the obtaining value method of above-mentioned LW;

(5) if in reference video image, search with target video image in corresponding image block, then carry out step (8), if the search less than with target video image in corresponding image block, then carry out step (6);

(6) parallax value with the respective image piece in the previous frame target video image is a current location, to the right reference video image is searched for, and search radius is LW+1～LW+DW2, and wherein the obtaining value method of DW2 is identical with the obtaining value method of above-mentioned LW;

(7) if in reference video image, search with target video image in corresponding image block, then carry out step (8), if the search less than with target video image in corresponding image block, then get the image block of the image block of mean pixel gray scale difference minimum in the target video image, carry out step (8) as correspondence;

(8), obtain the parallax between the respective image piece in image block in the target video image and the reference video image according to the position of the position of image block in the target video image and the reference image block that searches.