CN103096113B - Method of generating stereo image array of discrete view collection combined window intercept algorithm - Google Patents

Abstract

The method for generating a stereo element image array combining discrete viewpoint acquisition with a window interception algorithm belongs to the technical field of stereo image generation. The present invention includes the following steps: collecting a discrete viewpoint image array; calculating the position of an object in each discrete viewpoint image; The horizontal relative displacement of any two horizontally adjacent discrete viewpoint images in the discrete viewpoint image array and the vertical relative displacement of the interception window in any two vertically adjacent discrete viewpoint images in the discrete viewpoint image array; calculate the interception window size; calculate the interception window The position of the lower right corner in the first row and first column of the discrete viewpoint image array; the discrete viewpoint image array is intercepted to generate a sub-image array; the sub-image array is converted into a volume element image array. The invention is not limited by acquisition equipment, and can generate high-resolution stereo element image arrays of actual scenes. Compared with the traditional camera array direct acquisition method, the invention can greatly reduce the shooting cost and workload.

Description

Stereo Element Image Array Generation Method Based on Discrete Viewpoint Acquisition and Window Clipping Algorithm

technical field

The invention belongs to the technical field of stereoscopic image generation, and in particular relates to a method for generating stereoscopic element image arrays in a combined stereoscopic image system.

Background technique

For a long time, the acquisition of visual information of the displayed world mainly comes from single-camera capture, and this acquisition form cannot bring the human eye a sense of depth, three-dimensionality, and all-round understanding of objects. Driven by the development of related disciplines and new demands for new technologies, stereoscopic display technology has emerged as the times require. Stereoscopic display technology mainly includes stereoscopic display technology utilizing binocular parallax and true stereoscopic display technology. The stereoscopic display technology using binocular parallax can be further divided into naked-eye display and display using auxiliary equipment. Among them, the naked-eye display mainly uses grating display, and the display using auxiliary equipment mainly uses 3D glasses for display. At present, stereoscopic display technology with binocular parallax is widely used in movie theaters. This technology is easy to implement and low in cost. However, because this method transmits the images seen by the left and right eyes to the audience separately, forcing the audience to have a stereoscopic impression in the brain, it is very difficult. It is easy to cause visual fatigue, and this technology cannot achieve continuous multi-view parallax changes, so it is not an ideal stereoscopic display method; real stereoscopic display technologies mainly include holography, volumetric display technology and combined stereoscopic display technology. True stereoscopic display technology can reproduce all the information of the object being photographed in space, and the audience can obtain a stereoscopic effect by physiologically adjusting the focal length of the eyes without visual fatigue. Therefore, it has become the development trend of stereoscopic display technology. Compared with holography and volumetric display technology with limited temporal and spatial resolution, combined stereoscopic display technology can enable the display to restore the spatial structure and positional relationship of the scene being photographed in a wide field of view, and the viewer can experience it without any auxiliary equipment. It is an important part of the new generation of stereoscopic display technology to feel the stereoscopic scene of full parallax.

The combined stereo imaging system mainly includes two parts: acquisition and display. During the acquisition process, as shown in Figure 1, when the light emitted by the actual object is recorded on the recording medium through the lens array, we get a stereo element image array , each stereo element image in the array respectively records the image information of different positions and angles of the object to be photographed. During the display process, as shown in Figure 2, the three-dimensional element image array is presented in front of the lens array through a high-resolution flat panel display, and the lens array converges the light emitted from the three-dimensional element image array into a real three-dimensional scene in space.

The lens array acquisition method is the simplest and direct method to obtain the stereoscopic element image array. This method uses the lens array and a single recording medium to directly capture the stereoscopic element image array of the 3D object. However, in practical applications, this method has many shortcomings. , for example, lower display resolution, narrow viewing angle, and smaller depth of field. In response to the above problems, many improved acquisition methods have been proposed. J.-S Jang and B.Javidi et al. proposed the MALT method based on time division multiplexing, which obtains more stereoscopic element images by increasing the spatial sampling rate. Therefore, the resolution of the reconstructed image is effectively improved. At the same time, the two propose the SAII method, which can not only improve the display resolution, but also increase the viewing field of view. Although the above method improves the display effect of the combined stereoscopic imaging system to varying degrees on the basis of the lens array acquisition method, for a given image recording medium, there is always an irreconcilable relationship between the resolution and the number of stereoscopic element images. contradiction. Using a camera array for acquisition can well resolve this contradiction, and each camera in the array records a stereoscopic element image of the shooting scene respectively. However, as the display resolution continues to increase, the scale of the camera array is also increasing. If we want to obtain a voxel image array with a display resolution of 1024×768 and each voxel contains 20×20 pixels , you need 1024×768 cameras. Obviously, such a large-scale camera array is expensive and inconvenient to adjust. In view of the above shortcomings of the lens array acquisition method and the camera array acquisition method, we need to find a more effective and more suitable method for synthesizing the stereo element image array of the combined stereo image system. This method can generate high-resolution stereo Meta-image arrays, at the same time, do not require high experimental costs and complex conditioning efforts.

Contents of the invention

The object of the present invention is to provide a method for generating a stereoscopic element image array combining discrete viewpoint acquisition with a window interception algorithm, so as to realize the stereoscopic display of real scenes.

The present invention comprises the following steps:

1. Collect discrete viewpoint image arrays, comprising the following steps:

1.1 Initialization: Adjust the two tripod brackets supporting the stereo photography track to fix the height of the stereo photography track to a lower position, install the camera on the stereo photography track, the stereo photography track can make the camera move at a constant speed from right to left, experiment The personnel control the start and stop of the stereo photography track through the remote control;

1.2 Acquire discrete viewpoint image groups: use the level ruler to adjust the plane of the stereographic track to be parallel to the horizontal plane, and use the remote control to move the camera to the rightmost end of the stereographic track, hold down the camera shutter line and start the stereographic track at the same time, During the moving process, use the continuous shooting function of the camera to continuously collect multiple discrete viewpoint images of the subject, and arrange the collected discrete viewpoint images in a row from left to right in the order of shooting, that is, the stereo photography track is located at this height The discrete viewpoint image group collected at time;

1.3 Use the vertical ruler to raise the two tripod brackets supporting the stereoscopic photography track to raise the stereoscopic photography track by a fixed distance;

1.4 Repeat the shooting process of steps 1.1.2 and 1.1.3 to obtain multiple discrete viewpoint image groups;

1.5 Arrange all the discrete viewpoint image groups from top to bottom according to the order of collection, that is, put the first group of discrete viewpoint image groups collected first in the first row of the discrete viewpoint image array, The last group of discrete viewpoint image groups collected is placed in the last row of the discrete viewpoint image array;

2. Calculate the position of the subject in each discrete viewpoint image:

The position parameters of the object in each discrete viewpoint image include: the horizontal relative displacement of the object in any two horizontally adjacent discrete viewpoint images in the discrete viewpoint image array, The vertical relative displacement in the adjacent discrete viewpoint image, the position of the upper, lower, left and right boundaries of the object in the first row and the first column of the discrete viewpoint image array, the upper, lower, left and right positions of the object and the position of the right boundary in the last row and last column of discrete viewpoint images of the discrete viewpoint image array;

2.1 Determine the horizontal relative displacement of the shooting object in any two horizontally adjacent discrete viewpoint images in the discrete viewpoint image array _. For a discrete viewpoint image, first, DVI _1,1 is shifted pixel by pixel to the right, after each shift, calculate the peak signal-to-noise ratio of the overlapped part of DVI _1,1 and DVI _2,1 after shifting, peak signal-to-noise ratio defined as:

$\mathrm{<span\; class="notranslate">\; PSNR</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; the\; s</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 10</span>}<span\; class="notranslate">\; \&times;</span>{\mathrm{<span\; class="notranslate">\; log</span>}}_{\mathrm{<span\; class="notranslate">\; 10</span>}}<span\; class="notranslate">\; [</span>\frac{{\mathrm{<span\; class="notranslate">\; 255</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}{\mathrm{<span\; class="notranslate">\; MSE</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; the\; s</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; ]</span>$

In the formula, s-is the translation distance of DVI _1,1 , PSNR(s)-is the peak signal-to-noise ratio of the overlapping part of DVI _1,1 and DVI _2,1 after translation, MSE(s)-is the mean square error, Its definition is:

$\mathrm{<span\; class="notranslate">\; MSE</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; the\; s</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; the\; s</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; Y</span>}}\underset{\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; the\; s</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; Y</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{<span\; class="notranslate">\; [</span>{\mathrm{<span\; class="notranslate">\; DVI</span>}}_{\mathrm{<span\; class="notranslate">\; 1,1</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; DVI</span>}}_{\mathrm{<span\; class="notranslate">\; 2,1</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; the\; s</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ]</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}$

In the formula, x, y-are the horizontal and vertical position coordinates of pixels in DVI _1,1, respectively, and X, Y-are the number of pixels contained in the horizontal and vertical directions of the discrete viewpoint image, respectively;

Then, the moving distance corresponding to the maximum peak signal-to-noise ratio is taken as the horizontal relative displacement of the subject in any two horizontally adjacent discrete viewpoint images in the discrete viewpoint image array;

2.2 Determine the vertical relative displacement of the subject in any two vertically adjacent discrete viewpoint images in the discrete viewpoint image array. The calculation method is: first transpose DVI _1,1 and DVI _1,2 , and then follow the Calculate the horizontal relative displacement in any two horizontally adjacent discrete viewpoint images in the discrete viewpoint image array using the same calculation method;

2.3 Determine the position of the upper, lower, left and right boundaries of the subject in DVI _1,1 , the calculation method is: first, calculate the difference image of DVI _1,1 and DVI _2,1 and perform median filtering; Then, the positions of the upper, lower and left boundaries of all non-zero points in the difference image after median filtering are used as the positions of the upper, lower and left boundaries of the subject in DVI _{1, 1} , and the median filtered The value obtained after subtracting the horizontal relative displacement of the subject in any two horizontally adjacent discrete viewpoint images in the discrete viewpoint image array from the position of the right boundary of all non-zero points in the difference image is taken as the right boundary of the subject in DVI _{1 , the position in 1} ;

2.4 Determine the position of the upper, lower, left and right boundaries of the shooting object in the last row and last column of discrete viewpoint images in the discrete viewpoint image array, the calculation method is: first, calculate the last row and last column in the discrete viewpoint image array The difference image of the discrete viewpoint image in the discrete viewpoint image and the discrete viewpoint image in the last row and the penultimate column in the discrete viewpoint image array and perform median filtering; and the position of the right boundary are taken as the positions of the upper, lower and right boundaries of the subject in the last row and last column of discrete viewpoint images of the discrete viewpoint image array, and the left borders of all non-zero points in the median-filtered difference image The position of the subject plus the horizontal relative displacement of the subject in any two horizontally adjacent discrete viewpoint images in the discrete viewpoint image array is the value obtained as the left boundary of the subject in the last row and last column of the discrete viewpoint image array Discrete viewpoint position in the image;

3. Calculate the horizontal relative displacement of the interception window in any two horizontally adjacent discrete viewpoint images in the discrete viewpoint image array and the vertical relative displacement of the interception window in any two vertically adjacent discrete viewpoint images in the discrete viewpoint image array:

The expressions of the horizontal relative displacement of the interception window in any two horizontally adjacent discrete viewpoint images in the discrete viewpoint image array and the vertical relative displacement of the interception window in any two vertically adjacent discrete viewpoint images in the discrete viewpoint image array are:

MH=DH+delta

MV=DV+delta

In the formula, MH and MV- are the horizontal relative displacement of the interception window in any two horizontally adjacent discrete viewpoint images in the discrete viewpoint image array and the interception window in any two vertically adjacent discrete viewpoint images in the discrete viewpoint image array, respectively. , DH, DV- are the horizontal relative displacement of the subject in any two horizontally adjacent discrete viewpoint images in the discrete viewpoint image array and the horizontal relative displacement of the subject in any two vertically adjacent discrete viewpoint images in the discrete viewpoint image array The vertical relative displacement in the image, delta- is the depth factor;

According to actual needs, delta can take any value within the allowable range, and the value range of delta is:

delta_max=min[(MH_max-DH), (MV_max-DV)]

delta_min=0

In the formula, delta_max, delta_min- are the maximum value and minimum value of delta respectively, MH_max, MV_max- are the maximum value and The maximum value of the vertical relative displacement of the interception window in any two vertically adjacent discrete viewpoint images in the discrete viewpoint image array, min(.)-indicates the minimum value of all the values in the brackets;

The expressions of MH_max and MV_max are:

MH_max=min(X-IR,IL)/(M-1)

MV_max=min(Y-IB,IT)/(N-1)

In the formula, IR, IB- are the positions of the right and lower borders of the subject in DVI _1,1 respectively, IL and IT- are the left and upper borders of the subject respectively in the last row of the discrete viewpoint image array , the position in the last column of discrete viewpoint images, M, N-respectively the number of discrete viewpoint images contained in each row and each column in the discrete viewpoint image array;

4. Calculate the size of the interception window:

The expression for the size of the interception window is:

W=IR+(M-1)×MH-IL

H=IB+(N-1)×MV-IT

In the formula, W and H-respectively represent the width and height of the interception window;

5. Calculate the position of the lower right corner of the interception window in the discrete viewpoint image in the first row and first column of the discrete viewpoint image array:

The expression of the position of the lower right corner of the interception window in DVI _1,1 is:

PH=IR

PV=IB

In the formula, PH and PV- are respectively the horizontal and vertical position coordinates of the lower right corner of the interception window in DVI _1,1 ;

6. Intercept the discrete viewpoint image array to generate a sub-image array:

Use the interception window to intercept each discrete viewpoint image in the discrete viewpoint image array to generate a subimage array, the number of subimages contained in the subimage array is equal to the number of discrete viewpoint images contained in the discrete viewpoint image array, and they are located in the subimage array The expression of a sub-image in column i and row j is:

SI _i,j (u,v)=DVI _i,j (PH+(i-1)×MH-W+u, PV+(j-1)×MV-H+v)

In the formula, SI _{i, j} - is a sub-image in the i-th column and j-th row in the sub-image array, u=1, 2,..., W and v=1, 2,..., H-respectively pixel points Horizontal and vertical position coordinates in the sub-image;

7. Convert the array of sub-images into an array of volume element images:

The stereoscopic element image array converted from the sub-image array contains W×H stereoscopic element images, the size of each stereo element image is M pixels×N pixels, and it is located in the p-th column and q-th row of the stereo element image array. The expression of the stereo element image is:

EIp,q(r,t)=SIr,t(p,q)

In the formula, EI _{p, q} - is a stereo element image of the pth row and the qth row in the stereo element image array, r=1, 2,..., M and t=1, 2,..., N-respectively The horizontal and vertical position coordinates of pixels in the stereo image.

The invention aims at the acquisition principle of the stereo element image array in the combined stereo imaging system, and obtains the discrete viewpoint image array by using the acquisition method combining a single camera and a stereo photography track. Aiming at the imaging relationship between the discrete viewpoint image array and the sub-image array and the mapping relationship between the sub-image array and the stereo element image array, the present invention proposes a method for generating a stereo element image array that performs window interception on the discrete viewpoint image array , which can achieve the purpose of generating a high-resolution stereo element image array, and at the same time, does not require expensive acquisition equipment and heavy workload.

The invention can generate a stereoscopic primary image array of a relatively large shooting object, and the generated stereoscopic primary image array has a continuous viewing angle during a display process, and can truly reproduce the structural information of the shooting object. Compared with the lens array acquisition method, the present invention is not limited by the acquisition equipment, and can generate a high-resolution stereo elemental image array. Compared with the camera array acquisition method, the invention only uses one camera, which greatly reduces the shooting cost and adjustment workload.

Description of drawings

Figure 1 is a schematic diagram of the stereo element image array acquisition process of the combined stereo imaging system

Figure 2 is a schematic diagram of the stereo element image array display process of the combined stereo imaging system

Of which: 1. Actual object 2. Light 3. Lens array 4. Recording medium 5. Stereoscopic element image 6. Stereoscopic element image array 7. Lighting 8. Flat panel display 9. Stereoscopic image

Fig. 3 is a flow chart of the stereo element image array generation method combined with discrete viewpoint acquisition and window interception algorithm

Figure 4 is a schematic diagram of the discrete viewpoint image array acquisition platform

Figure 5 is a schematic diagram of a discrete viewpoint image array

Fig. 6 is the nine discrete viewpoint images of columns 1, 12 and 24 in rows 1, 12 and 24 of the enlarged discrete viewpoint image array

Fig. 7 is a flow chart of the calculation method of the position of the upper, lower, left and right boundaries of the object in DVI _1,1

Figure 8 is a schematic diagram of the sub-image array generated by the window truncation algorithm

Detailed ways

The present invention will be further described in detail below in conjunction with the accompanying drawings. The specific process (as shown in Figure 3) of the stereo element image array generation method (as shown in Figure 3) comprising the following steps:

1. Acquisition of discrete viewpoint image arrays

As shown in Figure 4, the acquisition platform of the discrete viewpoint image array is taken. The objects to be photographed are two toy trucks. The front truck is closer to the camera than the rear truck, and the front truck blocks the rear truck. The discrete viewpoint The acquisition process of the image array includes the following steps:

The first step: initialization. Adjust the two tripod brackets supporting the stereo photography track to fix the height of the stereo photography track to a lower position, install the camera on the stereo photography track, the stereo photography track can make the camera move from right to left at a constant speed, and the experimenter can control it by remote control The controller controls the start and stop of the stereographic track.

Step 2: Acquire discrete viewpoint image groups: Use the level ruler to adjust the plane of the stereographic track to be parallel to the horizontal plane, and use the remote control to move the camera to the rightmost end of the stereographic track, press and hold the camera shutter line and start the stereographic track at the same time , during the movement of the camera, use the continuous shooting function of the camera to continuously collect multiple discrete viewpoint images of the subject, and arrange the collected discrete viewpoint images in a row from left to right according to the sequence of shooting, that is, the stereoscopic photography track A set of discrete viewpoint images collected at that altitude;

Step 3: Utilize the vertical ruler to raise the two tripods supporting the stereoscopic photography track to raise the stereoscopic photography track by a fixed distance.

Step 4: Repeat the shooting process of Step 2 and Step 3 to obtain multiple discrete viewpoint image groups.

Step 5: Arrange all discrete viewpoint image groups from top to bottom according to the order of collection, that is, put the first group of discrete viewpoint image groups collected first in the discrete viewpoint image array. One row, the last collected set of discrete viewpoint image groups is placed in the last row of the discrete viewpoint image array. The generated discrete viewpoint image array is shown in Figure 5. For the convenience of observation, the nine discrete viewpoint images in the 1st, 12th and 24th rows of the discrete viewpoint image array are enlarged respectively in Figure 6. It can be seen from Fig. 6 that as the viewing angle moves, the discrete viewpoint image array presents a change process of the red truck from visible to invisible.

2. Calculate the position of the subject in each discrete view image

The position parameters of the object in each discrete viewpoint image include: the horizontal relative displacement of the object in any two horizontally adjacent discrete viewpoint images in the discrete viewpoint image array, The vertical relative displacement in the adjacent discrete viewpoint image, the position of the upper, lower, left and right boundaries of the object in the first row and the first column of the discrete viewpoint image array, the upper, lower, left and right positions of the object and the position of the right boundary in the last row, last column of discrete view images of the discrete view image array.

Let DVI _i,j denote a discrete viewpoint image located in column i and row j in the discrete viewpoint image array. The calculation method of the horizontal relative displacement of the subject in any two horizontally adjacent discrete viewpoint images in the discrete viewpoint image array is as follows: firstly, the DVI _1,1 is shifted to the right pixel by pixel, and after each shift, calculate the shifted The peak signal-to-noise ratio of the overlapping part of DVI _1,1 and DVI _2,1 , the peak signal-to-noise ratio is defined as:

$\mathrm{<span\; class="notranslate">\; PSNR</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; the\; s</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 10</span>}<span\; class="notranslate">\; \&times;</span>{\mathrm{<span\; class="notranslate">\; log</span>}}_{\mathrm{<span\; class="notranslate">\; 10</span>}}<span\; class="notranslate">\; [</span>\frac{{\mathrm{<span\; class="notranslate">\; 255</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}{\mathrm{<span\; class="notranslate">\; MSE</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; the\; s</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; ]</span>$

In the formula, s-is the translation distance of DVI _1,1 , PSNR(s)-is the peak signal-to-noise ratio of the overlapping part of DVI _1,1 -and DVI _2,1 after translation, MSE(s)-is the mean square error , whose definition is:

$\mathrm{<span\; class="notranslate">\; MSE</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; the\; s</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; the\; s</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; Y</span>}}\underset{\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; the\; s</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; Y</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{<span\; class="notranslate">\; [</span>{\mathrm{<span\; class="notranslate">\; DVI</span>}}_{\mathrm{<span\; class="notranslate">\; 1,1</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; DVI</span>}}_{\mathrm{<span\; class="notranslate">\; 2,1</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; the\; s</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ]</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}$

In the formula, x, y-are the horizontal and vertical position coordinates of pixels in DVI _{1, 1,} respectively, and X, Y-are the horizontal and vertical resolutions of the discrete viewpoint image, respectively.

Then, the moving distance corresponding to the peak signal-to-noise ratio is taken as the horizontal relative displacement of the photographed object in any two horizontally adjacent discrete viewpoint images in the discrete viewpoint image array.

The calculation method of the vertical relative displacement of the subject in any two vertically adjacent discrete viewpoint images in the discrete viewpoint image array is: first transpose DVI _{1, 1} and DVI _{1, 2} , and then follow the The horizontal relative displacement in any two horizontally adjacent discrete viewpoint images in the image array is calculated by the same calculation method.

The calculation method of the position of the upper, lower, left and right boundaries of the subject in DVI _1,1 is (as shown in Figure 7): first, calculate the difference image of DVI _1,1 and DVI _2,1 and perform Median filtering; then, with the position of the upper, lower and left borders of all non-zero points in the difference image after median filtering as the position of the upper, lower and left borders of the subject in DVI _{1, 1} , the middle The value obtained after subtracting the horizontal relative displacement of the subject in any two horizontally adjacent discrete viewpoint images in the discrete viewpoint image array from the position of the right boundary of all non-zero points in the value-filtered difference image is taken as the right side of the subject The position of the border in DVI _1,1 .

The calculation method for the position of the upper, lower, left and right sides of the object in the last row and last column of discrete viewpoint images in the discrete viewpoint image array is as follows: first, calculate the last row and last column of discrete viewpoint images in the discrete viewpoint image array and the difference image of the discrete viewpoint image in the last row and the penultimate column in the discrete viewpoint image array and perform median filtering; then, the upper, lower and right boundaries of all non-zero points in the median filtered difference image The position is taken as the position of the upper, lower and right boundaries of the shooting object in the last row and last column of discrete viewpoint images of the discrete viewpoint image array, and the positions of the left borders of all non-zero points in the median-filtered difference image are added to The value obtained after the horizontal relative displacement of the subject in any two horizontally adjacent discrete viewpoint images in the discrete viewpoint image array is taken as the position of the left boundary of the subject in the last row and last column of discrete viewpoint images of the discrete viewpoint image array .

3. Calculate the horizontal relative displacement of the interception window in any two horizontally adjacent discrete viewpoint images in the discrete viewpoint image array and the vertical relative displacement of the interception window in any two vertically adjacent discrete viewpoint images in the discrete viewpoint image array, where The expression is:

MH=DH+delta

MV=DV+delta

In the formula, MH and MV- are the horizontal relative displacement of the interception window in any two horizontally adjacent discrete viewpoint images in the discrete viewpoint image array and the interception window in any two vertically adjacent discrete viewpoint images in the discrete viewpoint image array, respectively. , DH, DV- are the horizontal relative displacement of the subject in any two horizontally adjacent discrete viewpoint images in the discrete viewpoint image array and the horizontal relative displacement of the subject in any two vertically adjacent discrete viewpoint images in the discrete viewpoint image array The vertical relative displacement in the image, delta- is the depth factor.

According to actual needs, delta can take any value within the allowable range, and the value range of delta is:

delta_max=min[(MH_max-DH), (MV_max-DV)]

delta_min=0

In the formula, delta_max, delta_min- are the maximum value and minimum value of delta respectively, MH_max, MV_max- are the maximum value and The maximum value of the vertical relative displacement of the interception window in any two vertically adjacent discrete viewpoint images in the discrete viewpoint image array, min(·)-means taking the minimum value of all the values in the brackets.

The expressions of MH_max and MV_max are:

MH_max=min(X-IR,IL)/(M-1)

MV_max=min(Y-IB,IT)/(N-1)

In the formula, IR, IB- are the positions of the right and lower borders of the subject in DVI _1,1 respectively, IL and IT- are the left and upper borders of the subject respectively in the last row of the discrete viewpoint image array , the position in the last column of discrete viewpoint images, M, N—respectively the number of discrete viewpoint images contained in each row and column of the discrete viewpoint image array.

4. Calculate the size of the interception window

The expression for the size of the interception window is:

W=IR+(M-1)×MH-IL

H=IB+(N-1)×MV-IT

In the formula, W and H- are the width and height of the interception window, respectively.

5. Calculate the position of the lower right corner of the interception window in the discrete viewpoint image in the first row and first column of the discrete viewpoint image array

The expression of the position of the lower right corner of the interception window in DVI _1,1 is:

PH=IR

PV=IB

In the formula, PH and PV- are the horizontal and vertical position coordinates of the lower right corner of the interception window in DVI _1,1 , respectively.

6. Intercept discrete viewpoint image array to generate sub-image array

As shown in Figure 8, it is a schematic diagram of generating a sub-image array. The white box in the figure represents the interception window, and the interception window is used to intercept each discrete viewpoint image in the discrete viewpoint image array to generate a sub-image array. The sub-image array contains The number of sub-images is equal to the number of discrete viewpoint images contained in the discrete viewpoint image array, and the expression of a sub-image located in the i-th column and j-th row in the sub-image array is:

SI _i,j (u,v)=DVI _i,j (PH+(i-1)×MH-W+u, PV+(j-1)×MV-H+v)

In the formula, SI _{i, j} - is a sub-image in the i-th column and j-th row in the sub-image array, u=1, 2,..., W and v=1, 2,..., H-respectively pixel points Horizontal and vertical position coordinates within the subimage.

7. Convert the sub-image array into a volume element image array

The stereoscopic element image array converted from the sub-image array contains W×H stereoscopic element images, each of which has a resolution of M×N, and is located in the p-th column and q-th row of a stereoscopic element image array. The expression for the meta image is:

EI _{p, q} (r, t) = SI _{r, t} (p, q)

In the formula, EI _{p, q} - is a stereo element image in column p and row q in the stereo element image array, r=1, 2,..., M and t=1, 2,..., N-respectively The horizontal and vertical position coordinates of pixels in the stereo image.

Claims (1)

1. a discrete view collection is in conjunction with the three-dimensional element image array generation method of window intercepting algorithm, it is characterized in that comprising the following steps:

1.1 gather discrete view pattern matrix, comprise the following steps:

1.1.1 initialization: regulate two A-frames supporting stereo track the height of stereo track to be fixed to a lower position, camera is arranged on stereo track, stereo track can make camera at the uniform velocity move from right to left, and experimenter controls startup and the stopping of stereo track by remote controller;

1.1.2 discrete view image sets is obtained: utilize level gauge the plane of stereo track to be adjusted to and plane-parallel, and with remote controller, camera is moved to the low order end of stereo track, pin camera shutter release cable and start stereo track simultaneously, in the moving process of camera, utilize multiple discrete view images of the continuous shooting function continuous acquisition reference object of camera, the sequencing of the discrete view image collected according to shooting is from left to right in line, namely obtains the discrete view image sets collected when stereo track is positioned at this height;

1.1.3 utilize Vertical surveyors' staff, raise two A-frames supporting stereo track, make stereo track increase a fixing distance;

1.1.4 repeat the shooting process of step 1.1.2 and 1.1.3, obtain multiple discrete view image sets;

1.1.5 by the sequencing of all discrete view image sets according to collection, be arranged in a discrete view pattern matrix from top to bottom, namely, collect at first one group of discrete view image sets is placed on the first row of discrete view pattern matrix, the one group of discrete view image sets finally collected is placed on last column of discrete view pattern matrix;

1.2 calculate the position of reference object in every width discrete view image:

The location parameter of reference object in every width discrete view image comprises: the horizontal relative displacement of reference object in discrete view pattern matrix in the horizontal adjacent discrete visual point image of any two width, the vertically opposite displacement of reference object in discrete view pattern matrix in any two width vertical adjacent discrete visual point image, reference object upper, under, left and right lateral boundaries is in the first row of discrete view pattern matrix, position in first row discrete view image, reference object upper, under, left and right lateral boundaries is in last column of discrete view pattern matrix, position in last row discrete view image,

1.2.1 determine the horizontal relative displacement of reference object in discrete view pattern matrix in the horizontal adjacent discrete visual point image of any two width, its computational methods are: establish DVI _{i, j}represent in discrete view pattern matrix and be positioned at the i-th row, a width discrete view image of jth row, first, by DVI _1,1pixel by pixel to right translation, after each translation, calculate the DVI after translation _1,1with DVI _2,1the Y-PSNR of lap, Y-PSNR is defined as:

$\mathrm{PSNR}\left(s\right)=10\&times;{\log }_{10}\left[\frac{{255}^2}{\mathrm{MSE}\left(s\right)}\right]$

In formula, s-is DVI _1,1translation distance, PSNR (s)-be the DVI after translation _1,1with DVI _2,1the Y-PSNR of lap, MSE (s)-be mean square error, its definition is:

$\mathrm{MSE}\left(s\right)=\frac{1}{(X-s)\&times;Y}\underset{x=0}{\overset{X-1-s}{\mathrm{\&Sigma;}}}\underset{y=0}{\overset{Y-1}{\mathrm{\&Sigma;}}}{[{\mathrm{DVI}}_{\mathrm{1,1}}(x,y)-{\mathrm{DVI}}_{\mathrm{2,1}}(x+s,y)]}^2$

In formula, x, y-are respectively pixel at DVI _1,1in horizontal and vertical position coordinates, the pixel quantity that the horizontal and vertical direction that X, Y-are respectively discrete view image comprises;

Then, displacement corresponding time Y-PSNR is maximum is as the horizontal relative displacement of reference object in discrete view pattern matrix in the horizontal adjacent discrete visual point image of any two width;

1.2.2 determine the vertically opposite displacement of reference object in discrete view pattern matrix in any two width vertical adjacent discrete visual point image, its computational methods are: first by DVI _1,1and DVI _1,2carry out transposition, then calculate according to the computational methods identical with the horizontal relative displacement of reference object in discrete view pattern matrix in any two width horizontal adjacent discrete visual point image;

1.2.3 determine that upper and lower, the left and right lateral boundaries of reference object is at DVI _1,1in position, its computational methods are: first, calculate DVI _1,1and DVI _2,1error image and carry out medium filtering; Then, using the position of the upper and lower and left margin of non-zero points all in the error image after medium filtering as the upper and lower of reference object and left border at DVI _1,1in position, the value obtained after the position of the right margin of non-zero points all in the error image after medium filtering being deducted the horizontal relative displacement of reference object in discrete view pattern matrix in the horizontal adjacent discrete visual point image of any two width as the right side boundary of reference object at DVI _1,1in position;

1.2.4 upper and lower, the position of left and right lateral boundaries in last column, last row discrete view image of discrete view pattern matrix of reference object is determined, its computational methods are: first, calculate the error image of the discrete view image of last column, row second from the bottom in last column in discrete view pattern matrix, last discrete view image arranged and discrete view pattern matrix and carry out medium filtering, then, upper by non-zero points all in the error image after medium filtering, upper as reference object of the position of lower and right margin, lower and right side boundary is in last column of discrete view pattern matrix, position in last row discrete view image, the value obtained after the position of the left margin of non-zero points all in the error image after medium filtering being added the horizontal relative displacement of reference object in discrete view pattern matrix in the horizontal adjacent discrete visual point image of any two width as the left border of reference object in last column of discrete view pattern matrix, position in last row discrete view image,

1.3 calculate and intercept the horizontal relative displacement of windows in discrete view pattern matrix in any two width horizontal adjacent discrete visual point image and intercept the vertically opposite displacement of window in discrete view pattern matrix in any two width vertical adjacent discrete visual point image:

Intercepting the horizontal relative displacement of window in discrete view pattern matrix in any two width horizontal adjacent discrete visual point image with the expression formula intercepting the vertically opposite displacement of window in discrete view pattern matrix in any two width vertical adjacent discrete visual point image is:

MH＝DH+delta

MV＝DV+delta

In formula, MH, MV-are respectively and intercept the horizontal relative displacement of window in discrete view pattern matrix in any two width horizontal adjacent discrete visual point image and the vertically opposite displacement of intercepting window in discrete view pattern matrix in any two width vertical adjacent discrete visual point image, DH, DV-are respectively the horizontal relative displacement of reference object in discrete view pattern matrix in any two width horizontal adjacent discrete visual point image and the vertically opposite displacement of reference object in discrete view pattern matrix in any two width vertical adjacent discrete visual point image, and delta-is the effect of depth factor;

According to actual needs, delta can carry out any value in allowed limits, and the span of delta is:

delta_max＝min[(MH_max-DH)，(MV_max-DV)]

delta_min＝0

In formula, delta_max, delta_min-are respectively the desirable maximum of delta and minimum value, MH_max, MV_max-are respectively the maximum intercepting the horizontal relative displacement of window in discrete view pattern matrix in any two width horizontal adjacent discrete visual point image and the maximum intercepting the vertically opposite displacement of window in discrete view pattern matrix in any two width vertical adjacent discrete visual point image, and the minimum value of all numerical value in bracket is got in min ()-expression;

The expression formula of MH_max and MV_max is:

MH_max＝min(X-IR,IL)/(M-1)

MV_max＝min(Y-IB,IT)/(N-1)

In formula, IR, IB-are respectively the right side of reference object and border, downside at DVI _1,1in position, IL, IT-are respectively left side and the position of boundary in last column, last row discrete view image of discrete view pattern matrix of reference object, and M, N-are respectively in discrete view pattern matrix the quantity often going and often arrange the discrete view image comprised;

1.4 calculate the size intercepting window:

The expression formula intercepting the size of window is:

W＝IR+(M-1)×MH-IL

H＝IB+(N-1)×MV-IT

In formula, W, H-are respectively the width and height that intercept window;

1.5 calculate the position of the lower right corner in the first row, first row discrete view image of discrete view pattern matrix intercepting window:

Intercept the lower right corner of window at DVI _1,1the expression formula of middle position is:

PH＝IR

PV＝IB

In formula, PH, PV-are respectively the lower right corner of intercepting window at DVI _1,1in horizontal and vertical position coordinates;

1.6 intercept discrete view pattern matrix spanning subgraph as array:

With intercepting window, intercepting spanning subgraph is carried out as array to the every width discrete view image in discrete view pattern matrix, the subgraph quantity comprised in subgraph array is equal with the discrete view amount of images comprised in discrete view pattern matrix, is arranged in that subgraph array i-th arranges, the expression formula of a width subgraph of jth row is:

SI _i，j(u，V)＝DVI _i，j(PH+(i-1)×MH-W+u，PV+(j-1)×MV-H+V)

In formula, SI _i,j-in subgraph array i-th row, jth row a width subgraph, u=1,2 ..., W and v=1,2 ..., H-is respectively the horizontal and vertical position coordinates of pixel in subgraph;

1.7 by array switching for subgraph one-tenth three-dimensional element pattern matrix:

Comprise W × H three-dimensional element image in the three-dimensional element image array that subgraph array converts to, the size of each three-dimensional element image is M pixel × N pixel, and the expression formula being arranged in the width three-dimensional element image that three-dimensional element image array p arranges, q is capable is:

EI _p，q(r，t)＝SI _r，t(p，q)

In formula, EI _{p, q}-be the width three-dimensional element image that p in three-dimensional element image array arranges, q is capable, r=1,2 ..., M and t=1,2 ..., N-is respectively the horizontal and vertical position coordinates of pixel in three-dimensional element image.