CN103974055B - 3D photo generation system and method - Google Patents

Abstract

The invention relates to a 3D photo generation system and a method thereof, wherein the 3D photo generation system comprises: the stereoscopic image input module is used for inputting a stereoscopic image, and the stereoscopic image comprises a left eye image and a right eye image; the depth estimation module is used for estimating the depth information of the stereo image and generating a depth map; the multi-view image reconstruction module is used for generating a multi-view image according to the depth map and the stereo image; and the image interlaced scanning module is used for adjusting the multi-viewpoint images and forming a mixed image. The 3D photo generation system and the method thereof remarkably simplify the 3D photo generation process and improve the quality of the 3D photo. The 3D photo generation system and the method thereof can be widely applied to various theme parks, tourist attractions and photo museums. More consumers will enjoy the fun of 3D photos.

Description

3D photo generation system and method

technical field

The present invention relates to a photo processing system and method, more specifically, to a 3D photo processing system and method.

Background technique

3D photos are usually produced using microlens technology. The lenses of the microlenses are arrays of magnifying lenses designed to magnify different images when viewed from slightly different angles. In order to generate a 3D photo, it is first necessary to generate multi-viewpoint images, say 12 or more viewpoint images, and then the multi-viewpoint images will be blended into a blended image. Blending multi-view images is a process of extracting suitable pixels from multi-view images and merging them into a new image, which contains multi-view information of the original image. The lenses of the microlenses are used to display these multi-view perspectives at different viewing angles. Finally, viewing through the lenticular plate, the viewer's left and right eyes can see different images, thus producing a 3D effect.

Currently, there are several different methods of generating 3D photos, of which the most common method is to manually convert 2D images into multi-viewpoint images, which takes hours or even days. Usually the operator needs to extract objects from the target image by creating skins, and then assign depth information to these skins according to his own judgment. The depth information is a separate grayscale image with the same size as the original 2D image, and the grayscale image uses gray to represent the depth of each part of the image. The manually created depth information is used to guide the computer to move the pixels of the original 2D image to form a new viewpoint map. Depth maps can produce strong 3D display effects.

Another method is to shoot objects from multiple viewpoints, however, this method is not a feasible method for moving objects. This method requires the arrangement of one or more cameras to acquire multi-viewpoint images. The image acquisition device needs to be positioned carefully so that the viewing angle of the output image is not too wide.

Multi-view images are used to reconstruct hybrid images, and most systems will directly construct hybrid images from data extracted from multi-view images. Since the final image is a subsample of each multi-view image, the resulting image cannot maintain the quality of the original image.

To sum up, the existing methods and systems for generating 3D photos have disadvantages such as long processing time and low quality of photos.

Contents of the invention

The object of the present invention is to provide a 3D photo generation system and method with fast processing speed and high photo quality.

The 3D photo generation system of the present invention includes: a stereoscopic image input module for inputting a stereoscopic image, and the stereoscopic image includes a left-eye image and a right-eye image;

a depth estimation module, configured to estimate the depth information of the stereo image and generate a depth map;

A multi-viewpoint image reconstruction module, configured to generate a multi-viewpoint image according to the depth map and the stereoscopic image;

The image interlaced scanning module is used to adjust the multi-viewpoint image and form a mixed image.

In the 3D photo generation system of the present invention, the depth estimation module includes:

The pixel matching module is used to compare the left-eye image and the right-eye image of the stereoscopic image and find the relative pixels between the left-eye image and the right-eye image, and estimate the optical flow of the pixel according to the optical flow constraint equation;

A depth information determination module, configured to find pixel displacement to determine pixel depth information according to the optical flow of the left-eye image and the right-eye image;

A depth map generating module, configured to generate a depth map according to the depth information.

In the 3D photo generation system of the present invention, the multi-viewpoint image reconstruction module includes:

A basic image selection module, configured to select the left-eye image, the right-eye image, or the left-eye image and the right-eye image of the stereoscopic image as the basic image;

Image number determination module, used to determine the number and disparity of required images according to requirements;

a pixel shifting module for shifting pixels of the base image according to the depth map to form a new image;

A hole filling module, configured to fill holes in the new image due to pixel loss;

The multi-viewpoint image generation module is used to generate multi-viewpoint images.

In the 3D photo generation system of the present invention, the image interlaced scanning module includes:

an image adjustment module, configured to adjust the size of the multi-view image;

a contrast adjustment module, configured to adjust the contrast of the adjusted multi-viewpoint image output by the image adjustment module;

An image synthesis module, which is used to merge the multi-viewpoint images after contrast adjustment to form a mixed image;

A mixed image output module, configured to output the mixed image.

The hole filling module uses an interpolation method to fill holes in the new image due to missing pixels.

3D photo generation method of the present invention, comprises the steps:

S1 inputs a stereoscopic image, the stereoscopic image includes a left-eye image and a right-eye image;

S2 estimates the depth information of the stereo image and generates a depth map;

S3 generates a multi-viewpoint image according to the depth map and the stereo image;

S4 adjusts the multi-viewpoint image and forms a mixed image.

In the 3D photo generation method of the present invention, the step S2 includes the following steps:

S21 comparing the left-eye image and the right-eye image of the stereoscopic image and finding relative pixels between the left-eye image and the right-eye image, and estimating the optical flow of the pixel according to the optical flow constraint equation;

S22 Find out the pixel displacement to determine the depth information of the pixel according to the optical flow of the left-eye image and the right-eye image;

S23 Generate a depth map according to the depth information.

In the 3D photo generating method of the present invention, the step S3 includes:

S31 selecting the left-eye image, the right-eye image, or the left-eye image and the right-eye image of the stereoscopic image as a base image;

S32 determines the number and disparity of required images according to requirements;

S33 moving pixels of the base image according to the depth map to form a new image;

S34 filling holes in the new image due to pixel loss;

S35 Generate a multi-viewpoint image.

In the 3D photo generating method of the present invention, the step S4 includes:

S41 adjusts the size of the multi-view image;

S42 adjusts the contrast of the multi-viewpoint image adjusted by the step S41;

S43 merging the contrast-adjusted multi-viewpoint images to form a mixed image;

S44 outputs the mixed image.

In the method for generating a 3D photo according to the present invention, in the step 34, an interpolation method is used to fill holes in the new image due to pixel loss.

Implementing the 3D photo generating system and method of the present invention has the following beneficial effects: the 3D photo generating system and method of the present invention significantly simplifies the 3D photo generating process and improves the quality of the 3D photo. The 3D photo generation system and method of the present invention can be widely used in various theme parks, tourist attractions and photo studios. Will make more consumers enjoy the fun brought by 3D photos.

Description of drawings

The present invention will be further described below in conjunction with accompanying drawing and embodiment, in the accompanying drawing:

Fig. 1 is a system block diagram of the 3D photo generation system of the present invention;

Fig. 2 is a block diagram of the depth estimation module in the 3D photo generation system of the present invention;

Fig. 3 is a block diagram of a multi-viewpoint image reconstruction module in the 3D photo generation system of the present invention;

Fig. 4 is the block diagram of the image interlaced scanning module in the 3D photo generation system of the present invention;

Fig. 5 is a flow chart of the 3D photo generating method of the present invention;

Fig. 6 is the flowchart of step S2 in the 3D photo generation method of the present invention;

Fig. 7 is the flowchart of step S3 in the 3D photo generation method of the present invention;

Fig. 8 is a flowchart of step S4 in the 3D photo generation method of the present invention;

9 is a schematic diagram of a stereoscopic image input by the 3D photo generation system of the present invention;

Fig. 10 is a schematic diagram of the comparison between the depth map and the original image formed by the 3D photo generation system of the present invention;

Figure 11 is the adjusted multi-view image;

Fig. 12 is a schematic diagram of a mixed image.

detailed description

In order to have a clearer understanding of the technical features, purposes and effects of the present invention, the specific implementation manners of the present invention will now be described in detail with reference to the accompanying drawings.

As shown in Figures 1 to 4, it is a system block diagram of an embodiment of the 3D photo generation system of the present invention, the 3D photo generation system includes a stereoscopic image input module 1, a depth estimation module 2, a multi-viewpoint image reconstruction module 3 and Image interlacing module4. Wherein, the stereoscopic image input module 1 is used for inputting the stereoscopic image, and the stereoscopic image includes a left-eye image and a right-eye image; the depth estimation module 2 is used for estimating the depth information of the stereoscopic image and generating a depth map; the multi-viewpoint image reconstruction module 3 is used for The depth map and the stereoscopic image generate a multi-viewpoint image; the image interlacing module 4 is used to adjust the multi-viewpoint image and form a mixed image.

In the 3D photo generation system of the present invention, the depth estimation module 2 further includes: a pixel matching module 21 , a depth information determination module 22 and a depth map generation module 23 . Wherein, the pixel matching module 21 is used to compare the left-eye image and the right-eye image of the stereoscopic image and find out the relative pixel between the left-eye image and the right-eye image, and estimate the optical flow of the pixel according to the optical flow constraint equation; A pixel refers to a pixel at the same pixel position of the left-eye image and the right-eye image. The depth information determination module 22 is used to find pixel displacement according to the optical flow of the left-eye image and the right-eye image to determine the depth information of the pixel; the depth map generation module 23 is used to generate a depth map according to the depth information.

In the 3D photo generation system of the present invention, the multi-view image reconstruction module 3 further includes: a basic image selection module 31, an image number determination module 32, a pixel shift module 33, a hole filling module 34 and a multi-view image generation module 35. Wherein, the basic image selection module 31 is used to select the left-eye image, the right-eye image or the left-eye image and the right-eye image of the stereoscopic image as the basic image; the image number determination module 32 is used to determine the number and parallax of the required images according to requirements; The pixel shifting module 33 is used to move the pixels of the base image according to the depth map to form a new image; the hole filling module 34 is used to fill holes in the new image due to missing pixels; the multi-viewpoint image generation module 35 is used to generate multi-viewpoint images.

In the 3D photo generation system of the present invention, the image interlaced scanning module 4 further includes: an image adjustment module 41 , a contrast adjustment module 42 , an image synthesis module 43 and a mixed image output module 44 . Wherein, the image adjustment module 41 is used to adjust the size of the multi-viewpoint image; the contrast adjustment module 42 is used to adjust the contrast of the adjusted multi-viewpoint image output by the image adjustment module; The viewpoint images are combined to form a mixed image; the mixed image output module 44 is configured to output the mixed image.

As shown in Figure 5 to Figure 8, it is a flow chart of the 3D photo generation method of the present invention, which includes the following steps:

S1 inputs a stereoscopic image, the stereoscopic image includes a left-eye image and a right-eye image;

S2 estimates the depth information of the stereo image and generates a depth map;

S3 generates a multi-viewpoint image according to the depth map and the stereo image;

S4 adjusts the multi-viewpoint images and forms a mixed image.

Wherein, step S2 further includes the following steps:

S21, comparing the left-eye image and the right-eye image of the stereoscopic image and finding the relative pixels between the left-eye image and the right-eye image, and estimating the optical flow of the pixel according to the optical flow constraint equation;

S22 Find out the pixel displacement to determine the depth information of the pixel according to the optical flow of the left-eye image and the right-eye image;

S23 Generate a depth map according to the depth information.

Step S3 further includes:

S31 selecting the left-eye image, the right-eye image, or the left-eye image and the right-eye image of the stereoscopic image as the base image;

S32 determines the number and disparity of required images according to requirements;

S33 moving pixels of the base image according to the depth map to form a new image;

S34 filling holes formed due to pixel loss in the new image;

S35 Generate a multi-viewpoint image.

Step S4 further includes:

S41 adjusts the size of the multi-view image;

S42 adjusts the contrast of the multi-viewpoint image adjusted in step S41;

S43 merging the contrast-adjusted multi-viewpoint images to form a mixed image;

S44 outputs the mixed image.

The composition of the 3D photo generation system of the present invention and the specific steps of the 3D photo generation method of the present invention have been introduced above, and how the 3D photo generation system and method of the present invention work will be described in detail below in conjunction with a specific example. The 3D photo generation system of the present invention uses stereoscopic images as input, automatically compares and calculates 3D information (also called a depth map) based on the stereoscopic images. The pixels of the original input image are then shifted based on the depth information to generate a multi-view image. In order to improve the quality of the final mixed image, the 3D photo generation system of the present invention will adjust the generated image to form a suitable size, and then merge the adjusted images together. The resulting mixed image can be displayed on a glasses-free 3D display device, or composited on a microlens plate to form a 3D photo.

In the 3D photo generation system of the present invention, the stereoscopic image input device 1 is used to input a stereoscopic image, and a stereoscopic image is also a stereogram, which can produce a three-dimensional visual effect. The stereoscopic image can be obtained by one or more technologies, and the stereoscopic image can also directly use a 3D image. In this example, the input stereoscopic image is a stereoscopic image including a left-eye image and a right-eye image, and the specific image is shown in FIG. 9 .

The depth estimation module 2 is used to analyze the depth information of the stereo image input by the stereo image input module 1 to reconstruct a multi-viewpoint image. The steps of depth estimation are shown in FIG. 6 . The depth estimation module 2 includes a pixel matching module 21 , a depth information determination module 22 and a depth map generation module 23 . Among them, the pixel matching module 21 is used for comparing the left-eye image and the right-eye image of the input stereoscopic image to find the relative pixels of the two, that is, the pixels at the same pixel position of the left-eye image and the right-eye image. Objects are displaced in the left-eye image and in the right-eye image of the stereoscopic image, which is also called parallax. To extract disparity, matching methods such as optical flow and stereo matching are usually used to find out the pixel displacement between the left-eye image and the right-eye image. Optical flow is the pattern of apparent motion of objects, surfaces, or edges in a visible scene caused by relative motion between an observer (such as glasses or a camera) and the scene. Optical flow estimation is to estimate the optical flow according to the optical flow constraint equation. In order to find matching pixels, the images need to be compared, following the well-known optical flow constraint equation:

$\frac{<span\; class="notranslate">\; \&PartialD;</span>\mathrm{<span\; class="notranslate">\; I</span>}}{<span\; class="notranslate">\; \&PartialD;</span>\mathrm{<span\; class="notranslate">\; x</span>}}{\mathrm{<span\; class="notranslate">\; V</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; +</span>\frac{<span\; class="notranslate">\; \&PartialD;</span>\mathrm{<span\; class="notranslate">\; I</span>}}{<span\; class="notranslate">\; \&PartialD;</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}}{\mathrm{<span\; class="notranslate">\; V</span>}}_{\mathrm{<span\; class="notranslate">\; the\; y</span>}}<span\; class="notranslate">\; +</span>\frac{<span\; class="notranslate">\; \&PartialD;</span>\mathrm{<span\; class="notranslate">\; I</span>}}{<span\; class="notranslate">\; \&PartialD;</span>\mathrm{<span\; class="notranslate">\; t</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}$

Among them, V _x , V _y are the x and y components of velocity or optical flow I(x, y, t), respectively, and is the derivative of the image at (x, y, t) in the corresponding direction. A coarse-to-fine strategy can be used to determine the optical flow of pixels. There are different robust methods for enhancing disparity estimation, such as high-precision optical flow estimation based on convolution theory.

After pixel matching, depth information can be conveyed by disparity information and camera parameters. The displacement of pixels can indicate depth formation. However, most 3D stereoscopic devices switch the camera or lens to a point. In other words, the direction of optical flow needs to be considered in the calculation of the depth of each pixel. The depth information determination module 22 is used to determine the depth information of a pixel, and the depth information of each pixel can be calculated using the following formula.

$\mathrm{<span\; class="notranslate">\; max</span>}\mathrm{<span\; class="notranslate">\; displacement</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; direction</span>}\sqrt{{\mathrm{<span\; class="notranslate">\; u</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; v</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}$

Where maxdisplacement is the maximum displacement of the pixel, direction is the direction of optical flow, and u and v are the optical flow vectors of each pixel in the x and y directions, respectively. This depth information can be used to reconstruct the 3D environment, eg a depth map. A depth map is represented by a grayscale image recognized by a computer. The depth map generation module 23 is used to generate a depth map. Generally, the depth value of a pixel is from 0 to 255, and the higher the depth value of a pixel, the closer it is to the viewer. In order to improve the quality of 3D photos, the 3D photo generation system of the present invention separates the foreground and the background in the depth map, and the system uses depth values in the range of 99 to 255 pixels to represent the foreground, and depth values in the range of 0 to 128 of pixels represent the background. The foreground depth information and the background depth information have a certain overlap, and in this embodiment, the overlap ranges from 99 to 128. The overlapping range of the foreground depth information and the background depth information can be adjusted by the user. This process increases the contrast between the foreground and background. Further, depth details of foreground main objects and background can be enhanced. Fig. 10 is a schematic diagram of separating foreground and background images.

The multi-view image reconstruction module 3 is used to reconstruct multi-view images, including: a basic image selection module 31 , an image number determination module 32 , a pixel shift module 33 , a hole filling module 34 and a multi-view image generation module 35 . The base image selection module 31 may select a left-eye image, a right-eye image, or a left-eye image and a right-eye image of a stereoscopic image as a base image to generate a multi-viewpoint image. The multi-viewpoint image reconstruction process is shown in Figure 7. If a single image is selected, such as a left-eye image or a right-eye image, then the generated image will be the left-eye image and right-eye image of the selected image. If it is necessary to generate 2N+ 1 image, the selected image will be the N+1th image, the generated images will be the 1st to Nth images and the N+2th to 2N+1th images. For example, if 9 images need to be generated, the selected image will be the 5th image and the generated images will be the 1st to 4th images and the 6th to 9th images. The image number determining module 32 is used to determine the number of images as required. On the other hand, if two images (left-eye image and right-eye image) are selected as base images to generate a multi-viewpoint image, that is, a stereo image is selected as a base image, the system will use the disparity and the number of images required to determine the selected The positions of the two images. In this embodiment, the system first determines the number of images to be generated. The number of multi-viewpoint images depends on the number of lenses per inch of the lenticular plate and the number of dots per inch of the printer. The number of multi-viewpoint images N=DPI/LPI, where DPI is the number of dots per inch of the printer, and LPI is the number of dots per inch of the lenticular plate Number of lenses. For example, the number of lenses per inch of the microlens plate is 50, the number of dots per inch is 600, and the number of images required is 600/50=12, so 12 images are required to form a suitable 3D image. The position of the initial stereo image is determined by the following formula:

where N is the number of multi-viewpoint images, D is the disparity of the initial stereo image, and d is the disparity of each viewpoint of the multi-viewpoint image to be generated. The initial stereo image is inserted into the appropriate position of the multi-view image. Other viewpoint images will be generated from the initial stereo image. This method will evenly distribute the multi-view images, i.e. these images have similar disparity, this method will also improve the quality of the final blended image.

After determining the number of images needed and the locations of all images, the system uses the depth map to generate multi-view images. The depth maps for the left and right eye images have been generated in the previous sections. These base images, such as the left eye image or the right eye image, shift pixels according to their own depth maps. The pixel shifting module 33 is used to shift the pixels of the base image to form a new image. Usually the depth value of the depth map is from 0 to 255, and 128 is the middle value, which is the intersection point of the base image. To simulate a left-eye image from the base image, pixels with depth values in the range 128 to 255 are shifted to the right, and pixels with depth values in the range 0 to 127 are shifted to the left. To simulate a left-eye view from the base image, pixels with depth values in the range 128 to 255 are shifted to the left, and pixels with depth values in the range 0 to 127 are shifted to the right. From 128 to 255, the higher the depth value of the pixel, the greater the distance the pixel moves. From 0 to 127, the smaller the depth value of the pixel, the greater the distance the pixel moves. Below is the pixel shift formula.

lx=x+parallax; rx=x-parallax

Where parallax is the parallax parameter with depth information of the image, lx is the x-axis coordinate of the left-eye image pixel, rx is the x-axis coordinate of the right-eye image pixel, and the pixel at the new left-eye image (lx, y) is the base image The pixel at (x,y), the pixel at (rx,y) of the new right eye image is the pixel at (x,y) of the base image. After appropriate shifting of the pixels, new left and right eye images are finally generated.

When the system generates a new image, the new image will lose some pixels. The process of processing these lost pixels is called hole filling. The hole filling module 34 is used to fill these holes. These holes generated by pixel movement can utilize adjacent pixels Fill by interpolation method, or use other appropriate hole filling methods to fill. The calculation formula of the interpolation method for the pixel value of the hole is as follows.

lengthh=endx-startx

$\mathrm{<span\; class="notranslate">\; weight</span>}<span\; class="notranslate">\; =</span>\frac{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; hole</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; startx</span>}<span\; class="notranslate">\; )</span>}{\mathrm{<span\; class="notranslate">\; length</span>}}$

pixelvalue=((sourceImage(endx,y)-sourceImage(startx,y))×weight+sourceImage(startx,y)

Among them, startx and endx are the start and end positions of the hole in this row, length is the length of the hole, holex is the x-axis position of the hole, weight is the weight value of the hole, and pixelvalue is the pixel value of the hole. After the holes are filled, the newly generated viewpoint images are ready, and the next step can be performed. The multi-viewpoint image generating module 35 can generate multi-viewpoint images according to the initial image and the newly generated images.

The image interlacing module 4 is used to adjust the previously generated multi-viewpoint images and form a mixed image, including an image adjustment module 41 , a contrast adjustment module 42 , an image synthesis module 43 and a mixed image output module 44 . FIG. 8 is a flowchart of forming a mixed image. This system can improve the quality of the final blended image. To improve the quality of the final blended image. The system first resizes each image to the appropriate width. The image adjustment module 41 is used for the size of the multi-view image. Take the aforementioned 12 images as an example, the final printed image has a width of 600 pixels per inch, since there are 12 images, each image per inch is adjusted to 600/12=50 pixels per inch, the adjusted image and The initial images have the same height. As shown in Figure 11, the 12 resized images have the same height as the original image, but have different widths. The system then increases the contrast of these adjusted images. The contrast adjustment module 42 is used to adjust the contrast. These two processes can increase the color detail of the final composite image.

The finally formed image is a mixed image of 12 images, and the image synthesis module 43 is used to form the mixed image. In this example, an image with 600 pixels per inch will be reconstructed. In order to cooperate with a lenticular plate with 50 LPI, the composite image will comprise 600/12=50 LPI. Each bar consists of 12 pixels. Figure 12 is a schematic illustration of a mixed strip of images. Pixels are extracted from 12 images in 12 to 1 order, usually the right eye view is the first row of these strips, the 12th image. In Figure 12, the first bar is the combination of the first row of each image, and so on, the second bar is the combination of the second row of each image.

In practice, most lenticular sheets do not have an ideal lens per inch count. For example, sometimes the LPI is 50.1 or 49.9 instead of 50. This causes distortion in the final 3D image. So the system ends up scaling the image to fit the actual lenticular plate. For example, ideally, a lenticular sheet would have 50 lenses per inch and be 10 inches wide. The width of the image is 50X12X10=6000. However, if the lenticular plate has a lens count of 50.1 and a width of 10 inches, the final image will have a width of 5988, which can be calculated by

${\mathrm{<span\; class="notranslate">\; Width</span>}}_{\mathrm{<span\; class="notranslate">\; actual</span>}}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; LPI</span>}}_{\mathrm{<span\; class="notranslate">\; tdeal</span>}}}{{\mathrm{<span\; class="notranslate">\; LPI</span>}}_{\mathrm{<span\; class="notranslate">\; actual</span>}}}<span\; class="notranslate">\; \&times;</span>{\mathrm{<span\; class="notranslate">\; Width</span>}}_{\mathrm{<span\; class="notranslate">\; tdeal</span>}}$

Among them, LPI _tdeal is the number of lenses per inch of the microlens plate under ideal conditions. In this example, its value is 50. LPI _actual is the actual number of lenses per inch of the microlens plate. In this example, its value is 50.1, and Width _tdeal is The ideal width of the image at 50 lenses per inch is 6000, and Width _actual is the actual width of the image at 50.1 lenses per inch, which is 5988. The mixed image output module 44 is used for the formed mixed image.

The hybrid image can be combined with a lenticular plate to create a 3D photo. The image can be directly printed on the lenticular plate, the printed image can be composited on the lenticular plate and put into a lenticular plate photo frame, or combined with the lenticular plate by any other suitable method.

The 3D photo generating system and method of the present invention significantly simplifies the 3D photo generating process and improves the quality of the 3D photo. The 3D photo generation system and method of the present invention use a stereogram as an input, and the existing 3D camera and 3D lens can be used as a stereoscopic image shooting device. Using image processing technology to reconstruct 3D information from stereoscopic images and improve the quality of 3D photos. This can generate multi-view images very quickly and efficiently and can improve the quality of the generated images. In order to further improve the quality of the mixed image, the 3D photo generation system and method of the present invention firstly adjust the size of the multi-view image, which will increase the color details of the output mixed image. The 3D photo generation system and method of the present invention can be widely used in various theme parks, tourist attractions and photo studios. Will make more consumers enjoy the fun brought by 3D photos.

Embodiments of the present invention have been described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementations, and the above-mentioned specific implementations are only illustrative, rather than restrictive. Those of ordinary skill in the art will Under the enlightenment of the present invention, many forms can also be made without departing from the gist of the present invention and the protection scope of the claims, and these all belong to the protection of the present invention.

Claims (6)

1. a 3D photo generation system, it is characterised in that, comprising:

Stereo-picture load module, for inputting stereo-picture, described stereo-picture comprises left-eye image and eye image;

Depth estimation module, for estimating the degree of depth information of described stereo-picture and generate depth map;

Multi-view image rebuilds module, for generating multi-view image according to described depth map and described stereo-picture;

Image staggered scanning module, for described multi-view image is adjusted and form vision-mix;

Described multi-view image is rebuild module and is comprised:

Base image selects module, for selecting image based on the left-eye image of described stereo-picture, eye image or left-eye image and eye image;

Picture number determination module, for determining number and the parallax of required image according to demand;

Pixel mobile module, for move described base image according to described depth map pixel to form new images;

Fill out hole module, for filling in described new images owing to pixel loses the hole formed;

Multi-view image generation module, for generating multi-view image;

Described image staggered scanning module comprises:

Image adjustment module, for adjusting the size of described multi-view image;

Setting contrast module, for adjusting the contrast gradient of multi-view image after adjustment that described image adjustment module exports;

Image synthesis unit, forms vision-mix for being merged by the multi-view image after setting contrast;

Vision-mix output module, for exporting described vision-mix;

Described image output module is also for the microlens plate by described vision-mix ratio Scale to Fit reality, and the developed width of described vision-mix is by following formulae discovery:

Wherein: LPI_tdaalFor the per inch number of lenses of ideally microlens plate, LPI_actualFor the actual per inch number of lenses of microlens plate, Width_tdealIt is the desired width of vision-mix, Width_actualIt it is the developed width of vision-mix.

2. 3D photo generation system according to claim 1, it is characterised in that, described depth estimation module comprises:

Pixel matching module, for comparing the left-eye image of described stereo-picture and eye image and find out the generic pixel between described left-eye image and eye image, find out the light stream of pixel according to optical flow constraint equation;

Degree of depth information determination module, for the light stream according to described left-eye image and eye image, finds out pixel displacement to determine the degree of depth information of pixel;

Depth map generation module, for generating depth map according to described degree of depth information.

3. 3D photo generation system according to claim 1, it is characterised in that, described in fill out hole module and adopt method of interpolation to fill in described new images owing to pixel loses the hole formed.

4. a 3D photograph generation method, it is characterised in that, comprise the steps:

S1 inputs stereo-picture, and described stereo-picture comprises left-eye image and eye image;

S2 estimates the degree of depth information of described stereo-picture and generates depth map;

S3 generates multi-view image according to described depth map and described stereo-picture;

Described multi-view image is adjusted and forms vision-mix by S4;

Described step S3 comprises:

S31 selects image based on the left-eye image of described stereo-picture, eye image or left-eye image and eye image;

S32 determines number and the parallax of required image according to demand;

S33 moves the pixel of described base image to form new images according to described depth map;

S34 fills in described new images owing to pixel loses the hole formed;

S35 generates multi-view image;

Described step S4 comprises:

S41 adjusts the size of described multi-view image;

The contrast gradient of the multi-view image of S42 adjustment after described step S41 adjusts;

Multi-view image after setting contrast is merged formation vision-mix by S43;

S44 exports described vision-mix;

Described step S44 comprises:

By the microlens plate of described vision-mix ratio Scale to Fit reality, the developed width of described vision-mix is by following formulae discovery:

Wherein: LPI_tdealFor the per inch number of lenses of ideally microlens plate, LPI_actualFor the actual per inch number of lenses of microlens plate, Width_tdealIt is the desired width of vision-mix, Width_actualIt it is the developed width of vision-mix.

5. 3D photograph generation method according to claim 4, it is characterised in that, described step S2 comprises the steps:

Left-eye image and the eye image of the more described stereo-picture of S21 also find out the generic pixel between described left-eye image and eye image, go out the light stream of pixel according to optical flow constraint equation estimation;

S22, according to the light stream of described left-eye image and eye image, finds out pixel displacement to determine the degree of depth information of pixel;

S23 generates depth map according to described degree of depth information.

6. 3D photograph generation method according to claim 4, it is characterised in that, described step S34 adopt method of interpolation fill in described new images owing to pixel loses the hole formed.