CN107358645A - Product method for reconstructing three-dimensional model and its system - Google Patents

Abstract

A method and system for reconstructing a three-dimensional model of a product. After collecting a product image and calculating the homography matrix of the infinite plane corresponding to the product image, the mapping between the pixel points of the product image and the coordinate points of the actual scene is obtained through the Cholesky decomposition method. Matrix; then establish the network flow corresponding to the product, calculate the optimal value of the energy function, and obtain the depth information of the product surface; finally establish a cube according to the depth information and the mapping matrix and divide the cube into voxels, and then update each cube The TSDF of the voxel is rendered and projected to realize the reconstruction of the three-dimensional model. The present invention does not need a calibration object, has high robustness, reduces the influence of imaging distortion, and the three-dimensional model has high reliability and robustness.

Description

Product 3D Model Reconstruction Method and System

technical field

The present invention relates to a technology in the field of three-dimensional modeling, in particular to a method and system for reconstructing a three-dimensional model of a product based on two-dimensional images.

Background technique

3D reconstruction refers to 3D modeling of objects to display products more realistically and intuitively. Image-based 3D reconstruction can break through the real-time bottleneck and has been well developed. The existing 3D reconstruction requires high equipment precision and many restrictions in the camera calibration process. The modeling process in the 3D reconstruction process is complicated, the speed is slow, and the reconstruction accuracy cannot meet the requirements, so it cannot be applied to the 3D reconstruction of the actual product display model.

Contents of the invention

The present invention aims at the defect that the existing technology does not have depth information optimization processing, and does not process smooth items and occlusion items, resulting in poor modeling effect, and proposes a product 3D model reconstruction method and system without calibration objects , high robustness, reducing the influence of imaging distortion, and the 3D model has high reliability and robustness.

The present invention is achieved through the following technical solutions:

The invention relates to a method for reconstructing a three-dimensional model of a product. After collecting the product image and calculating the homography matrix of the infinite plane corresponding to the product image, the mapping between the pixel points of the product image and the coordinate points of the actual scene is obtained through the Cholesky decomposition method. Matrix; then establish the network flow corresponding to the product, calculate the optimal value of the energy function, and obtain the depth information of the product surface; finally establish a cube according to the depth information and the mapping matrix and divide the cube into voxels, and then update each cube The TSDF (truncated signed distance function) of the voxel is rendered and projected to realize the reconstruction of the 3D model.

The mapping matrix is obtained in the following way:

1) Establish the homography matrix H _∞ of the infinite plane, and have Solve the homography matrix H _∞ ;

2) According to H _∞ = KR _t K ⁻¹ , the mapping matrix K between the pixel points of the product image and the coordinate points of the actual scene is obtained by using the Cholesky decomposition method.

The depth information of the product surface is obtained in the following ways:

a) Establish a virtual network of the product, and establish an energy function based on the pixel coordinates of the product image;

b) Assign values to the grids in the virtual network through similarity cost and smoothing cost to form network flow;

c) Optimizing the energy function by using the solution algorithm of the maximum flow/minimum cut problem to obtain the depth information of the product surface.

Described energy function is:

E(f)＝∑ _p∈P [I _l (Tranl(x _p ,y _p ,f _p ))-I _r (Tranr(x _p ,y _p ,f _p ))] ² +∑ _{(p,q) ∈N} u _{p,q} |f _p -f _q |, where: I _l and I _r are the pixel matrix of the product image, (x _p , y _p ) are the grid point coordinates on the base plane, and Tran is the coordinate system Conversion function, f is the mapping relationship between pixels and labels, P is the set of all pixels in the defined image, and p and q are individual pixel points in the pixel set P.

The reconstruction specifically includes the following steps:

i) Perform bilateral filtering on the depth information;

ii) Obtain the depth map according to the depth information, and back-project the depth map to obtain the vertex map and the normal vector of each vertex;

iii) converting the product image pixels to the world coordinate system according to the mapping matrix K;

iv) Create a cube and perform voxel division, and update the TSDF of each voxel;

v) Perform rendering and projection according to TSDF to generate a 3D model of the product.

The truncated signed distance function is a voxel to the nearest surface of the built model, i.e. the signed distance of the surface of the model, that is, the symbol represents the front and back relationship with respect to the surface; since the reconstruction space is regarded as a cube, the voxel is the volume element Abbreviation, is the smallest unit of digital data in three-dimensional space segmentation, similar to the pixel of a two-dimensional image. A voxel represents a series of voxels with (x, y) coordinates determined (only the z coordinate changes), and the negative number of the final TSDF indicates that it is outside the reconstructed object, 0 indicates that it is on the surface of the reconstructed object, and the inside of the reconstructed object is a positive number.

The invention relates to a product three-dimensional model reconstruction system, comprising: a camera calibration module, a depth information acquisition module and a model building module, wherein: the camera calibration module collects product images, and obtains a mapping matrix between image pixels and coordinate points of an actual scene The depth information acquisition module obtains the depth information of the product; the model building module receives the mapping matrix and the depth information, and obtains the three-dimensional model of the product through rendering and projection.

Description of drawings

Fig. 1 is a schematic flow chart of the present invention.

detailed description

This embodiment relates to a product 3D model reconstruction system that implements the above method, including: a camera calibration module, a depth information acquisition module, and a model building module, wherein: the camera calibration module collects product images to obtain image pixels and coordinate points of the actual scene The mapping matrix between them; the depth information acquisition module obtains the depth information of the product; the model building module receives the mapping matrix and depth information, and obtains the three-dimensional model of the product through rendering and projection.

As shown in Figure 1, the product 3D model reconstruction method for the above system includes the following steps:

Implementation settings: For the show car, control the camera to do a translation movement and 2 arbitrary movements, and take 4 photos (the photo pixel is 1200w);

Software and hardware requirements: Intel(R) Core(TM) i5-3210M CPU@2.5GHz, graphics card GTX970.

1) Solve the homography matrix H _∞ of the infinite plane corresponding to the product image, and use the Cholesky decomposition method to obtain the mapping matrix K between the pixel points of the product image and the coordinate points of the actual scene. The camera that collects product images adopts translational motion and multiple arbitrary motions to take photos to obtain product images.

1.1) Establish the homography matrix H _∞ of the infinite plane.

1.2) According to the equations Solve the homography matrix H _∞ , where: e ₁ and e ₂ are the extreme points of the product image after motion, H ₁ and H ₂ are the homography matrices of the space plane, a ₁ and a ₂ are scalars, X ₁ and X ₂ are column vectors .

1.3) According to the obtained different homography matrices H _∞ , according to the formula H _∞ =KR _t K ^-1 , and use the Cholesky decomposition method to solve the mapping matrix K.

2) Establish the network flow corresponding to the product, calculate the optimal value of the energy function, and obtain the depth information of the product surface.

2.1) Establish a virtual network of products. According to the coordinate position of the product in the world coordinate system, a three-dimensional virtual network is established, and the product to be reconstructed is completely wrapped in it, and the front cut plane is used as the base plane of the entire three-dimensional network, and the network points on each base plane are located The position of the object point on the product surface is determined on which section, each section corresponds to a label, so that the problem of depth information acquisition is transformed into the problem of labeling the depth of each network point on the base plane of the virtual three-dimensional network.

2.2) An energy function is established based on the pixel coordinates of the product image. The energy function is used to represent the property information of the image, and it mainly consists of three parts: the data constraint item, the smoothness constraint item and the occlusion item. The constraints of the data constraints are: when the grid points on the base plane are not correctly labeled, the image points in the image pixel coordinate system projected by the potential object points at the incorrect depth labels in the world coordinate system express The pixel information is inconsistent; only when the assigned depth label conforms to its real depth, the image point reflects the pixel information of the same object point, and the cost of this depth label is the smallest. The smooth constraint represents the constraint relationship between adjacent pixel pairs. When the gap between adjacent pixels is large, the smooth constraint item will increase, which will lead to an increase in the energy function. The smooth constraint item reflects the fragmentation by this method. smoothness. The constraint condition of the occlusion item is: when the cost of all the data items of the depth label is greater than a certain threshold, that is, the point on the surface of the object is set to be occluded. By increasing the parameter value of its smooth constraint, this point can refer to other The depth of surrounding object points is smoothed.

2.3) Assign values to the meshes of the virtual network through the similarity cost and the smoothing cost to form a network flow. The similarity cost is obtained by SAD local matching.

2.4) The energy function is optimized by using the solution algorithm of the maximum flow/minimum cut problem, and the depth information of the product surface is obtained.

The algorithm for solving the maximum flow/minimum cut problem includes, but is not limited to, the Push-Relabel method and the Ford-Fulkerson method.

3) Create a cube according to the depth information and the mapping matrix K, divide the cube into voxels, and then render and project to obtain the 3D model of the product.

3.1) Bilateral filtering is performed on the depth information, and noise reduction has been performed.

3.2) Obtain the depth map according to the depth information, and back-project the depth map to obtain the vertex map and the normal vector of each vertex.

3.3) Convert the product image pixels to the world coordinate system according to the mapping matrix K.

3.4) Create a cube and perform voxel division, and update the TSDF of each voxel. For each frame of product image, transform each voxel into the camera coordinate system and project it to the coordinate point of the product image. If it is within the projection range, update TSDF.

3.5) Perform rendering and projection according to TSDF to generate a 3D model of the product.

Compared with the prior art, the present invention does not require a calibration object, has high robustness, reduces the influence of imaging distortion, and has high reliability and robustness of the 3D model, and only requires ordinary commercial GPUs for computing resources. 7.3% faster.

The above specific implementation can be partially adjusted in different ways by those skilled in the art without departing from the principle and purpose of the present invention. The scope of protection of the present invention is subject to the claims and is not limited by the above specific implementation. Each implementation within the scope is bound by the invention.

Claims (7)

1. a kind of product method for reconstructing three-dimensional model, it is characterised in that by gathering product image, and it is corresponding to calculate product image Plane at infinity homography matrix after, pass through the coordinate that Cholesky decomposition methods obtain product image slices vegetarian refreshments and actual scene Mapping matrix between point；Then network flow corresponding to the product is established, the optimal value of computation energy function, obtains product table The depth information in face；Cube is finally established according to depth information and mapping matrix and voxel division is carried out to cube, and then By updating the TSDF of each cube volume elements and being rendered and projected, the reconstruction of threedimensional model is realized.

2. product method for reconstructing three-dimensional model according to claim 1, it is characterized in that, described mapping matrix, by with Under type obtains：

1.1) the homography matrix H of plane at infinity is established_∞, and haveSolve homography matrix H_∞；

1.2) according to H_∞=KR_tK-¹, the coordinate of product image slices vegetarian refreshments and actual scene is solved using Cholesky decomposition methods Mapping matrix K between point.

3. product method for reconstructing three-dimensional model according to claim 1, it is characterized in that, the depth letter of described product surface Breath, is obtained in the following manner：

2.1) virtual network of product is established, and energy function is established with the pixel point coordinates of product image；

2.2) by the way that similarity cost and smooth cost are the grid assignment in virtual network to form network flow；

2.3) energy function is optimized using the solution annual reporting law of max-flow/minimal cut problem, obtains the depth information of product surface.

4. product method for reconstructing three-dimensional model according to claim 3, it is characterized in that, described energy function be E (f)= ∑_p∈P[I_l(Tranl(x_p, y_p,f_p))-I_r(Tranr(x_p, y_p,f_p))]²+∑_(p,q)∈Nu_{{ p, q }}|f_p-f_q|, wherein：I_lAnd I_rFor production The picture element matrix of product image, (x_p,y_p) it is mesh point coordinate value in the plane of base, Tran is coordinate system transfer function, and f is pixel Point is the set of all pixels of the image of definition to the mapping relations between label, P, and p, q are picture single in pixel set P Vegetarian refreshments.

5. product method for reconstructing three-dimensional model according to claim 4, it is characterized in that, described max-flow/minimal cut is asked The solution annual reporting law of topic includes：Push-Relabel methods and Ford-Fulkerson methods.

6. product method for reconstructing three-dimensional model according to claim 3, it is characterized in that, described reconstruction, specifically include with Lower step：

3.1) bilateral filtering is carried out to depth information；

3.2) depth map is obtained according to depth information, carrying out back projection to depth map obtains the normal direction on vertex graph and each summit Amount；

3.3) product image pixel is transformed into by world coordinate system according to mapping matrix K；

3.4) establish cube and carry out voxel division, and update the TSDF of each volume elements；

3.5) rendered and projected according to TSDF, generate product threedimensional model.

A kind of 7. product reconstructing three-dimensional model system for realizing the above method, it is characterised in that including：Camera calibration module, depth Data obtaining module and model building module are spent, wherein：Camera calibration module gathers product image, obtains image slices vegetarian refreshments and reality Mapping matrix between the coordinate points of border scene；Depth Information Acquistion module obtains the depth information of product；Model building module Receive mapping matrix and depth information, rendered projection obtain the threedimensional model of product.