CN106548509A - A kind of 3-dimensional image generation method based on CUDA and three-dimensional imaging load - Google Patents

Abstract

The invention discloses a three-dimensional image generation method, which is used for rapidly merging point cloud data and image data acquired by three-dimensional imaging loads to generate a three-dimensional image. Including: the initialization of the task by the host, including the allocation of memory and video memory, reading of data, etc.; and then divide the task into two parts, serial and parallel. The correspondence of the image and the fusion output of the final three-dimensional coordinate information and texture information, the parallel part is mainly to interpolate the vertical flight direction elevation value, along the flight direction elevation value interpolation and collinear equation solution plane with high parallelism and intensive calculation The coordinates are implemented based on the CUDA parallel computing architecture. After the calculation is completed, the results are mapped back to the host-side memory for fusion output.

Description

A 3D Image Generation Method Based on CUDA and 3D Imaging Load

technical field

The invention relates to an image generation method, in particular to a three-dimensional image generation method based on CUDA and three-dimensional imaging load.

Background technique

Airborne lidar can quickly obtain the three-dimensional point coordinates of ground objects and return the intensity of the reflected signal of the ground objects, but it is difficult to obtain the texture information of the ground objects, which brings great challenges to the processing and understanding of airborne laser point cloud data. difficulty. In order to solve this problem, LiDAR system and CCD imaging system are often installed on the same platform at home and abroad, and stereoscopic imaging is realized based on point cloud and image registration. However, this method has disadvantages such as complex data processing, long operation cycle, and low degree of automation, which restrict the direct registration speed of laser point cloud and image.

In order to quickly realize the fusion of point cloud and image to generate a three-dimensional image of the target area, an active and passive three-dimensional imaging payload integrating a lidar system and a CCD camera came into being. The 3D imaging payload is a new type of payload that can collect lidar data and CCD images at the same time. By using a common optical path optical system to integrate the linear array lidar and the linear array CCD camera, and based on the synchronization control unit, the synchronization between the lidar array and the corresponding CCD array is guaranteed. Simultaneous scanning and imaging, based on the front-end alignment relationship, the rapid fusion of laser point cloud data and CCD images is realized.

CUDA (Compute Unified Device Architecture, CUDA for short) is a computing unified device architecture proposed by NVIDIA in 2007. CUDA is a development environment for GPU computing. It is a brand-new software and hardware architecture. GPU can be regarded as a device for parallel data computing, and the calculations performed are allocated and managed. CUDA-based parallel algorithms have been widely used in image processing, terrain rendering, hydrological models and other fields, and have achieved high speed-up ratios in most applications. Although the current fast fusion processing method of 3D imaging load generates a 3D image of the target area, the entire processing flow usually adopts a serial processing method, which has a huge amount of calculation and low operating efficiency, and does not make full use of the laser linear array push-broom imaging method. The obtained point cloud data is distributed in regular ranks and columns, the index method is simple, and the processing process has the characteristics of high parallelism. It is necessary to explore a parallel computing method based on CUDA suitable for fast fusion processing of 3D imaging loads. Improve the generation speed of 3D images to meet the needs of real-time processing.

Contents of the invention

(1) Technical problems to be solved

Based on the CUDA framework, the present invention aims at the defect of low efficiency of fast fusion operation in the prior art, and provides a parallel processing method capable of rapidly merging 3D imaging load point clouds and images to generate 3D images in real time, further improving the processing of the prior art Speed, in order to achieve the purpose of generating 3D images in real time. This method is also applicable to the data processing of other line array laser radars.

(2) Technical solution

The present invention provides a three-dimensional image generation method, which is used to quickly fuse point cloud data acquired by a three-dimensional imaging payload with image data to generate a three-dimensional image, and the three-dimensional imaging payload uses a laser linear array push-broom imaging method to obtain data distributed in regular rows and columns , characterized by including:

The host end processes the data distributed in regular rows and columns;

The host side applies for the memory space of the device side, and copies the processed data distributed in regular rows and columns to the device side memory;

The device side performs interpolation calculation of elevation values and collinear equations to solve plane coordinates for multiple rows of data at the same time, so as to obtain three-dimensional coordinate results;

The device side maps the completed result back to the host side memory;

The host side performs fusion output to generate a 3D image.

In the above solution, the simultaneous calculation of multiple sets of data by the device side is implemented based on the CUDA parallel computing architecture.

In the above solution, the host side processes the data distributed in regular rows and columns, including:

The host side performs leak-trapping processing on the point cloud data in the data distributed in regular rows and columns;

The host side groups and processes each row of data with regular row and column distribution.

In the above solution, before the host side processes the data distributed in regular rows and columns, it also includes:

The host end reads in the regularly distributed image data and point cloud data, POS data, and the correspondence table between laser detectors and CCD pixels acquired by the 3D imaging payload.

The above scheme also includes:

The host side copies the POS data to the memory of the device side.

In the above solution, the POS data includes instantaneous position data and attitude data when the image is imaged.

In the above scheme, the host end performs fusion output to generate a three-dimensional image, and the host end outputs the three-dimensional coordinate results point by point according to the correspondence table between the laser probe and the CCD pixel, and the R, R, and G, B pixel values to complete the generation of three-dimensional images.

In the above solution, the device end is a GPU.

In the above solution, the interpolation calculation of the elevation value includes the interpolation calculation of the elevation vertically to the flight direction and the interpolation calculation of the elevation along the flight direction.

In the above scheme, the collinear equation is:

In the formula, X _A , Y _A are the geodetic plane coordinates corresponding to the CCD pixel to be solved, (X _S , Y _S , Z _S ) is the position of the optical center of the CCD in the geodetic coordinate system during imaging, a ₁ , a ₂ , a ₃ , b ₁ , b ₂ , b ₃ , c ₁ , c ₂ , c ₃ are the rotation matrix elements of the CCD optical center relative to the earth coordinate system at the imaging moment, x, y are the image coordinates of the CCD pixel, and f is The focal length of the camera.

(3) Beneficial effects

The present invention has the following beneficial effects:

1. Based on the CUDA parallel computing architecture, the present invention designs a parallel processing method that can complete fast fusion processing of 3D imaging loads in real time to generate 3D images, and solves the problem of low computing efficiency in the prior art.

2. The present invention utilizes GPU and CPU to process linear array laser point cloud data in cooperation, and provides a feasible solution for fast processing of large-scale linear array laser point cloud data.

Description of drawings

Fig. 1 schematically shows a flowchart of a method for generating a 3D image according to an embodiment of the present invention.

detailed description

In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with specific embodiments and with reference to the accompanying drawings.

The present invention provides a three-dimensional image generation method, and provides a parallel computing method for the current low efficiency of serial processing of three-dimensional imaging loads. The parallel computing method in the embodiment of the present invention adopts the CUDA parallel computing architecture, and also A CPU multi-core parallel approach can be used instead.

The three-dimensional image generation method mentioned in the present invention, based on the three-dimensional imaging load point, provides a parallel processing method that can quickly fuse the three-dimensional imaging load point cloud and image in real time to generate a three-dimensional image, and further improve the processing speed of the existing technology to achieve The purpose of generating 3D images in real time. This method is also applicable to data processing acquired by other types of linear array lidar payloads.

The interpolation of the elevation value mentioned in the present invention may be interpolation methods such as nearest neighbor interpolation, linear interpolation, and spline interpolation.

Fig. 1 schematically shows a flowchart of a method for generating a 3D image according to an embodiment of the present invention.

As shown in the figure, in step S1, the host-side CPU performs task initialization. According to the embodiment of the present invention, a certain space is sorted out from the CPU memory of the host computer, the part of the memory is initialized, and the memory and video memory are allocated.

In step S2, the host-side CPU reads data. According to the embodiment of the present invention, the host-side CPU reads the laser point cloud data and image data, calibration parameters, POS data, and the correspondence table between the laser probe and the CCD pixel acquired by the three-dimensional imaging payload. Among them, the POS data includes the instantaneous position data and attitude data (X _S , Y _S , Z _S , Roll, Pitch, Heading) during image imaging, and the calibration parameters include the correspondence table between the laser radar detector and the CCD pixel and The inner orientation element of the image. The 3D imaging payload uses the laser linear array push-broom imaging method to obtain point cloud data and image data, and the data is distributed in regular rows and columns, that is, every time an image is taken, a corresponding row of data is obtained.

In step S3, the host-side CPU fills in the empty and defective cloud data. According to the embodiment of the present invention, due to the influence of the reflectivity of ground objects and equipment failures, the phenomenon of laser leakage and element leakage will occur. Point cloud data, interpolating the elevation value of the missing points in the row direction in the laser grid point cloud data.

In step S4, the CPU on the host side corresponds and groups the data. According to the embodiment of the present invention, the CPU at the host end divides the data into two groups with laser corresponding CCD numbers of 34 and 35 according to the correspondence table between laser detectors and CCD pixels, so as to facilitate subsequent development of parallel threads.

In step S5, the host-side CPU applies for device-side GPU memory space, and copies the data into the memory. According to the embodiment of the present invention, the CPU at the host end applies for the memory space of the GPU at the device end, and the elevation coordinate Z, image data, and position data (X _s , Y _s , Z _s ) and attitude angle data (Roll, Pitch, Heading) are copied to the memory of the GPU.

In step S6, device-side GPU resources are initialized. According to the embodiment of the present invention, the device-side GPU sorts out a certain memory space, and initializes the space, and prepares to store the contents copied by the host-side.

In step S7, the device-side GPU performs vertical interpolation calculation on the data in the direction of flight. According to the embodiment of the present invention, the interpolation method is linear interpolation, the interpolation in the vertical flight direction is divided into two groups, the number of laser lines is the number of block thread blocks, and considering the calculation optimization and combined access conditions provided by the CUDA architecture, thread The number of threads is preferably a multiple of 32, so set the number of threads to 64, and start CUDA multi-threading to interpolate the vertical flight direction of the elevation Z value.

In step S8, the device-side GPU performs elevation interpolation calculation along the flight direction on the data. According to the embodiment of the present invention, the method of interpolation is linear interpolation, and the elevation Z value interpolation along the flight direction is performed, the number of laser points in each row is the number of Block thread blocks, and the scanning magnification 64 in the time of laser and CCD is the number of threads , start CUDA multithreading to complete the interpolation.

In step S9, the device-side GPU performs a collinear equation solution on the data to calculate the plane coordinates. According to the embodiment of the present invention, based on the image point coordinates of the CCD image point and the elevation value of the corresponding ground point, the corresponding position and attitude information, combined with the orientation elements in the image, its X and Y coordinates are calculated according to the collinear conditional equation, and the solution is The formula is as follows:

In the formula, X _A , Y _A are the geodetic plane coordinates corresponding to the CCD pixel to be solved, (X _S , Y _S , Z _S ) is the position of the optical center of the CCD in the geodetic coordinate system during imaging, a ₁ , a ₂ , a ₃ , b ₁ , b ₂ , b ₃ , c ₁ , c ₂ , c ₃ are the rotation matrix elements of the CCD optical center relative to the earth coordinate system at the imaging moment, x, y are the image coordinates of the CCD pixel, and f is The focal length of the camera.

Start the CUDA multi-thread with the number of image lines as the number of Block threads and the number of CCD pixels as the number of Thread threads, read the position and posture information into the shared memory to speed up the memory access speed during parallelism, and obtain the plane coordinate results and results of the calculation. Elevation coordinate value.

In step S10, the device-side GPU maps the result back to the host-side CPU. According to the embodiment of the present invention, the device side maps the obtained three-dimensional coordinate result back to the memory of the host side.

In step S11, the host generates a 3D image. According to the embodiment of the present invention, the host terminal outputs the three-dimensional coordinates and the R, G, and B pixel values of the corresponding CCD pixel point by point according to the calculated three-dimensional coordinate results and according to the corresponding relationship table between the laser probe and the CCD pixel , to complete the generation of the 3D image.

In step S12, the host terminal outputs the 3D image result.

The specific embodiments described above have further described the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above descriptions are only specific embodiments of the present invention, and are not intended to limit the present invention. Within the spirit and principles of the present invention, any modifications, equivalent replacements, improvements, etc., shall be included in the protection scope of the present invention.

Claims (10)

1. a kind of 3-dimensional image generation method, for the cloud data and the image data rapid fusion that obtain three-dimensional imaging load 3-dimensional image is generated, the three-dimensional imaging load obtains the data of regular ranks distribution using laser linear array push-scanning image mode, Characterized in that, including：

Host side is processed to the data of the regular ranks distribution；

Host side application equipment end memory headroom, by the data copy of the regular ranks distribution for processing into equipment end memory；

Equipment end carries out the calculating of elevation value interpolation simultaneously to multirow data and collinearity equation resolves plane coordinates, to obtain three-dimensional Coordinate result；

Result of the equipment end by after the completion of maps back main frame end memory；

Host side carries out fusion output, generates 3-dimensional image.

2. 3-dimensional image generation method according to claim 1, it is characterised in that the equipment end is simultaneously to multi-group data Carrying out calculating is realized based on CUDA parallel computations framework.

3. 3-dimensional image generation method according to claim 1, it is characterised in that the host side is to the regular ranks The data of distribution are processed, including：

Cloud data in the data that host side is distributed to regular ranks carries out mending-leakage process；

Each row of the data that host side is distributed to regular ranks carries out packet transaction.

4. 3-dimensional image generation method according to claim 1, it is characterised in that the host side is to the regular ranks The data of distribution carry out before processing, also include：

Host side read in the image data that the regular ranks that three-dimensional imaging load obtains are distributed and cloud data, POS data and Laser visits unit's table corresponding with CCD pixels.

5. 3-dimensional image generation method according to claim 1, it is characterised in that also include：

Host side is copied to the POS data in equipment end memory.

6. 3-dimensional image generation method according to claim 5, it is characterised in that the POS data includes video imaging When instantaneous position data and attitude data.

7. 3-dimensional image generation method according to claim 1, it is characterised in that the host side carries out fusion output life Into 3-dimensional image be host side by the three-dimensional coordinate result, visit unit's table corresponding with CCD pixels according to the laser, pointwise is exported R, G, B pixel value of three-dimensional coordinate and correspondence CCD pixels, completes the generation of 3-dimensional image.

8. 3-dimensional image generation method according to claim 1, it is characterised in that the equipment end is GPU.

9. 3-dimensional image generation method according to claim 1, it is characterised in that the elevation value interpolation is calculated including hanging down Fly nonstop to line direction elevation interpolation to calculate and calculate along heading elevation interpolation.

10. 3-dimensional image generation method according to claim 1, it is characterised in that the collinearity equation is：

$\left\{\begin{array}{c}{X}_A=(Z_A-Z_S)\frac{a_{1}x+a_{2}y-a_{3}f}{c_{1}x+c_{2}y-c_{3}f}+X_S\\ Y_A=(Z_A-Z_S)\frac{b_{1}x+b_{2}y-b_{3}f}{c_{1}x+c_{2}y-c_{3}f}+Y_S\end{array}\right.$

In formula, X_A, Y_AFor the corresponding the earth plane coordinates of demand solution CCD pixel, (X_S, Y_S, Z_S) be imaging when CCD optical centre Position under earth coordinates, a₁, a₂, a₃, b₁, b₂, b₃, c₁, c₂, c₃Sit relative to the earth for imaging moment CCD optical centres The spin matrix element of mark system, the image coordinate of x, y for CCD pixels, focal lengths of the f for camera.