CN112669436A - Deep learning sample generation method based on 3D point cloud - Google Patents
Deep learning sample generation method based on 3D point cloud Download PDFInfo
- Publication number
- CN112669436A CN112669436A CN202011563204.8A CN202011563204A CN112669436A CN 112669436 A CN112669436 A CN 112669436A CN 202011563204 A CN202011563204 A CN 202011563204A CN 112669436 A CN112669436 A CN 112669436A
- Authority
- CN
- China
- Prior art keywords
- color
- point
- virtual camera
- point cloud
- pixel
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000005286 illumination Methods 0.000 claims abstract description 33
- 238000000034 method Methods 0.000 claims abstract description 33
- 238000012937 correction Methods 0.000 claims abstract description 27
- 238000013135 deep learning Methods 0.000 claims abstract description 25
- 239000011159 matrix material Substances 0.000 claims description 14
- 238000001514 detection method Methods 0.000 claims description 11
- 238000013519 translation Methods 0.000 claims description 7
- 238000004364 calculation method Methods 0.000 claims description 6
- 230000003287 optical effect Effects 0.000 claims description 5
- 235000002566 Capsicum Nutrition 0.000 claims description 3
- 239000006002 Pepper Substances 0.000 claims description 3
- 235000016761 Piper aduncum Nutrition 0.000 claims description 3
- 235000017804 Piper guineense Nutrition 0.000 claims description 3
- 235000008184 Piper nigrum Nutrition 0.000 claims description 3
- 150000003839 salts Chemical class 0.000 claims description 3
- 239000003086 colorant Substances 0.000 claims description 2
- 244000203593 Piper nigrum Species 0.000 claims 1
- 238000010276 construction Methods 0.000 abstract description 3
- 230000036544 posture Effects 0.000 description 9
- 238000013527 convolutional neural network Methods 0.000 description 3
- 241000722363 Piper Species 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000010606 normalization Methods 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a deep learning sample generation method based on 3D point cloud, which comprises the following steps: obtaining a color point cloud data set of a target to be detected; constructing a virtual camera model; setting the focal length and the posture of a virtual camera, inputting each point in a color point cloud data set into a virtual camera model to obtain a pixel coordinate for each point, and generating an image output by the virtual camera model for a target to be detected; calculating an illumination correction coefficient, superposing the illumination correction coefficient on color components of the color point cloud data set, and inputting points in the color point cloud data set after virtual illumination change into a virtual camera model one by one to obtain an image of the target to be detected after the virtual illumination change; adding noise to obtain a new image based on the obtained image; and repeating the steps until a preset number of images are obtained and used as deep learning samples of the target to be detected. The invention simplifies the construction difficulty of the sample and improves the richness of the sample.
Description
Technical Field
The application belongs to the field of deep learning artificial intelligence, and particularly relates to a deep learning sample generation method based on 3D point cloud.
Background
With the rapid development of deep learning and computer vision, both have been widely used in many fields. Object Detection (Object Detection), which is a more common application in the two combined applications, has become an important ring in image understanding, and the task of the Object Detection is to find all interested objects (objects) in an image, and determine the position and size of each Object, which is one of the core problems in the field of machine vision.
At present, in a target detection algorithm based on a convolutional neural network, weight parameters in the convolutional neural network need to be trained on a large number of data samples, and the more data is, the better the regularization effect of the trained parameters is. However, in reality, detection and identification of a specific target, such as image detection of professional equipment, image detection of a specific defect, and the like, are greatly limited due to the lack of a large number of data set references and the lack of sufficient data samples, and thus, the application of the convolutional neural network is greatly limited.
Disclosure of Invention
The application aims to provide a deep learning sample generation method based on a 3D point cloud, which simplifies the sample construction difficulty and improves the sample abundance.
In order to achieve the purpose, the technical scheme adopted by the application is as follows:
a deep learning sample generation method based on 3D point cloud is used for generating a deep learning sample of a target to be detected in target detection, and comprises the following steps:
and 6, repeating the steps 3-5 until a preset number of images are obtained and used as deep learning samples of the target to be detected.
Several alternatives are provided below, but not as an additional limitation to the above general solution, but merely as a further addition or preference, each alternative being combinable individually for the above general solution or among several alternatives without technical or logical contradictions.
Preferably, the scanning precision of the 3D scanner is not lower than 1 mm.
Preferably, the constructing the virtual camera model includes:
the virtual camera model is constructed as follows:
in the formula, ZcIs a proportionality constant, u, v are the pixel coordinates of the image, fx、fyIs the focal length of the virtual camera, u0、v0Is the optical center coordinate of the virtual camera, R is the rotation matrix of the virtual camera's pose relative to the world coordinate system, t is the translation matrix of the virtual camera's pose relative to the world coordinate system, xw、yw、zwThe position of a point in a world coordinate system;
wherein the rotation matrix R is defined by euler angles (α, β, γ) as follows:
in the formula, the angles of alpha, beta and gamma are 0 to 360;
wherein the translation matrix t can be expressed as:
in the formula, tx、ty、tzIs the coordinate of the virtual camera center in the world coordinate system.
Preferably, the inputting each point in the color point cloud data set into the virtual camera model to obtain a pixel coordinate for each point includes:
said colour point cloud data set being
And is
Wherein (x)i,yi,zi) Is the three-dimensional coordinate of point i, (r)i,gi,bi) Is the color component of point i, n is the number of points;
three-dimensional coordinates (x) of point ii,yi,zi) As (x)w,yw,zw) Inputting the pixel coordinate (u) corresponding to the point i into the virtual camera modeli,vi) And pixel coordinate (u)i,vi) Is the color component (r) of point ii,gi,bi)；
If the same pixel coordinate is obtained by corresponding a plurality of points in the color point cloud data set, calculating a plurality of points and the center (t) of the virtual camera respectivelyx,ty,tz) And (4) taking the color component corresponding to the point with the minimum distance as the color of the pixel coordinate.
Preferably, the generating an image output by the virtual camera model for the target to be detected includes:
acquiring pixel coordinates aiming at each point output by the virtual camera model and colors corresponding to the pixel coordinates to obtain an initial image;
traversing all pixel coordinates in the initial image, judging whether a pixel with the color of 0 exists or not, and if not, outputting the initial image as an image of a target to be detected; otherwise, color filling is carried out on the pixel with the color of 0, and the image after the color filling is finished is taken as the image of the target to be detected to be output.
Preferably, the color padding for the pixel with the color of 0 includes:
let the pixel coordinate of the pixel to be filled with color 0 be (u)k,vk)；
Taking the pixel coordinate (u)k,vk) Effective pixel θ { (u) within a rectangular region as a centerq,vq),-d≤uq-uk≤d,-d≤vq-vk≤d,(rq,gq,bq)≠0}，(uq,vq) Is the pixel coordinate of the effective pixel, (r)q,gq,bq) Is the color of the active pixel;
calculate the pixel coordinate as (u)k,vk) Is (r) as the fill colork,gk,bk)：
In the formula (d)qkIs an effective pixel (u)q,vq) And the pixel (u) to be filledk,vk) The distance between them.
Preferably, the calculating the illumination correction coefficient, and superimposing the illumination correction coefficient on the color component of the color point cloud data set to obtain the color point cloud data set after the virtual illumination is changed includes:
for each point (x) to be correctedi,yi,zi) The illumination correction factor is calculated as follows:
Id＝IKd(N·L)
in the formula IdAs the illumination correction factor, I is the external illumination (R)I,GI,BI)，RI、GI、BIIs between 0 and 1, KdIs a reflection coefficient (K)r,Kg,Kb)，Kr、Kg、KbThe value of (1) is between 0 and 1, N is a normalized normal vector of a point to be corrected, and L is a vector normalized by the center of the virtual camera;
the calculation process of the normalized normal vector N of the point to be corrected is as follows:
acquiring a point (x) to be correctedi,yi,zi) Carrying out plane fitting on 4 points closest to the periphery to obtain a plane AX + BY + CZ + D which is 0;
obtaining a normalized normal vector according to the plane obtained by fitting
The calculation process of the vector L normalized by the virtual camera center is as follows:
let the virtual camera center and the point (x) to be correctedi,yi,zi) A distance of dvThen vector normalized by virtual camera center
The calculated illumination correction coefficient IdWith the point (x) to be correctedi,yi,zi) Corresponding color component (r)i,gi,bi) Multiplying to obtain (I)dri,Idgi,Idbi) As a point (x)i,yi,zi) Color after virtual illumination change;
and (4) performing illumination correction on each point one by one to obtain a color point cloud data set after virtual illumination change.
Preferably, the noise is gaussian noise and/or salt and pepper noise.
According to the 3D point cloud-based deep learning sample generation method, a data set picture is automatically generated by 3D point cloud scanning and a computer vision geometric model method, and the richness of a sample is improved by changing the shooting posture of a virtual camera, the change of virtual illumination and adding random noise; the virtual modeling mode is utilized to save the process of actually acquiring and processing the image, thereby greatly simplifying the difficulty of sample construction.
Drawings
Fig. 1 is a flowchart of a 3D point cloud-based deep learning sample generation method according to the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the description of the present application herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application.
In order to solve the problems of high sample generation difficulty, high cost and long time consumption in the prior art, the embodiment provides the deep learning sample generation method based on the 3D point cloud, which is used for generating the deep learning sample of the target to be detected in the target detection through the 3D point cloud, so that the sample generation difficulty is effectively reduced, and the richness of the sample is improved.
As shown in fig. 1, the method for generating a deep learning sample based on a 3D point cloud in this embodiment includes the following steps:
In order to ensure that the 3D point cloud can restore the form of the real target to a large extent, the scanning accuracy of the 3D scanner is set to be not less than 1mm in this embodiment. The color point cloud data obtained by scanning are
And is
Wherein (x)i,yi,zi) Is the three-dimensional coordinate of point i, (r)i,gi,bi) Is the color component of point i and n is the number of points.
in the formula, ZcIs a proportionality constant, u, v are the pixel coordinates of the image, fx、fyIs the focal length of the virtual camera, u0、v0Is the optical center coordinate of the virtual camera, R is the rotation matrix of the virtual camera's pose relative to the world coordinate system, t is the translation matrix of the virtual camera's pose relative to the world coordinate system, xw、yw、zwIs the position of a point in the world coordinate system.
Wherein the rotation matrix R is defined by euler angles (α, β, γ) as follows:
in the formula, the angles of alpha, beta and gamma are 0 to 360.
Wherein the translation matrix t can be expressed as:
in the formula, tx、ty、tzIs the coordinate of the virtual camera center in the world coordinate system.
And 3, setting the focal length and the posture of the virtual camera, inputting each point in the color point cloud data set into the virtual camera model to obtain a pixel coordinate for each point, and generating an image output by the virtual camera model for the target to be detected.
Different images can be obtained for the same target under different focal lengths and postures of the camera, so that the target image under multiple angles is obtained through the posture of the focal length of the virtual camera, and samples are enriched.
Wherein the focal length f of the virtual camerax、fyThe value can be 2000, and of course, any focal length value can be set according to needs. Optical center coordinate u of virtual camera0、v0Depending on the pixel size of the sample picture to be generated, for example, a picture of 600 × 600 pixels is to be generated, the optical center coordinates are (300 ).
The rotation matrix R can be obtained by taking (alpha, beta, gamma) at random, and the translation matrix t can be obtained by taking t at randomx、ty、tzAnd (4) obtaining.
After the focal length and the posture of the virtual camera are set, an image can be generated through the 3D point cloud, and the generation of a sample of a target to be detected is achieved. The process of generating an image in one embodiment is as follows:
three-dimensional coordinates (x) of point ii,yi,zi) As (x)w,yw,zw) Inputting the pixel coordinate (u) corresponding to the point i into the virtual camera modeli,vi) And pixel coordinate (u)i,vi) Is the color component (r) of point ii,gi,bi)。
If the same pixel coordinate is obtained by corresponding a plurality of points in the color point cloud data set, calculating a plurality of points and the center (t) of the virtual camera respectivelyx,ty,tz) And (4) taking the color component corresponding to the point with the minimum distance as the color of the pixel coordinate.
And traversing each point in the color point cloud data set, acquiring the pixel coordinate of each point output by the virtual camera model and the color corresponding to each pixel coordinate, and obtaining the image of the target to be detected at the moment as an initial image.
The method is limited by the scanning precision and environmental interference of a 3D scanner, so that points in a color point cloud data set may not completely reflect a target to be detected, namely, a currently obtained initial image may have a hole to be filled, and therefore hole filling detection needs to be performed on the initial image:
traversing all pixel coordinates in the initial image, judging whether a pixel (a hole pixel) with the color of 0 exists or not, and if not, outputting the initial image as an image of the target to be detected; otherwise, color filling is carried out on the pixel with the color of 0, and the image after the color filling is finished is taken as the image of the target to be detected to be output.
In the color filling, the color of any adjacent pixel of the hole pixel may be used as the color of the hole pixel, or the color of a pixel at a distance and in a direction from the hole pixel may be used as the color of the hole pixel. In order to improve the effectiveness of color filling and avoid color overshoot, in one embodiment, a color filling method is provided as follows:
let the pixel coordinate of the pixel to be filled with color 0 be (u)k,vk)。
Taking the pixel coordinate (u)k,vk) Effective pixel θ { (u) within a rectangular region as a centerq,vq),-d≤uq-uk≤d,-d≤vq-vk≤d,(rq,gq,bq)≠0}，(uq,vq) Is the pixel coordinate of the effective pixel, (r)q,gq,bq) Is the color of the active pixel; typically d has a value of 5.
Calculate the pixel coordinate as (u)k,vk) Is (r) as the fill colork,gk,bk)：
In the formula (d)qkIs an effective pixel (u)q,vq) And the pixel (u) to be filledk,vk) Is a distance therebetween, i.e.
And 4, calculating an illumination correction coefficient, superposing the illumination correction coefficient on the color components of the color point cloud data set to obtain a color point cloud data set after virtual illumination change, and inputting points in the color point cloud data set after the virtual illumination change into the virtual camera model one by one to obtain an image of the target to be detected after the virtual illumination change.
Considering that when a sample of a target to be detected is obtained in an actual environment, the ambient light can affect the imaging color of the target to be detected to a greater extent, so that the illumination correction coefficient is introduced in the embodiment, and the change of the ambient light is simulated by using the illumination correction coefficient, so that a richer sample is obtained.
Compared with the method for obtaining the sample in the actual environment, the method for obtaining the sample in the actual environment can simulate the illumination of various environments through the illumination correction coefficient, does not need to match the illumination condition in the actual environment, and can obtain more samples in the illumination environment more simply, conveniently and quickly.
In one embodiment, obtaining the image of the target to be detected after the virtual illumination is changed includes the following steps:
for each point (x) to be correctedi,yi,zi) The illumination correction factor is calculated as follows:
Id＝IKd(N·L)
in the formula IdAs the illumination correction factor, I is the external illumination (R)I,GI,BI)，RI、GI、BIIs between 0 and 1, KdIs a reflection coefficient (K)r,Kg,Kb)，Kr、Kg、KbThe value of (a) is between 0 and 1, N is a normalized normal vector of a point to be corrected, and L is a vector normalized by the center (observation position) of the virtual camera.
The calculation process of the normalized normal vector N of the point to be corrected is as follows:
acquiring a point (x) to be correctedi,yi,zi) And performing plane fitting on the nearest 4 points around the plane to obtain a plane AX + BY + CZ + D which is 0.
Obtaining a normalized normal vector according to the plane obtained by fitting
The calculation process of the vector L of the observation position normalization is as follows:
let the observed position and the point (x) to be correctedi,yi,zi) A distance of dvThen observe the vector of the position normalization
The calculated illumination correction coefficient IdWith the point (x) to be correctedi,yi,zi) Corresponding color component (r)i,gi,bi) Multiplying to obtain (I)dri,Idgi,Idbi) As a point (x)i,yi,zi) Color after change in virtual lighting.
And (4) performing illumination correction on each point one by one to obtain a color point cloud data set after virtual illumination change.
And after the color point cloud data set with the changed virtual illumination is obtained, inputting the color point cloud data set with the changed virtual illumination into the virtual camera model to obtain an image with the changed virtual illumination. It should be noted that the process of inputting the color point cloud data set after the virtual illumination change into the virtual camera model to obtain the image is the same as the process of directly inputting the color point cloud data set into the virtual camera model to obtain the image, and the focal length and the posture which are the same as those set in step 3 can be adopted, and the focal length and the posture can also be reset, which is not described herein again.
And 5, adding noise to obtain a new image based on the images obtained in the steps 3 and 4. The noise can be gaussian noise and/or salt and pepper noise to expand the sample and make the sample more consistent with the image collected in the actual environment.
And 6, repeating the steps 3-5 until a preset number of images are obtained and used as deep learning samples of the target to be detected.
According to the embodiment, the focal length and the posture of the virtual camera are continuously changed, the virtual illumination environment is obtained, abundant and real samples can be obtained based on point cloud data of the target to be detected, and the sample obtaining difficulty and cost are greatly simplified.
It should be noted that, according to actual needs, operations such as image cropping and scaling may be performed on the obtained image, for example, using a findContour function of opencv to obtain an image contour, using a boundingRect function to obtain a minimum bounding rectangle of the contour, and cropping the image so as to conform to the size of the minimum bounding rectangle. And then scaling the image according to the sample picture size requirement.
The technical features of the embodiments described above may be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the embodiments described above are not described, but should be considered as being within the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (8)
1. A deep learning sample generation method based on 3D point cloud is used for generating a deep learning sample of a target to be detected in target detection, and is characterized in that the deep learning sample generation method based on the 3D point cloud comprises the following steps:
step 1, scanning a target to be detected by using a 3D scanner to obtain a color point cloud data set of the target to be detected, wherein each point in the color point cloud data set has a three-dimensional coordinate and a color component;
step 2, constructing a virtual camera model;
step 3, setting the focal length and the posture of a virtual camera, inputting each point in the color point cloud data set into the virtual camera model to obtain a pixel coordinate for each point, and generating an image output by the virtual camera model for a target to be detected;
step 4, calculating an illumination correction coefficient, superposing the illumination correction coefficient on the color components of the color point cloud data set to obtain a color point cloud data set after virtual illumination change, and inputting points in the color point cloud data set after virtual illumination change into the virtual camera model one by one to obtain an image of the target to be detected after virtual illumination change;
step 5, adding noise to obtain a new image based on the images obtained in the step 3 and the step 4;
and 6, repeating the steps 3-5 until a preset number of images are obtained and used as deep learning samples of the target to be detected.
2. The method for generating 3D point cloud-based deep learning samples according to claim 1, wherein the scanning accuracy of the 3D scanner is not less than 1 mm.
3. The method for generating 3D point cloud-based deep learning samples according to claim 1, wherein the constructing a virtual camera model comprises:
the virtual camera model is constructed as follows:
in the formula, ZcIs a proportionality constant, u, v are the pixel coordinates of the image, fx、fyIs the focal length of the virtual camera, u0、v0Is the optical center coordinate of the virtual camera, R is the rotation matrix of the virtual camera's pose relative to the world coordinate system, t is the translation matrix of the virtual camera's pose relative to the world coordinate system, xw、yw、zwThe position of a point in a world coordinate system;
wherein the rotation matrix R is defined by euler angles (α, β, γ) as follows:
in the formula, the angles of alpha, beta and gamma are 0 to 360;
wherein the translation matrix t can be expressed as:
in the formula, tx、ty、tzIs the coordinate of the virtual camera center in the world coordinate system.
4. The method of claim 1, wherein the inputting each point in the color point cloud data set into the virtual camera model to obtain pixel coordinates for each point comprises:
said colour point cloud data set being
And is
Wherein (x)i,yi,zi) Is the three-dimensional coordinate of point i, (r)i,gi,bi) Is the color component of point i, n is the number of points;
three-dimensional coordinates (x) of point ii,yi,zi) As (x)w,yw,zw) Inputting the pixel coordinate (u) corresponding to the point i into the virtual camera modeli,vi) And pixel coordinate (u)i,vi) Is the color component (r) of point ii,gi,bi)；
If the same pixel coordinate is obtained by corresponding a plurality of points in the color point cloud data set, calculating a plurality of points and the center (t) of the virtual camera respectivelyx,ty,tz) And (4) taking the color component corresponding to the point with the minimum distance as the color of the pixel coordinate.
5. The method for generating the deep learning sample based on the 3D point cloud according to claim 4, wherein the generating the image output by the virtual camera model for the target to be detected comprises:
acquiring pixel coordinates aiming at each point output by the virtual camera model and colors corresponding to the pixel coordinates to obtain an initial image;
traversing all pixel coordinates in the initial image, judging whether a pixel with the color of 0 exists or not, and if not, outputting the initial image as an image of a target to be detected; otherwise, color filling is carried out on the pixel with the color of 0, and the image after the color filling is finished is taken as the image of the target to be detected to be output.
6. The method for generating deep learning samples based on 3D point cloud according to claim 5, wherein the color filling of the pixel with color 0 comprises:
let the pixel coordinate of the pixel to be filled with color 0 be (u)k,vk)；
Taking the pixel coordinate (u)k,vk) Effective pixel θ { (u) within a rectangular region as a centerq,vq),-d≤uq-uk≤d,-d≤vq-vk≤d,(rq,gq,bq)≠0}，(uq,vq) Is the pixel coordinate of the effective pixel, (r)q,gq,bq) Is the color of the active pixel;
calculate the pixel coordinate as (u)k,vk) Is (r) as the fill colork,gk,bk)：
In the formula (d)qkIs an effective pixel (u)q,vq) And the pixel (u) to be filledk,vk) The distance between them.
7. The method of claim 1, wherein the calculating an illumination correction coefficient and superimposing the illumination correction coefficient on color components of the color point cloud data set to obtain a color point cloud data set after virtual illumination change comprises:
for each point (x) to be correctedi,yi,zi) The illumination correction factor is calculated as follows:
Id＝IKd(N·L)
in the formula IdAs the illumination correction factor, I is the external illumination (R)I,GI,BI)，RI、GI、BIIs between 0 and 1, KdIs a coefficient of reflection(Kr,Kg,Kb)，Kr、Kg、KbThe value of (1) is between 0 and 1, N is a normalized normal vector of a point to be corrected, and L is a vector normalized by the center of the virtual camera;
the calculation process of the normalized normal vector N of the point to be corrected is as follows:
acquiring a point (x) to be correctedi,yi,zi) Carrying out plane fitting on 4 points closest to the periphery to obtain a plane AX + BY + CZ + D which is 0;
obtaining a normalized normal vector according to the plane obtained by fitting
The calculation process of the vector L normalized by the virtual camera center is as follows:
let the virtual camera center and the point (x) to be correctedi,yi,zi) A distance of dvThen vector normalized by virtual camera center
The calculated illumination correction coefficient IdWith the point (x) to be correctedi,yi,zi) Corresponding color component (r)i,gi,bi) Multiplying to obtain (I)dri,Idgi,Idbi) As a point (x)i,yi,zi) Color after virtual illumination change;
and (4) performing illumination correction on each point one by one to obtain a color point cloud data set after virtual illumination change.
8. The method of generating deep learning samples based on 3D point cloud of claim 1, wherein the noise is gaussian noise and/or salt and pepper noise.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011563204.8A CN112669436B (en) | 2020-12-25 | 2020-12-25 | Deep learning sample generation method based on 3D point cloud |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011563204.8A CN112669436B (en) | 2020-12-25 | 2020-12-25 | Deep learning sample generation method based on 3D point cloud |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112669436A true CN112669436A (en) | 2021-04-16 |
| CN112669436B CN112669436B (en) | 2024-10-18 |
Family
ID=75409199
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011563204.8A Active CN112669436B (en) | 2020-12-25 | 2020-12-25 | Deep learning sample generation method based on 3D point cloud |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112669436B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113237896A (en) * | 2021-06-08 | 2021-08-10 | 诚丰家具有限公司 | Furniture board dynamic monitoring system and method based on light source scanning |
| CN113333998A (en) * | 2021-05-25 | 2021-09-03 | 绍兴市上虞区武汉理工大学高等研究院 | Automatic welding system and method based on cooperative robot |
| CN115100360A (en) * | 2022-07-28 | 2022-09-23 | 中国电信股份有限公司 | Image generation method and device, storage medium and electronic equipment |
| CN116758124A (en) * | 2023-06-16 | 2023-09-15 | 北京代码空间科技有限公司 | 3D model correction method and terminal equipment |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109635843A (en) * | 2018-11-14 | 2019-04-16 | 浙江工业大学 | A kind of three-dimensional object model classification method based on multi-view image |
| CN110853075A (en) * | 2019-11-05 | 2020-02-28 | 北京理工大学 | A visual tracking and localization method based on dense point cloud and synthetic view |
| CN111476882A (en) * | 2020-03-26 | 2020-07-31 | 哈尔滨工业大学 | A Browser-Oriented Robot Virtual Graphic Modeling Method |
-
2020
- 2020-12-25 CN CN202011563204.8A patent/CN112669436B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109635843A (en) * | 2018-11-14 | 2019-04-16 | 浙江工业大学 | A kind of three-dimensional object model classification method based on multi-view image |
| CN110853075A (en) * | 2019-11-05 | 2020-02-28 | 北京理工大学 | A visual tracking and localization method based on dense point cloud and synthetic view |
| CN111476882A (en) * | 2020-03-26 | 2020-07-31 | 哈尔滨工业大学 | A Browser-Oriented Robot Virtual Graphic Modeling Method |
Non-Patent Citations (2)
| Title |
|---|
| KEVIN LESNIAK .ETC: "Dynamic Rendering of Remote Indoor Environments Using Real-Time Point Cloud Data", 《JOURNAL OF COMPUTING AND INFORMATION SCIENCE IN ENGINEERING》, vol. 18, no. 3, 12 June 2018 (2018-06-12), pages 1 - 25 * |
| 李勇 等: "三维点云场景数据获取机器场景理解关键技术综述", 《激光与光电子学进展》, vol. 56, no. 4, 21 December 2018 (2018-12-21), pages 21 - 34 * |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113333998A (en) * | 2021-05-25 | 2021-09-03 | 绍兴市上虞区武汉理工大学高等研究院 | Automatic welding system and method based on cooperative robot |
| CN113237896A (en) * | 2021-06-08 | 2021-08-10 | 诚丰家具有限公司 | Furniture board dynamic monitoring system and method based on light source scanning |
| CN113237896B (en) * | 2021-06-08 | 2024-02-20 | 诚丰家具有限公司 | Furniture board dynamic monitoring system and method based on light source scanning |
| CN115100360A (en) * | 2022-07-28 | 2022-09-23 | 中国电信股份有限公司 | Image generation method and device, storage medium and electronic equipment |
| CN115100360B (en) * | 2022-07-28 | 2023-12-01 | 中国电信股份有限公司 | Image generation method and device, storage medium and electronic equipment |
| CN116758124A (en) * | 2023-06-16 | 2023-09-15 | 北京代码空间科技有限公司 | 3D model correction method and terminal equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112669436B (en) | 2024-10-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109461180B (en) | Three-dimensional scene reconstruction method based on deep learning | |
| US20210004973A1 (en) | Image processing method, apparatus, and storage medium | |
| CN112669436B (en) | Deep learning sample generation method based on 3D point cloud | |
| WO2022095721A1 (en) | Parameter estimation model training method and apparatus, and device and storage medium | |
| CN107945267B (en) | Method and equipment for fusing textures of three-dimensional model of human face | |
| CN113345063B (en) | PBR three-dimensional reconstruction method, system and computer storage medium based on deep learning | |
| CN109697688A (en) | A kind of method and apparatus for image procossing | |
| CN112784621B (en) | Image display method and device | |
| CN110648274B (en) | Method and device for generating fisheye image | |
| CN109636890B (en) | Texture fusion method and device, electronic equipment, storage medium and product | |
| CN109711472B (en) | Training data generation method and device | |
| CN108509887A (en) | A kind of acquisition ambient lighting information approach, device and electronic equipment | |
| CN112802208B (en) | Three-dimensional visualization method and device in terminal building | |
| US10169891B2 (en) | Producing three-dimensional representation based on images of a person | |
| CN113723317A (en) | Reconstruction method and device of 3D face, electronic equipment and storage medium | |
| CN110458932A (en) | Image processing method, device, system, storage medium and image scanning apparatus | |
| CN108364292A (en) | A kind of illumination estimation method based on several multi-view images | |
| CN113205560A (en) | Calibration method, device and equipment of multi-depth camera and storage medium | |
| CN116681839B (en) | Live three-dimensional target reconstruction and singulation method based on improved NeRF | |
| CN111382618B (en) | Illumination detection method, device, equipment and storage medium for face image | |
| CN118736092A (en) | A method and system for rendering virtual human at any viewing angle based on three-dimensional Gaussian splashing | |
| Schneider et al. | Efficient global illumination for morphable models | |
| CN109785429A (en) | A kind of method and apparatus of three-dimensional reconstruction | |
| WO2022217470A1 (en) | Hair rendering system based on deep neural network | |
| Malleson et al. | Single-view RGBD-based reconstruction of dynamic human geometry |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |