CN115511718A - PCB image correction method and device, terminal equipment and storage medium - Google Patents
PCB image correction method and device, terminal equipment and storage medium Download PDFInfo
- Publication number
- CN115511718A CN115511718A CN202110631938.3A CN202110631938A CN115511718A CN 115511718 A CN115511718 A CN 115511718A CN 202110631938 A CN202110631938 A CN 202110631938A CN 115511718 A CN115511718 A CN 115511718A
- Authority
- CN
- China
- Prior art keywords
- depth
- dimensional
- map
- substrate area
- depth map
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 57
- 238000003702 image correction Methods 0.000 title claims abstract description 37
- 238000003860 storage Methods 0.000 title claims abstract description 20
- 239000000758 substrate Substances 0.000 claims abstract description 117
- 238000003062 neural network model Methods 0.000 claims description 25
- 238000012937 correction Methods 0.000 claims description 24
- 238000012549 training Methods 0.000 claims description 13
- 238000007781 pre-processing Methods 0.000 claims description 12
- 238000004590 computer program Methods 0.000 claims description 10
- 238000012790 confirmation Methods 0.000 claims description 4
- 230000009466 transformation Effects 0.000 claims description 3
- 238000005259 measurement Methods 0.000 claims description 2
- 238000001514 detection method Methods 0.000 abstract description 20
- 230000008569 process Effects 0.000 abstract description 15
- 230000003287 optical effect Effects 0.000 abstract description 8
- 238000010586 diagram Methods 0.000 description 17
- 230000006870 function Effects 0.000 description 13
- 238000012545 processing Methods 0.000 description 10
- 238000004519 manufacturing process Methods 0.000 description 4
- 238000013528 artificial neural network Methods 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 3
- 238000005452 bending Methods 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000009826 distribution Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000003384 imaging method Methods 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 239000003990 capacitor Substances 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 210000002569 neuron Anatomy 0.000 description 1
- 150000003071 polychlorinated biphenyls Chemical class 0.000 description 1
- 230000008707 rearrangement Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/80—Geometric correction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
- G06T2207/30141—Printed circuit board [PCB]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Health & Medical Sciences (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Image Processing (AREA)
Abstract
The embodiment of the invention discloses a PCB image correction method, a device, terminal equipment and a storage medium, wherein the method confirms a corresponding substrate area according to a depth map and a two-dimensional map of a PCB to be detected; confirming three-dimensional coordinate data of the substrate area based on the depth information of the depth map and the coordinates of the two-dimensional map; randomly selecting a set number of points from the three-dimensional coordinate data for fitting to obtain a reference surface; and correcting the depth of each point in the depth map by taking a preset reference plane as a reference on the basis of the reference plane to obtain a corrected depth map. The method comprises the steps of confirming a substrate area by integrating depth information in a depth map and color information in a two-dimensional map, fitting based on three-dimensional coordinate data of the substrate area, correcting the depth of each point in the depth map by taking a preset reference plane as a reference, ensuring an image obtained in an optical detection process, finally obtaining the depth data which is as accurate as possible, and effectively reducing the occurrence of false detection results.
Description
Technical Field
The embodiment of the invention relates to the technical field of optical quality detection, in particular to a PCB image correction method, a device, terminal equipment and a storage medium.
Background
In the field of 3-dimensional automatic Optical Inspection, a target object (mainly for a PCB) such as a PCB (Printed Circuit Board) is scanned by a three-dimensional imaging method such as structured light, so as to obtain depth information of the target object, and the depth information is combined with a two-dimensional texture image acquired by a camera and compared with standard qualified data in a database, so that defect detection of the PCB is realized. Typically, a 3DAOI machine will have a reference plane (height 0) set before it leaves the factory, which can be considered as parallel to the plane of the machine guide rail, and the depth information finally obtained by the sensor module is the relative depth with respect to this reference plane.
When the inventor uses the existing detection equipment to carry out optical detection, due to the bending of PCB element pins and a substrate, a PCB placed on a guide rail is not necessarily parallel to a reference plane, so that the difference between the acquired depth and the depth required to be acquired is overlarge, and the false detection is caused.
Disclosure of Invention
The invention provides a PCB image correction method, a device, equipment and a storage medium, which aim to solve the technical problem of inaccurate detection caused by deformation or inclination in PCB detection in the prior art.
In a first aspect, an embodiment of the present invention provides a method for correcting a PCB image, including:
confirming a corresponding substrate area according to the depth map and the two-dimensional map of the PCB to be detected;
confirming three-dimensional coordinate data of the substrate area based on the depth information of the depth map and the coordinates of the two-dimensional map;
randomly selecting a set number of points from the three-dimensional coordinate data for fitting to obtain a reference surface;
and correcting the depth of each point in the depth map by taking a preset reference plane as a reference on the basis of the reference plane to obtain a corrected depth map.
In a second aspect, an embodiment of the present invention provides a method for correcting a PCB image, including:
confirming a corresponding substrate area according to the depth map and the two-dimensional map of the PCB to be detected;
training depth information related to the depth map and the substrate area and pixel coordinates related to the substrate area in the two-dimensional map as samples to obtain a BP neural network model, wherein the BP neural network model is used for outputting height values of the pixel coordinates corresponding to a preset reference surface;
inputting the pixel coordinates in the two-dimensional graph into the BP neural network model to obtain the height value of each pixel coordinate corresponding to the preset reference surface;
and taking the difference value between the depth information of the depth map and the corresponding height value as the corrected depth information.
In a third aspect, an embodiment of the present invention further provides a PCB image correction device, including:
the substrate area determining unit is used for determining a corresponding substrate area according to the depth map and the two-dimensional map of the PCB to be detected;
a coordinate data confirmation unit configured to confirm three-dimensional coordinate data of the substrate region based on the depth information of the depth map and the coordinates of the two-dimensional map;
the coordinate data fitting unit is used for randomly selecting a set number of points from the three-dimensional coordinate data to fit to obtain a reference plane;
and the depth map correction unit is used for correcting the depth of each point in the depth map by taking a preset reference plane as a reference based on the reference plane to obtain a corrected depth map.
In a fourth aspect, an embodiment of the present invention further provides a PCB image correction apparatus, including:
the substrate area determining unit is used for determining a corresponding substrate area according to the depth map and the two-dimensional map of the PCB to be detected;
the model training unit is used for training depth information related to the depth map and the substrate area and pixel coordinates related to the substrate area in the two-dimensional map as samples to obtain a BP neural network model, and the BP neural network model is used for outputting height values of the pixel coordinates corresponding to a preset reference surface;
the height value output unit is used for inputting the pixel coordinates in the two-dimensional graph into the BP neural network model to obtain the height value of each pixel coordinate corresponding to the preset reference surface;
and the depth information correction unit is used for taking the difference value between the depth information of the depth map and the corresponding height value as the corrected depth information.
In a fifth aspect, an embodiment of the present invention further provides a terminal device, including:
one or more processors;
a memory for storing one or more programs;
when the one or more programs are executed by the one or more processors, the terminal device is caused to implement the PCB image correction method according to the first aspect or the second aspect.
In a sixth aspect, the present invention further provides a computer readable storage medium, on which a computer program is stored, where the computer program is executed by a processor to implement the PCB image correction method according to the first or second aspect.
In the method, the corresponding substrate area is confirmed according to the depth map and the two-dimensional map of the PCB to be detected; confirming three-dimensional coordinate data of the substrate area based on the depth information of the depth map and the coordinates of the two-dimensional map; randomly selecting a set number of points from the three-dimensional coordinate data to perform fitting to obtain a reference surface; and correcting the depth of each point in the depth map by taking a preset reference plane as a reference on the basis of the reference plane to obtain a corrected depth map. The method comprises the steps of confirming a substrate area by integrating depth information in a depth map and color information in a two-dimensional map, fitting a datum plane based on three-dimensional coordinate data of the substrate area, correcting the depth of each point in the depth map by taking a preset reference plane as a reference according to the datum plane, ensuring an image obtained in an optical detection process, finally obtaining the depth data as accurate as possible, and effectively reducing the occurrence of false detection results.
Drawings
Fig. 1 is a flowchart of a method for correcting a PCB image according to an embodiment of the present invention;
FIG. 2 is a schematic view illustrating a substrate tilted state;
FIG. 3 is a schematic diagram of a calibration process for a PCB image with a planar reference surface according to an embodiment of the present invention;
FIG. 4 is a schematic diagram illustrating a calibration process for a PCB image with a curved reference surface according to an embodiment of the present invention;
FIG. 5 is a flowchart of a PCB image correction method according to an embodiment of the present invention;
FIG. 6 is a schematic structural diagram of a BP neural network model according to an embodiment of the present invention;
fig. 7 is a schematic structural diagram of a PCB image correction apparatus according to an embodiment of the present invention;
fig. 8 is a schematic structural diagram of a PCB image correction device according to an embodiment of the present invention;
fig. 9 is a schematic structural diagram of a terminal device according to an embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are for purposes of illustration and not limitation. It should be further noted that, for the convenience of description, only some of the structures related to the present invention are shown in the drawings, not all of the structures.
It should be noted that, for the sake of brevity, this description does not exhaust all alternative embodiments, and it should be understood by those skilled in the art after reading this description that any combination of features may constitute an alternative embodiment as long as the features are not mutually inconsistent.
For example, in one embodiment of the embodiment, one technical feature is recited: in another embodiment of the present invention, another technical feature is described in which a first substrate region is obtained by dividing a depth map by depth information of the depth map: by statistically confirming the second substrate area in the two-dimensional image through the color channel in the two-dimensional image, after reading the present specification, it should be understood by those skilled in the art that an embodiment having both of these features is also an alternative embodiment, that is, in the specific implementation process, the first substrate area is obtained based on the depth information, and the second substrate area is obtained based on the color channel statistics, so as to confirm the substrate area of the PCB to be tested.
Each example will be described in detail below.
Fig. 1 is a flowchart of a method for correcting a PCB image, which is used in a terminal device, according to an embodiment of the present invention, and as shown in the figure, the method for correcting a PCB image includes:
step S110: and confirming the corresponding substrate area according to the depth map and the two-dimensional map of the PCB to be tested.
In the actual production process, in order to reduce the production cost, a PCB usually completes a large number of production tasks at one time, that is, a batch of PCBs to be tested have almost the same surface shape and layout, and based on this, when a first product is produced, the first product can be used as a sample of the PCB to be tested to correspondingly mark a substrate area from a depth map and a two-dimensional map thereof, and then, the substrate area is used for carrying out related image correction. After each PCB is produced, when the PCB to be detected enters the detection equipment for detection, the detection equipment scans through three-dimensional imaging methods such as structured light and the like, and a depth map and a two-dimensional map are obtained through corresponding detection. The substrate area is identified in the depth map and/or the two-dimensional map.
In a specific implementation process, step S110 may further be implemented by steps S111 to S113:
step S111: and preprocessing the depth map of the PCB to be detected to obtain a first substrate area.
When the depth map of the PCB to be detected is preprocessed, the depth map of the PCB to be detected is mainly divided based on a preset height threshold value, and a part, lower than the height threshold value, in the depth map is extracted to obtain a first substrate area. That is, by processing the depth map, the component regions corresponding to the higher parts of the PCB, such as the chip region, the capacitor region, etc., in the depth map are removed.
Step S112: and preprocessing the two-dimensional graph of the PCB to be detected to obtain a second substrate area.
The preprocessing of the two-dimensional graph of the PCB to be detected mainly identifies the area corresponding to the substrate according to the color characteristics of the PCB, for the PCB, the area of the substrate is generally a mask area corresponding to green, and the identified green area in continuous strip distribution can be used as the mask area.
In a specific implementation process, step S112 may also be implemented by steps S1121 through S1123:
step S1121: and respectively carrying out gray level histogram statistics on the three color channels of the two-dimensional image.
Step S1122: and confirming the area to be masked corresponding to each color channel based on the preset gray scale range value.
Step S1123: and obtaining a second substrate area according to the intersection of the areas to be masked corresponding to each color channel.
For the collected two-dimensional image, actually an RGB texture image of a PCB to be detected, gray level histogram statistics is carried out on three color channels of R, G and B respectively, and a gray level range [ I ] with the largest number of pixels is obtained based on a set gray level range value 1 ,I 2 ]And then the corresponding mask area mask (i, j) to be masked is obtained.
R, G and B color channels can respectively obtain the corresponding mask regions to be masked R (i,j)、mask G (i, j) and mask B (i, j), the final mask region mask (i.e. the second substrate region) is:
mask＝mask R (i,j)&mask G (i,j)&mask B (i,j)。
step S113: confirming the substrate region based on an intersection region of the first substrate region and the second substrate region.
There may be different changes in processing details during the specific intersection region processing. For example, the first substrate area and the second substrate area are calculated respectively, and then the intersection of the first substrate area and the second substrate area is taken; the first substrate area may be calculated first, and the second substrate area may be calculated within the range of the first substrate area, that is, on the basis of the first substrate area obtained by dividing based on the height threshold, only the two-dimensional map corresponding to the area is preprocessed to obtain the second substrate area, and at this time, the intersection of the three channels is the second substrate area and also corresponds to the final substrate area.
Step S120: and confirming the three-dimensional coordinate data of the substrate area based on the depth information of the depth map and the coordinates of the two-dimensional map.
The z coordinate of the three-dimensional point of the substrate area can be directly obtained through the depth information in the most originally obtained depth map, and in addition, in the detection equipment, a two-dimensional map is generally obtained through acquiring by a telecentric lens; and carrying out geometric transformation on the pixel coordinates of the two-dimensional graph correspondingly based on the parameters of the telecentric lens to obtain the x coordinate and the y coordinate in the three-dimensional coordinate data, and taking the corresponding x coordinate, the corresponding y coordinate and the corresponding z coordinate as the three-dimensional coordinate data of the three-dimensional point of the substrate area.
Step S130: and randomly selecting a set number of points from the three-dimensional coordinate data to perform fitting to obtain a reference plane.
In the scheme, for a large amount of three-dimensional coordinate data corresponding to the substrate area, each point is not fitted to one surface, and a set number of points are randomly selected from the three-dimensional coordinate data for fitting to obtain the reference surface.
In order to ensure the accuracy of the fitting result, a plurality of groups of three-dimensional coordinate data can be randomly selected for fitting for a plurality of times, and the optimal result is selected as a reference surface. That is, in the specific implementation process, step S130 may be implemented by steps S131 to S134:
step S131: and randomly selecting a preset number of points from the three-dimensional coordinate data to perform fitting to obtain at least two initial reference planes.
Step S132: and calculating the distance from each point to each initial datum plane according to the three-dimensional coordinate data.
Step S133: and counting effective points corresponding to each initial datum plane, wherein the effective points are points of which the distance from the initial datum plane is less than a preset distance threshold.
Step S134: and taking the initial reference surface with the largest effective point as the reference surface of the substrate area.
And fitting each group of three-dimensional coordinate data in the same way as the existing fitting mode. And further calculating the distance between the three-dimensional coordinate data and each initial datum plane for the plurality of initial datum planes obtained by fitting, recording the points with the distance smaller than a preset distance threshold as effective points of the initial datum planes, wherein the more effective points of the initial datum planes, the closer the fitting result is to the distribution of the points, so that the initial datum planes with the most effective points are used as datum planes of the substrate area, and the influence of abnormal points on the final fitting result can be effectively avoided through multiple random sampling, and the obtained result is more accurate. And describing the initial reference plane obtained by fitting and the finally confirmed reference plane by a space equation.
Step S140: and correcting the depth of each point in the depth map by taking a preset reference plane as a reference on the basis of the reference plane to obtain a corrected depth map.
The reference surface is a plane or a curved surface, and the spatial equation of the plane is k 0 x+k 1 y+z+k 2 =0, the space equation of the curved surface is k 3 x 2 +k 4 y 2 +k 5 xy+k 6 x+k 7 y+k 8 +z＝0；
The correcting the depth of each point in the depth map by taking a preset reference plane as a reference based on the reference plane to obtain a corrected depth map includes:
if the reference plane is a plane, the point in the depth map is corrected by a first depth correction formula as follows:
if the reference surface is a curved surface, the points in the depth map are corrected by a second depth correction formula as follows:
wherein (i, j) is a pixel coordinate corresponding to the midpoint (x, y) of the three-dimensional coordinate data, h' (i, j) is a correction depth corresponding to the midpoint (x, y) of the three-dimensional coordinate data, h (i, j) is a measurement depth corresponding to the midpoint (x, y) of the three-dimensional coordinate data, mi = x, mj = y, and m is an object resolution (um/pixel) of a camera used for acquiring the two-dimensional map.
For plane k 0 x+k 1 y+z+k 2 =0, a correction model can be constructed as shown in fig. 3, where the reference plane obtained by fitting is Π 1 Reference plane is pi 0 Normal vector of the reference plane is
The normal vector of the reference plane is
The angle between the two planes is set as θ, and can be calculated as:
as shown in fig. 3, the actual height of a certain point in the PCB with respect to the reference plane is h', but the measured height of the detection device is h, which is obtained by the spatial equation of the reference plane:
in the three-dimensional coordinate data, (x, y) is a physical coordinate, the physical coordinate (x, y) is expressed by a pixel coordinate (i, j), mi = x, mj = y, (i, j) is a pixel coordinate, and m is an object resolution of a camera which acquires the two-dimensional map, so that a first depth correction formula when a reference plane is a plane can be obtained:
for curved surface k 3 x 2 +k 4 y 2 +k 5 xy+k 6 x+k 7 y+k 8 + z =0, a correction model can be established as shown in fig. 4, where the fitted base plane is S1 and the reference plane is Π 0 The normal vector of the reference plane is
The included angle between the actual height h 'and the measured height h is theta, and because theta is generally smaller than theta during PCB substrate correction, the normal vector at the position of h' can be approximate to the normal vector of the curved surface of the measured position, namely the normal vector
Further it can be calculated that:
similar to the corresponding calculation of fig. 3:
finally, a second correction formula is obtained:
confirming a corresponding substrate area according to the depth map and the two-dimensional map of the PCB to be detected; confirming three-dimensional coordinate data of the substrate area based on the depth information of the depth map and the coordinates of the two-dimensional map; randomly selecting a set number of points from the three-dimensional coordinate data for fitting to obtain a reference surface; and correcting the depth of each point in the depth map by taking a preset reference plane as a reference based on the reference plane to obtain a corrected depth map. The method comprises the steps of confirming a substrate area by integrating depth information in a depth map and color information in a two-dimensional map, fitting a datum plane based on three-dimensional coordinate data of the substrate area, correcting the datum plane to the reference plane by taking a preset reference plane as a reference according to the datum plane, and correcting other areas to corresponding height values at the same time, so that the depth of each point in the depth map is corrected, an image obtained in an optical detection process is ensured, the depth data which is as accurate as possible can be obtained finally, and the occurrence of false detection results is effectively reduced.
Fig. 5 is a flowchart of another PCB image correction method according to an embodiment of the present invention, where as shown in the figure, the PCB image correction method includes:
step S210: and confirming the corresponding substrate area according to the depth map and the two-dimensional map of the PCB to be tested.
Step S220: and training the depth information associated with the depth map and the substrate area and the pixel coordinate associated with the substrate area in the two-dimensional map as a sample to obtain a BP neural network model, wherein the BP neural network model is used for outputting a height value of the pixel coordinate corresponding to a preset reference surface.
Step S230: and inputting the pixel coordinates in the two-dimensional graph into the BP neural network model to obtain the height value of each pixel coordinate corresponding to the preset reference surface.
Step S240: and taking the difference value between the depth information of the depth map and the corresponding height value as the corrected depth information.
For more complex PCB basal plane cases, BP neural network can be adopted to fit the relation between the pixel coordinates (i, j) and the depth information z in the two-dimensional map. Specifically, the depth map pixel points of the substrate region segmented in step S210 are used to train the three-layer BP neural network to establish a mapping relationship between the pixel coordinates (i, j) and the depth information z, that is, a model of the reference plane of the PCB. In a specific training process, a BP neural network model as shown in fig. 6 may be adopted to down-sample a pixel point region to speed up the training process. And (3) outputting a pixel coordinate (i, j) of a substrate area divided by an input sample of the BP neural network model, wherein the output sample is a corresponding height value, correcting the connection weight and the neuron threshold value of each layer in the BP neural network model layer by layer through error back propagation to gradually reduce the error, and repeating training until the error meets the precision requirement.
Inputting the pixel coordinates in the two-dimensional image into the trained BP neural network model, correspondingly obtaining the height value of each pixel coordinate corresponding to the preset reference surface, wherein the correction result of each point in the depth image is the actual depth h' (i, j) of the point, and directly correcting the height value obtained through the BP neural network, namely:
h'(i,j)＝h(i,j)-z(i,j)
wherein h (i, j) is the depth information corresponding to the point (i, j) and z (i, j) is the height value corresponding to the point (i, j) and calculated by the BP neural network model.
For the more complex PCB basal plane condition, the BP neural network model is trained, the height value can be obtained corresponding to each point, and correction is carried out based on the height value, so that the problem of depth map deviation caused by substrate inclination or bending is effectively corrected, and further false detection is reduced.
Fig. 7 is a schematic structural diagram of a PCB image correction device according to an embodiment of the present invention. Referring to fig. 7, the PCB image correction apparatus includes: a substrate region determining unit 310, a coordinate data confirming unit 320, a coordinate data fitting unit 330, and a depth map correcting unit 340.
The substrate area determining unit 310 is configured to determine a corresponding substrate area according to the depth map and the two-dimensional map of the PCB to be tested; a coordinate data confirmation unit 320 for confirming three-dimensional coordinate data of the substrate region based on the depth information of the depth map and the coordinates of the two-dimensional map; a coordinate data fitting unit 330, configured to randomly select a set number of points from the three-dimensional coordinate data to perform fitting, so as to obtain a reference plane; the depth map correcting unit 340 is configured to correct the depth of each point in the depth map by using a preset reference plane as a reference based on the reference plane, so as to obtain a corrected depth map.
On the basis of the above embodiment, the coordinate data fitting unit 330 includes:
the initial fitting module is used for randomly selecting a preset number of points from the three-dimensional coordinate data to perform fitting to obtain at least two initial reference surfaces;
the distance calculation module is used for calculating the distance from each point to each initial datum plane according to the three-dimensional coordinate data;
the effective point counting module is used for counting effective points corresponding to each initial datum plane, and the effective points are points of which the distance from the initial datum plane is smaller than a preset distance threshold;
and the reference surface confirming module is used for taking the initial reference surface with the most effective points as the reference surface of the substrate area.
On the basis of the above embodiment, the substrate area determining unit 310 includes:
the first preprocessing module is used for preprocessing a depth map of the PCB to be detected to obtain a first substrate area;
the second preprocessing module is used for preprocessing the two-dimensional graph of the PCB to be detected to obtain a second substrate area;
and the intersection confirming module is used for confirming the substrate area based on the intersection area of the first substrate area and the second substrate area.
On the basis of the foregoing embodiment, the first preprocessing module is specifically configured to segment a depth map of a PCB to be tested based on a preset height threshold, and extract a portion of the depth map that is lower than the height threshold to obtain a first substrate area.
On the basis of the above embodiment, the second preprocessing module includes:
the gray scale statistics submodule is used for respectively carrying out gray scale histogram statistics on three color channels of the two-dimensional image;
the undetermined area confirming submodule is used for confirming the undetermined mask area corresponding to each color channel based on a preset gray scale range value;
and the intersection area confirming submodule is used for obtaining a second substrate area according to the intersection of the areas to be masked corresponding to each color channel.
On the basis of the embodiment, the two-dimensional image is acquired through a telecentric lens;
the coordinate data confirming unit is specifically configured to perform geometric transformation on pixel coordinates of the two-dimensional map based on parameters of the telecentric lens to obtain an x coordinate and a y coordinate in the three-dimensional coordinate data, acquire depth information of the substrate area corresponding to the depth map as a z coordinate, and use the corresponding x coordinate, y coordinate, and z coordinate as the three-dimensional coordinate data of the substrate area.
On the basis of the above embodiment, the reference plane is a plane or a curved surface, and the spatial equation of the plane is k 0 x+k 1 y+z+k 2 =0, curved surface having a spatial equation of k 3 x 2 +k 4 y 2 +k 5 xy+k 6 x+k 7 y+k 8 +z＝0；
The depth map correction unit 340 includes:
a first correction module, configured to, if the reference plane is a plane, correct a point in the depth map by using a first depth correction formula as follows:
a second correction module, configured to, if the reference plane is a curved surface, correct a point in the depth map by using a second depth correction formula as follows:
wherein, (i, j) is a pixel coordinate corresponding to a midpoint (x, y) of the three-dimensional coordinate data, h' (i, j) is a corrected depth corresponding to the midpoint (x, y) of the three-dimensional coordinate data, h (i, j) is a measured depth corresponding to the midpoint (x, y) of the three-dimensional coordinate data, mi = x, mj = y, and m is an object-side resolution of a camera used for acquiring the two-dimensional image.
The PCB image correction device provided by the embodiment of the invention is contained in the terminal equipment, can be used for executing the PCB image correction method provided by the embodiment, and has corresponding functions and beneficial effects.
It should be noted that, in the embodiment of the PCB image correction apparatus, the included units and modules are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be implemented; in addition, specific names of the functional units are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present invention.
Fig. 8 is a schematic structural diagram of a PCB image correction apparatus according to an embodiment of the present invention. Referring to fig. 8, the PCB image correction apparatus includes: a substrate region determining unit 410, a model training unit 420, a height value output unit 430, and a depth information correcting unit 440.
The device comprises a substrate area determining unit, a processing unit and a processing unit, wherein the substrate area determining unit is used for determining a corresponding substrate area according to a depth map and a two-dimensional map of a PCB to be tested;
the model training unit is used for training depth information related to the depth map and the substrate area and pixel coordinates related to the substrate area in the two-dimensional map as samples to obtain a BP neural network model, and the BP neural network model is used for outputting height values of the pixel coordinates corresponding to a preset reference surface;
the height value output unit is used for inputting the pixel coordinates in the two-dimensional graph into the BP neural network model to obtain the height value of each pixel coordinate corresponding to the preset reference surface;
and the depth information correction unit is used for taking the difference value between the depth information of the depth map and the corresponding height value as the corrected depth information.
The PCB image correction device provided by the embodiment of the invention is contained in the terminal equipment, can be used for executing the PCB image correction method provided by the embodiment, and has corresponding functions and beneficial effects.
It should be noted that, in the embodiment of the PCB image correction apparatus, the included units and modules are only divided according to functional logic, but not limited to the above division, as long as the corresponding functions can be realized; in addition, specific names of the functional units are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present invention.
Fig. 9 is a schematic structural diagram of a terminal device according to an embodiment of the present invention. As shown in fig. 9, the electronic device includes a processor 510, a memory 520, an input device 530, an output device 540, and a communication device 550; the number of the processors 510 in the electronic device may be one or more, and one processor 510 is taken as an example in fig. 9; the processor 510, the memory 520, the input device 530, the output device 540, and the communication device 550 in the electronic apparatus may be connected by a bus or other means, and the bus connection is exemplified in fig. 9.
The memory 520 is a computer readable storage medium, and can be used to store software programs, computer executable programs, and modules, such as program instructions/modules corresponding to the PCB image correction method in the embodiment of the present invention (for example, the substrate area determination unit 310, the coordinate data confirmation unit 320, the coordinate data fitting unit 330, and the depth map correction unit 340 in the PCB image correction apparatus), and of course, program instructions/modules for implementing functions thereof corresponding to different electronic devices are located in the corresponding electronic devices. The processor 310 executes various functional applications and data processing of the electronic device by executing software programs, instructions and modules stored in the memory 320, that is, implements the PCB image correction method described above.
The memory 320 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required for at least one function; the storage data area may store data created according to use of the electronic device, and the like. Further, the memory 320 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some examples, the memory 320 can further include memory located remotely from the processor 310, which can be connected to electronic devices through a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The input device 330 may be used to receive input numeric or character information and generate key signal inputs related to user settings and function control of the electronic apparatus. The output device 340 may include a display device such as a display screen.
The electronic equipment comprises the PCB image correction device, can be used for executing any PCB image correction method, and has corresponding functions and beneficial effects.
Embodiments of the present invention also provide a storage medium containing computer-executable instructions, which when executed by a computer processor, are configured to perform operations related to the PCB image correction method provided in any of the embodiments of the present application, and have corresponding functions and advantages.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product.
Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and so forth) having computer-usable program code embodied therein. The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks. These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory. The memory may include forms of volatile memory in a computer readable medium, random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). The memory is an example of a computer-readable medium.
Computer-readable media, including both permanent and non-permanent, removable and non-removable media, may implement the information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrases "comprising a," "8230," "8230," or "comprising" does not exclude the presence of additional identical elements in the process, method, article, or apparatus comprising the element.
It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.
Claims (12)
1. A PCB image correction method is characterized by comprising the following steps:
confirming a corresponding substrate area according to the depth map and the two-dimensional map of the PCB to be detected;
confirming three-dimensional coordinate data of the substrate area based on the depth information of the depth map and the coordinates of the two-dimensional map;
randomly selecting a set number of points from the three-dimensional coordinate data for fitting to obtain a reference surface;
and correcting the depth of each point in the depth map by taking a preset reference plane as a reference on the basis of the reference plane to obtain a corrected depth map.
2. The method of claim 1, wherein said randomly selecting a set number of points from the three-dimensional coordinate data to fit to obtain a reference plane comprises:
randomly selecting a preset number of points from the three-dimensional coordinate data for fitting to obtain at least two initial datum planes;
calculating the distance from each point to each initial datum plane according to the three-dimensional coordinate data;
counting effective points corresponding to each initial datum plane, wherein the effective points are points of which the distance from the initial datum plane is smaller than a preset distance threshold;
and taking the initial reference surface with the largest effective point as the reference surface of the substrate area.
3. The method of claim 1, wherein identifying the corresponding substrate area according to the depth map and the two-dimensional map of the PCB to be tested comprises:
preprocessing a depth map of a PCB to be detected to obtain a first substrate area;
preprocessing a two-dimensional graph of the PCB to be detected to obtain a second substrate area;
confirming the substrate region based on an intersection region of the first substrate region and the second substrate region.
4. The method according to claim 3, wherein the preprocessing is performed on the depth map of the PCB to be tested to obtain a first substrate area, specifically:
based on a preset height threshold value, a depth map of the PCB to be tested is segmented, and a part of the depth map, which is lower than the height threshold value, is extracted to obtain a first substrate area.
5. The method according to claim 3, wherein the two-dimensional map of the PCB to be tested is preprocessed to obtain a second substrate region, specifically:
carrying out gray level histogram statistics on three color channels of the two-dimensional image respectively;
confirming the area to be masked corresponding to each color channel based on a preset gray scale range value;
and obtaining a second substrate area according to the intersection of the areas to be masked corresponding to each color channel.
6. The method of claim 1, wherein the two-dimensional map is acquired by a telecentric lens;
confirming three-dimensional coordinate data of the substrate area based on the depth information of the depth map and the coordinates of the two-dimensional map, specifically:
and performing geometric transformation on the pixel coordinates of the two-dimensional map based on the parameters of the telecentric lens to obtain an x coordinate and a y coordinate in the three-dimensional coordinate data, acquiring depth information of the substrate area corresponding to the depth map as a z coordinate, and taking the corresponding x coordinate, y coordinate and z coordinate as the three-dimensional coordinate data of the substrate area.
7. The method of claim 1, wherein the reference plane is a plane or a curved plane, and the spatial equation of the plane is k 0 x+k 1 y+z+k 2 =0, the space equation of the curved surface is k 3 x 2 +k 4 y 2 +k 5 xy+k 6 x+k 7 y+k 8 +z＝0；
The correcting the depth of each point in the depth map by taking a preset reference plane as a reference based on the reference plane to obtain a corrected depth map comprises:
if the reference plane is a plane, the points in the depth map are corrected by a first depth correction formula as follows:
if the reference surface is a curved surface, the points in the depth map are corrected by a second depth correction formula as follows:
wherein (i, j) is a pixel coordinate corresponding to a midpoint (x, y) of the three-dimensional coordinate data, h' (i, j) is a correction depth corresponding to the midpoint (x, y) of the three-dimensional coordinate data, h (i, j) is a measurement depth corresponding to the midpoint (x, y) of the three-dimensional coordinate data, mi = x, mj = y, and m is an object resolution of a camera used for acquiring the two-dimensional map.
8. A PCB image correction method is characterized by comprising the following steps:
confirming a corresponding substrate area according to the depth map and the two-dimensional map of the PCB to be detected;
training depth information related to the depth map and the substrate area and pixel coordinates related to the substrate area in the two-dimensional map as samples to obtain a BP neural network model, wherein the BP neural network model is used for outputting height values of the pixel coordinates corresponding to a preset reference surface;
inputting the pixel coordinates in the two-dimensional graph into the BP neural network model to obtain the height value of each pixel coordinate corresponding to the preset reference surface;
and taking the difference value between the depth information of the depth map and the corresponding height value as the corrected depth information.
9. A PCB image correction apparatus, comprising:
the substrate area determining unit is used for determining a corresponding substrate area according to the depth map and the two-dimensional map of the PCB to be detected;
a coordinate data confirmation unit configured to confirm three-dimensional coordinate data of the substrate region based on the depth information of the depth map and the coordinates of the two-dimensional map;
the coordinate data fitting unit is used for randomly selecting a set number of points from the three-dimensional coordinate data to fit to obtain a reference plane;
and the depth map correction unit is used for correcting the depth of each point in the depth map by taking a preset reference plane as a reference on the basis of the reference plane to obtain a corrected depth map.
10. A PCB image correction apparatus, comprising:
the substrate area determining unit is used for determining a corresponding substrate area according to the depth map and the two-dimensional map of the PCB to be detected;
the model training unit is used for training depth information associated with the depth map and the substrate area and pixel coordinates associated with the substrate area in the two-dimensional map as samples to obtain a BP neural network model, and the BP neural network model is used for outputting height values of the pixel coordinates corresponding to a preset reference surface;
the height value output unit is used for inputting the pixel coordinates in the two-dimensional graph into the BP neural network model to obtain the height value of each pixel coordinate corresponding to the preset reference surface;
and a depth information correction unit for taking the difference between the depth information of the depth map and the corresponding height value as corrected depth information.
11. A terminal device, comprising:
one or more processors;
a memory for storing one or more programs;
when executed by the one or more processors, cause the terminal device to implement the PCB image correction method of any of claims 1-8.
12. A computer-readable storage medium, on which a computer program is stored, which program, when executed by a processor, carries out the PCB image correction method according to any one of claims 1-8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110631938.3A CN115511718A (en) | 2021-06-07 | 2021-06-07 | PCB image correction method and device, terminal equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110631938.3A CN115511718A (en) | 2021-06-07 | 2021-06-07 | PCB image correction method and device, terminal equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115511718A true CN115511718A (en) | 2022-12-23 |
Family
ID=84500214
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110631938.3A Pending CN115511718A (en) | 2021-06-07 | 2021-06-07 | PCB image correction method and device, terminal equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115511718A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118052861A (en) * | 2024-02-20 | 2024-05-17 | 上海赫立智能机器有限公司 | Zero plane acquisition method, system, medium and electronic equipment |
| CN118657698A (en) * | 2024-08-21 | 2024-09-17 | 苏州华兴源创科技股份有限公司 | Fusion image correction method, device and imaging system |
-
2021
- 2021-06-07 CN CN202110631938.3A patent/CN115511718A/en active Pending
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118052861A (en) * | 2024-02-20 | 2024-05-17 | 上海赫立智能机器有限公司 | Zero plane acquisition method, system, medium and electronic equipment |
| CN118657698A (en) * | 2024-08-21 | 2024-09-17 | 苏州华兴源创科技股份有限公司 | Fusion image correction method, device and imaging system |
| CN118657698B (en) * | 2024-08-21 | 2024-11-05 | 苏州华兴源创科技股份有限公司 | Fusion image correction method and device and imaging system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113240630B (en) | Speckle image quality evaluation method and device, terminal equipment and readable storage medium | |
| CN113160161A (en) | Method and device for detecting defects at edge of target | |
| US10841561B2 (en) | Apparatus and method for three-dimensional inspection | |
| CN113160223A (en) | Contour determination method, determination device, detection device and storage medium | |
| CN115511718A (en) | PCB image correction method and device, terminal equipment and storage medium | |
| CN107230212B (en) | Vision-based mobile phone size measuring method and system | |
| CN117218062A (en) | Defect detection method and device, electronic equipment and storage medium | |
| CN113781392A (en) | Glue road detection method, electronic device and storage medium | |
| CN112581541B (en) | Parameter evaluation method, device and electronic device | |
| CN111435080B (en) | Water level measuring method, device and system | |
| CN115205287A (en) | Gear digital measurement and evaluation cloud method and system | |
| CN111336938A (en) | Robot and object distance detection method and device thereof | |
| CN118351097A (en) | Method and device for detecting glue path quality, electronic equipment and storage medium | |
| CN117830251A (en) | Defect analysis method, defect analysis device and electronic equipment | |
| CN109212546B (en) | Method and device for calculating depth direction measurement error of binocular camera | |
| CN110874837A (en) | Automatic defect detection method based on local feature distribution | |
| CN111695551B (en) | Dial reading method, device, computer equipment and readable storage medium | |
| RU2519005C1 (en) | Method of prestart check of printboards | |
| CN113838146A (en) | Method and device for verifying calibration precision of camera module and method and device for testing camera module | |
| CN112802119A (en) | Factory ranging inspection method, system and equipment based on binocular camera | |
| CN109754365B (en) | Image processing method and device | |
| CN118424162B (en) | A laser rangefinder polygon area measurement method, device and storage medium | |
| RU2351091C2 (en) | Method of automatic detection and correction of radial distortion on digital images | |
| CN116608816B (en) | Calibration method and device for calibrating device of small-angle measuring instrument | |
| CN118864616B (en) | Parameter optimization device and parameter optimization method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| DD01 | Delivery of document by public notice |
Addressee: Guangzhou Shiyuan Electronic Technology Company Limited Person in charge of patents Document name: Notification of qualified procedures |
|
| DD01 | Delivery of document by public notice | ||
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |