CN116152208A - Defect detection method, device, equipment and storage medium - Google Patents

Abstract

The present disclosure provides a defect detection method, device, equipment, and storage medium, by obtaining a point cloud of a component to be tested and a point cloud of a target component template, wherein the target component template point cloud is a point cloud for judging the component to be tested Whether there is a reference template point cloud of the attachment defect; respectively rebuilding the part to be measured point cloud and the target part template point cloud into a part to be measured depth map and a target part template depth map; by analyzing the part to be measured depth The image and the target component template depth image are subjected to binarization processing and Blob analysis to determine whether there is an attachment defect in the point cloud of the component to be tested, which not only effectively improves the defect detection rate, but also saves a lot of manpower and material resources.

Description

Defect detection method, device, equipment and storage medium

technical field

The present disclosure relates to the field of computer technology, and in particular to a defect detection method, device, equipment and storage medium.

Background technique

The defect detection method of the notebook internal structure based on 3D vision can be understood as a point cloud defect detection problem. Specifically, based on the specified area in the template image, the point cloud registration method is used to find the corresponding matching area in the image to be tested and perform defect detection. . Commonly used methods for 3D visual inspection in the prior art include: a defect detection method based on deep learning and a defect detection method based on template comparison.

Among them, although the neural network training method based on 3D point cloud has a certain effect on defect detection in theory, it is very difficult to collect a large number of defect samples in the actual implementation process, resulting in a decrease in detection accuracy.

The template comparison defect detection method based on 3D point cloud is to extract the standard template and the position of the component in the point cloud to be tested for point cloud information comparison, so as to judge whether there is an attachment defect in the component. This method has the problem of missed detection and false detection, especially when there is a high protrusion near the position of the attached part or a slight deviation of the attached part, the defect detection effect is very poor.

Contents of the invention

The present disclosure provides a defect detection method, device, equipment and storage medium to at least solve the above technical problems existing in the prior art.

According to a first aspect of the present disclosure, a defect detection method is provided, the method comprising:

Obtaining the component to-be-tested point cloud and acquiring the target component template point cloud, wherein the target component template point cloud is a reference template point cloud for judging whether the component to-be-tested point cloud has an attachment defect;

Respectively reconstructing the point cloud of the part to be measured and the point cloud of the target part template into a depth map of the part to be measured and a depth map of the target part template;

By performing binarization processing and Blob analysis on the depth map of the part to be tested and the depth map of the target part template, it is determined whether there is an attachment defect in the point cloud of the part to be tested.

In a possible implementation manner, the step of determining whether there is an attachment defect in the point cloud of the part to be tested by performing binarization processing and Blob analysis on the depth map of the part to be tested and the depth map of the target part template includes: :

Based on the depth mean value corresponding to the component template point cloud, the component to-be-measured depth map and the target component template depth map are respectively binarized to obtain the component to-be-tested binary map and the target component template binary map;

Respectively acquire a first preset area in the binary image of the component to be tested and a second preset area in the binary image of the target component template;

Comparing the area information of the first preset area and the second preset area to determine whether there is an attachment defect in the point cloud of the component to be tested, wherein the area information includes at least one of the following: width, height and area.

In a possible implementation manner, the comparing the area information of the first preset area and the second preset area to determine whether there is an attachment defect in the point cloud of the component to be tested includes:

When the width difference between the first preset area and the second preset area is less than a preset width threshold, the height difference between the first preset area and the second preset area is less than a preset high threshold And when the area difference between the first preset area and the second preset area is less than a preset area threshold, it is determined that there is no attachment defect in the component to-be-tested point cloud; otherwise, it is determined that the component to-be-tested point cloud The cloud has an attachment flaw.

In a possible implementation manner, the acquisition of the target component template point cloud includes:

Obtaining a point cloud to be measured that includes at least one attached part to be tested and a template point cloud of the sample attached part, wherein the template point cloud is marked with a label of each sample attached part;

Based on the point cloud registration algorithm and the point cloud to be measured containing at least one attached part to be tested, the template point cloud is calibrated to obtain a corrected template point cloud;

On the corrected template point cloud, at least one sample part template point cloud is determined, and each sample part template point cloud is sequentially used as a target part template point cloud.

In a possible implementation manner, said acquiring the template point cloud of the sample attachment part includes:

Determining the source point cloud data of the sample attachment part by using the original 3D design drawing of the sample attachment part or photographing the sample attachment part by a 3D camera;

Modeling the source point cloud data of the sample attachment component to obtain a template point cloud of the sample attachment component.

In a possible implementation manner, the acquiring point cloud of the component to be measured includes:

performing dimensionality reduction processing on the corrected template point cloud and the measured point cloud containing at least one attached component to be measured, to obtain a dimensionality-reduced template point cloud and a dimensionality-reduced point cloud to be measured;

In the dimension reduction template point cloud, determine a dimension reduction component template point cloud corresponding to the target component template point cloud;

Based on the neighborhood minimum value method, in the dimensionality reduction point cloud to be measured, determine the dimensionality reduction component point cloud corresponding to the dimensionality reduction component template point cloud, and pass the dimensionality reduction measurement point cloud and the The mapping relationship between the point clouds to be measured that contains at least one attached part to be tested, on the point cloud to be measured that contains at least one attached part to be tested, it is determined to correspond to the point cloud of the dimensionality reduction component The region of the point cloud to be measured is used as the point cloud of the component to be measured.

In a possible implementation manner, the respectively reconstructing the point cloud of the component to be measured and the point cloud of the target component template into a depth map of the component to be measured and a depth map of the target component template includes:

Obtain the coordinates of each voxel in the point cloud of the component to be measured and the point cloud of the target component template respectively, wherein the coordinates include an X value, a Y value and a Z value;

Using the X value and Y value of each voxel in the part to be measured point cloud as the coordinate value of the part to be measured depth map, and using the Z value as the pixel value of the part to be measured depth map to reconstruct the The depth map of the component to be tested;

Using the X value and Y value of each voxel in the target part template point cloud as the coordinate value of the target part template depth map, and using the Z value as the pixel value of the target part template depth map to reconstruct the Target part template depth map.

According to a second aspect of the present disclosure, a defect detection device is provided, the device comprising:

The point cloud acquisition module is used to acquire the point cloud of the component to be tested and the target component template point cloud, wherein the target component template point cloud is a reference template point cloud for judging whether the component to be tested point cloud has an attachment defect;

A depth map reconstruction module, configured to reconstruct the part to-be-measured point cloud and the target part template point cloud into a part to-be-measured depth map and a target part template depth map;

The defect analysis module is configured to perform binarization processing and Blob analysis on the depth map of the part to be tested and the depth map of the target part template to determine whether there is an attachment defect in the point cloud of the part to be tested.

In a possible implementation, the defect analysis module is specifically used for:

Based on the depth mean value corresponding to the component template point cloud, the component to-be-measured depth map and the target component template depth map are respectively binarized to obtain the component to-be-tested binary map and the target component template binary map;

Respectively acquire a first preset area in the binary image of the component to be tested and a second preset area in the binary image of the target component template;

Comparing the area information of the first preset area and the second preset area to determine whether there is an attachment defect in the point cloud of the component to be tested, wherein the area information includes at least one of the following: width, height and area.

In a possible implementation, the defect analysis module is also specifically used for:

When the width difference between the first preset area and the second preset area is less than a preset width threshold, the height difference between the first preset area and the second preset area is less than a preset high threshold And when the area difference between the first preset area and the second preset area is less than a preset area threshold, it is determined that there is no attachment defect in the component to-be-tested point cloud; otherwise, it is determined that the component to-be-tested point cloud The cloud has an attachment flaw.

In one possible implementation, the point cloud acquisition module is specifically used for:

Obtaining a point cloud to be measured that includes at least one attached part to be tested and a template point cloud of the sample attached part, wherein the template point cloud is marked with a label of each sample attached part;

Based on the point cloud registration algorithm and the point cloud to be measured containing at least one attached part to be tested, the template point cloud is calibrated to obtain a corrected template point cloud;

On the corrected template point cloud, at least one sample part template point cloud is determined, and each sample part template point cloud is sequentially used as a target part template point cloud.

In one possible implementation, the point cloud acquisition module is also specifically used for:

Determining the source point cloud data of the sample attachment part by using the original 3D design drawing of the sample attachment part or photographing the sample attachment part by a 3D camera;

Modeling the source point cloud data of the sample attachment component to obtain a template point cloud of the sample attachment component.

In one possible implementation, the point cloud acquisition module is also specifically used for:

performing dimensionality reduction processing on the corrected template point cloud and the measured point cloud containing at least one attached component to be measured, to obtain a dimensionality-reduced template point cloud and a dimensionality-reduced point cloud to be measured;

In the dimension reduction template point cloud, determine a dimension reduction component template point cloud corresponding to the target component template point cloud;

Based on the neighborhood minimum value method, in the dimensionality reduction point cloud to be measured, determine the dimensionality reduction component point cloud corresponding to the dimensionality reduction component template point cloud, and pass the dimensionality reduction measurement point cloud and the The mapping relationship between the point clouds to be measured that contains at least one attached part to be tested, on the point cloud to be measured that contains at least one attached part to be tested, it is determined to correspond to the point cloud of the dimensionality reduction component The region of the point cloud to be measured is used as the point cloud of the component to be measured.

In a possible implementation manner, the depth map reconstruction module is specifically used for:

Obtain the coordinates of each voxel in the point cloud of the component to be measured and the point cloud of the target component template respectively, wherein the coordinates include an X value, a Y value and a Z value;

Using the X value and Y value of each voxel in the part to be measured point cloud as the coordinate value of the part to be measured depth map, and using the Z value as the pixel value of the part to be measured depth map to reconstruct the The depth map of the component to be tested;

Using the X value and Y value of each voxel in the target part template point cloud as the coordinate value of the target part template depth map, and using the Z value as the pixel value of the target part template depth map to reconstruct the Target part template depth map.

According to a third aspect of the present disclosure, an electronic device is provided, including:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,

The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor, to enable the at least one processor to perform the methods described in the present disclosure.

According to a fourth aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions for causing the computer to execute the method described in the present disclosure.

A defect detection method, device, equipment, and storage medium of the present disclosure obtain a component to-be-tested point cloud and a target component template point cloud, wherein the target component template point cloud is used to determine whether the component to-be-tested point cloud is There is a reference template point cloud with an attachment defect; respectively rebuilding the part to be measured point cloud and the target part template point cloud into a part to be measured depth map and a target part template depth map; by analyzing the part to be measured depth map Performing binarization processing and Blob analysis with the depth map of the target component template to determine whether there is an attachment defect in the point cloud of the component to be tested, which not only effectively improves the defect detection rate, but also saves a lot of manpower and material resources.

It should be understood that what is described in this section is not intended to identify key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will be readily understood through the following description.

Description of drawings

The above and other objects, features and advantages of exemplary embodiments of the present disclosure will become readily understood by reading the following detailed description with reference to the accompanying drawings. In the drawings, several embodiments of the present disclosure are shown by way of illustration and not limitation, in which:

In the drawings, the same or corresponding reference numerals denote the same or corresponding parts.

FIG. 1 shows a schematic diagram of the implementation flow of a defect detection method provided by Embodiment 1 of the present disclosure;

FIG. 2 shows a schematic flow diagram of a defect detection method provided by Embodiment 2 of the present disclosure;

FIG. 3 shows a framework diagram of the implementation process of a defect detection method provided by Embodiment 2 of the present disclosure;

FIG. 4 shows a schematic structural diagram of a defect detection device provided in Embodiment 3 of the present disclosure;

FIG. 5 shows a schematic diagram of the composition and structure of an electronic device according to an embodiment of the present disclosure.

Detailed ways

In order to make the purpose, features, and advantages of the present disclosure more obvious and understandable, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present disclosure. Obviously, the described The embodiments are only some of the embodiments of the present disclosure, but not all of them. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without making creative efforts belong to the protection scope of the present disclosure.

Embodiment one

FIG. 1 is a flow chart of a defect detection method provided in Embodiment 1 of the present disclosure. The method can be executed by a defect detection device provided in an embodiment of the present disclosure, and the device can be implemented in software and/or hardware. The method specifically includes:

S110. Obtain a point cloud of the component to be tested and a target component template point cloud.

Wherein, the target component template point cloud may be a reference template point cloud for judging whether there is an attachment defect in the component to be tested, which is recorded as ROI_model. The point cloud of the component to be tested can be a point cloud obtained by shooting the component to be tested with a 3D camera, which is recorded as ROI_dst.

Specifically, different attachment positions in the notebook require different types of attachment components, and the diversity and complexity of the attachment positions themselves will seriously affect the effect of component attachment detection. There are different requirements for the types of attached parts. For example, the attached parts in the notebook can be conductive cloth, foot pads, iron sheets, iron rods, wire drawing, etc., and due to different manufacturers, it is easy to cause different differences in the same type of attached parts. Color; For the diversity and complexity of the attachment position, for example, the attachment position may be a flat position, or a corner position, and there may be interference around the attachment position, etc., all of which will affect the detection accuracy.

In an embodiment of the present disclosure, obtaining the template point cloud of the target part includes: obtaining the point cloud to be measured containing at least one attached part to be tested and obtaining the template point cloud of the sample attached part, wherein the template point cloud is marked with The label of each sample attachment part; based on the point cloud registration algorithm and the test point cloud containing at least one test attachment part, the template point cloud is calibrated to obtain the corrected template point cloud; the corrected template On the point cloud, at least one sample part template point cloud is determined, and each sample part template point cloud is sequentially used as the target part template point cloud.

Wherein, the point cloud registration algorithm may be an algorithm based on voxel segmentation and matching for extracting feature points. Exemplarily, the point cloud registration algorithm adopted in this embodiment may be the closest point iterative algorithm (Iterative Closest Point, ICP ). The template point cloud can be a source point cloud that has not yet been calibrated and contains the identification of each sample attachment part, denoted as pt_model. The point cloud to be tested is a point cloud that includes multiple parts to be tested, and is used to measure whether there are defects in the point cloud, which is denoted as pt_dst.

Specifically, in order to obtain an accurate template point cloud, this embodiment needs to first obtain a test point cloud including at least one attachment part to be tested, and a template point cloud with the identification of each sample attachment part. In theory, if the template point cloud and the point cloud to be tested are taken under the conditions of the same detection mechanism, the same position, the same camera, and the same design parameters, no correction is required. However, in the actual operation process, due to the slight deviation that may exist in the above-mentioned various links, as well as the interference of factors such as design accuracy deviation, there may be deviations between the template point cloud and the point cloud to be measured. Therefore, in order to solve the above problems in this embodiment, The template point cloud is calibrated based on the point cloud registration algorithm and the measured point cloud containing at least one attached part to be tested, so that the template point cloud is corrected according to the measured point cloud, thereby obtaining an accurately calibrated template point cloud , denoted as pt_model'.

Specifically, since there is at least one sample attachment part point cloud on the calibrated template point cloud in this embodiment, this embodiment can sequentially extract and segment each sample attachment part point cloud, and use the divided independent parts as The point cloud of the target part template is convenient for subsequent comparison with the point cloud of the corresponding attached part to be tested.

In the embodiment of the present disclosure, obtaining the template point cloud of the sample attachment part includes: determining the source point cloud data of the sample attachment part through the original 3D design drawing of the sample attachment part or by taking pictures of the sample attachment part through a 3D camera; The source point cloud data of the sample-attached part is modeled to obtain the template point cloud of the sample-attached part.

Wherein, the source point cloud data can be obtained through a 3D camera or a 3D design drawing, and contains point cloud data of all sample attachment parts.

Specifically, in order to obtain the template point cloud of the sample attachment part, in this embodiment, the source point cloud containing all the sample attachment parts can be obtained through the original 3D design drawing of the sample attachment part or the sample attachment part photographed by a 3D camera Data, and then the obtained source point cloud data is processed by modeling to obtain the template point cloud containing each attached part, that is, the template point cloud of the sample attached part, and then mark each sample attached part to facilitate Subsequent segmentation comparisons are performed. Exemplarily, in this embodiment, if the source point cloud data of the attached parts of the sample is obtained by directly shooting with a 3D camera, modeling can be performed manually, and each attached part to be tested can be marked.

In an embodiment of the present disclosure, obtaining the point cloud of the component to be measured includes: performing dimensionality reduction processing on the corrected template point cloud and the point cloud to be measured containing at least one attached component to be tested, to obtain the dimensionality reduction template point cloud and Reduce the dimensionality of the point cloud to be measured; in the dimensionality reduction template point cloud, determine the dimensionality reduction component template point cloud corresponding to the target component template point cloud; based on the neighborhood minimum method, in the dimensionality reduction point cloud to be measured, determine and The dimensionality reduction part point cloud corresponding to the dimensionality reduction part template point cloud, and through the mapping relationship between the dimensionality reduction point cloud to be measured and the point cloud to be measured containing at least one attached part to be On the point cloud of the component to be measured, determine the area of the point cloud to be measured corresponding to the point cloud of the dimensionality reduction component as the point cloud of the component to be measured.

Among them, the dimensionality reduction template point cloud can be a template point cloud with a Z-axis coordinate value of 0, which is recorded as pt_model". . The neighborhood minimum method can be a method for matching the dimensionality-reduced point cloud to be measured with the corresponding dimensionality reduction component template point cloud. Exemplarily, the neighborhood minimum method used in this embodiment is 8 Neighborhood minimum method. The dimension reduction part template point cloud refers to the independent part area segmented in the dimension reduction template point cloud, which is recorded as ROI_model”[N]. The dimension reduction part point cloud can be an attached part point with a Z-axis coordinate value of 0 cloud, corresponding to the point cloud of the dimensionality reduction part template, denoted as ROI_dst”[N]. The point cloud of the component to be tested may be a point cloud area corresponding to the point cloud of the dimensionality-reduced component on the point cloud to be tested, which is denoted as ROI_dst[N].

Specifically, due to the large amount of information in the point cloud, the system operation speed is slow, and because of the interference of the height difference in the notebook structure itself, it is easy to cause system judgment errors. Therefore, in order to improve the operating efficiency of the system and reduce the judgment error rate of the system, this paper Embodiment The obtained corrected template point cloud and the point cloud to be measured containing at least one attached part to be tested are subjected to dimensionality reduction processing, that is, the Z-axis coordinate values of these point clouds are set to 0, thereby obtaining the Z-axis coordinate value The template point cloud with a value of 0, and the point cloud to be measured containing at least one attached part to be tested with a Z-axis coordinate value of 0, to obtain a reduced-dimensional template point cloud and a reduced-dimensional measured point cloud. Since the dimensionality reduction template point cloud is a holistic template, in the dimensionality reduction template point cloud, the corresponding dimensionality reduction component template point cloud can be determined according to the selected local target component template point cloud. In the dimensionality reduction point cloud to be measured, this embodiment uses the neighborhood minimum value method, according to the dimensionality reduction component template point cloud and the X coordinate and Y coordinate of the dimensionality reduction component point cloud, thereby obtaining the corresponding value of the dimensionality reduction component template point cloud The dimensionality reduction part point cloud of . Then in this embodiment, according to the mapping relationship between the dimensionality reduction point cloud to be measured and the point cloud to be measured that contains at least one attached component to be measured, on the point cloud to be measured, it is determined that the point cloud corresponding to the dimensionality reduction component The area of the point cloud to be tested, that is, the point cloud of the component to be tested.

In this embodiment, the dimensionality reduction component template point cloud that needs to be matched is acquired on the dimensionality reduction template point cloud after dimensionality reduction processing, and then the dimensionality reduction component point cloud is obtained through two-dimensional matching, and finally the dimensionality reduction component point cloud is combined with The mapping relationship of the three-dimensional point cloud to be measured and the point cloud of the component to be measured can be determined, which can eliminate the interference of the height of the structure itself in the notebook, and at the same time reduce the possibility of judgment errors.

S120. Reconstruct the point cloud of the part to be tested and the point cloud of the target part template into a depth map of the part to be tested and a depth map of the target part template, respectively.

Wherein, the depth map of the part to be tested can be obtained through the point cloud of the part to be tested, and is used to generate the image of the binary image of the part to be tested, denoted as dep_dst. The depth map of the target part template can be obtained through the point cloud of the target part template, and is used to generate the image of the binary image of the target part template, denoted as dep_model.

In the embodiment of the present disclosure, respectively rebuilding the point cloud of the part to be measured and the point cloud of the target part template into the depth map of the part to be measured and the depth map of the target part template, including: respectively obtaining the point cloud of the part to be measured and the point cloud of the target part template The coordinates of each voxel, wherein the coordinates include X value, Y value and Z value; the X value and Y value of each voxel in the part to be measured point cloud are used as the coordinate value of the part to be measured depth map, and the Z value is used as The pixel value of the depth map of the part to be measured is used to reconstruct the depth map of the part to be measured; the X value and the Y value of each voxel in the point cloud of the target part template are used as the coordinate values of the depth map of the target part template, and the Z value is used as the target part Pixel values of the stencil depth map to reconstruct the target part stencil depth map.

Specifically, in order to determine whether there is an abnormality in the attachment of the component to be tested, this embodiment needs to first convert the point cloud of the component to be tested and the point cloud of the target component template into a component to be tested depth map and Target part template depth map. Specifically, firstly, through the point cloud of the part to be measured and the point cloud of the target part template, the coordinate values of the X, Y, and Z axes of each voxel corresponding to it are obtained, and then the X, Y coordinates of each voxel in the part to be measured point cloud are obtained The axis coordinate value is used as the coordinate value of the component to be measured depth map after dimensionality reduction, and the Z-axis coordinate value of each voxel is used as the pixel value of the component to be measured depth map, so as to obtain the reconstructed component to be measured depth map. Similarly, in this embodiment, the X and Y axis coordinate values of each voxel in the point cloud of the target component template are used as the coordinate values of the depth map of the target component template after dimensionality reduction, and the Z axis coordinate values of each voxel are used as the target The pixel value of the depth map of the part template, so as to obtain the reconstructed depth map of the target part template.

S130. By performing binarization processing and Blob analysis on the depth map of the part to be tested and the depth map of the target part template, determine whether there is an attachment defect in the point cloud of the part to be tested.

Wherein, the binarization process may be an operation of converting the depth image of the component to be tested and the depth image of the template of the target component into a grayscale image. Blob analysis can be the operation of analyzing physical characteristics such as the width, height and area of the binary image of the component to be tested and the binary image of the target component template. Wherein, the Blob analysis method adopted in this embodiment can be in the Open cv algorithm library FindContours.

Specifically, in order to determine whether there is an attachment defect in the point cloud of the component to be tested, the obtained depth map of the component to be tested and the depth map of the target component template are obtained in this embodiment through binarization processing, thereby obtaining the binary image of the component to be tested and the binary image of the target component template, and then use the obtained binary image of the component to be tested and the binary image of the target component template to determine whether there is an attachment defect in the point cloud of the component to be tested by means of Blob analysis.

The existing defect detection method is a neural network training method based on 3D point cloud. This method has a certain effect on the detection of whether there is an attachment defect, but it requires a lot of manpower to collect, label and train samples. This method is not only time-consuming and laborious but also Not easy to operate. In addition, this method has a good detection effect on standard attachment and no high protrusions around the attachment position, but it has a poor defect detection effect on the presence of high protrusions near the attachment position or a small deviation in attachment. The method adopted in this embodiment not only effectively solves the problem of low detection accuracy when there is a high protrusion near the position of the attached part or the attached part has a slight offset, but also makes up for the neural network training based on 3D point cloud. The method has the disadvantage of wasting a lot of manpower and material resources, and also effectively improves the speed of algorithm detection.

Embodiment two

Fig. 2 is a flow chart of a defect detection method provided by Embodiment 2 of the present disclosure. The embodiment of the present disclosure is based on the above-mentioned embodiments, wherein the depth map of the component to be tested and the depth map of the target component template are binarized processing and Blob analysis to determine whether there is an attachment defect in the point cloud of the part to be tested, including: based on the mean value of the depth corresponding to the point cloud of the part template, the depth map of the part to be tested and the depth map of the target part template are binarized respectively to obtain The binary image of the part to be tested and the binary image of the target part template; respectively obtain the first preset area in the binary image of the part to be tested and the second preset area in the binary image of the target part template; the first preset area Compared with the area information of the second preset area, it is determined whether there is an attachment defect in the point cloud of the component to be tested, wherein the area information includes at least one of the following: width, height, and area. The method specifically includes:

S210. Obtain a point cloud of the component to be tested and a target component template point cloud.

S220. Reconstruct the point cloud of the part to be tested and the point cloud of the target part template into a depth map of the part to be tested and a depth map of the target part template, respectively.

S230. Based on the average depth value corresponding to the point cloud of the part template, perform binarization processing on the depth map of the part to be tested and the depth map of the target part template, respectively, to obtain the binary map of the part to be tested and the binary map of the target part template.

Wherein, the binary image of the component to be tested may be a grayscale image obtained by binarizing the point cloud of the component to be tested, which is recorded as bin_dep_model. The binary image of the target part template can be a grayscale image obtained by binarizing the point cloud of the target part template, which is denoted as bin_dep_dst.

Specifically, in order to simplify operations and improve work efficiency, this embodiment obtains the Z-axis coordinate value of the point cloud of the part template, so that the mean depth value corresponding to the point cloud of the part template can be obtained, and then based on the mean value of the depth of the point cloud of the part template, the obtained The component-to-be-tested depth map and the target part template depth map are binarized to obtain the corresponding component-to-be-tested binary map and the target part template binary map. Wherein, the binary image of the component to be tested and the binary image of the target component template are grayscale images of black and white.

S240. Respectively acquire a first preset area in the binary image of the component to be tested and a second preset area in the binary image of the target component template.

Wherein, the first preset area may be a white area in the binary image of the component to be tested. The second preset area may be a white area in the binary image of the target component template.

In this embodiment, the mean value of the depth corresponding to the point cloud of the component template is used as a reference, and converted into a binary image of the component to be tested and a binary image of the target component template. Since the attached component area will be higher than the area of the internal structure of the notebook, it will be displayed as a white area in the above binary image. In this embodiment, the white area in the binary image of the component to be tested can be directly used as the first preset area. The white area in the binary image of the target component template is used as the second preset area.

S250. Comparing the area information of the first preset area and the second preset area to determine whether there is an attachment defect in the point cloud of the component to be tested.

Wherein, the area information includes at least one of the following: width, height and area.

Specifically, this embodiment compares the first preset area in the binary image of the component to be tested with the second preset area in the binary image of the target component template through Blob analysis, that is, the first Compare the width, height and area of the white area in the preset area and the second preset area, so as to determine whether there is a defect in the attached part in the binary image of the part to be tested, and then determine whether there is an attachment defect in the point cloud of the part to be tested .

In an embodiment of the present disclosure, the area information of the first preset area and the second preset area are compared to determine whether there is an attachment defect in the point cloud of the component to be tested, including: when the first preset area and the second preset area The width difference of the area is less than the preset width threshold, the height difference between the first preset area and the second preset area is less than the preset high threshold, and the area difference between the first preset area and the second preset area is less than the preset When the area threshold is , it is determined that there is no attachment defect in the point cloud of the part to be tested; otherwise, it is determined that there is an attachment defect in the point cloud of the part to be tested.

Wherein, the preset wide threshold, the preset high threshold, and the preset area threshold may be any values set according to actual conditions, which are not limited in this embodiment.

Specifically, this embodiment compares the obtained width, height and area of the white area in the first preset area with the width, height, and area of the white area in the second preset area, if the first preset area and When the differences of the width, height and area of the white area in the corresponding second preset area are all smaller than the corresponding preset threshold, it is determined that there is no attachment defect in the point cloud of the component to be measured. If the difference between the width, height and area of the white area in the first preset area and the corresponding second preset area, any one of which is greater than or equal to the corresponding preset threshold, it is determined that the component is to be tested. The cloud has an attachment flaw.

Fig. 3 is a frame diagram of the implementation process of a defect detection method provided by Embodiment 2 of the present disclosure. Since the existing defect detection method has a good detection effect on standard attachment and parts with no high protrusions around the attachment position, but for When there is a high protrusion near the attachment position or when the attachment is slightly offset, the defect detection effect is poor. Therefore, this embodiment provides an effective solution to the above problems. The detailed steps are as follows:

1. Read the 3D template point cloud pt_model as the source point cloud, read the 3D test point cloud pt_dst as the target point cloud, and perform point cloud registration to obtain the 3D transformation matrix T;

2. Cross-multiply the 3D template point cloud pt_model and the 3D transformation matrix T to obtain the corrected template point cloud pt_model', as shown in formula [1];

pt_model'=pt_model×T[1]

3. Extract the attached part area of the template point cloud pt_model', and divide it into independent part areas ROI_model[N] (ie, the target part template point cloud), where N is the total number of independent part areas contained in the template information number;

4. Perform dimensionality reduction processing on the template point cloud pt_model’ and the point cloud to be measured pt_dst respectively, that is, set the z coordinate value to 0 to obtain the dimensionality reduction template point cloud pt_model” and the dimensionality reduction point cloud pt_dst” to be measured;

5. Extract the attached part area of the dimension reduction template point cloud pt_model", and divide it into independent part areas ROI_model"[N] (ie, the dimension reduction part template point cloud), N is the independent part area contained in the template information The total number of;

6. Based on each independent component ROI_model”[N], through the 8-neighborhood minimum method, obtain the corresponding component point cloud ROI_dst”[N] (that is, the dimensionality reduction component point cloud) in the dimensionality reduction point cloud pt_dst”, By further mapping to the point cloud pt_dst to be measured, the point cloud ROI_dst[N] corresponding to the point cloud ROI_model[N] of the target component template can be obtained;

7. Convert the target part template point cloud ROI_model[i] and the component to be measured point cloud ROI_dst[i] into depth maps dep_model[i] (ie, the depth map of the target part template) and dep_dst[i] (ie, the depth of the component to be measured ), where i is any independent component area.

8. Based on the depth in the target part template point cloud ROI_model[i], binarize the target part template depth map dep_model[i] and the part to-be-tested depth map dep_dst[i] to obtain the part to-be-tested binary map bin_dep_model[ i] and target component template binary map bin_dep_dst[i]. Its calculation method is as formula [2], [3]:

Where bin_dep_model[i](x,y) is the pixel value corresponding to the template depth binary image, bin_dep_dst[i](x,y) is the pixel value corresponding to the depth binary image to be measured, and ROI_model[i]_z is the target component The z-axis depth value of the template point cloud, dep_dst[i](x,y) is the depth value of each pixel in the depth map of the component to be measured, and mean represents the average value;

9. Perform Blob analysis on the bin_dep_model[i] binary image and bin_dep_dst[i] binary image respectively, and obtain information such as the width, height, and area of the white area;

10. When the information such as the width, height and area of the white area of the template and the binary image to be tested are all relatively small, it is considered that the area to be tested is normally attached to the component; otherwise, the component in the area to be tested is considered to be not attached, and the calculation method is as follows Formula [4];

Among them, flag_pass is the normal flag, 0 means abnormal, 1 means normal, w_mdl, h_mdl, area_mdl are the width, height and area of the white area of the template depth binary image respectively, w_dst, h_dst, area_dst are the white color of the depth binary image to be tested respectively Area width, height, and area, T_w, T_h, and T_area are the width threshold, height threshold, and area threshold of the white area of the binary image, respectively.

This embodiment not only effectively solves the problem of low detection accuracy when there is a high protrusion near the position of the attached part or the attached part is slightly offset, but also makes up for the waste of a lot of manpower and the time spent on the neural network training method based on the 3D point cloud. The shortcomings of material resources can improve the detection efficiency.

Embodiment three

Fig. 4 is a schematic structural diagram of a defect detection device provided by an embodiment of the present disclosure, the device specifically includes:

The point cloud acquisition module 410 is used to acquire the point cloud of the component to be tested and the target component template point cloud, wherein the target component template point cloud is a reference template point cloud for judging whether the component to be tested point cloud has an attachment defect;

Depth map reconstruction module 420, for respectively reconstructing the point cloud of the part to be measured and the point cloud of the target part template into a depth map of the part to be measured and a depth map of the target part template;

The defect analysis module 430 is configured to perform binarization processing and Blob analysis on the depth map of the part to be tested and the depth map of the target part template to determine whether there is an attachment defect in the point cloud of the part to be tested.

In a possible implementation manner, the defect analysis module 430 is specifically used for:

Based on the mean value of the depth corresponding to the part template point cloud, the depth map of the part to be tested and the depth map of the target part template are respectively binarized to obtain the binary map of the part to be tested and the binary map of the target part template;

Respectively obtain a first preset area in the binary image of the component to be tested and a second preset area in the binary image of the target component template;

Comparing the area information of the first preset area and the second preset area to determine whether there is an attachment defect in the point cloud of the part to be tested, wherein the area information includes at least one of the following: width, height and area.

In a possible implementation manner, the defect analysis module 430 is also specifically used for:

When the width difference between the first preset area and the second preset area is less than the preset width threshold, the height difference between the first preset area and the second preset area is less than the preset high threshold, and the first preset area and the second preset area When the area difference between the two preset areas is less than the preset area threshold, it is determined that there is no attachment defect in the point cloud of the part to be tested; otherwise, it is determined that there is an attachment defect in the point cloud of the part to be tested.

In a possible implementation manner, the point cloud acquisition module 410 is specifically used for:

Obtaining a point cloud to be measured containing at least one attached part to be tested and a template point cloud of the sample attached part, wherein the template point cloud is marked with labels of each sample attached part;

Calibrate the template point cloud based on the point cloud registration algorithm and the point cloud to be measured including at least one attached part to be tested to obtain a corrected template point cloud;

On the corrected template point cloud, at least one sample part template point cloud is determined, and each sample part template point cloud is sequentially used as a target part template point cloud.

In a possible implementation manner, the point cloud acquisition module 410 is also specifically used for:

Determine the source point cloud data of the sample-attached part through the original 3D design drawing of the sample-attached part or take pictures of the sample-attached part through a 3D camera;

The source point cloud data of the sample-attached part is modeled to obtain the template point cloud of the sample-attached part.

In a possible implementation manner, the point cloud acquisition module 410 is also specifically used for:

performing dimensionality reduction processing on the corrected template point cloud and the point cloud to be measured containing at least one attached part to be tested, to obtain the dimensionality reduction template point cloud and the dimensionality reduction point cloud to be measured;

In the dimension reduction template point cloud, determine the dimension reduction component template point cloud corresponding to the target component template point cloud;

Based on the neighborhood minimum method, in the dimensionality reduction point cloud to be measured, determine the dimensionality reduction component point cloud corresponding to the dimensionality reduction component template point cloud, and through the dimensionality reduction to be measured point cloud and at least one to-be-tested attachment The mapping relationship between the point clouds to be measured of the parts, on the point cloud to be measured that contains at least one attached part to be tested, determine the area of the point cloud to be measured corresponding to the point cloud of the dimensionality-reduced part as the point cloud to be measured of the part .

In a possible implementation manner, the depth map reconstruction module 420 is specifically used for:

Respectively obtain the coordinates of each voxel in the point cloud of the part to be measured and the point cloud of the target part template, wherein the coordinates include X value, Y value and Z value;

Using the X value and Y value of each voxel in the part to be measured point cloud as the coordinate value of the part to be measured depth map, and using the Z value as the pixel value of the part to be measured depth map to reconstruct the part to be measured depth map;

The X value and Y value of each voxel in the point cloud of the target part template are used as the coordinate value of the depth map of the target part template, and the Z value is used as the pixel value of the depth map of the target part template to reconstruct the depth map of the target part template.

According to the embodiments of the present disclosure, the present disclosure also provides an electronic device and a readable storage medium.

FIG. 5 shows a schematic block diagram of an example electronic device 500 that may be used to implement embodiments of the present disclosure. Electronic device is intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other suitable computers. Electronic devices may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are by way of example only, and are not intended to limit implementations of the disclosure described and/or claimed herein.

As shown in FIG. 5 , the device 500 includes a computing unit 501 that can execute according to a computer program stored in a read-only memory (ROM) 502 or loaded from a storage unit 508 into a random-access memory (RAM) 503. Various appropriate actions and treatments. In the RAM 503, various programs and data necessary for the operation of the device 500 can also be stored. The computing unit 501 , ROM 502 and RAM 503 are connected to each other through a bus 504 . An input/output (I/O) interface 505 is also connected to the bus 504 .

Multiple components in the device 500 are connected to the I/O interface 505, including: an input unit 506, such as a keyboard, a mouse, etc.; an output unit 507, such as various types of displays, speakers, etc.; a storage unit 508, such as a magnetic disk, an optical disk, etc. ; and a communication unit 509, such as a network card, a modem, a wireless communication transceiver, and the like. The communication unit 509 allows the device 500 to exchange information/data with other devices over a computer network such as the Internet and/or various telecommunication networks.

The computing unit 501 may be various general-purpose and/or special-purpose processing components having processing and computing capabilities. Some examples of computing units 501 include, but are not limited to, central processing units (CPUs), graphics processing units (GPUs), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, digital signal processing processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 501 executes various methods and processes described above, such as a defect detection method. For example, in some embodiments, the defect detection method may be implemented as a computer software program tangibly embodied on a machine-readable medium, such as storage unit 508 . In some embodiments, part or all of the computer program may be loaded and/or installed onto the device 500 via the ROM 502 and/or the communication unit 509 . When the computer program is loaded into RAM 503 and executed by computing unit 501, one or more steps of the defect detection method described above may be performed. Alternatively, in other embodiments, the computing unit 501 may be configured to execute the defect detection method in any other suitable manner (for example, by means of firmware).

Various implementations of the systems and techniques described above herein may be implemented in digital electronic circuitry, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chips ( SOC), Complex Programmable Logic Device (CPLD), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include being implemented in one or more computer programs executable and/or interpreted on a programmable system including at least one programmable processor, the programmable processor Can be special-purpose or general-purpose programmable processor, can receive data and instruction from storage system, at least one input device, and at least one output device, and transmit data and instruction to this storage system, this at least one input device, and this at least one output device an output device.

Program codes for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special purpose computer, or other programmable data processing devices, so that the program codes, when executed by the processor or controller, make the functions/functions specified in the flow diagrams and/or block diagrams Action is implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media would include one or more wire-based electrical connections, portable computer discs, hard drives, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, compact disk read only memory (CD-ROM), optical storage, magnetic storage, or any suitable combination of the foregoing.

To provide for interaction with the user, the systems and techniques described herein can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user. ); and a keyboard and pointing device (eg, a mouse or a trackball) through which a user can provide input to the computer. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and can be in any form (including Acoustic input, speech input or, tactile input) to receive input from the user.

The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., as a a user computer having a graphical user interface or web browser through which a user can interact with embodiments of the systems and techniques described herein), or including such backend components, middleware components, Or any combination of front-end components in a computing system. The components of the system can be interconnected by any form or medium of digital data communication, eg, a communication network. Examples of communication networks include: Local Area Network (LAN), Wide Area Network (WAN) and the Internet.

A computer system may include clients and servers. Clients and servers are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, a server of a distributed system, or a server combined with a blockchain.

It should be understood that steps may be reordered, added or deleted using the various forms of flow shown above. For example, each step described in the present disclosure may be executed in parallel, sequentially, or in a different order, as long as the desired result of the technical solution disclosed in the present disclosure can be achieved, no limitation is imposed herein.

In addition, the terms "first" and "second" are used for descriptive purposes only, and cannot be interpreted as indicating or implying relative importance or implicitly specifying the quantity of indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include at least one of these features. In the description of the present disclosure, "plurality" means two or more, unless otherwise specifically defined.

The above is only a specific implementation of the present disclosure, but the scope of protection of the present disclosure is not limited thereto. Anyone skilled in the art can easily think of changes or substitutions within the technical scope of the present disclosure. should fall within the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be determined by the protection scope of the claims.

Claims (10)

1. A method of defect detection, the method comprising:

acquiring a part to-be-measured point cloud and a target part template point cloud, wherein the target part template point cloud is a reference template point cloud for judging whether the part to-be-measured point cloud has an attachment defect or not;

reconstructing the component to-be-measured point cloud and the target component template point cloud into a component to-be-measured depth map and a target component template depth map respectively;

and determining whether the cloud of the part to be measured point has an attachment defect or not by carrying out binarization processing and Blob analysis on the part to be measured depth map and the target part template depth map.

2. The method according to claim 1, wherein the determining whether the component to-be-measured point cloud has an attachment defect by performing binarization processing and Blob analysis on the component to-be-measured depth map and the target component template depth map includes:

respectively carrying out binarization processing on the part to-be-detected depth map and the target part template depth map based on the depth mean value corresponding to the part template point cloud to obtain a part to-be-detected binary map and a target part template binary map;

respectively acquiring a first preset area in the binary image to be tested of the component and a second preset area in the binary image of the target component template;

Comparing the area information of the first preset area with the area information of the second preset area to determine whether the cloud of the part to be measured points has an attachment defect, wherein the area information comprises at least one of the following components: wide, high, and area.

3. The method of claim 2, wherein comparing the area information of the first preset area and the second preset area to determine whether the component-to-be-measured-point cloud has an attachment defect comprises:

when the width difference value of the first preset area and the second preset area is smaller than a preset width threshold value, the height difference value of the first preset area and the second preset area is smaller than a preset height threshold value, and the area difference value of the first preset area and the second preset area is smaller than a preset area threshold value, determining that the cloud to be tested of the component has no attachment defect; otherwise, determining that the cloud of the part to be measured points has attachment defects.

4. A method according to claim 1 or 3, wherein the obtaining a target part template point cloud comprises:

acquiring a point cloud to be detected comprising at least one attaching part to be detected and a template point cloud of a sample attaching part, wherein labels of all the sample attaching parts are marked in the template point cloud;

Calibrating the template point cloud based on a point cloud registration algorithm and the point cloud to be detected comprising at least one attached component to be detected, so as to obtain a corrected template point cloud;

and determining at least one sample part template point cloud on the corrected template point cloud, and taking each sample part template point cloud as a target part template point cloud in sequence.

5. The method of claim 4, wherein the obtaining a template point cloud of the sample application component comprises:

determining source point cloud data of the sample attached component by shooting the sample attached component through an original 3D design drawing of the sample attached component or through a 3D camera;

modeling the source point cloud data of the sample attached component to obtain a template point cloud of the sample attached component.

6. The method of claim 5, wherein the acquiring the component-to-be-measured point cloud comprises:

performing dimension reduction processing on the corrected template point cloud and the point cloud to be measured containing at least one attached component to be measured to obtain dimension reduction template point cloud and dimension reduction point cloud to be measured;

determining a dimension reduction component template point cloud corresponding to the target component template point cloud in the dimension reduction template point cloud;

And determining a dimension reduction component point cloud corresponding to the dimension reduction component template point cloud in the dimension reduction component point cloud based on a neighborhood minimum method, and determining a to-be-measured point cloud area corresponding to the dimension reduction component point cloud as a component to-be-measured point cloud on the to-be-measured point cloud containing at least one to-be-measured attachment component through a mapping relation between the dimension reduction to-be-measured point cloud and the to-be-measured point cloud containing at least one to-be-measured attachment component.

7. The method of claim 6, wherein reconstructing the component-to-be-measured point cloud and the target component template point cloud into a component-to-be-measured depth map and a target component template depth map, respectively, comprises:

respectively acquiring coordinates of each voxel in the component to-be-measured point cloud and the target component template point cloud, wherein the coordinates comprise an X value, a Y value and a Z value;

taking the X value and the Y value of each voxel in the component to-be-measured point cloud as coordinate values of the component to-be-measured depth map, and taking the Z value as pixel values of the component to-be-measured depth map to reconstruct the component to-be-measured depth map;

and taking the X value and the Y value of each voxel in the target component template point cloud as coordinate values of the target component template depth map, and taking the Z value as pixel values of the target component template depth map so as to reconstruct the target component template depth map.

8. A defect detection apparatus, the apparatus comprising:

the device comprises a point cloud acquisition module, a target component template point cloud, a target component detection module and a target component detection module, wherein the point cloud acquisition module is used for acquiring a component to-be-detected point cloud and acquiring the target component template point cloud, wherein the target component template point cloud is a reference template point cloud for judging whether the component to-be-detected point cloud has an attachment defect or not;

the depth map reconstruction module is used for reconstructing the component to-be-measured point cloud and the target component template point cloud into a component to-be-measured depth map and a target component template depth map respectively;

and the defect analysis module is used for determining whether the cloud of the part to be measured points has attachment defects or not by carrying out binarization processing and Blob analysis on the part to be measured depth map and the target part template depth map.

9. An electronic device, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.

10. A non-transitory computer readable storage medium storing computer instructions for causing a computer to perform the method of any one of claims 1-7.

Images