CN110288651A - Tolerance visual inspection method, device and computing device for large-size workpieces - Google Patents

Abstract

The invention discloses a tolerance visual inspection method for large-size workpieces, which is suitable for execution in computing equipment. The method includes the steps of: acquiring an image of a target part in the large-size workpiece, and extracting a picture including mounting holes from the image. The target area of interest; extract the sub-pixel edges of the target part and the target area respectively, and perform contour tracking on the extracted sub-pixel edges to obtain multiple feature contours; perform linear edge and/or elliptical arc shape on multiple feature contours Edge fitting, obtains line segment set and/or elliptical arc set; Extracts at least one feature point of line segment set and/or elliptical arc set, and calculates the workpiece coordinates of each feature point; And calculate the target position and The geometrical dimensions and tolerances of each mounting hole.

Description

Tolerance visual inspection method, device and computing device for large-size workpieces

technical field

The invention relates to the technical field of visual inspection, in particular to a tolerance visual inspection method, device and computing device for large-sized workpieces.

Background technique

The dimensional and geometrical tolerance detection of large-sized workpieces has a higher and higher demand in the field of machinery manufacturing. It is especially important for the position detection of the mounting holes on the workpiece surface. The current mainstream detection methods include manual and three-coordinate measuring instruments. The manual method requires the use of special detection equipment for measurement, which has many problems such as high cost, low efficiency, inaccurate detection results, and inability to achieve full inspection. The CMM with high measurement accuracy is very expensive, troublesome to maintain, and limited in application.

Because machine vision technology has the advantages of non-contact, large amount of information, high cost performance, and easy operation, it has been applied by many scholars to the field of size detection. The mainstream method for the size measurement of large-sized workpieces is to capture a sequence of images of large-sized workpieces based on a camera, which adopts the method of image stitching to obtain a panorama of large-sized workpieces, and finally determines the corresponding results and calibration values of each item according to the panorama. The actual size of each pixel corresponds to the size of the large-sized part to be measured. However, the detection accuracy of this method has accumulated errors, and the detection accuracy is not enough, which affects the accuracy of the detection results.

Therefore, it is necessary to provide a DUT size detection method with higher detection accuracy.

SUMMARY OF THE INVENTION

To this end, the present invention provides a tolerance visual inspection method, device and computing device for large-sized workpieces to solve or at least alleviate the above problems.

According to an aspect of the present invention, there is provided a tolerance visual inspection method for a large-sized workpiece, suitable for execution in a computing device, the method comprising the steps of: acquiring an image of a target portion in the large-sized workpiece, and extracting an image from the image The target area of interest including the mounting hole; extract the sub-pixel edges of the target part and the target area respectively, and perform contour tracking on the extracted sub-pixel edges to obtain multiple feature contours; Or elliptical arc edge fitting, obtain line segment set and/or elliptical arc set; extract at least one feature point of line segment set and/or elliptical arc set, and calculate the workpiece coordinates of each feature point; and according to the workpiece coordinates of each feature point Calculate the geometric dimensions and tolerances of the target site and each mounting hole in it.

Optionally, in the tolerance visual inspection method according to the present invention, the method further includes the step of calculating the geometric dimensions and tolerances of the large-sized workpiece by synthesizing different target parts of the large-sized workpiece and the geometrical dimensions and tolerances of the mounting holes therein.

Optionally, in the tolerance visual inspection method according to the present invention, the step of extracting the target area including the mounting hole from the image includes: converting the image of the target part into a grayscale image, and performing adaptive thresholding on the grayscale image. Segment and extract target regions from the segmented image.

Optionally, in the tolerance visual detection method according to the present invention, the method further includes the steps of: performing distortion correction on the image of the target part, and performing filtering processing and morphological processing on the grayscale image.

Optionally, in the tolerance visual detection method according to the present invention, the step of performing contour tracking on the extracted sub-pixel edges includes: performing Freenman chain code tracking on the extracted sub-pixel edges, and filtering noise edges to obtain multiple feature outline.

Optionally, in the tolerance visual detection method according to the present invention, the step of filtering noise edges includes: respectively setting an area threshold and a length threshold, and determining the feature contours whose actual area is less than the area threshold or whose actual length is less than the length threshold. Set to Noise Edge.

Optionally, in the tolerance visual inspection method according to the present invention, it further comprises the steps of: setting a first threshold and a second threshold of the aspect ratio of the circumscribed rectangle, and setting the feature contour whose actual aspect ratio is greater than the second threshold. It is a straight edge, and the feature contour whose actual aspect ratio is less than the first threshold is set as an elliptical arc edge.

Optionally, in the tolerance visual inspection method according to the present invention, the step of performing linear edge/or circular arc line segment fitting on the plurality of feature contours includes: performing linear edge culling on the feature contours belonging to the linear edge. The least squares fitting of error points is performed to obtain a line segment set; and/or the elliptical arc edge is subjected to the least squares fitting of excluding coarse error points for the feature contour belonging to the elliptical arc edge to obtain an elliptical arc set.

Optionally, in the tolerance visual inspection method according to the present invention, the feature points of the elliptical arc set include the center point and each vertex of the ellipse, and the feature points of the line segment set include two end points and midpoints of each line segment in the line segment set.

Optionally, in the tolerance visual inspection method according to the present invention, the step of acquiring an image of a certain target part of a large-sized workpiece includes: acquiring an image of the target part from a camera, and acquiring the camera to position the robot to measure the image when the image is captured. The tool coordinate system of the head; when the camera takes the image, the positioning robot adjusts the working distance of the industrial camera according to the distance information measured by the ranging sensor to ensure that the imaging plane of the camera is parallel to the surface to be detected of the large-size workpiece .

Optionally, in the tolerance visual inspection method according to the present invention, the conversion relationship between the tool coordinates and the workpiece coordinates is stored in the computing device, and the step of calculating the workpiece coordinates of each feature point includes: determining the image coordinates of each feature point. The tool coordinates are calculated, and the workpiece coordinates of each feature point are calculated according to the conversion relationship.

Optionally, in the tolerance visual inspection method according to the present invention, the origin of the tool coordinate system coincides with the origin of the image coordinate system, and the xOy plane of the tool coordinate system coincides with the plane where the image coordinate system is located; the conversion relationship is:

C _i =R ₁ ·(Q·V _i )+T ₁

Among them, C _i and V _i are the workpiece coordinates and tool coordinates of the ith feature point, respectively, T ₁ and R ₁ are the translation matrix and rotation matrix from the tool coordinate system to the workpiece coordinate system when the image of the target part is captured, Q is the pixel equivalent matrix, R _h and R _v are the horizontal and vertical pixel equivalents of the camera, respectively.

Optionally, in the tolerance visual inspection device according to the present invention, the size, shape, position and tolerance of the target portion and each mounting hole therein include the external dimension of the target portion, the positioning size and external size of each mounting hole in the target portion, and the Dimensional deviation.

Optionally, in the tolerance visual inspection device according to the present invention, the external dimension of the target part is calculated by combining with the center point of the outer contour line segment of the workpiece; the external dimension of the mounting hole is calculated by combining with each vertex of the mounting hole; the positioning size of the mounting hole is calculated. It is calculated by combining the coordinates of the center point of the reference datum line segment and the center point of the ellipse.

Optionally, in the tolerance visual inspection method according to the present invention, it also includes the step of: setting the identification of the dimension measurement parameters, the identification including the workpiece outer dimension identification, the positioning dimension identification and the outer dimension identification of each mounting hole, and the identification of each dimension. The size identifier is stored in association with the calculated corresponding size value.

According to yet another aspect of the present invention, there is provided another method for visual inspection of tolerances of large-sized workpieces, suitable for execution in a computing device, the method comprising the step of: acquiring a sequence of images of the large-sized workpieces, the sequence images including the large-sized workpieces. Images of different parts; using the above-mentioned tolerance visual inspection method for large-size workpieces to visually inspect each frame of sequence images to obtain the shape and position dimensions and tolerances of different parts; and synthesizing the shape and position dimensions and tolerances of all parts, Calculate the geometric dimensions and tolerances of large workpieces.

According to yet another aspect of the present invention, there is provided a tolerance visual inspection device for a large-sized workpiece, suitable for resident in a computing device, the device comprising: a target area extraction module, suitable for acquiring an image of a certain target part in the large-sized workpiece , and extract the target area of interest including the mounting hole from the image; the feature contour extraction module is suitable for extracting the sub-pixel edges of the target part and the target area respectively, and performs contour tracking on the extracted sub-pixel edges to obtain multiple Feature contour; Feature contour fitting module, suitable for fitting multiple feature contours with linear edge/or circular arc line segment to obtain line segment set and/or elliptical arc set; Feature point extraction module, suitable for extracting line segment set and /or at least one feature point of the elliptical arc set, and calculate the workpiece coordinates of each feature point; and a shape and size calculation module, suitable for calculating the shape size and tolerance of the target part and each mounting hole according to the workpiece coordinates of each feature point .

Optionally, in the tolerance visual inspection device according to the present invention, the shape, position and size calculation module is also suitable for synthesizing different target parts of the large-size workpiece and the shape and position dimensions and tolerances of each mounting hole therein to calculate the shape of the large-size workpiece. Bit dimensions and tolerances.

Optionally, in the tolerance visual inspection device according to the present invention, the target area extraction module is adapted to convert the image of the target part into a grayscale image, perform adaptive threshold segmentation on the grayscale image, and extract the image from the segmented image. the target area.

Optionally, in the tolerance visual inspection apparatus according to the present invention, the target area extraction module is further adapted to perform distortion correction on the image of the target portion, and perform filtering processing and morphological processing on the grayscale image.

According to yet another aspect of the present invention, there is provided a computing device comprising: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by one or more When executed by a processor, one or more programs are executed by the processor to implement the steps of the above-mentioned method for tolerance visual inspection of large-sized workpieces.

According to yet another aspect of the present invention, there is provided a computer-readable storage medium storing one or more programs including instructions that, when executed by a computing device, implement the tolerances for large-sized workpieces as described above Steps of a visual inspection method.

According to the technical solution of the present invention, the target area of interest including the mounting hole is extracted from the image of the target part of the large-sized workpiece, and the line segment set and/or ellipse are obtained by performing edge fitting on multiple feature contours of the target part and the target area Arc collection. Then, according to the workpiece coordinates of each feature point in the line segment set and/or the elliptical arc set, the size, shape, position and tolerance of the target part and each mounting hole are calculated. This method accurately calculates the geometric dimensions and tolerances of each target part through image processing.

Moreover, the present invention can also acquire images of all target parts of the workpiece to generate sequence images, and process each frame of sequence images to obtain the shape, position, size and tolerance of the corresponding target parts in each frame of images. Then, the results of all the sequence images are combined to obtain the shape, position and tolerance of the large-sized workpiece. This method avoids the cumulative accuracy problem of calibrating each item method based on the spliced panorama, and can obtain the accurate shape and size of the workpiece and its various parts, and the image processing method improves the detection efficiency of the workpiece size.

The above description is only an overview of the technical solutions of the present invention, in order to be able to understand the technical means of the present invention more clearly, it can be implemented according to the content of the description, and in order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand , the following specific embodiments of the present invention are given.

Description of drawings

To achieve the above and related objects, certain illustrative aspects are described herein in conjunction with the following description and drawings, which are indicative of the various ways in which the principles disclosed herein may be practiced, and all aspects and their equivalents are intended to be within the scope of the claimed subject matter. The above and other objects, features and advantages of the present disclosure will become more apparent by reading the following detailed description in conjunction with the accompanying drawings. Throughout this disclosure, the same reference numbers generally refer to the same parts or elements.

FIG. 1 shows a structural diagram of a tolerance visual inspection system 100 according to an embodiment of the present invention;

FIG. 2 shows a structural diagram of a computing device 200 according to an embodiment of the present invention;

FIG. 3 shows a flowchart of a method 300 for visual tolerance inspection of large-sized workpieces according to an embodiment of the present invention;

Figures 4a to 4d respectively show sequence images of multiple target parts of a large-sized workpiece;

Figures 5a to 5e respectively show schematic diagrams of positioning the mounting holes of the target site in Figure 5c;

FIG. 6 and FIG. 7 respectively show flowcharts of tolerance visual inspection methods 600 and 700 for large-sized workpieces according to another embodiment of the present invention; and

FIG. 8 shows a structural diagram of a tolerance visual inspection device 800 for large-sized workpieces according to an embodiment of the present invention.

Detailed ways

Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure will be more thoroughly understood, and will fully convey the scope of the present disclosure to those skilled in the art.

FIG. 1 shows a structural diagram of a tolerance visual inspection system 100 of the present invention. As shown in FIG. 1, the system 100 includes a positioning robot 110, a collection module and a computing device. The acquisition module is installed on the positioning robot 110, and the acquisition module includes a camera 120 and a ranging sensor, and both the camera 120 and the ranging sensor are connected to the computing device.

The positioning robot 110 drives the acquisition module to move to a predetermined position of the surface to be inspected, so that the camera 120 can photograph the surface to be inspected on the large-sized workpiece at the predetermined position to obtain an image of the surface to be inspected. According to one embodiment, the positioning robot 110 may be a robotic arm, and the end of the robotic arm is fixedly installed with the acquisition module. The arm span of the robotic arm may be 1420mm, which is of course not limited to this, and may also be other values.

The camera 120 may be an industrial camera, such as a CCD camera, a CMOS camera, etc., of course, it is not limited thereto. The camera 120 may perform camera distortion correction using Zhang Zhengyou's calibration method before image acquisition, so as to avoid lens distortion caused by deviations in manufacturing precision and assembly process of the lens. The ranging sensor may be a laser ranging sensor, which moves along the edge of the surface to be detected under the driving of the positioning robot 110 and records the distance data from the camera to the surface to be detected. The positioning robot 110 adjusts the distance between the camera 120 and the surface to be inspected according to the distance data, so that the imaging plane of the camera 120 is parallel to the surface to be inspected of the large-sized workpiece, so as to ensure that a clear image of the surface to be inspected can be collected. This image acquisition method combined with laser ranging can also avoid the perspective distortion problem of "near large and far small" caused by the non-parallel imaging plane and the surface to be measured, and improve the quality of image acquisition.

The computing device may acquire the image captured by the camera 120 and the tool coordinate system of the camera 120 when capturing the image, and determine the size, shape, position and tolerance of the DUT according to the captured image and the tool coordinate system. The computing device can also determine the parallelism or perpendicularity of the surface to be measured according to the distance data measured by the distance measuring sensor. The test piece usually includes a plurality of surfaces to be tested, and each surface to be tested is distributed with several mounting holes. The mounting holes can be round holes, square holes, threaded holes, assembly holes, etc. No restrictions. The structural dimensions of the DUT may include external dimensions and positioning dimensions. The positioning size refers to the size between two structures on the DUT, such as the size between two holes, so that the relative positions of the holes on the DUT are determined according to the positioning size. The external dimension refers to the size of the structure on the test piece, such as the overall size of the workpiece, the diameter of the hole, the radius and so on. Geometric tolerances usually include features such as straightness, parallelism, perpendicularity, and inclination.

FIG. 2 shows a structural block diagram of a computing device 200 according to an embodiment of the present invention. In basic configuration 202 , computing device 200 typically includes system memory 206 and one or more processors 204 . Memory bus 208 may be used for communication between processor 204 and system memory 206 .

Depending on the desired configuration, the processor 204 may be any type of process including, but not limited to, a microprocessor (μP), a microcontroller (μC), a digital information processor (DSP), or any combination thereof. Processor 204 may include one or more levels of cache, such as L1 cache 210 and L2 cache 212 , processor core 214 , and registers 216 . Exemplary processor cores 214 may include arithmetic logic units (ALUs), floating point units (FPUs), digital signal processing cores (DSP cores), or any combination thereof. The exemplary memory controller 218 may be used with the processor 204 , or in some implementations, the memory controller 218 may be an internal part of the processor 204 .

Depending on the desired configuration, system memory 206 may be any type of memory including, but not limited to, volatile memory (such as RAM), non-volatile memory (such as ROM, flash memory, etc.), or any combination thereof. System memory 206 may include operating system 220 , one or more applications 222 , and program data 224 . In some embodiments, application 222 may be arranged to operate with program data 224 on an operating system. The program data 224 includes instructions, in the computing device 200 according to the present invention, the program data 224 includes instructions for performing the tolerance vision inspection methods 300, 600 and/or 700 for large-sized workpieces.

Computing device 200 may also include an interface bus 240 that facilitates communication from various interface devices (eg, output device 242 , peripheral interface 244 , and communication device 246 ) to base configuration 102 via bus/interface controller 230 . Example output devices 242 include graphics processing unit 248 and audio processing unit 250 . They may be configured to facilitate communication via one or more A/V ports 252 with various external devices such as displays or speakers. Example peripheral interfaces 244 may include serial interface controller 254 and parallel interface controller 256, which may be configured to facilitate communication via one or more I/O ports 258 and input devices such as keyboard, mouse, pen, etc. , voice input devices, touch input devices) or other peripherals (eg printers, scanners, etc.) The example communication device 246 may include a network controller 160 that may be arranged to facilitate communication via one or more communication ports 264 with one or more other computing devices 262 over a network communication link.

A network communication link may be one example of a communication medium. Communication media may typically embody computer readable instructions, data structures, program modules in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media. A "modulated data signal" can be a signal of which one or more of its data sets or whose alterations can be made in such a way as to encode information in the signal. By way of non-limiting example, communication media may include wired media, such as wired or leased line networks, and various wireless media, such as acoustic, radio frequency (RF), microwave, infrared (IR), or other wireless media. The term computer readable medium as used herein may include both storage media and communication media.

Computing device 200 can be implemented as a server, such as a file server, database server, application server, and WEB server, etc., or as part of a small-sized portable (or mobile) electronic device such as a cellular phone, a personal digital Assistants (PDAs), personal media player devices, wireless web browsing devices, personal headsets, application specific devices, or hybrid devices that may include any of the above. Computing device 200 may also be implemented as a personal computer including desktop computer and notebook computer configurations. In some embodiments, computing device 200 is configured to perform methods 300, 600, and/or 700 for tolerance visual inspection of large-sized workpieces.

FIG. 3 shows a schematic flowchart of a method 300 for visual tolerance inspection of large-sized workpieces according to an embodiment of the present invention. The method 300 is performed in a computing device, such as in the computing device 200, to detect the geometric dimensions and tolerances of large-sized workpieces.

As shown in FIG. 3, the method 300 begins at step S310. In step S310, an image of a certain target part in the large-sized workpiece is acquired, and a target area of interest including a mounting hole is extracted from the image. The mounting holes may be, for example, threaded holes, mounting holes, or the like.

In practical applications, the large-sized workpiece can be divided into multiple target parts, and the images of each target part can be obtained to form a sequence image, and each target part may have mounting holes. Image processing is performed on each frame of image separately, and the size, shape, position and tolerance of each target part and its mounting holes can be obtained. Dimensions and tolerances. As shown in 4a to 4d, the sequence images of the left pillar of a subway screen door are respectively shown, corresponding to the four target parts from top to bottom.

Usually, when the camera captures images, it can capture images of multiple parts of the workpiece in sequence, and the upper and lower edges of the multiple parts can be just connected or overlapped by a certain distance, which is not limited in the present invention. In addition, the DUT usually includes a plurality of surfaces to be measured. When collecting images of the multiple surfaces to be measured, try to ensure that the path of the positioning robot 110 does not repeat as much as possible (the principle of the shortest time), so as to improve the efficiency of image collection. Specifically, a reference surface can be selected from all the surfaces to be detected, and after the reference surface is detected, the structural dimensions of the remaining surfaces to be detected and their perpendicularity relative to the reference surface are detected.

According to one embodiment, the image of the target part can be converted into a grayscale image, and after adaptive threshold segmentation is performed on the grayscale image, the target region is extracted from the segmented image. Further, it is also possible to perform distortion correction on the image of the target portion first, and then perform grayscale conversion on the image after distortion correction. In addition, filtering processing and morphological processing can also be performed on the converted grayscale image. Filter processing such as median filter processing, Gaussian filter processing, etc. Morphological processing such as opening operation processing and closing operation processing and the like. The open operation is used to remove dark stains on the surface of the workpiece and to enhance the texture characteristics of the threaded and assembly hole areas to be measured. The gray value of the void feature is too large, and the open operation can further increase the gray value of the void area and increase the void area. The closed operation is used to eliminate light-colored oil stains and bright spots on the surface of the workpiece. The light and dark contrast of the image can be significantly enhanced after the opening and closing operation processing, and then adaptive threshold segmentation such as OSTO Otsu segmentation is performed on the image to obtain the approximate area of the threaded hole and the assembly hole, and then the minimum circumscribed rectangle of the area can be calculated as the inclusion. The ROI target area for this mounting hole.

There are various methods for image distortion correction, grayscale conversion, filtering processing, morphological processing, adaptive threshold segmentation and ROI extraction of the target region of interest. The present invention is not limited to this, and all image processing methods that can realize the corresponding image processing function The methods are all within the protection scope of the present invention.

Subsequently, in step S320, the sub-pixel edges of the target part and the target area are respectively extracted, and contour tracking is performed on the extracted sub-pixel edges to obtain multiple feature contours.

Here, the main purpose is to obtain the edge contour of the workpiece and the contours of multiple mounting holes. The sub-pixel edge extraction can be implemented using any existing technology, for example, an interpolation method is used to extract the sub-pixel edge, which is not limited in the present invention.

According to one embodiment, when performing contour tracking, Freenman chain code tracking may be performed on the extracted sub-pixel edges, and noise edges are filtered to obtain multiple feature contours. Among them, the Freeman chain code is a method of describing the curve or boundary by the coordinates of the starting point of the curve and the direction code of the boundary point. The commonly used chain codes are divided into 4-connected chain codes and 8-connected chains according to the number of adjacent directions of the central pixel point. code. In addition, when filtering noise edges, an area threshold and a length threshold can be set respectively, and feature contours whose actual area is less than the area threshold or whose actual length is less than the length threshold are set as noise edges.

According to another embodiment, a first threshold value and a second threshold value of the aspect ratio of the circumscribed rectangle may also be set (the first threshold value is smaller than the second threshold value), and the feature contour whose actual aspect ratio is greater than the second threshold value is set as a straight line shape edge, and set the feature contour whose actual aspect ratio is less than the first threshold as elliptical arc edge. Among them, the straight edge such as the outermost edge of the workpiece and the edge of the square hole, etc. (if the width of the side beam needs to be measured, the edge of the beam needs to be measured), the elliptical arc edge such as the edge of the oval or circular hole, etc. (in addition to the screen door There will be some mounting holes on the bottom surface consisting of two straights and two semi-circular bends).

Subsequently, in step S330, linear edge and/or elliptical arc edge fitting is performed on the plurality of feature contours to obtain a line segment set and/or an elliptical arc set.

Specifically, for the feature contour belonging to the linear edge, the least squares fitting method in which the linear edge is eliminated and the coarse error point is eliminated may be performed to obtain a line segment set. For the feature contours belonging to the elliptical arc edge, the least squares fitting can be performed on the elliptical arc edge to remove the coarse error points, and the elliptical arc set can be obtained. That is to say, large error points are eliminated when the straight line fitting and elliptic arc fitting of the least square method are performed, and the accuracy of the line fitting is improved.

Subsequently, in step S340, at least one feature point of the line segment set and/or the elliptical arc set is extracted, and the workpiece coordinates of each feature point are calculated.

The feature points of the elliptical arc set may include, for example, the center point and each vertex of the ellipse (eg, the most edge points in four directions, up, down, left, and right), so as to determine the position and size of the ellipse. The set of elliptical arcs can be an independent single-segment elliptical arc (such as a semicircular transition arc at the top corner of the workpiece), or a complete ellipse (such as a complete ellipse corresponding to a mounting hole). The set of elliptical arcs may also be several scattered elliptical arcs that belong to the same ellipse (eg, multiple scattered elliptical arcs corresponding to the mounting holes), and corresponding ellipses can be obtained by fitting these scattered elliptical arcs. The feature points of the line segment set may include, for example, two end points and midpoints of each line segment in the line segment set, and the position and length of each line segment, as well as the width and height of the workpiece can be determined according to the feature points.

According to one embodiment, the conversion relationship between tool coordinates and workpiece coordinates is stored in the computing device, so that after the image coordinates V _i (x, y, 0) of each feature point are determined, the tool coordinates B can be determined according to the image coordinates. _i (x, y, z), and then the workpiece coordinates of each feature point can be calculated according to the conversion relationship between the tool coordinates and the workpiece coordinates.

Figure 1 is marked with the specific positional relationship between the tool coordinate system Tool and the workpiece coordinate system Base. If the workpiece has multiple surfaces to be measured, set the workpiece coordinate system and tool coordinate system once for each surface to be measured. The tool coordinate system Base can be manually defined according to the workpiece and used to determine the coordinates of each point on the workpiece. Tool coordinate system Tool is the coordinate system of the positioning robot measuring head (ie acquisition module, including industrial cameras and ranging sensors). The initial position is usually obtained by coincidence or translation with the tool coordinate system. Specifically, the tool coordinate system can be set from the workpiece coordinate system. Translated by h units along the x-axis. The tool coordinate system Tool moves with the movement of the probe. In addition, the origin of the tool coordinate system coincides with the origin of the image coordinate system, and the xOy plane of the tool coordinate system coincides with the plane where the image coordinate system is located. The large-sized workpiece in Figure 1 is a subway screen door. The origin of the Base coordinate system is located in the lower left corner of the workpiece. In the initial state, the origin of the Tool coordinate system is located in the upper left corner of the workpiece. The difference between the two origins is h units on the x-axis. When the measuring head moves to the detection position in Figure 1, the origin of the Tool coordinate system moves to the upper right corner of the workpiece.

According to one embodiment, the conversion relationship between tool coordinates and workpiece coordinates is:

C _i =R ₁ ·(Q·V _i )+T ₁

Among them, C _i and V _i are the workpiece coordinates and tool coordinates of the ith feature point, respectively, T ₁ and R ₁ are the translation matrix and rotation matrix from the tool coordinate system to the workpiece coordinate system when the image of the target part is captured, Q is the pixel equivalent matrix, R _h and R _v are the horizontal and vertical pixel equivalents of the camera, respectively. Here, the pixel equivalent is the actual size corresponding to a pixel on the image, and the pixel equivalent corresponding to the image can be obtained by calibrating the size of the standard part at the inspection site. For example, a 1*1mm square standard part is placed in the imaging field of view of the camera 120, and the square standard part is photographed as an image, and the number of pixels in the 1*1mm area on the image is calculated after image processing, so that , the ratio of the size (1mm) to the number of pixels is the pixel equivalent.

Next, in step S350, according to the workpiece coordinates of each feature point, the target position and the geometrical dimension and tolerance of each mounting hole therein are calculated.

Wherein, the size, shape, position and tolerance of the target part and each installation hole therein include the outer dimension of the target part, the positioning dimension and outer dimension of each installation hole in the target part, and the deviation of each dimension. Specifically, the outer dimension of the target portion is calculated in combination with the center point of the outer contour line segment of the workpiece, for example, the width is calculated according to the center point of the left and right line segments. The outer dimension of the mounting hole is calculated by combining the vertices of the mounting hole, for example, the diameter or radius is calculated according to the left and right vertices and the center point of the circle. The positioning size of the mounting hole is calculated by combining the coordinates of the center point of the reference datum line segment outline and the center point of the ellipse. The deviation of each dimension can be calculated by comparing the measured actual dimension value with the preset standard value.

Figures 5a to 5e respectively show the basic process of locating the threaded hole through image processing, in which Figure 5a is the original image, and Figure 5b is the extraction of the ROI region of the threaded hole based on the morphologically processed image, including three ROI target regions . Figure 5c shows the image segmentation for the extracted ROI area, and Figure 5d shows the least squares fitting of the extracted sub-pixel edges to remove the coarse error points from the arc edge to obtain three fitted ellipses. Fig. 5e is the calculation result of the pixel coordinates of the center of the threaded hole, wherein the coordinates of the center of the three threaded holes are A(x1, y1), B(x2, y2), and C(x3, y3). It should be understood that the z-axis coordinate of each point in the image can be considered to be 0. In addition, the width of the section of the jamb can also be calculated as D according to the feature point coordinates of the two left and right line segments of the section of the jamb in the image (eg, the midpoint of the two line segments).

In addition, the present invention can also set the identification of the size measurement parameters, the identification can include the workpiece outer dimension identification, the positioning dimension identification of each mounting hole and the outer dimension identification, and each size identification and the calculated corresponding size value are stored in association with each other. . For example, it can be defined that the workpiece outer dimension identification M_type=0, the threaded hole positioning dimension identification M_type=1, the assembly hole positioning and the outer dimension identification M_type=2. The M_type value can be determined according to the size measurement type, and is stored in association with the corresponding workpiece outer dimension (E_flag=0), threaded hole positioning dimension (E_flag=1), assembly hole positioning and outer dimension (E_flag=2).

FIG. 6 shows a schematic flowchart of a method 600 for visual tolerance inspection of large-size workpieces according to another embodiment of the present invention. The method 600 is performed in a computing device, such as the computing device 200, to detect the geometrical dimensions and tolerances of the part under test.

As shown in FIG. 6, the method 600 begins at step S610. In step S610, a sequence of images of the large-size workpiece is acquired, and the sequence of images includes images of different parts of the large-size workpiece.

Subsequently, in step S620 , the tolerance visual inspection method for large-sized workpieces as in the method 300 is respectively used to perform tolerance visual inspection on each frame of sequence images to obtain the size, shape, position and tolerance of different parts. For example, the tolerance visual inspection is performed on the sequence images of the left pillars of the four screen doors shown in Figures 4a to 4d, respectively, to obtain the external dimensions and positioning dimensions of the corresponding structural parts in each image, such as the endpoints and midpoints of the left and right line segments, The width of each pillar, etc., and the center point and radius of each mounting hole, etc.

Next, in step S630, the dimensions, shape, position and tolerance of all parts are integrated, and the size, shape, position and tolerance of the large-sized workpiece are calculated. That is, the size, shape, position and tolerance of the line segment structure, hole structure, elliptical arc structure, etc. in each image in Figures 4a to 4d are combined to obtain the size, shape, position and tolerance of the left pillar of the screen door.

FIG. 7 shows a detailed flowchart of a method 700 for tolerance visual inspection of large-sized workpieces according to another embodiment of the present invention, which is executed in a computing device, such as the computing device 200 .

As shown in FIG. 7, the method 700 begins at step S710. In step S710, a sequence of images I _0-N is acquired, and the corresponding tool coordinate system R _i in each image acquisition is recorded, i=0-N. The N sequence images correspond to N parts of the DUT, and the N parts together constitute the DUT.

Subsequently, in step S720, for any frame of image, distortion correction, grayscale conversion and morphological processing are sequentially performed on the image to create at least one rectangular ROI target area of the image.

Subsequently, in step S730, sub-edge pixel detection and Freenman chain code-based contour tracking are performed on each target area in the image to obtain multiple feature contours corresponding to each target area.

Subsequently, in step S740, the linear edge and/or the elliptical arc edge is judged on the multiple feature contours of each target area in the image, and the linear edge and/or the elliptical arc edge are fitted to obtain Sets of line segments and/or elliptical arcs for each target region in this image.

Then, in step S750, the feature points of the line segment set and/or the elliptical arc set in the image are determined, the image coordinates of each feature point are calculated, and the image coordinates are converted into workpiece coordinates.

Subsequently, in step S760, for other frame images, the operations in steps S720 to S750 are sequentially performed on them to obtain the workpiece coordinates of each feature point in the other frame images respectively.

Next, in step S770, the size, shape, position and tolerance of the DUT are calculated according to the workpiece coordinates of each feature point in all the sequence images.

That is, in the method 700, the workpiece coordinates of the feature points in all the sequence images are obtained, and then the size, shape, position and tolerance calculation are carried out uniformly. The specific operation details of each image processing and calculation have been disclosed in detail in the description based on other drawings. , which will not be repeated here.

FIG. 8 shows a tolerance visual inspection apparatus 800 for large-size workpieces according to an embodiment of the present invention, and the apparatus 800 may be included in the computing device 200 shown in FIG. 2 . As shown in FIG. 8 , the apparatus 800 includes a target region extraction module 810 , a feature contour extraction module 820 , a feature contour fitting module 830 , a feature point extraction module 840 and a shape and size calculation module 850 .

The target area extraction module 810 acquires an image of a certain target part in the large-sized workpiece, and extracts a target area of interest including mounting holes from the image. The target area extraction module 810 may perform processing corresponding to the processing described in step S310 above, which will not be repeated here.

The feature contour extraction module 820 extracts the sub-pixel edges of the target part and the target area respectively, and performs contour tracking on the extracted sub-pixel edges to obtain multiple feature contours. The feature contour extraction module 820 may perform processing corresponding to the processing described in step S320 above, which will not be repeated here.

The feature contour fitting module 830 performs linear edge/or circular arc line segment fitting on the plurality of feature contours to obtain a line segment set and/or an elliptical arc set. The feature contour fitting module 830 may perform processing corresponding to the processing described in step S330 above, which will not be repeated here.

The feature point extraction module 840 extracts at least one feature point of the line segment set and/or the elliptical arc set, and calculates the workpiece coordinates of each feature point. The feature point extracting module 840 may perform processing corresponding to the processing described in step S340 above, which will not be repeated here.

The shape, position and dimension calculation module 850 calculates the size, shape, position and tolerance of the target part and each installation hole in the target part according to the workpiece coordinates of each feature point. The shape, position and dimension calculation module 850 can also calculate the size, shape, position and tolerance of the large-size workpiece by integrating different target parts of the large-size workpiece and the size, shape, position and tolerance of each mounting hole therein. The shape, position and size calculation module 850 may perform processing corresponding to the processing described in step S350 above, which will not be repeated here.

According to the technical solution of the present invention, an image processing method based on adaptive threshold processing, median filtering, morphological processing, and sub-pixel edge detection algorithms is established by using robots to cooperate with machine vision and laser rangefinders to measure large dimensions. , process the image of the large-sized workpiece, and extract and identify the feature information of the large-sized workpiece by combining the linear edge and elliptical edge fitting algorithms that remove the coarse error points, and finally obtain the various dimensions of the workpiece. The calculation accuracy is far far to meet the measurement needs.

A7. The method according to any one of A1-A6, further comprising the steps of: setting a first threshold value and a second threshold value of the aspect ratio of the circumscribed rectangle, and setting a feature contour whose actual aspect ratio is greater than the second threshold value It is a straight edge, and the feature contour whose actual aspect ratio is less than the first threshold is set as an elliptical arc edge. A8. The method according to A7, wherein the step of performing linear edge/or arc-shaped line segment fitting on the plurality of feature contours includes: performing linear edge culling on the feature contours belonging to the linear edge to remove coarse error points The least squares fitting is performed to obtain a line segment set; and/or the feature contour belonging to the elliptical arc edge is subjected to the least squares fitting of the elliptical arc edge excluding coarse error points to obtain an elliptical arc set. A9. The method according to any one of A1-A8, wherein the feature points of the elliptical arc set include the center point and each vertex of the ellipse, and the feature points of the line segment set include two points of each line segment in the line segment set endpoints and midpoints.

A10. The method according to any one of A1-A9, wherein the step of acquiring an image of a certain target part of the large-size workpiece includes: acquiring an image of the target part from a camera, and acquiring the image of the target part when the camera is shooting The tool coordinate system of the robot's measuring head is positioned during the image; wherein, when the camera captures the image, the positioning robot adjusts the working distance of the camera according to the distance information measured by the ranging sensor, so as to ensure that the imaging plane of the camera is the same as that of the camera. The surfaces to be detected of the large-sized workpiece are parallel. A11. The method according to any one of A1-A10, wherein a conversion relationship between tool coordinates and workpiece coordinates is stored in the computing device, and the step of calculating the workpiece coordinates of each feature point includes: according to each feature point The tool coordinates are determined by the image coordinates, and the workpiece coordinates of each feature point are calculated according to the conversion relationship. A12. The method of A11, wherein the origin of the tool coordinate system coincides with the origin of the image coordinate system, and the xOy plane of the tool coordinate system coincides with the plane where the image coordinate system is located; the conversion relationship is:

C _i =R ₁ ·(Q·V _i )+T ₁

Among them, C _i and V _i are the workpiece coordinates and tool coordinates of the ith feature point, respectively, T ₁ and R ₁ are the translation matrix and rotation matrix from the tool coordinate system to the workpiece coordinate system when the image of the target part is captured, Q is the pixel equivalent matrix, R _h and R _v are the horizontal and vertical pixel equivalents of the camera, respectively.

A13. The method according to any one of A1-A12, wherein the shape and position dimensions and tolerances of the target part and each mounting hole therein include the external dimension of the target part, the positioning dimension and the external shape of each mounting hole in the target part size, and the deviation of each size. A14. The method according to A13, wherein the outer dimension of the target part is calculated by combining with the center point of the outer contour line segment of the workpiece; the outer dimension of the mounting hole is calculated by combining with the vertices of the mounting hole; The positioning dimension is calculated by combining the center point coordinates of the reference datum line segment outline and the center point coordinates of the ellipse. A15. The method according to A13, further comprising the step of: setting the identification of the dimension measurement parameters, the identification includes the identification of the outer dimension of the workpiece, the identification of the positioning dimension of each mounting hole and the identification of the outer dimension, and calculating the identification of each dimension with the calculated The corresponding size value of , is stored associatively.

B18. The device according to B17, wherein the shape, position and size calculation module is further adapted to calculate the shape, position and size of the large-size workpiece by synthesizing different target parts of the large-size workpiece and the shape, position, size and tolerance of each mounting hole therein. Bit dimensions and tolerances. B19. The device according to B17, wherein the target region extraction module is adapted to convert the image of the target portion into a grayscale image, perform adaptive threshold segmentation on the grayscale image, and extract the image from the segmented image the target area. B20. The device according to B19, wherein the target region extraction module is further adapted to perform distortion correction on the image of the target portion, and perform filtering processing and morphological processing on the grayscale image.

The various techniques described herein can be implemented in conjunction with hardware or software, or a combination thereof. Thus, the method and apparatus of the present invention, or certain aspects or portions of the method and apparatus of the present invention, may take the form of an embedded tangible medium, such as a removable hard disk, a USB stick, a floppy disk, a CD-ROM, or any other machine-readable storage medium. in the form of program code (ie, instructions) that, when the program is loaded into a machine, such as a computer, and executed by the machine, the machine becomes an apparatus for practicing the invention.

Where the program code is executed on a programmable computer, the computing device typically includes a processor, a storage medium readable by the processor (including volatile and nonvolatile memory and/or storage elements), at least one input device, and at least one output device. Wherein, the memory is configured to store program codes; the processor is configured to execute the tolerance visual inspection method for large-size workpieces of the present invention according to the instructions in the program codes stored in the memory.

By way of example and not limitation, readable media include readable storage media and communication media. Readable storage media store information such as computer readable instructions, data structures, program modules or other data. Communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media. Combinations of any of the above are also included within the scope of readable media.

In the specification provided herein, the algorithms and displays are not inherently related to any particular computer, virtual system, or other device. Various general purpose systems may also be used with examples of the present invention. The structure required to construct such a system is apparent from the above description. Furthermore, the present invention is not directed to any particular programming language. It should be understood that various programming languages may be used to implement the inventions described herein, and that the descriptions of specific languages above are intended to disclose the best mode for carrying out the invention.

In the description provided herein, numerous specific details are set forth. It will be understood, however, that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.

Similarly, it is to be understood that in the above description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together into a single embodiment, figure, or its description. This disclosure, however, should not be interpreted as reflecting an intention that the invention as claimed requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the Detailed Description are hereby expressly incorporated into this Detailed Description, with each claim standing on its own as a separate embodiment of this invention.

Those skilled in the art will appreciate that the modules or units or components of the apparatus in the examples disclosed herein may be arranged in the apparatus as described in this embodiment, or alternatively may be positioned differently from the apparatus in this example in one or more devices. The modules in the preceding examples may be combined into one module or further divided into sub-modules.

Those skilled in the art will appreciate that the modules in the device in the embodiment can be adaptively changed and arranged in one or more devices different from the embodiment. The modules or units or components in the embodiments may be combined into one module or unit or component, and further they may be divided into multiple sub-modules or sub-units or sub-assemblies. All features disclosed in this specification (including accompanying claims, abstract and drawings) and any method so disclosed may be employed in any combination unless at least some of such features and/or procedures or elements are mutually exclusive. All processes or units of equipment are combined. Each feature disclosed in this specification (including accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.

Furthermore, those skilled in the art will appreciate that although some of the embodiments described herein include certain features, but not others, included in other embodiments, that combinations of features of different embodiments are intended to be within the scope of the invention within and form different embodiments. For example, in the following claims, any of the claimed embodiments may be used in any combination.

Furthermore, some of the described embodiments are described herein as methods or combinations of method elements that can be implemented by a processor of a computer system or by other means for performing the described functions. Thus, a processor having the necessary instructions for implementing the method or method element forms means for implementing the method or method element. Furthermore, an element of an apparatus embodiment described herein is an example of a means for carrying out the function performed by the element for the purpose of carrying out the invention.

As used herein, unless otherwise specified, the use of the ordinal numbers "first," "second," "third," etc. to describe common objects merely refers to different instances of similar objects, and is not intended to imply such The objects being described must have a given order in time, space, ordinal, or in any other way.

While the invention has been described in terms of a limited number of embodiments, those skilled in the art will appreciate, having the benefit of the above description, that other embodiments are conceivable within the scope of the invention thus described. Furthermore, it should be noted that the language used in this specification has been principally selected for readability and teaching purposes, rather than to explain or define the subject matter of the invention. Accordingly, many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the appended claims. This disclosure is intended to be illustrative and not restrictive with regard to the scope of the present invention, which is defined by the appended claims.

Claims (10)

1. A tolerance visual inspection method for a large-sized workpiece, suitable for execution in a computing device, the method comprising the steps: Acquiring an image of a target part in the large-size workpiece, and extracting a target area of interest including mounting holes from the image; Extracting the sub-pixel edges of the target part and the target area respectively, and performing contour tracking on the extracted sub-pixel edges to obtain multiple feature contours; performing linear edge and/or elliptical arc edge fitting on the plurality of feature contours to obtain a line segment set and/or an elliptical arc set; Extracting at least one feature point of the line segment set and/or elliptical arc set, and calculating the workpiece coordinates of each feature point; and According to the workpiece coordinates of each feature point, the geometrical dimension and tolerance of the target part and each mounting hole therein are calculated. 2. The method of claim 1, further comprising the step of: The geometrical dimensions and tolerances of the large-sized workpiece are calculated by synthesizing the geometrical dimensions and tolerances of different target parts of the large-sized workpiece and each mounting hole therein. 3. The method of claim 1, wherein the step of extracting a target area containing mounting holes from the image comprises: Convert the image of the target part into a grayscale image, perform adaptive threshold segmentation on the grayscale image, and extract the target area from the segmented image. 4. The method of claim 3, further comprising the step of: Distortion correction is performed on the image of the target part, and filtering processing and morphological processing are performed on the grayscale image. 5. The method according to any one of claims 1-4, wherein the step of performing contour tracking on the extracted sub-pixel edges comprises: Freenman chain code tracking is performed on the extracted sub-pixel edges, and noise edges are filtered to obtain multiple feature contours. 6. The method of claim 5, wherein the step of filtering noisy edges comprises: The area threshold and the length threshold are respectively set, and the feature contour whose actual area is smaller than the area threshold or whose actual length is smaller than the length threshold is set as the noise edge. 7. A tolerance visual inspection method for large-sized workpieces, suitable for execution in a computing device, the method comprising the steps of: acquiring a sequence of images of the large-sized workpiece, the sequence of images including images of different parts of the large-sized workpiece; Adopt the method as described in any one of claim 1-6 to carry out tolerance visual inspection to each frame sequence image respectively, obtain the shape and position size and tolerance of different parts; And The size, shape, position and tolerance of all parts are integrated, and the shape, position, size and tolerance of the large-sized workpiece are calculated. 8. A tolerance visual inspection device for large-sized workpieces, adapted to reside in a computing device, the device comprising: a target area extraction module, adapted to acquire an image of a certain target part in the large-sized workpiece, and extract the target area of interest including the mounting hole from the image; a feature contour extraction module, adapted to extract the sub-pixel edges of the target part and the target area respectively, and perform contour tracking on the extracted sub-pixel edges to obtain multiple feature contours; a feature contour fitting module, adapted to perform linear edge/or circular arc line segment fitting on the plurality of feature contours to obtain a line segment set and/or an elliptical arc set; A feature point extraction module, adapted to extract at least one feature point of the line segment set and/or the elliptical arc set, and calculate the workpiece coordinates of each feature point; and The shape and position size calculation module is suitable for calculating the shape and position size and tolerance of the target part and each installation hole in it according to the workpiece coordinates of each feature point. 9. A computing device comprising: one or more processors; memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs when executed by the processors implement the claims The steps of any one of 1-7. 10. A computer-readable storage medium storing one or more programs comprising instructions that, when executed by a computing device, implement the method of any of claims 1-7 step.